Commit Graph

95 Commits

Author SHA1 Message Date
mergify[bot]
02d36d0be0 Quiet notification logs when no subscriptions (#13629) (#13636)
(cherry picked from commit 3e4acba72f)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-11-17 08:22:12 +00:00
mergify[bot]
c6818f8faf Disable the PubSub vote subscription by default (#13599)
The --rpc-pubsub-enable-vote-subscription flag may be used to enable it.
The current vote subscription is problematic because it emits a
notification for *every* vote, so hundreds a second in a real cluster.
Critically it's also missing information about *who* is voting,
rendering all those notifications practically useless.

Until these two issues can be resolved, the vote subscription is not
much more than a potential DoS vector.

(cherry picked from commit 5d72e52ad0)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-14 21:51:15 +00:00
mergify[bot]
965e6dfc9a Add counter metrics to rpc-subscriptions (#13596) (#13597)
(cherry picked from commit 88ae321d3f)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-11-14 20:53:38 +00:00
mergify[bot]
41eab22117 Send pubsub metrics to metrics server (#13584) (#13585)
(cherry picked from commit 34bf80ba9c)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-11-13 20:44:59 +00:00
Michael Vines
5ad0ccdfe1 cargo fmt 2020-10-21 20:27:57 -07:00
Michael Vines
422bb3c526 Run codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand 2020-10-21 20:27:57 -07:00
mergify[bot]
5b322a995f Rpc -> proper optimistic confirmation (#12514) (#12537)
* Add service to track the most recent optimistically confirmed bank

* Plumb service into ClusterInfoVoteListener and ReplayStage

* Clean up test

* Use OptimisticallyConfirmedBank in RPC

* Remove superfluous notifications from RpcSubscriptions

* Use crossbeam to avoid mpsc recv_timeout panic

* Review comments

* Remove superfluous last_checked_slots, but pass in OptimisticallyConfirmedBank for complete correctness

(cherry picked from commit 89621adca7)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-09-29 03:49:18 +00:00
mergify[bot]
e85c792f70 Add RPC notify and banking keys debug (bp #12396) (#12452)
* Add RPC notify and banking keys debug (#12396)

(cherry picked from commit 68e5a2ef56)

# Conflicts:
#	core/src/validator.rs

* Rebase

Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2020-09-25 21:05:21 +00:00
mergify[bot]
398f12dcc5 Program subscriptions now properly check results len and token program id (#12139) (#12141)
(cherry picked from commit 4431080066)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-09-09 22:23:11 +00:00
Justin Starry
58312655b4 Fix signature subscription panic (#12077) (#12092) 2020-09-07 10:31:25 +00:00
mergify[bot]
aa3bdd3730 Revert signature-notification format change (bp #12032) (#12038)
* Revert signature-notification format change (#12032)

* Use untagged RpcSignatureResult enum to avoid breaking downstream consumers of current signature subscriptions

* Clean up client duplication

* Clippy

(cherry picked from commit 39246f9dd7)

# Conflicts:
#	core/src/rpc_pubsub.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-09-04 01:34:49 +00:00
mergify[bot]
a5832366a7 Detect and notify when deserializable shreds are available (bp #11816) (#11988)
* Detect and notify when deserializable shreds are available (#11816)

* Add logic to check for complete data ranges

* Add RPC signature notification

Co-authored-by: Carl <carl@solana.com>
(cherry picked from commit 1c1a3f979d)

# Conflicts:
#	accounts-bench/Cargo.toml
#	core/src/rpc_pubsub.rs

* Fix conflicts

Co-authored-by: carllin <wumu727@gmail.com>
Co-authored-by: Carl <carl@solana.com>
2020-09-02 06:38:10 +00:00
mergify[bot]
2a649e990d Update spl-token to v2.0 (bp #11884) (#11897)
* Update spl-token to v2.0 (#11884)

* Update account-decoder to spl-token v2.0

* Update transaction-status to spl-token v2.0

* Update rpc to spl-token v2.0

* Update getTokenSupply to pull from Mint directly

* Fixup to spl-token v2.0.1

(cherry picked from commit 76be36c9ce)

# Conflicts:
#	Cargo.lock
#	account-decoder/Cargo.toml
#	core/Cargo.toml

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-08-28 23:36:53 +00:00
mergify[bot]
d9ae092637 Re-do rent collection check on rent-exempt account (bp #11349) (#11655)
* Re-do rent collection check on rent-exempt account (#11349)

* wip: re-do rent collection check on rent-exempt account

* Let's see how the ci goes

* Restore previous code

* Well, almost all new changes are revertable

* Update doc

* Add test and gating

* Fix tests

* Fix tests, especially avoid to change abi...

* Fix more tests...

* Fix snapshot restore

* Align to _new_ with better uninitialized detection

(cherry picked from commit 23fa84b322)

# Conflicts:
#	core/src/rpc_subscriptions.rs

* Fix conflicts

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-08-17 07:20:24 +00:00
mergify[bot]
1589a41178 Add config param to specify offset/length for single and program account info (bp #11515) (#11518)
* Add config param to specify offset/length for single and program account info (#11515)

* Add config param to specify dataSlice for account info and program accounts

* Use match instead of if

(cherry picked from commit 88ca04dbdb)

# Conflicts:
#	cli/src/cli.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-08-11 00:03:18 +00:00
mergify[bot]
23a381b995 Fix parsing of spl-token Mint (#11512) (#11514)
* Add failing test

* Fix jsonParsed mint

(cherry picked from commit da210ddd51)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-08-10 21:51:20 +00:00
mergify[bot]
3e483314b6 Decode native-program and sysvar accounts (bp #11463) (#11485)
* Decode native-program and sysvar accounts (#11463)

* Pass pubkey in to account-decoder for sysvars

* Decode sysvar accounts

* Decode config accounts; move validator-info lower

* Decode stake accounts

* Review comments

* Stringify any account lamports and epochs that can be set to u64::MAX

(cherry picked from commit a9f76862fb)

# Conflicts:
#	Cargo.lock
#	account-decoder/Cargo.toml

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-08-09 09:18:17 +00:00
mergify[bot]
fa9aa0a1d7 Token Accounts: return ui_amount, decimals with decoded account (#11407) (#11453)
* Return ui_amount, decimals from token client methods

* Return ui_amount, decimals in RPC jsonParsed token accounts

* Fixup docs

* Return ui_amount, decimals in pubsub jsonParsed token accounts

* Remove unnecessary duplicate struct

* StringAmount rename

(cherry picked from commit b7c2681903)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-08-07 19:05:35 +00:00
mergify[bot]
5841e4d665 Long-term ledger storage with BigTable (bp #11222) (#11392)
* ledger-storage-bigtable boilerplate

(cherry picked from commit 9d2293bb32)

* $ wget https://pki.goog/roots.pem -O pki-goog-roots.pem

(cherry picked from commit 1617a025ce)

* Add access_token module

(cherry picked from commit 59d266a111)

* Add root_ca_certificate

(cherry picked from commit faa016e4b7)

* Add build-proto

(cherry picked from commit c31e1f5bf0)

* UiTransactionEncoding is now copy

(cherry picked from commit 494968be66)

* Increase timeout

(cherry picked from commit 57dfebc5ba)

* Add build-proto/build.sh output

(cherry picked from commit 54dae6ba2c)

* Supress doctest errors

(cherry picked from commit 019c75797d)

* Add compression

(cherry picked from commit 243e05d59f)

* Add bigtable

(cherry picked from commit 6e0353965a)

* Add configuration info

(cherry picked from commit 98cca1e774)

* Add ledger-tool bigtable subcommands

(cherry picked from commit f9049d6ee4)

# Conflicts:
#	ledger-tool/Cargo.toml

* Make room for tokio 0.2

(cherry picked from commit b876fb84ba)

# Conflicts:
#	core/Cargo.toml

* Setup a tokio 0.2 runtime for RPC usage

(cherry picked from commit 0e02740565)

# Conflicts:
#	core/Cargo.toml

* Plumb Bigtable ledger storage into the RPC subsystem

(cherry picked from commit dfae9a9864)

# Conflicts:
#	core/Cargo.toml

* Add RPC transaction history design

(cherry picked from commit e56ea138c7)

* Simplify access token refreshing

(cherry picked from commit 1f7af14386)

* Report block status more frequently

(cherry picked from commit 22c46ebf96)

* after -> before

(cherry picked from commit 227ea934ff)

* Rebase

* Cargo.lock

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-08-06 04:06:44 +00:00
Tyera Eulberg
c90de8978d Add encoding and filters parameters to rpc Subscriptions (#11065)
* Plumb account configs and enable encoding

* Enable filters for pubsub program accounts

* Update docs
2020-07-23 13:38:28 -06:00
carllin
73f3d04798 Add replay votes to gossip vote tracking (#11119)
* Plumb replay vote channel for notifying vote listener of replay votes

* Keep gossip only notification for debugging gossip in the future

Co-authored-by: Carl <carl@solana.com>
2020-07-20 17:29:07 -07:00
Tyera Eulberg
a255b0fc86 Rename CacheSlotInfo to CommitmentSlots (#11111) 2020-07-17 17:54:49 +00:00
Tyera Eulberg
fdff681bcc Simplify BlockCommitmentCache slot info (#11106)
* Refactor BlockCommitmentCache, store a CacheSlotInfo

* Comma
2020-07-17 09:24:51 -06:00
Greg Fitzgerald
ebadbce920 Drop bank from BlockCommitmentCache (#10959)
* Remove bank reference from BlockCommitmentCache

* Don't use a Bank to create BlockCommitmentCache

* Rename recent_slot to slot
2020-07-09 00:50:13 +00:00
Greg Fitzgerald
2fdbb97244 Rename largest_confirmed_root to highest_confirmed_root (#10947) 2020-07-07 23:59:46 +00:00
Tyera Eulberg
d97850f1d9 Add account-decoder utilities (#10846)
* Fix comment and make less pub

* Add account-decoder crate and use to decode vote and system (nonce) accounts

* Update docs

* Rename RpcAccount struct

* s/Rpc/Display

* Call it jsonParsed and update docs

* Revert "s/Rpc/Display"

This reverts commit 6e7149f503.

* s/Rpc/Ui

* Add tests

* Ui more things

* Comments
2020-07-01 04:55:11 +00:00
Greg Fitzgerald
50b3fa83a0 Move BankCommitmentCache to solana_runtime (#10816)
* Remove Blockstore member variable from BlockCommitmentCache

* Hoist is_confirmed_rooted() to its only caller

BlockCommitmentCache no longer depends on Blockstore

* Move BlockCommitmentCache to solana_runtime
2020-06-25 22:06:58 -06:00
Justin Starry
39984cdcc3 Wait until bank is frozen before sending RPC notifications (#10654) 2020-06-18 00:44:51 +08:00
Greg Fitzgerald
6ee222363e Move BankForks to solana_runtime (#10637)
* Move BankForks to solana_runtime

* Update imports
2020-06-17 15:27:03 +00:00
Greg Fitzgerald
8dd5384d6d Split commitment module (#10541)
automerge
2020-06-12 17:16:10 -07:00
Greg Fitzgerald
2eb6f498a8 Remove redundant BankForks parameter (#10537) 2020-06-12 11:04:17 -06:00
Tyera Eulberg
2928c5d103 Add SingleGossip commitment level to use for subscriptions (#10147)
automerge
2020-05-22 12:55:17 -07:00
Justin Starry
10d60288e8 Update default subscription commitment level from recent to single (#10161)
* Update default subscription commitment level from recent to single

* Update jsonrpc docs

* Fix failing tests
2020-05-22 00:33:24 +08:00
Tyera Eulberg
bac4aec16f Trigger RPC notifications after block commitment cache update (#10077)
* Fixup commitment-aggregation metric

* Trigger notifications after commitment-cache update

* Fixup fn name

* Add single-confirmation commitment level

* Rename to highest_confirmed_slot

* Pass commitment-cache info directly to notifications

* Use match

* Update commitment docs

* Update out of date pubsub docs
2020-05-18 12:49:01 -06:00
Reisen
bfcfbab818 Add Vote PubSub endpoint for live gossip votes. (#10045)
* Add Vote PubSub endpoint for live gossip votes.

* Updated tests for Vote RPC and Vote Listener

* Add JSON RPC documentation for Vote RPC.

* Base58 encode hash in Vote RPC response.
2020-05-17 15:01:08 -06:00
Kristofer Peterson
58ef02f02b 9951 clippy errors in the test suite (#10030)
automerge
2020-05-15 09:35:43 -07:00
Tyera Eulberg
754c65c066 Refactor RPC subscriptions account handling (#9888)
* Switch subscriptions to use commitment instead of confirmations

* Add bank method to return account and last-modified slot

* Add last_modified_slot to subscription data and use to filter account subscriptions

* Update tests to non-zero last_notified_slot

* Add accounts subscriptions to test; fails at higher tx load

* Pass BankForks to RpcSubscriptions

* Use BankForks on add_account_subscription to properly initialize last_notified_slot

* Bundle subscriptions

* Check for non-equality

* Use commitment to initialize last_notified_slot; revert context.slot chage
2020-05-07 00:23:06 -06:00
Tyera Eulberg
9d91cca73c Remove commented code 2020-04-28 15:47:41 -06:00
Tyera Eulberg
d5abff82e0 Add largest_confirmed_root to BlockCommitmentCache (#9640)
* Add largest_confirmed_root to BlockCommitmentCache

* clippy

* Add blockstore to BlockCommitmentCache to check root

* Add rooted_stake helper fn and test

* Nodes that are behind should correctly id confirmed roots

* Simplify rooted_stake collector
2020-04-22 12:22:09 -06:00
Michael Vines
b58338b066 Make rpc_subscriptions.rs tests serial (#9556)
automerge
2020-04-17 10:48:39 -07:00
Michael Vines
ad0997e15f RPC: add err field to TransactionStatus, alongside the now deprecated status field (#9296)
automerge
2020-04-04 16:13:26 -07:00
Tyera Eulberg
50fa577af8 Use cluster confirmations in rpc and pubsub (#9138)
* Add runtime methods to simply get status and slot

* Add helper function to get slot confirmation_count from BlockCommitmentCache

* Return cluster confirmations in getSignatureStatus

* Remove use of invalid get_signature_confirmation_status

* Remove unused methods

* Update pubsub to use cluster confirmations

* Fix test_check_signature_subscribe failure

* Refactor confirmations to read commitment cache only once

* Review comments

* Use bank, root from BlockCommitmentCache

* Update docs

* Add metric for block-commitment aggregations

Co-authored-by: Justin Starry <justin@solana.com>
2020-03-30 17:53:25 -06:00
Justin Starry
8bbf6e3f54 Fix race in RPC subscriptions test (#9142) 2020-03-29 01:58:51 +08:00
Justin Starry
c1a3b6ecc2 Add RPC subscription api for rooted slots (#9118)
automerge
2020-03-27 09:33:40 -07:00
Justin Starry
8f38bc7dc0 Refactor how pubsub subscriptions are added (#9042) 2020-03-25 00:53:32 +08:00
Justin Starry
eed8087d87 Respect confirmations param for signature subscription notifications (#9019)
automerge
2020-03-23 17:00:34 -07:00
Justin Starry
4b4819cd07 Add slot context to rpc pubsub notifications (#9001)
automerge
2020-03-23 05:34:42 -07:00
Michael Vines
18c1f0dfe9 Remove stub core/src/genesis_utils.rs (#8999) 2020-03-21 10:54:40 -07:00
Justin Starry
8ec8204a30 Run pubsub test poller in tokio runtime (#8494) 2020-03-03 09:44:39 +08:00
Justin Starry
8839dbfe5b Use runtime executor to send pubsub notifications (#8353)
automerge
2020-02-25 20:23:54 -08:00