* Optimize RPC pubsub for multiple clients with the same subscription (#18943)
* reimplement rpc pubsub with a broadcast queue
* update tests for new pubsub implementation
* fix: fix review suggestions
* chore(rpc): add additional pubsub metrics
* integrate max subscriptions check into SubscriptionTracker to reduce locking
* separate subscription control from tracker
* limit memory usage of items in pubsub broadcast queue, improve error handling
* add more pubsub metrics
* add final count metrics to pubsub
* add metric for total number of subscriptions
* fix small review suggestions
* remove by_params from SubscriptionTracker and add node_progress_watchers map instead
* add subscription tracker tests
* add metrics for number of pubsub notifications as a counter
* ignore clippy lint in TokenCounter
* fix underflow in token counter
* reduce queue capacity in pubsub tests
* fix(rpc): fix test timeouts
* fix race in account subscription test
* Add RpcSubscriptions::new_for_tests
Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 65227f44dc)
# Conflicts:
# Cargo.lock
# core/Cargo.toml
# core/src/replay_stage.rs
# core/src/validator.rs
# replica-node/src/replica_node.rs
# rpc/Cargo.toml
* Fix conflicts (and standardize naming to make future subscription backports easier
Co-authored-by: Pavel Strakhov <ri@idzaaus.org>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
* rpc: performance fix for getProgramAccounts
The accounts were gradually pushed into a vector, which produced
significant slowdowns for very large responses.
* rpc: rewrite loops using iterators
Co-authored-by: Christian Kamm <ckamm@delightful-solutions.de>
(cherry picked from commit f1bbf1d8b0)
Co-authored-by: Christian Kamm <mail@ckamm.de>
* Only encode data with <128 bytes in bs58
* Use same MAX_BS58_BYTES const in rpc
* Fix test
* Spell out base
Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 967746abbf)
Co-authored-by: sakridge <sakridge@gmail.com>
* stake: Remove v2 program references (#19308)
* stake: Remove v2 program references
* Remove stake v2 feature, along with stake rewrite
(cherry picked from commit 73aa004c59)
# Conflicts:
# runtime/src/bank.rs
* Fix merge conflict
* Fix uses of old `stake` function
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
* Fixing missing pubsub notification for programSubscribe and logsSubscribe (#19092)
#18587: programSubscribe is missing notifications randomly. The issue is because of two reasons
Not all rooted slots get OptimisticallyConfirmed notifications
The OptimisticallyConfirmed notifications can be out of order for slots: slot A and B with A < B can see notification for B first before A.
Summary of Changes
Changed OptimisticallyConfirmedBankTracker to send notifications for parent banks if they have not been notified yet. We use a new variable last_notified_slot to track that.
Tests:
With my validator running against testnet, before the fix, it was failing 75% of time, with the fix, it is passing consistently. Using the program mentioned in #18587.
(cherry picked from commit 1a372a792e)
* Use v1.7 api
Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
* Use last_valid_block_height in services and client apps (#19163)
* Add deprecated tag to Bank::get_blockhash_last_valid_slot
* Update SendTransactionService to use last_valid_block_height
* Update solana-tokens to use last_valid_block_height
* Remove dangling file
* Update solana program to use last_valid_block_height
* Update Banks crates to use last_valid_block_height
(cherry picked from commit 5970083b4d)
# Conflicts:
# cli/src/program.rs
* Fix conflict
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
* Refactor account encoding to povide helper w/out querying Bank
* Use new method in get_program_accounts to properly return length err
(cherry picked from commit 3eecb6f4ae)
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Refactoring: Unify account_deps and accounts (#17898)
* Changes ThisInvokeContext::get_account() to use accounts instead of pre_accounts.
* Adds explicit keys to accounts to make them symmetric to account_deps.
* Appends account_deps to accounts in transaction loading and removes account_deps everywhere else.
(cherry picked from commit 7462c27d07)
# Conflicts:
# program-test/src/lib.rs
# runtime/src/bank.rs
# runtime/src/message_processor.rs
* fix conflicts
Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Justin Starry <justin@solana.com>
* Report 404 instead of 502 when a snapshot file is not found
This provides the client with more useful information about why their
request failed
(cherry picked from commit 40d696fcbc)
* As a last resort try to download an uncompressed snapshot
`solana-test-validator` creates uncompressed snapshots and it can be
useful to attach another validator to a `solana-test-validator` cluster
from time to time
(cherry picked from commit e36247d187)
Co-authored-by: Michael Vines <mvines@gmail.com>
* Refactor stake program into solana_program (#17906)
* Move stake state / instructions into solana_program
* Update account-decoder
* Update cli and runtime
* Update all other parts
* Commit Cargo.lock changes in programs/bpf
* Update cli stake instruction import
* Allow integer arithmetic
* Update ABI digest
* Bump rust mem instruction count
* Remove useless structs
* Move stake::id() -> stake::program::id()
* Re-export from solana_sdk and mark deprecated
* Address feedback
* Run cargo fmt
* Run cargo fmt post cherry-pick
