* removes id and shred_version from CrdsGossip (#18505)
ClusterInfo is the gateway to CrdsGossip function calls, and it already
has node's pubkey and shred version (full ContactInfo and Keypair in
fact).
Duplicating these data in CrdsGossip adds redundancy and possibility for
bugs should they not be consistent with ClusterInfo.
(cherry picked from commit 4e1333fbe6)
# Conflicts:
# gossip/src/cluster_info.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* Refactoring: Unify account_deps and accounts (#17898)
* Changes ThisInvokeContext::get_account() to use accounts instead of pre_accounts.
* Adds explicit keys to accounts to make them symmetric to account_deps.
* Appends account_deps to accounts in transaction loading and removes account_deps everywhere else.
(cherry picked from commit 7462c27d07)
# Conflicts:
# program-test/src/lib.rs
# runtime/src/bank.rs
# runtime/src/message_processor.rs
* fix conflicts
Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Justin Starry <justin@solana.com>
Current implementation of weighted_shuffle:
https://github.com/solana-labs/solana/blob/b08f8bd1b/gossip/src/weighted_shuffle.rs#L11-L37
uses a heuristic which results in biased samples.
For example, if the weights are [1, 10, 100], then the 3rd index should
come first 100 times more often than the 1st index. However,
weighted_shuffle is picking the 3rd index 200+ times more often than the
1st index, showing a disproportional bias in favor of higher weights.
This commit implements weighted shuffle using binary indexed tree to
maintain cumulative sum of weights while sampling. The resulting samples
are demonstrably unbiased and precisely proportional to the weights.
Additionally the iterator interface allows to skip computations when
not all indices are processed.
Of the use cases of weighted_shuffle, changing turbine code requires
feature-gating to keep the cluster in sync. That is not updated in
this commit, but can be done together with future updates to turbine.
(cherry picked from commit dba42c57b4)
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
The repair-peers cache is reset each time repair service loop runs,
and so computed repeatedly for the same slots:
https://github.com/solana-labs/solana/blob/d2b07dca9/core/src/repair_service.rs#L275
This commit uses an LRU cache to persists repair-peers for each slot.
In addition to LRU eviction rules, in order to avoid re-using outdated
data, each entry also has 10 seconds TTL.
(cherry picked from commit a0551b4054)
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Broadcast stage and retransmit stage should arrange nodes on turbine
broadcast tree in exactly same order. Additionally any changes to this
ordering (e.g. updating how unstaked nodes are handled) requires feature
gating to keep the cluster in sync.
Current implementation is scattered out over several public methods and
exposes too much of implementation details (e.g. usize indices into
peers vector) which makes code changes and checking for feature
activations more difficult.
This commit encapsulates turbine peer computations into a new struct,
and only exposes two public methods, get_broadcast_peer and
get_retransmit_peers, for call-sites.
(cherry picked from commit 04787be8b1)
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* Report 404 instead of 502 when a snapshot file is not found
This provides the client with more useful information about why their
request failed
(cherry picked from commit 40d696fcbc)
* As a last resort try to download an uncompressed snapshot
`solana-test-validator` creates uncompressed snapshots and it can be
useful to attach another validator to a `solana-test-validator` cluster
from time to time
(cherry picked from commit e36247d187)
Co-authored-by: Michael Vines <mvines@gmail.com>
I wanted to use this sysvar to get a recent block hash, but I didn't
know whether the first or the last entry contains the most recent block
hash.
By calling it for mainnet, printing the results, and comparing that to
the recent blocks on solanabeach.io/blocks, I discovered that the
entries are ordered from most recent to least recent. Document this to
save future readers the trouble.
(cherry picked from commit 94ab0eb49f)
Co-authored-by: Ruud van Asseldonk <ruud@chorus.one>
* Add proposal for supporting big transactions
* account index program
* fix formatting
* review feedback
* Add cost changes section
* Add cost section and more attack details
* fix lint
* document metadata changes
* nit
* rpc details
* add index meta struct
* add additional proposal and chagne title
* rename proposal file
* rename to address map and rewrite tx format
* no more appends, limit mapping size to 256
* update dos section
* add note about readonly
* restructure message to use enum
* cleanup
(cherry picked from commit 191519188d)
Co-authored-by: Justin Starry <justin@solana.com>
