* filters crds values obtained through gossip by their shred version (#18072)
filter_by_shred_version does not check the shred-version of the owner of
the crds-value. It only checks the shred-version of the node which is
relaying the value:
https://github.com/solana-labs/solana/blob/5cc073420/gossip/src/cluster_info.rs#L2274-L2289
So crds-values with different shred versions can still pass through this
function as long as they are relayed by a node with matching shred
version; and so, a single node can bridge different shred values
through-out the cluster.
(cherry picked from commit 69a5f0e6cd)
# Conflicts:
# gossip/src/cluster_info.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* Keep track of dirty stores on remove accounts to clean
and not zero_lamport key set
* Only dirty when count==0?
* Add another clean
(cherry picked from commit 3b1738c000)
Co-authored-by: sakridge <sakridge@gmail.com>
* adds shred-version to ip-echo-server response
When starting a validator, the node initially joins gossip with
shred_verison = 0, until it adopts the entrypoint's shred-version:
https://github.com/solana-labs/solana/blob/9b182f408/validator/src/main.rs#L417
Depending on the load on the entrypoint, this adopting entrypoint
shred-version through gossip sometimes becomes very slow, and causes
several problems in gossip because we have to partially support
shred_version == 0 which is a source of leaking crds values from one
cluster to another. e.g. see
https://github.com/solana-labs/solana/pull/17899
and the other linked issues there.
In order to remove shred_version == 0 from gossip, this commit adds
shred-version to ip-echo-server response. Once the entrypoints are
updated, on validator start-up, if --expected_shred_version is not
specified we will obtain shred-version from the entrypoint using
ip-echo-server.
(cherry picked from commit 598093b5db)
# Conflicts:
# Cargo.lock
# net-utils/Cargo.toml
# programs/bpf/Cargo.lock
* removes backport merge conflicts
* obtains shred-version from entrypoint's ip-echo-server in validator-main
(cherry picked from commit 58e115275a)
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* eliminate flatten and sort in hash calculation
* reduce critical section time
* remove now no-longer necessary test code
* conflict with reset bins to 0 pr
(cherry picked from commit bf97627021)
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Crds values of nodes with different shred versions are creeping into
gossip table resulting in runtime issues as the one addressed in:
https://github.com/solana-labs/solana/pull/17899
This commit works towards enforcing more checks and filtering based on
shred version by adding necessary mapping and api to gossip table.
Once populated, pubkey->shred-version mapping persists as long as there
are any values associated with the pubkey.
(cherry picked from commit 5a99fa3790)
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* Make account shrink configurable #17544 (#17778)
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true
Fixes#17544
(cherry picked from commit 269d995832)
# Conflicts:
# core/tests/snapshots.rs
# ledger/src/bank_forks_utils.rs
# runtime/src/accounts_db.rs
# runtime/src/snapshot_utils.rs
* fix some merge errors
Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
* Skip shrink when it doesn't save anything (#17405)
(cherry picked from commit 14c52ab018)
# Conflicts:
# runtime/src/accounts_db.rs
* fix merge error
Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
