* verify bank hash on startup with ledger tool option (#17939)
(cherry picked from commit f558b9b6bf)
# Conflicts:
# core/tests/snapshots.rs
# ledger/src/bank_forks_utils.rs
# runtime/src/snapshot_utils.rs
* fix merge errors
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
* add metrics for startup
* roll timings up higher
* fix test
* fix duplicate
(cherry picked from commit 471b34132e)
# Conflicts:
# ledger/src/bank_forks_utils.rs
# runtime/src/snapshot_utils.rs
conflicts because #17778 is not present.
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Refactor a few functions that are on the load-from-snapshot path, to facilitate
adding in incremental snapshots more easily.
Additionally, add some tests and doc comments.
(cherry picked from commit 1953543274)
Co-authored-by: Brooks Prumo <brooks@solana.com>
* calculate_capitalization uses hash calculation
* feedback
* remove debugging code, clean up slot math
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
* use slots returned from get_snapshot_storages to sort
* add tests
(cherry picked from commit 9388aaca15)
# Conflicts:
# runtime/src/accounts_db.rs
# runtime/src/sorted_storages.rs
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
Account data hashing used to use different ways of hashing on different
clusters. That is no longer the case, but the old code still existed.
This commit removes that old, now used code.
**NOTE** The golden hash values in bank.rs needed to be updated. Since
the original code that selected the hash algorithm used `if >` instead
of `if >=`, this meant that the genesis block's hash _always_ used the
old hashing method, which is no longer valid.
Validated by running `cargo test` successfully.
* lazy calculate account hash
* push to bg thread
* remove deadlock
* logs
* format
* some cleanup on aisle 9
* format, fix up some metrics
* fix test, remove legacy function only there for tests
* cleanup
* remove unused store_hasher
* Switch to crossbeam
* clippy
* format
* use iter()
* rework from feedback
* hash_slot -> slot
* hash(cluster_type)
Co-authored-by: Carl Lin <carl@solana.com>
* Re-use accounts_db stores
Creating files and dropping mmap areas can be expensive
* Add test for storage finder
Can encounter an infinite loop when the store is too small, but
smaller than the normal store size.
* Fix storage finding
* Check for strong_count == 1
* try_recycle helper
