* removes packet-count metrics from retransmit stage
Working towards sending shreds (instead of packets) to retransmit stage
so that shreds recovered from erasure codes are as well retransmitted.
Following commit will add these metrics back to window-service, earlier
in the pipeline.
(cherry picked from commit bf437b0336)
# Conflicts:
# core/src/retransmit_stage.rs
* adds packet/shred count stats to window-service
Adding back these metrics from the earlier commit which removed them
from retransmit stage.
(cherry picked from commit 8198a7eae1)
* removes erroneous uses of Arc<...> from retransmit stage
(cherry picked from commit 6e413331b5)
# Conflicts:
# core/src/retransmit_stage.rs
# core/src/tvu.rs
* sends shreds (instead of packets) to retransmit stage
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
(cherry picked from commit 3efccbffab)
# Conflicts:
# core/src/retransmit_stage.rs
* returns completed-data-set-info from insert_data_shred
instead of opaque (u32, u32) which are then converted to
CompletedDataSetInfo at the call-site.
(cherry picked from commit 3c71670bd9)
# Conflicts:
# ledger/src/blockstore.rs
* retransmits shreds recovered from erasure codes
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.
This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.
(cherry picked from commit 7a8807b8bb)
# Conflicts:
# core/src/retransmit_stage.rs
# core/src/window_service.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* stake: Remove v2 program references (#19308)
* stake: Remove v2 program references
* Remove stake v2 feature, along with stake rewrite
(cherry picked from commit 73aa004c59)
# Conflicts:
# runtime/src/bank.rs
* Fix merge conflict
* Fix uses of old `stake` function
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
* backport new column families from master to 1.6 (#18743)
* backporting bank_hash and program_costs column families from master to 1.6 for rocksdb backward compatibility
* missed a line to allow dead code
* include code for purge
* Exclude stubbed ProgramCosts column from compaction (#18840)
Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
This feature requires clang to support -march=native on M1. Before this change,
cargo build would output a build error when building for aarch64.
(cherry picked from commit 8a9b7f5ef2)
# Conflicts:
# ledger/Cargo.toml
* Make account shrink configurable #17544 (#17778)
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true
Fixes#17544
(cherry picked from commit 269d995832)
# Conflicts:
# core/tests/snapshots.rs
# ledger/src/bank_forks_utils.rs
# runtime/src/accounts_db.rs
# runtime/src/snapshot_utils.rs
* fix some merge errors
Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
* verify bank hash on startup with ledger tool option (#17939)
(cherry picked from commit f558b9b6bf)
# Conflicts:
# core/tests/snapshots.rs
# ledger/src/bank_forks_utils.rs
# runtime/src/snapshot_utils.rs
* fix merge errors
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
* add metrics for startup
* roll timings up higher
* fix test
* fix duplicate
(cherry picked from commit 471b34132e)
# Conflicts:
# ledger/src/bank_forks_utils.rs
# runtime/src/snapshot_utils.rs
conflicts because #17778 is not present.
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Refactor a few functions that are on the load-from-snapshot path, to facilitate
adding in incremental snapshots more easily.
Additionally, add some tests and doc comments.
(cherry picked from commit 1953543274)
Co-authored-by: Brooks Prumo <brooks@solana.com>
* Refactor stake program into solana_program (#17906)
* Move stake state / instructions into solana_program
* Update account-decoder
* Update cli and runtime
* Update all other parts
* Commit Cargo.lock changes in programs/bpf
* Update cli stake instruction import
* Allow integer arithmetic
* Update ABI digest
* Bump rust mem instruction count
* Remove useless structs
* Move stake::id() -> stake::program::id()
* Re-export from solana_sdk and mark deprecated
* Address feedback
* Run cargo fmt
* Run cargo fmt post cherry-pick
* calculate_capitalization uses hash calculation
* feedback
* remove debugging code, clean up slot math
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
* Avoid full-range compactions with periodic filtered b.g. ones (#16697)
* Update rocksdb to v0.16.0
* Promote the infrequent and important log to info!
* Force background compaction by ttl without manual compaction
* Fix test
* Support no compaction mode in test_ledger_cleanup_compaction
* Fix comment
* Make compaction_interval customizable
* Avoid major compaction with periodic filtering...
* Adress lazy_static, special cfs and range check
* Clean up a bit and add comment
* Add comment
* More comments...
* Config code cleanup
* Add comment
* Use .conflicts_with()
* Nullify unneeded delete_range ops for special CFs
* Some clean ups
* Clarify the locking intention
* Ensure special CFs' consistency with PurgeType::CompactionFilter
* Fix comment
* Fix bad copy paste
* Fix various types...
* Don't use tuples
* Add a unit test for compaction_filter
* Fix typo...
* Remove flag and just use new behavior always
* Fix wrong condition negation...
* Doc. about no set_last_purged_slot in purge_slots
* Write a test and fix off-by-one bug....
* Apply suggestions from code review
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Follow up to github review suggestions
* Fix line-wrapping
* Fix conflict
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 1f97b2365f)
# Conflicts:
# ledger/src/blockstore_db.rs
* Fix conflict
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
