* excludes epoch-slots from nodes with unknown or different shred version (#17899)
Inspecting TDS gossip table shows that crds values of nodes with
different shred-versions are creeping in. Their epoch-slots are
accumulated in ClusterSlots causing bogus slots very far from current
root which are not purged and so cause ClusterSlots keep consuming more
memory:
https://github.com/solana-labs/solana/issues/17789https://github.com/solana-labs/solana/issues/14366#issuecomment-769896036https://github.com/solana-labs/solana/issues/14366#issuecomment-832754654
This commit updates ClusterInfo::get_epoch_slots, and discards entries
from nodes with unknown or different shred-version.
Follow up commits will patch gossip not to waste bandwidth and memory
over crds values of nodes with different shred-version.
(cherry picked from commit 985280ec0b)
# Conflicts:
# core/src/cluster_info.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* persists repair-peers cache across repair service loops (#18400)
The repair-peers cache is reset each time repair service loop runs,
and so computed repeatedly for the same slots:
https://github.com/solana-labs/solana/blob/d2b07dca9/core/src/repair_service.rs#L275
This commit uses an LRU cache to persists repair-peers for each slot.
In addition to LRU eviction rules, in order to avoid re-using outdated
data, each entry also has 10 seconds TTL.
(cherry picked from commit a0551b4054)
# Conflicts:
# core/src/repair_service.rs
# core/src/serve_repair.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* Fixing missing pubsub notification for programSubscribe and logsSubscribe (#19092)
#18587: programSubscribe is missing notifications randomly. The issue is because of two reasons
Not all rooted slots get OptimisticallyConfirmed notifications
The OptimisticallyConfirmed notifications can be out of order for slots: slot A and B with A < B can see notification for B first before A.
Summary of Changes
Changed OptimisticallyConfirmedBankTracker to send notifications for parent banks if they have not been notified yet. We use a new variable last_notified_slot to track that.
Tests:
With my validator running against testnet, before the fix, it was failing 75% of time, with the fix, it is passing consistently. Using the program mentioned in #18587.
(cherry picked from commit 1a372a792e)
# Conflicts:
# core/src/optimistically_confirmed_bank_tracker.rs
# core/src/rpc_subscriptions.rs
* Fix conflicts
Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
* Avoid full-range compactions with periodic filtered b.g. ones (#16697)
* Update rocksdb to v0.16.0
* Promote the infrequent and important log to info!
* Force background compaction by ttl without manual compaction
* Fix test
* Support no compaction mode in test_ledger_cleanup_compaction
* Fix comment
* Make compaction_interval customizable
* Avoid major compaction with periodic filtering...
* Adress lazy_static, special cfs and range check
* Clean up a bit and add comment
* Add comment
* More comments...
* Config code cleanup
* Add comment
* Use .conflicts_with()
* Nullify unneeded delete_range ops for special CFs
* Some clean ups
* Clarify the locking intention
* Ensure special CFs' consistency with PurgeType::CompactionFilter
* Fix comment
* Fix bad copy paste
* Fix various types...
* Don't use tuples
* Add a unit test for compaction_filter
* Fix typo...
* Remove flag and just use new behavior always
* Fix wrong condition negation...
* Doc. about no set_last_purged_slot in purge_slots
* Write a test and fix off-by-one bug....
* Apply suggestions from code review
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Follow up to github review suggestions
* Fix line-wrapping
* Fix conflict
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 1f97b2365f)
# Conflicts:
# Cargo.lock
# ledger/src/blockstore_db.rs
* Fix conflicts
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
* Add block height to ConfirmedBlock structs (#17523)
* Add BlockHeight CF to blockstore
* Rename CacheBlockTimeService to be more general
* Cache block-height using service
* Fixup previous proto mishandling
* Add block_height to block structs
* Add block-height to solana block
* Fallback to BankForks if block time or block height are not yet written to Blockstore
* Add docs
* Review comments
(cherry picked from commit ab581dafc2)
# Conflicts:
# core/src/replay_stage.rs
# core/src/tvu.rs
# core/src/validator.rs
* Fix conflicts
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
* simulateTransaction can now return accounts modified by the simulation
(cherry picked from commit cbce440af4)
# Conflicts:
# rpc/src/parsed_token_accounts.rs
* rebase
Co-authored-by: Michael Vines <mvines@gmail.com>
* Add a flag to simulateTransaction to use most recent blockhash
(cherry picked from commit 96cef5260c)
* rename flag
(cherry picked from commit e14f3eb529)
* sigVerify conflicts with replace, add tests
(cherry picked from commit 660d37aadf)
Co-authored-by: Justin Starry <justin@solana.com>
* Add blockstore-root-scan for api nodes on boot
* Ensure cluster-confirmed root and parents are set as root in blockstore in load_frozen_forks()
* Plumb rpc-scan-and-fix-roots validator flag
