Commit Graph

236 Commits

Author SHA1 Message Date
mergify[bot]
24ee0b3934 Avoid full-range compactions with periodic filtered b.g. ones (backport #16697) (#17956)
* Avoid full-range compactions with periodic filtered b.g. ones (#16697)

* Update rocksdb to v0.16.0

* Promote the infrequent and important log to info!

* Force background compaction by ttl without manual compaction

* Fix test

* Support no compaction mode in test_ledger_cleanup_compaction

* Fix comment

* Make compaction_interval customizable

* Avoid major compaction with periodic filtering...

* Adress lazy_static, special cfs and range check

* Clean up a bit and add comment

* Add comment

* More comments...

* Config code cleanup

* Add comment

* Use .conflicts_with()

* Nullify unneeded delete_range ops for special CFs

* Some clean ups

* Clarify the locking intention

* Ensure special CFs' consistency with PurgeType::CompactionFilter

* Fix comment

* Fix bad copy paste

* Fix various types...

* Don't use tuples

* Add a unit test for compaction_filter

* Fix typo...

* Remove flag and just use new behavior always

* Fix wrong condition negation...

* Doc. about no set_last_purged_slot in purge_slots

* Write a test and fix off-by-one bug....

* Apply suggestions from code review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Follow up to github review suggestions

* Fix line-wrapping

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 1f97b2365f)

# Conflicts:
#	Cargo.lock
#	ledger/src/blockstore_db.rs

* Fix conflicts

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-06-15 08:49:13 +00:00
Tyera Eulberg
dd2d119d2b v1.6: Ensure cluster-confirmed roots are set on boot (#17442)
* Add blockstore-root-scan for api nodes on boot

* Ensure cluster-confirmed root and parents are set as root in blockstore in load_frozen_forks()

* Plumb rpc-scan-and-fix-roots validator flag
2021-05-24 20:16:37 +00:00
mergify[bot]
4313240b1b Return error for excluded secondary-index keys (backport #17193) (#17215)
* Return error for excluded secondary-index keys (#17193)

* Add runtime helpers to check secondary indexes for key

* Add custom rpc error

* Check secondary-index key inclusion in rpc

* Clone complete AccountSecondaryIndexes into rpc to avoid bank query

(cherry picked from commit 27004f1b76)

# Conflicts:
#	core/src/rpc.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-05-13 23:04:01 +00:00
mergify[bot]
2367f561dc include/exclude keys on account secondary index (backport #17110) (#17179)
* include/exclude keys on account secondary index (#17110)

* AccountSecondaryIndexes.include/exclude

* use normal scan if key is not indexed

* add a test to ask for a scan for an excluded secondary index

* fix up cli args

(cherry picked from commit 7d96f78821)

# Conflicts:
#	runtime/src/accounts_db.rs
#	runtime/src/accounts_index.rs

* resolve merge conflicts

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-05-12 15:09:46 +00:00
mergify[bot]
970bba495f Add --tower argument to specify where tower files are persisted (#17060)
(cherry picked from commit 9ba2c53b85)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-05-05 20:37:36 +00:00
mergify[bot]
13e176a633 getLeaderSchedule now supports filtered results based on validator identity (#16731)
(cherry picked from commit 6004c0abf5)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-04-22 02:29:01 +00:00
mergify[bot]
e15ddbb979 Add port and gossip options to solana-test-validator (#16696) (#16698)
(cherry picked from commit 0924c2d070)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-04-21 03:46:52 +00:00
Michael Vines
6a7ce8500b canonicalize authorized voter filepath
(cherry picked from commit 05ad979a2d)
2021-04-12 20:01:56 -07:00
mergify[bot]
8ee294639a validator: Add authorized-voter add/remove-all commands (bp #16492) (#16496)
* Clean up build warning

(cherry picked from commit 17a173ebb5)

* Add authorized-voter add/remove-all commands

(cherry picked from commit 2229b70c4e)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-04-13 00:07:06 +00:00
Michael Vines
10e8f3ab32 Fix up App formatting
(cherry picked from commit ef30943c5c)
2021-04-11 23:36:31 -07:00
Michael Vines
f1bc7ec4fa wait-for-restart-window works again for unstaked nodes
(cherry picked from commit a679aebc82)
2021-04-04 12:59:13 -07:00
Michael Vines
b7efc2373c wait-for-restart-window now indicates how far away the next restart window is
(cherry picked from commit c8c89dd5f7)
2021-04-02 23:23:16 -07:00
mergify[bot]
d940c5b1a3 Skip leader slots until a vote lands (#15607) (#16147)
(cherry picked from commit b99ae8f334)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-03-26 19:07:24 +00:00
mergify[bot]
eeef9f4e59 Separate snapshot location (bp #15840) (#15956)
* Add option for separate snapshot location

(cherry picked from commit 6126878f509c69e23480a5ec22b3271e2b16e072)
(cherry picked from commit 0209d334bd)

* Apply suggestions from code review

Co-authored-by: Michael Vines <mvines@gmail.com>
(cherry picked from commit cfb01e26dd)

* add missed suggestion

(cherry picked from commit a43b3674c7)

* Revert to snapshots

Co-authored-by: Michael Vines <mvines@gmail.com>
(cherry picked from commit 0b42379ed7)

* Revert to snapshots 2

(cherry picked from commit 20b53eb4b4)

* Revert to removing only tmp-

(cherry picked from commit a5d144b00f)

Co-authored-by: DimAn <diman@diman.io>
Co-authored-by: DimAn <andiman7000@gmail.com>
2021-03-17 20:25:18 +00:00
mergify[bot]
4bcf976ecd Fix delinquent stake display (#15839)
(cherry picked from commit eab182188a)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-13 20:27:25 +00:00
Michael Vines
aa2b2d6b75 Default --ledger arg to "ledger" for solana-validator and solana-ledger-tool 2021-03-11 13:09:30 -08:00
Michael Vines
ac8ccee6b8 Add --force and --monitor options to exit subcommand 2021-03-11 05:17:31 +00:00
Michael Vines
c836cd85c3 Default to the BPF JIT. Use the solana-validator --no-bpf-jit flag to disable 2021-03-10 08:37:35 -08:00
Michael Vines
76ddc93358 The --identity argument is now required 2021-03-06 19:08:37 -08:00
Michael Vines
04d11ca6c6 Slow down solana-validator monitor refresh interval when talking to a real node 2021-03-07 00:51:02 +00:00
sakridge
d09112fa6d PoH batch size calibration (#15717) 2021-03-05 16:01:21 -08:00
Michael Vines
bd13262b42 Add validator startup process reporting before RPC is available 2021-03-05 08:03:36 -08:00
Michael Vines
24ab84936e Break up RPC API into three categories: minimal, full and admin 2021-03-04 16:39:44 -08:00
Jeff Washington (jwash)
22d2bb9a9c invert default behavior of index hashing (#15689) 2021-03-03 18:12:18 -06:00
Michael Vines
d47f1fae40 Check delinquency before restarting 2021-02-26 13:34:04 -08:00
Michael Vines
0dc482e987 Add wait-for-restart-window subcommand 2021-02-26 13:34:04 -08:00
sakridge
05409e51ce Increase tpu coalescing and add parameter (#15536)
Should create larger entries on average
2021-02-26 09:15:45 -08:00
Trent Nelson
7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Jeff Washington (jwash)
ba02452d75 add validator flag no-accounts-db-index-hashing (#15350)
* add validator flag no_accounts_db_index_hashing

* add validator flag no_accounts_db_index_hashing
2021-02-16 21:13:48 +00:00
sakridge
5b8f046c67 More configurable rocksdb compaction (#15213)
rocksdb compaction can cause long stalls, so
make it more configurable to try and reduce those stalls
and also to coordinate between multiple nodes to not induce
stall at the same time.
2021-02-14 10:16:30 -08:00
Michael Vines
f34b8643c7 Add |solana-validator monitor| subcommand (#15118) 2021-02-05 22:39:23 -08:00
Jeff Washington (jwash)
600ff0d915 calculate hash from store instead of index (#15034)
* calculate hash from store instead of index

* restore update hash in abs
2021-02-04 09:00:33 -06:00
behzad nouri
0ad063f4e9 adds flag to disable duplicate instance check (#15006) 2021-02-03 16:26:17 +00:00
Tyera Eulberg
cbb8b79a60 Add validator flag to opt in to cpi and logs storage (#14922)
* Add validator flag to opt in to cpi and logs storage

* Default TestValidator to opt-in; allow using in multinode-demo

* No clone

Co-authored-by: Carl Lin <carl@solana.com>
2021-02-01 14:00:51 -07:00
Michael Vines
49c908dc50 Add "init" subcommand 2021-02-01 08:12:28 -08:00
Michael Vines
8993ac0c74 Surface faucet start failures to the user of solana-test-validator 2021-01-28 22:35:58 +00:00
sakridge
5bf5a5ec41 Enable accounts caching by default (#14854)
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-26 12:06:39 -08:00
Tyera Eulberg
ffa5c7dcc8 Deprecate commitment variants (#14797)
* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments
2021-01-26 19:23:07 +00:00
Tyera Eulberg
4964b0fe61 Cli: default to single gossip (#14673)
* Init cli RpcClient with chosen commitment; default to single_gossip

* Fill in missing client methods

* Cli tests: make RpcClient commitment specific

* Simply rpc_client calls, using configured commitment

* Check validator vote account with single-gossip commitment
2021-01-19 15:33:03 -07:00
Ryo Onodera
dcaa025822 Configure Bigtable's timeout, enabling by default (#14657)
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-19 13:57:16 +00:00
Michael Vines
11daaadc93 Add --rpc-threads argument 2021-01-13 13:34:46 -08:00
carllin
6dfad0652f Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Michael Vines
3bd4c38a84 Clarify log message, the remote snapshot might not actually be newer 2021-01-11 11:53:13 -08:00
Michael Vines
7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
carllin
5affd8aa72 Add secondary indexes (#14212) 2020-12-31 18:06:03 -08:00
Michael Vines
04bf5ce830 Minor help improvements 2020-12-31 07:55:26 +00:00
Trent Nelson
fe667db910 validator: Add experimental flag to select PoH pinned core 2020-12-29 19:15:44 -07:00
sakridge
2074e407cd Add poh speed check and tick speed calibration (#14292) 2020-12-29 09:35:57 -08:00
Michael Vines
9ddd6f08e8 Persist gossip contact info 2020-12-27 20:46:54 -08:00
Michael Vines
65dcb3dc81 Add log message for when a local snapshot is too old 2020-12-22 19:39:17 -08:00