165 Commits

Author SHA1 Message Date
Lijun Wang
54f0fc9f0f Use type alias for DownloadProgress callback (#17518)
Convert to use type alias for the callback and cascade the changes to callers. Thanks @jeffwashington for the help making it possible.
Changed the closure for the progress update in the validator main to FnMut and modify the abort count in the closure which is more reliable.
2021-05-26 13:26:07 -07:00
Tyera Eulberg
9a5330b7eb Move gossip modules into solana-gossip crate (#17352)
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
2021-05-26 09:15:46 -06:00
carllin
3dfe87973b Propagate dead slots up to replay (#17227) 2021-05-25 13:43:47 -07:00
Lijun Wang
4c17243157 snapshot download enhancement (#17415)
1. Allow the validator bootstrap code to specify the minimal snapshot download speed. If the snapshot download speed is detected below that, a different RPC can be retried. The default is 10MB/sec.

2. To prevent spinning on a number of sub-optimal choices and not making progress, the abort/retry logic is implemented with the following safe guards:
2.1 at maximum we do this retry for 5 times -- this number is configurable with default 5.
2.2 if the download in one notification round (5 second) is more than 2%, do not do retry -- it is not as bad anyway.
2.3 if the remaining estimate time is less than 1 minutes, do not abort retry as it will be done quickly anyway.
2.4 We do this abort/retry logic only at the first notification to avoid wasting download efforts -- the reasoning is being opportunistic and too greedy may not achieve overall shorter download time.

3. The download_snapshot and download_file is modified with the option allowing caller to notified of download progress via a callback. This allows the business logic of retrying to the place it belongs.
2021-05-25 09:32:12 -07:00
Tao Zhu
0781fe1b4f Upgrade Rust to 1.52.0 (#17096)
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script

* upgrade to Rust 1.52.1

* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
Lijun Wang
9c42a89a43 Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Michael Vines
9ba2c53b85 Add --tower argument to specify where tower files are persisted 2021-05-05 12:20:39 -07:00
carllin
bc7e741514 Integrate gossip votes into switching threshold (#16973) 2021-05-04 00:51:42 -07:00
carllin
b5d30846d6 Retry latest vote if expired (#16735) 2021-04-28 11:46:16 -07:00
Jeff Washington (jwash)
6d160768d7 delete vest program (#16795) 2021-04-26 08:50:46 -05:00
carllin
4c94f8933f Ingest votes from gossip into fork choice (#16560) 2021-04-21 14:40:35 -07:00
Michael Vines
2229b70c4e Add authorized-voter add/remove-all commands 2021-04-12 15:55:28 -07:00
sakridge
b99ae8f334 Skip leader slots until a vote lands (#15607) 2021-03-25 18:54:51 -07:00
Justin Starry
e817a6db00 Add timeout for local cluster partition tests (#16123)
* Add timeout for local cluster partition tests

* fix optimistic conf test logs

* Bump instruction count assertions
2021-03-25 13:27:07 -06:00
Michael Vines
a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Michael Vines
0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Trent Nelson
24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
Jeff Washington (jwash)
8a3135d17b Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
sakridge
d09112fa6d PoH batch size calibration (#15717) 2021-03-05 16:01:21 -08:00
Michael Vines
bd13262b42 Add validator startup process reporting before RPC is available 2021-03-05 08:03:36 -08:00
Tyera Eulberg
1fc8836631 Add timeout to prevent infinite loop (#15715) 2021-03-05 21:37:46 +09:00
Michael Vines
24ab84936e Break up RPC API into three categories: minimal, full and admin 2021-03-04 16:39:44 -08:00
Tyera Eulberg
a4f0033bd7 Remove ValidatorConfig derive Clone, and fix local-cluster tests (#15647)
* Remove ValidatorConfig derive Clone

* Add local-cluster ValidatorConfig helpers

* Fix benches
2021-03-03 04:21:30 +00:00
Trent Nelson
7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Michael Vines
9ba69a7381 Adapt to fs_extra 1.2.0 2021-02-17 12:44:58 -08:00
Jeff Washington (jwash)
600ff0d915 calculate hash from store instead of index (#15034)
* calculate hash from store instead of index

* restore update hash in abs
2021-02-04 09:00:33 -06:00
behzad nouri
0ad063f4e9 adds flag to disable duplicate instance check (#15006) 2021-02-03 16:26:17 +00:00
Tom Parker-Shemilt
01230a0105 Remove serial_test_derive dependency (#14891) 2021-01-28 22:35:31 -07:00
Tyera Eulberg
ffa5c7dcc8 Deprecate commitment variants (#14797)
* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments
2021-01-26 19:23:07 +00:00
Michael Vines
bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
Michael Vines
7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
Jeff Washington (jwash)
938d482135 consolidate constants related to time for future refactoring (#14440) 2021-01-07 09:49:24 -06:00
Michael Vines
ace360ade2 Multiple entrypoint support 2020-12-22 18:35:31 -08:00
Michael Vines
0b92720fdb Don't require increased open file limit in solana-test-validator
Travis CI in particular does not allow the open file limit to be
increased.
2020-12-16 20:43:08 -08:00
Jack May
582418de5e Cleanup ledger builtins (#14083) 2020-12-15 12:23:49 -08:00
Michael Vines
1e977ac025 Bump version to v1.6.0 2020-12-15 18:28:04 +00:00
Michael Vines
7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
dependabot[bot]
88d950e5a8 chore: bump log from 0.4.8 to 0.4.11 (#13691)
* chore: bump log from 0.4.8 to 0.4.11

Bumps [log](https://github.com/rust-lang/log) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
Michael Vines
bbad3fe501 TestValidator now implements Drop, no need to close() it 2020-12-11 04:17:38 +00:00
Tyera Eulberg
10c81a2448 Remove rpc_banks from validator (#13882)
* Remove rpc_banks from validator

* Bump abi-digest
2020-12-02 03:25:09 +00:00
Michael Vines
61ab2072bd Clean up default commitment handling for subscriptions 2020-11-23 22:54:47 -08:00
carllin
791fb17437 Prevent scans on unrooted slots from seeing partial clean (#13628)
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 13:01:04 -08:00
carllin
9bb11a2dcc Cleanup (#13728)
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 03:17:19 -08:00
sakridge
598e5f58d5 Add wait for max stake command (#13532) 2020-11-12 13:48:34 -08:00
carllin
9821a7754c Discard pre hard fork persisted tower if hard-forking (#13536)
* Discard pre hard fork persisted tower if hard-forking

* Relax config.require_tower

* Add cluster test

* nits

* Remove unnecessary check

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-12 23:29:04 +09:00
Justin Starry
c24fbb6f8b Fix duplicate records of inner instructions (#13380)
* Fix duplicate records of inner instructions

* fix tests

* fix clippy

* Remove bad_inner_instructions
2020-11-05 15:23:52 +08:00
sakridge
7ee0c9e68a Add non-progress bar download status (#13370) 2020-11-04 10:32:27 -08:00
Ryo Onodera
1df15d85c3 Fix tower/blockstore unsync due to external causes (#12671)
* Fix tower/blockstore unsync due to external causes

* Add and clean up long comments

* Clean up test

* Comment about warped_slot_history

* Run test_future_tower with master-only/master-slave

* Update comments about false leader condition
2020-10-30 19:31:23 +09:00
Ryo Onodera
0264147d42 Clean up opt conf verifier and vote state tracker (#13081)
* Clean up opt conf verifier and vote state tracker

* Update test to follow new message and some knob

* Rename
2020-10-24 10:19:12 +09:00
Michael Vines
959880db60 Remove unused pubkey::Pubkey imports 2020-10-21 19:08:13 -07:00