Commit Graph

121 Commits

Author SHA1 Message Date
Carl Lin
e3ccf417b2 Fix flaky test 2022-02-15 01:58:49 -05:00
mergify[bot]
86e369ec8e uses Option<Slot> for SlotMeta.parent_slot (backport #21808) (#22936)
* uses Option<Slot> for SlotMeta.parent_slot (#21808)

SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.

(cherry picked from commit 8d980f07ba)

# Conflicts:
#	core/src/repair_generic_traversal.rs
#	ledger-tool/src/main.rs
#	ledger/src/blockstore.rs

* removes mergify merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-02-04 16:30:31 +00:00
mergify[bot]
430cdf679e rpc: use minimal mode by default (backport #22734) (#22878)
* rpc: use minimal mode by default

(cherry picked from commit eac4a6df68)

# Conflicts:
#	core/src/validator.rs
#	local-cluster/tests/common.rs
#	local-cluster/tests/local_cluster.rs
#	replica-node/tests/local_replica.rs
#	rpc/src/rpc_service.rs
#	transaction-dos/src/main.rs

* test-validator-bin: reinstate full rpc method set

Co-authored-by: Trent Nelson <trent@solana.com>
2022-02-03 21:11:58 +00:00
mergify[bot]
fdc1b046bc Set the correct root in block commitment cache initialization (#22750) (#22756)
* Set the correct root in block commitment cache initialization

* clean up test

* bump

(cherry picked from commit d9c259a231)

# Conflicts:
#	replica-node/src/replica_node.rs

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-27 02:49:28 +00:00
mergify[bot]
54db774203 Split up local cluster tests into separate CI steps (backport #22295) (#22302)
* Split up local cluster tests into separate CI steps (#22295)

* Split up local cluster tests into separate CI steps

* Update buildkite-pipeline.sh

(cherry picked from commit 0e1afcbb26)

# Conflicts:
#	local-cluster/tests/local_cluster.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-06 15:26:56 +00:00
mergify[bot]
3b895104f3 ClusterInfoVoteListener send only missing votes to BankingStage (backport #20873) (#22135)
* ClusterInfoVoteListener send only missing votes to BankingStage (#20873)

(cherry picked from commit b30c94ce55)

# Conflicts:
#	core/src/cluster_info_vote_listener.rs
#	core/src/verified_vote_packets.rs
#	gossip/src/cluster_info.rs
#	local-cluster/tests/local_cluster.rs
#	runtime/src/bank.rs

* Resolve conflicts

* Remove Select

* Fixup tests

Co-authored-by: carllin <carl@solana.com>
2021-12-30 06:38:49 +00:00
Tyera Eulberg
b7cf6e7d9a Revert "Revert "Fixup flaky tests (backport #21617) (#21646)""
This reverts commit 156caeb710.
2021-12-16 19:28:48 -07:00
Tyera Eulberg
9f53f3455a Revert "Revert "Reformat imports to a consistent style for imports""
This reverts commit d7377d4794.
2021-12-16 19:28:48 -07:00
Trent Nelson
d7377d4794 Revert "Reformat imports to a consistent style for imports"
This reverts commit 139d15cd84.
2021-12-13 12:46:23 -06:00
Trent Nelson
156caeb710 Revert "Fixup flaky tests (backport #21617) (#21646)"
This reverts commit 922c74caea.
2021-12-13 12:46:23 -06:00
mergify[bot]
922c74caea Fixup flaky tests (backport #21617) (#21646)
* Fix fork race condition in optimistic violation tower tests (#19192)

* Fix fork race condition in optimistic violation tower tests

* clippy

* pr comments

* Fixup flaky tests (#21617)

* Fixup flaky tests

* Fixup listeners

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
Co-authored-by: carllin <carl@solana.com>
2021-12-07 05:00:49 +00:00
Michael Vines
139d15cd84 Reformat imports to a consistent style for imports
rustfmt.toml configuration:
        imports_granularity = "One"
         group_imports = "One"
2021-12-03 09:41:09 -08:00
Ivan Mironov
8072635967 Add validator options to change priority of snapshot packager and RPC threads (v1.8) (#21020)
* Add function for changing thread's nice value

Linux only.

* Add validator option to change niceness of snapshot packager thread

* Add validator option to change niceness of RPC server threads

Fixes https://github.com/solana-labs/solana/issues/14556

* Run `./scripts/cargo-for-all-lock-files.sh tree`
2021-11-15 09:50:45 -08:00
mergify[bot]
21cd423e67 Rename "trusted" to "known" in validators/ (backport #21197) (#21255)
* Rename "trusted" to "known" in `validators/` (#21197)

* Replaced trusted with known validator

* Format Convention

(cherry picked from commit b0ca335463)

# Conflicts:
#	core/src/accounts_hash_verifier.rs
#	core/src/serve_repair.rs
#	rpc/src/rpc_service.rs
#	validator/src/bootstrap.rs

* Fix conflicts

Co-authored-by: Michael Keleti <16996410+mkeleti@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-11-12 21:36:53 +00:00
mergify[bot]
eacc69efba adds validator flag to allow private ip addresses (backport #18850) (#18975)
* adds validator flag to allow private ip addresses (#18850)

(cherry picked from commit d2d5f36a3c)

# Conflicts:
#	accounts-cluster-bench/Cargo.toml
#	bench-tps/Cargo.toml
#	cli/Cargo.toml
#	core/benches/cluster_info.rs
#	core/src/banking_stage.rs
#	core/src/broadcast_stage.rs
#	core/src/broadcast_stage/broadcast_duplicates_run.rs
#	core/src/broadcast_stage/fail_entry_verification_broadcast_run.rs
#	core/src/broadcast_stage/standard_broadcast_run.rs
#	core/src/cluster_slots_service.rs
#	core/src/repair_service.rs
#	core/src/tvu.rs
#	core/src/validator.rs
#	dos/Cargo.toml
#	gossip/src/cluster_info.rs
#	gossip/src/crds_gossip_pull.rs
#	gossip/src/crds_gossip_push.rs
#	gossip/src/gossip_service.rs
#	local-cluster/Cargo.toml
#	local-cluster/src/cluster_tests.rs
#	local-cluster/tests/local_cluster.rs
#	rpc/Cargo.toml
#	rpc/src/rpc.rs
#	tokens/Cargo.toml
#	validator/Cargo.toml
#	validator/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-29 21:43:24 +00:00
mergify[bot]
daf2c3c155 Consider all peers as potential candidates during pull-request in case of offline nodes (#18333) (#18372)
* Try all peers during pull-request in case of offline nodes

* fix clippy err

(cherry picked from commit f4fb5de545)

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2021-07-01 22:56:20 +00:00
mergify[bot]
56cd963fd7 Ignore flaky tests (#18278) (#18282)
(cherry picked from commit 639a61d639)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-06-28 23:13:25 +00:00
mergify[bot]
9c1a6bed7b Fix flaky optimistic violation detection cluster test (#18027) (#18126)
* Fix flaky optimistic violation detection cluster test

* Add small sleep to avoid tight loop

(cherry picked from commit 423e0d90ee)

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2021-06-22 05:07:33 +00:00
mergify[bot]
0e7512a225 Fix Nightly Clippy Warnings (backport #18065) (#18070)
* chore: cargo +nightly clippy --fix -Z unstable-options

(cherry picked from commit 6514096a67)

# Conflicts:
#	core/src/banking_stage.rs
#	core/src/cost_model.rs
#	core/src/cost_tracker.rs
#	core/src/execute_cost_table.rs
#	core/src/replay_stage.rs
#	core/src/tvu.rs
#	ledger-tool/src/main.rs
#	programs/bpf_loader/build.rs
#	rbpf-cli/src/main.rs
#	sdk/cargo-build-bpf/src/main.rs
#	sdk/cargo-test-bpf/src/main.rs
#	sdk/src/secp256k1_instruction.rs

* chore: cargo fmt

(cherry picked from commit 789f33e8db)

* Updates BPF program assert_instruction_count tests.

(cherry picked from commit c1e03f3410)

# Conflicts:
#	programs/bpf/tests/programs.rs

* Resolve conflicts

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-18 20:02:48 +00:00
Jon Cinque
0c90307677 v1.7: Stake refactor (Manual backport of #17906) (#17978)
* Refactor stake program into solana_program (#17906)

* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt

* Run cargo fmt post cherry-pick
2021-06-15 23:43:22 +02:00
mergify[bot]
738df79394 Add local cluster tests that broadcast duplicate slots (#13995) (#17863)
* Add duplicate node local cluster test

* fix clippy

* remove dupe test

(cherry picked from commit 050bb5446d)

Co-authored-by: Justin Starry <justin@solana.com>
2021-06-10 03:17:44 +00:00
Lijun Wang
54f0fc9f0f Use type alias for DownloadProgress callback (#17518)
Convert to use type alias for the callback and cascade the changes to callers. Thanks @jeffwashington for the help making it possible.
Changed the closure for the progress update in the validator main to FnMut and modify the abort count in the closure which is more reliable.
2021-05-26 13:26:07 -07:00
Tyera Eulberg
9a5330b7eb Move gossip modules into solana-gossip crate (#17352)
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
2021-05-26 09:15:46 -06:00
carllin
3dfe87973b Propagate dead slots up to replay (#17227) 2021-05-25 13:43:47 -07:00
Lijun Wang
4c17243157 snapshot download enhancement (#17415)
1. Allow the validator bootstrap code to specify the minimal snapshot download speed. If the snapshot download speed is detected below that, a different RPC can be retried. The default is 10MB/sec.

2. To prevent spinning on a number of sub-optimal choices and not making progress, the abort/retry logic is implemented with the following safe guards:
2.1 at maximum we do this retry for 5 times -- this number is configurable with default 5.
2.2 if the download in one notification round (5 second) is more than 2%, do not do retry -- it is not as bad anyway.
2.3 if the remaining estimate time is less than 1 minutes, do not abort retry as it will be done quickly anyway.
2.4 We do this abort/retry logic only at the first notification to avoid wasting download efforts -- the reasoning is being opportunistic and too greedy may not achieve overall shorter download time.

3. The download_snapshot and download_file is modified with the option allowing caller to notified of download progress via a callback. This allows the business logic of retrying to the place it belongs.
2021-05-25 09:32:12 -07:00
Lijun Wang
9c42a89a43 Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
carllin
bc7e741514 Integrate gossip votes into switching threshold (#16973) 2021-05-04 00:51:42 -07:00
carllin
b5d30846d6 Retry latest vote if expired (#16735) 2021-04-28 11:46:16 -07:00
Jeff Washington (jwash)
6d160768d7 delete vest program (#16795) 2021-04-26 08:50:46 -05:00
carllin
4c94f8933f Ingest votes from gossip into fork choice (#16560) 2021-04-21 14:40:35 -07:00
Justin Starry
e817a6db00 Add timeout for local cluster partition tests (#16123)
* Add timeout for local cluster partition tests

* fix optimistic conf test logs

* Bump instruction count assertions
2021-03-25 13:27:07 -06:00
Jeff Washington (jwash)
8a3135d17b Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
Tyera Eulberg
1fc8836631 Add timeout to prevent infinite loop (#15715) 2021-03-05 21:37:46 +09:00
Michael Vines
24ab84936e Break up RPC API into three categories: minimal, full and admin 2021-03-04 16:39:44 -08:00
Tyera Eulberg
a4f0033bd7 Remove ValidatorConfig derive Clone, and fix local-cluster tests (#15647)
* Remove ValidatorConfig derive Clone

* Add local-cluster ValidatorConfig helpers

* Fix benches
2021-03-03 04:21:30 +00:00
Trent Nelson
7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Jeff Washington (jwash)
600ff0d915 calculate hash from store instead of index (#15034)
* calculate hash from store instead of index

* restore update hash in abs
2021-02-04 09:00:33 -06:00
Tom Parker-Shemilt
01230a0105 Remove serial_test_derive dependency (#14891) 2021-01-28 22:35:31 -07:00
Tyera Eulberg
ffa5c7dcc8 Deprecate commitment variants (#14797)
* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments
2021-01-26 19:23:07 +00:00
Michael Vines
bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
Michael Vines
7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
Michael Vines
0b92720fdb Don't require increased open file limit in solana-test-validator
Travis CI in particular does not allow the open file limit to be
increased.
2020-12-16 20:43:08 -08:00
Jack May
582418de5e Cleanup ledger builtins (#14083) 2020-12-15 12:23:49 -08:00
Michael Vines
7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
Michael Vines
61ab2072bd Clean up default commitment handling for subscriptions 2020-11-23 22:54:47 -08:00
carllin
791fb17437 Prevent scans on unrooted slots from seeing partial clean (#13628)
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 13:01:04 -08:00
carllin
9bb11a2dcc Cleanup (#13728)
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 03:17:19 -08:00
sakridge
598e5f58d5 Add wait for max stake command (#13532) 2020-11-12 13:48:34 -08:00
carllin
9821a7754c Discard pre hard fork persisted tower if hard-forking (#13536)
* Discard pre hard fork persisted tower if hard-forking

* Relax config.require_tower

* Add cluster test

* nits

* Remove unnecessary check

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-12 23:29:04 +09:00
Justin Starry
c24fbb6f8b Fix duplicate records of inner instructions (#13380)
* Fix duplicate records of inner instructions

* fix tests

* fix clippy

* Remove bad_inner_instructions
2020-11-05 15:23:52 +08:00