Commit Graph

29 Commits

Author SHA1 Message Date
mergify[bot]
7584262f47 add validator option --accounts-db-skip-shrink (backport #19028) (#19128)
* add validator option --accounts-db-skip-shrink (#19028)

* add validator option --accounts-db-skip-shrink

* typo

(cherry picked from commit 3280ae3e9f)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	ledger/src/blockstore_processor.rs
#	replica-node/src/replica_node.rs
#	runtime/src/snapshot_utils.rs

* Fix conflicts

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-10 10:07:27 -06:00
mergify[bot]
eacc69efba adds validator flag to allow private ip addresses (backport #18850) (#18975)
* adds validator flag to allow private ip addresses (#18850)

(cherry picked from commit d2d5f36a3c)

# Conflicts:
#	accounts-cluster-bench/Cargo.toml
#	bench-tps/Cargo.toml
#	cli/Cargo.toml
#	core/benches/cluster_info.rs
#	core/src/banking_stage.rs
#	core/src/broadcast_stage.rs
#	core/src/broadcast_stage/broadcast_duplicates_run.rs
#	core/src/broadcast_stage/fail_entry_verification_broadcast_run.rs
#	core/src/broadcast_stage/standard_broadcast_run.rs
#	core/src/cluster_slots_service.rs
#	core/src/repair_service.rs
#	core/src/tvu.rs
#	core/src/validator.rs
#	dos/Cargo.toml
#	gossip/src/cluster_info.rs
#	gossip/src/crds_gossip_pull.rs
#	gossip/src/crds_gossip_push.rs
#	gossip/src/gossip_service.rs
#	local-cluster/Cargo.toml
#	local-cluster/src/cluster_tests.rs
#	local-cluster/tests/local_cluster.rs
#	rpc/Cargo.toml
#	rpc/src/rpc.rs
#	tokens/Cargo.toml
#	validator/Cargo.toml
#	validator/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-29 21:43:24 +00:00
mergify[bot]
0e7512a225 Fix Nightly Clippy Warnings (backport #18065) (#18070)
* chore: cargo +nightly clippy --fix -Z unstable-options

(cherry picked from commit 6514096a67)

# Conflicts:
#	core/src/banking_stage.rs
#	core/src/cost_model.rs
#	core/src/cost_tracker.rs
#	core/src/execute_cost_table.rs
#	core/src/replay_stage.rs
#	core/src/tvu.rs
#	ledger-tool/src/main.rs
#	programs/bpf_loader/build.rs
#	rbpf-cli/src/main.rs
#	sdk/cargo-build-bpf/src/main.rs
#	sdk/cargo-test-bpf/src/main.rs
#	sdk/src/secp256k1_instruction.rs

* chore: cargo fmt

(cherry picked from commit 789f33e8db)

* Updates BPF program assert_instruction_count tests.

(cherry picked from commit c1e03f3410)

# Conflicts:
#	programs/bpf/tests/programs.rs

* Resolve conflicts

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-18 20:02:48 +00:00
mergify[bot]
e9c234d89f Make account shrink configurable #17544 (backport #17778) (#18013)
* Make account shrink configurable #17544 (#17778)

1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544

(cherry picked from commit 269d995832)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/accounts_db.rs
#	runtime/src/snapshot_utils.rs

* fix some merge errors

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-17 17:32:03 +00:00
mergify[bot]
5c495ad1b0 ledger tool limit_load_slot_count_from_snapshot avoids assert failures (backport #17974) (#18008)
* ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974)

(cherry picked from commit dbd4dc04b0)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/blockstore_processor.rs
#	runtime/benches/accounts.rs
#	runtime/src/bank.rs

* fix merge errors

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-17 01:32:50 +00:00
mergify[bot]
12e92dd59d verify bank hash on startup with ledger tool option (backport #17939) (#17996)
* verify bank hash on startup with ledger tool option (#17939)

(cherry picked from commit f558b9b6bf)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/snapshot_utils.rs

* fix merge errors

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-16 14:46:34 +00:00
mergify[bot]
61573756f8 add metrics for startup (#17913) (#17975)
* add metrics for startup

    * roll timings up higher

    * fix test

    * fix duplicate

    (cherry picked from commit 471b34132e)

    # Conflicts:
    #       ledger/src/bank_forks_utils.rs
    #       runtime/src/snapshot_utils.rs
conflicts because #17778 is not present.

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 01:40:56 +00:00
mergify[bot]
d6b83e3b0a add metrics to handle_snapshot_requests (#17937) (#17945)
(cherry picked from commit e6bbd4b3f0)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-14 22:56:14 +00:00
Jeff Washington (jwash)
6b9d8d41a3 add --limit_load_slot_count_from_snapshot to ledger-tool (#17417) 2021-05-26 10:36:12 -05:00
Tyera Eulberg
9a5330b7eb Move gossip modules into solana-gossip crate (#17352)
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
2021-05-26 09:15:46 -06:00
Lijun Wang
9c42a89a43 Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Jeff Washington (jwash)
f39dda00e0 type AccountSecondaryIndexes = HashSet (#17108) 2021-05-10 14:22:48 +00:00
Michael Vines
5df36aec7d Pacify clippy 2021-02-19 20:08:41 -08:00
Trent Nelson
7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Jeff Washington (jwash)
ba02452d75 add validator flag no-accounts-db-index-hashing (#15350)
* add validator flag no_accounts_db_index_hashing

* add validator flag no_accounts_db_index_hashing
2021-02-16 21:13:48 +00:00
carllin
990bb426a9 Fix flaky test test_concurrent_snapshot_packaging (#15252) 2021-02-11 16:03:51 -08:00
Jeff Washington (jwash)
fabecdc86c use thread pool for non-index hash calculations (#15149) 2021-02-05 19:48:55 +00:00
behzad nouri
6fd5ec0e4c caches descendants in bank forks (#15107) 2021-02-05 18:00:45 +00:00
Jeff Washington (jwash)
600ff0d915 calculate hash from store instead of index (#15034)
* calculate hash from store instead of index

* restore update hash in abs
2021-02-04 09:00:33 -06:00
Michael Vines
bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
carllin
6dfad0652f Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Michael Vines
a95675a7ce Avoid tmp snapshot backlog in SnapshotPackagerService under high load (#14516) 2021-01-11 10:21:15 -08:00
Michael Vines
7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
carllin
5affd8aa72 Add secondary indexes (#14212) 2020-12-31 18:06:03 -08:00
carllin
55fc963595 Move slot cleanup to AccountsBackgroundService (#13911)
* Move bank drop to AccountsBackgroundService

* Send to ABS on drop instead, protects against other places banks are dropped

* Fix Abi

* test

Co-authored-by: Carl Lin <carl@solana.com>
2020-12-13 01:22:34 +00:00
Michael Vines
73111b005f Reduce the number of snapshots 2020-12-01 11:13:37 -08:00
Ryo Onodera
c0675968b1 Support Debug Bank (#13017) 2020-10-21 01:05:45 +09:00
carllin
c879e7c1ad Fix fee mismatch on snapshot deserialize (#12697)
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-08 23:44:41 -07:00
carllin
06f84c65f1 Fix rooted accounts cleanup, simplify locking (#12194)
Co-authored-by: Carl Lin <carl@solana.com>
2020-09-28 16:04:46 -07:00