212 Commits

Author SHA1 Message Date
mergify[bot]
b6d040b0d7 Remove deprecated slow epoch boundary methods (backport #21568) (#21595)
* Remove deprecated slow epoch boundary methods (#21568)

(cherry picked from commit 1430b58a6d)

# Conflicts:
#	core/src/replay_stage.rs
#	core/src/tvu.rs
#	core/src/validator.rs
#	local-cluster/src/validator_configs.rs
#	runtime/src/bank.rs
#	runtime/src/stakes.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-12-03 15:54:33 -05:00
Michael Vines
139d15cd84 Reformat imports to a consistent style for imports
rustfmt.toml configuration:
        imports_granularity = "One"
         group_imports = "One"
2021-12-03 09:41:09 -08:00
Michael Vines
008139f506 Bump version to v1.8.7 2021-12-01 11:44:18 -08:00
Tyera Eulberg
336ee01aae Bump version to v1.8.6 (#21329) 2021-11-17 17:16:20 -07:00
mergify[bot]
064cce41f7 add --no-os-network-stats-reporting option (backport #21296) (#21303)
* add --no-os-network-stats-reporting option (#21296)

(cherry picked from commit d5de0c8e12)

# Conflicts:
#	core/src/system_monitor_service.rs
#	ledger-tool/src/main.rs
#	validator/src/main.rs

* resolve merge conflicts

Co-authored-by: Jeff Biseda <jbiseda@gmail.com>
2021-11-16 23:00:54 +00:00
sakridge
80c3591391 Bump version to 1.8.5 (#21295) 2021-11-16 11:58:57 -07:00
Ivan Mironov
8072635967 Add validator options to change priority of snapshot packager and RPC threads (v1.8) (#21020)
* Add function for changing thread's nice value

Linux only.

* Add validator option to change niceness of snapshot packager thread

* Add validator option to change niceness of RPC server threads

Fixes https://github.com/solana-labs/solana/issues/14556

* Run `./scripts/cargo-for-all-lock-files.sh tree`
2021-11-15 09:50:45 -08:00
mergify[bot]
21cd423e67 Rename "trusted" to "known" in validators/ (backport #21197) (#21255)
* Rename "trusted" to "known" in `validators/` (#21197)

* Replaced trusted with known validator

* Format Convention

(cherry picked from commit b0ca335463)

# Conflicts:
#	core/src/accounts_hash_verifier.rs
#	core/src/serve_repair.rs
#	rpc/src/rpc_service.rs
#	validator/src/bootstrap.rs

* Fix conflicts

Co-authored-by: Michael Keleti <16996410+mkeleti@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-11-12 21:36:53 +00:00
Tyera Eulberg
09ef4d12f7 Bump version to 1.8.4 (#21232) (#21235)
Co-authored-by: sakridge <sakridge@gmail.com>
2021-11-10 15:20:41 -07:00
Tyera Eulberg
74684a107c Revert "Bump version to 1.8.4 (#21232)"
This reverts commit 19b3ba0442.
2021-11-10 11:19:39 -07:00
sakridge
19b3ba0442 Bump version to 1.8.4 (#21232) 2021-11-10 16:38:44 +01:00
Tyera Eulberg
8dd3c1ece1 Bump version to v1.8.3 (#21040) 2021-10-28 11:17:11 -06:00
Trent Nelson
23b6ce7980 Bump version to 1.8.2 2021-10-21 00:43:40 -06:00
mergify[bot]
8cba6cca76 rpc-send-tx-svc server-side retry knobs (backport #20818) (#20830)
* rpc-send-tx-svc: add with_config constructor

(cherry picked from commit fe098b5ddc)

# Conflicts:
#	Cargo.lock
#	core/Cargo.toml
#	replica-node/Cargo.toml
#	rpc/src/rpc_service.rs
#	rpc/src/send_transaction_service.rs
#	validator/Cargo.toml

* rpc-send-tx-svc: server-side retry knobs

(cherry picked from commit 2744a2128c)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-10-21 02:15:03 +00:00
Lijun Wang
50cb612ae1 Accountsdb stream plugin improvement (#20419) (#20573)
* Accountsdb stream plugin improvement (#20419)

Support using connection pooling and use multiple threads to do Postgres db operations. The performance is improved from 1500 RPS to 40,000 RPS measured during validator start.

Support multiple plugins at the same time.

* Fixed a fmt issue
2021-10-10 15:24:12 -07:00
Lijun Wang
7d0494fcaa Merge AccountsDb plugin framework to v1.8 (#20518)
Merge AccountsDb plugin framework to v1.8 (#20518)
Summary of Changes

Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows

Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.

The code comprises 4 major parts:

accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-10-07 14:15:05 -07:00
Tao Zhu
348ba57b12 Bump version to 1.8.1 2021-10-06 17:57:06 -07:00
Trent Nelson
a4df784e82 Bump version to 1.8.0 2021-10-06 15:48:23 -06:00
Justin Starry
d922971ec6 Optimize stakes cache and rewards at epoch boundaries (backport #20432) (#20472)
* Optimize stakes cache and rewards at epoch boundaries (backport #20432)

* fix conflicts
2021-10-06 16:15:27 +00:00
Tyera Eulberg
734b380cdb Bump version to v1.7.15 (#20338) 2021-09-30 10:51:34 -06:00
sakridge
fec15f69f4 Increment 1.7 version (#20316) 2021-09-29 15:37:45 -04:00
sakridge
70d556782b Bump 1.7 version (#19943) 2021-09-16 13:16:09 -06:00
Tyera Eulberg
f73a61d2ec Bump version to 1.7.12 2021-08-27 16:24:24 +00:00
Trent Nelson
ab5d032634 Bump version to v1.7.11 2021-08-12 06:55:18 +00:00
mergify[bot]
7584262f47 add validator option --accounts-db-skip-shrink (backport #19028) (#19128)
* add validator option --accounts-db-skip-shrink (#19028)

* add validator option --accounts-db-skip-shrink

* typo

(cherry picked from commit 3280ae3e9f)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	ledger/src/blockstore_processor.rs
#	replica-node/src/replica_node.rs
#	runtime/src/snapshot_utils.rs

* Fix conflicts

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-10 10:07:27 -06:00
Trent Nelson
b7f1f19d8e Bump version to v1.7.10 2021-07-31 01:19:33 -06:00
mergify[bot]
b142ef5f8b Improve target CPU feature checks (backport #18940) (#18958)
* Improve check for Apple M1 silicon under Rosetta

(cherry picked from commit 59641623d1)

# Conflicts:
#	core/src/validator.rs

* test-validator: start logging asap

(cherry picked from commit ee65ffb3c8)

* test-validator: move feature check earlier in startup

(cherry picked from commit e641f257ef)

* validator: add avx2 runtime check

(cherry picked from commit c435f7b3e3)

* validator: start logging asap

(cherry picked from commit ed8285c096)

* validator: check target CPU features earlier

(cherry picked from commit 8ed0cd0fff)

* validator: remove disused cuda config argument

(cherry picked from commit 71f6d839f9)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-07-30 06:47:22 +00:00
mergify[bot]
eacc69efba adds validator flag to allow private ip addresses (backport #18850) (#18975)
* adds validator flag to allow private ip addresses (#18850)

(cherry picked from commit d2d5f36a3c)

# Conflicts:
#	accounts-cluster-bench/Cargo.toml
#	bench-tps/Cargo.toml
#	cli/Cargo.toml
#	core/benches/cluster_info.rs
#	core/src/banking_stage.rs
#	core/src/broadcast_stage.rs
#	core/src/broadcast_stage/broadcast_duplicates_run.rs
#	core/src/broadcast_stage/fail_entry_verification_broadcast_run.rs
#	core/src/broadcast_stage/standard_broadcast_run.rs
#	core/src/cluster_slots_service.rs
#	core/src/repair_service.rs
#	core/src/tvu.rs
#	core/src/validator.rs
#	dos/Cargo.toml
#	gossip/src/cluster_info.rs
#	gossip/src/crds_gossip_pull.rs
#	gossip/src/crds_gossip_push.rs
#	gossip/src/gossip_service.rs
#	local-cluster/Cargo.toml
#	local-cluster/src/cluster_tests.rs
#	local-cluster/tests/local_cluster.rs
#	rpc/Cargo.toml
#	rpc/src/rpc.rs
#	tokens/Cargo.toml
#	validator/Cargo.toml
#	validator/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-29 21:43:24 +00:00
Michael Vines
548ddff7ed Bump version to v1.7.9 2021-07-24 11:23:44 -06:00
Ryo Onodera
1cc8de0fed Bump version to v1.7.8 (#18866) 2021-07-24 01:14:03 +09:00
Trent Nelson
19049ca91b Bump version to v1.7.7 2021-07-17 08:42:22 +00:00
sakridge
551dc0a74c Bump 1.7 version (#18723) 2021-07-16 09:43:18 -06:00
mergify[bot]
daf2c3c155 Consider all peers as potential candidates during pull-request in case of offline nodes (#18333) (#18372)
* Try all peers during pull-request in case of offline nodes

* fix clippy err

(cherry picked from commit f4fb5de545)

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2021-07-01 22:56:20 +00:00
Trent Nelson
4466aa39c4 Bump version to v1.7.5 2021-06-30 22:55:01 -06:00
mergify[bot]
56cd963fd7 Ignore flaky tests (#18278) (#18282)
(cherry picked from commit 639a61d639)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-06-28 23:13:25 +00:00
Trent Nelson
597429ab3e Bump version to v1.7.4 2021-06-22 19:57:34 +00:00
mergify[bot]
9c1a6bed7b Fix flaky optimistic violation detection cluster test (#18027) (#18126)
* Fix flaky optimistic violation detection cluster test

* Add small sleep to avoid tight loop

(cherry picked from commit 423e0d90ee)

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2021-06-22 05:07:33 +00:00
mergify[bot]
0e7512a225 Fix Nightly Clippy Warnings (backport #18065) (#18070)
* chore: cargo +nightly clippy --fix -Z unstable-options

(cherry picked from commit 6514096a67)

# Conflicts:
#	core/src/banking_stage.rs
#	core/src/cost_model.rs
#	core/src/cost_tracker.rs
#	core/src/execute_cost_table.rs
#	core/src/replay_stage.rs
#	core/src/tvu.rs
#	ledger-tool/src/main.rs
#	programs/bpf_loader/build.rs
#	rbpf-cli/src/main.rs
#	sdk/cargo-build-bpf/src/main.rs
#	sdk/cargo-test-bpf/src/main.rs
#	sdk/src/secp256k1_instruction.rs

* chore: cargo fmt

(cherry picked from commit 789f33e8db)

* Updates BPF program assert_instruction_count tests.

(cherry picked from commit c1e03f3410)

# Conflicts:
#	programs/bpf/tests/programs.rs

* Resolve conflicts

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-18 20:02:48 +00:00
Stephen Akridge
d159ae9342 Bump version to v1.7.3 2021-06-17 15:34:50 -06:00
mergify[bot]
e9c234d89f Make account shrink configurable #17544 (backport #17778) (#18013)
* Make account shrink configurable #17544 (#17778)

1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544

(cherry picked from commit 269d995832)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/accounts_db.rs
#	runtime/src/snapshot_utils.rs

* fix some merge errors

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-17 17:32:03 +00:00
Jon Cinque
0c90307677 v1.7: Stake refactor (Manual backport of #17906) (#17978)
* Refactor stake program into solana_program (#17906)

* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt

* Run cargo fmt post cherry-pick
2021-06-15 23:43:22 +02:00
mergify[bot]
738df79394 Add local cluster tests that broadcast duplicate slots (#13995) (#17863)
* Add duplicate node local cluster test

* fix clippy

* remove dupe test

(cherry picked from commit 050bb5446d)

Co-authored-by: Justin Starry <justin@solana.com>
2021-06-10 03:17:44 +00:00
Ryo Onodera
48e565038a Bump version to v1.7.2 (#17831) 2021-06-08 10:29:39 +00:00
mergify[bot]
cd6e1d921c update dependence version for gag to latest support windows (#17801) (#17825)
* update dependence version for gag to leatest support windows

* fix compile on windows

* add Cargo.lock

(cherry picked from commit e0ab5ee4f8)

Co-authored-by: Govlzkoy <gotope@users.noreply.github.com>
2021-06-08 07:48:39 +00:00
mergify[bot]
e247625025 Create solana-poh and move remaining rpc modules to solana-rpc (backport #17698) (#17745)
* Create solana-poh and move remaining rpc modules to solana-rpc (#17698)

* Create solana-poh crate

* Move BigTableUploadService to solana-ledger

* Add solana-rpc to workspace

* Move dependencies to solana-rpc

* Move remaining rpc modules to solana-rpc

* Single use statement solana-poh

* Single use statement solana-rpc

(cherry picked from commit 544b3c0d17)

# Conflicts:
#	Cargo.lock
#	banking-bench/Cargo.toml
#	core/Cargo.toml
#	core/benches/banking_stage.rs
#	local-cluster/Cargo.toml
#	rpc/Cargo.toml
#	stake-monitor/Cargo.toml
#	validator/Cargo.toml

* Fix conflicts & versions

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-04 18:19:08 +00:00
mergify[bot]
893df9b277 Rename ValidatorExit and move to sdk (#17728) (#17729)
(cherry picked from commit 3a647c4bea)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-06-04 04:38:49 +00:00
Michael Vines
975c942ea7 Bump version to v1.7.1 2021-06-02 05:21:14 +00:00
Lijun Wang
54f0fc9f0f Use type alias for DownloadProgress callback (#17518)
Convert to use type alias for the callback and cascade the changes to callers. Thanks @jeffwashington for the help making it possible.
Changed the closure for the progress update in the validator main to FnMut and modify the abort count in the closure which is more reliable.
2021-05-26 13:26:07 -07:00
Tyera Eulberg
9a5330b7eb Move gossip modules into solana-gossip crate (#17352)
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
2021-05-26 09:15:46 -06:00
carllin
3dfe87973b Propagate dead slots up to replay (#17227) 2021-05-25 13:43:47 -07:00