413 Commits

Author SHA1 Message Date
Tyera Eulberg
e9bef42502 Bump version to v1.7.18 (#21011) 2021-10-26 23:02:09 -06:00
Justin Starry
e3e1396c1d Bump version to v1.7.17 (#20896) 2021-10-23 09:48:15 -04:00
mergify[bot]
4453e2937f solana-test-validator --log now includes version/argument information (#20892)
(cherry picked from commit 86bf071d77)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-10-22 21:45:34 +00:00
mergify[bot]
e333cd48e8 rpc-send-tx-svc server-side retry knobs (backport #20818) (#20829)
* rpc-send-tx-svc: add with_config constructor

(cherry picked from commit fe098b5ddc)

# Conflicts:
#	Cargo.lock
#	core/Cargo.toml
#	replica-node/Cargo.toml
#	rpc/src/rpc_service.rs
#	rpc/src/send_transaction_service.rs
#	validator/Cargo.toml

* rpc-send-tx-svc: server-side retry knobs

(cherry picked from commit 2744a2128c)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-10-21 00:13:46 +00:00
mergify[bot]
84644ce994 Ignore delinquent stake on exit (#20367) (#20381)
* add --ignore-delinquency flag to validator exit and wait-for-restart-window sub commands

* Fix a merge issue

* Add missing variable declaration

* Remove empty line to help CI checks pass

* run rustfmt

* Change argument wording for clarity and verbosity

* Change --ignore-delinquent-stake to --max-delinquent-stake

* cargo fmtgit add validator/src/main.rsgit add validator/src/main.rs

* Adjust per mvines

* Formatting

* Improve input validation

* Please automate cargo fmt somehow

(cherry picked from commit fc5dd7f3bc)

Co-authored-by: Michael <68944931+michaelh-laine@users.noreply.github.com>
2021-10-12 19:14:30 +00:00
mergify[bot]
353cb8efa5 solana-validator wait-for-restart-window command now accepts an optional --identity argument (backport #18684) (#20609)
* wait-for-restart-window command now accepts an optional --identity argument

(cherry picked from commit c418e8f370)

# Conflicts:
#	validator/src/main.rs

* Fixed cherry-pick conflicts

Co-authored-by: Michael Vines <mvines@gmail.com>
Co-authored-by: Steven Czabaniuk <steven@solana.com>
2021-10-12 06:15:30 +00:00
Trent Nelson
33c24ec3ae Bump version to 1.7.16 (#20519) 2021-10-07 18:29:40 -06:00
Justin Starry
d922971ec6 Optimize stakes cache and rewards at epoch boundaries (backport #20432) (#20472)
* Optimize stakes cache and rewards at epoch boundaries (backport #20432)

* fix conflicts
2021-10-06 16:15:27 +00:00
Michael Vines
00b03897e1 Default --rpc-bind-address to 127.0.0.1 when --private-rpc is provided and --bind-address is not
(cherry picked from commit 221343e849)
2021-10-04 16:58:46 -07:00
mergify[bot]
51593a882b Properly enable unprefixed_malloc_on_supported_platforms in tikv-jemallocator (#20351) (#20354)
Trivial typo fix.

Fixes: 4bf6d0c4d7 ("adds unprefixed_malloc_on_supported_platforms to jemalloc (#20317)")
(cherry picked from commit 8ae88632cb)

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
2021-09-30 20:26:11 +00:00
Tyera Eulberg
734b380cdb Bump version to v1.7.15 (#20338) 2021-09-30 10:51:34 -06:00
mergify[bot]
5d8429d953 adds unprefixed_malloc_on_supported_platforms to jemalloc (#20317) (#20325)
Without this feature jemalloc is used only for Rust code but not for
bundled C/C++ libraries (like rocksdb).
https://github.com/solana-labs/solana/issues/14366#issuecomment-930404992

(cherry picked from commit 4bf6d0c4d7)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-09-29 22:49:47 +00:00
sakridge
fec15f69f4 Increment 1.7 version (#20316) 2021-09-29 15:37:45 -04:00
sakridge
257ddbeee1 Tpu vote 1.7 (#20187)
* Add separate vote processing tpu port

* Add feature to send to tpu vote port

* Add vote rejecting sigverify mode

* use packet.meta.is_simple_vote_tx in place of deserialization

* consolidate code that identifies vote tx atcommon path for cpu and gpu

* new key for feature set

* banking forward tpu vote

* add tpu vote port to dockerfile and other review changes

* Simplify thread id compare

* fix a test; updated cluster_info ABI change

Co-authored-by: Tao Zhu <tao@solana.com>
2021-09-29 18:12:58 +02:00
mergify[bot]
085f5f945d uses tikv_jemallocator::Jemalloc as the global allocator (backport #20149) (#20166)
* uses tikv_jemallocator::Jemalloc as the global allocator (#20149)

https://github.com/solana-labs/solana/pull/16346
switched default allocator from jemalloc to system allocator, but that
has shown regressions in form of higher ram usage causing nodes go OOM:
https://discord.com/channels/428295358100013066/439194979856809985/890413193858539580

This commit sets jemalloc as the default allocator.

(cherry picked from commit 2cf081d863)

# Conflicts:
#	Cargo.lock

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-09-24 17:37:57 +00:00
mergify[bot]
b112e4a8aa windows: Make solana-test-validator work (backport #20099) (#20123)
* windows: Make solana-test-validator work (#20099)

* windows: Make solana-test-validator work

The important changes to get this going on Windows:

* ledger lock needs to be done on a file instead of the directory
* IPC service needs to use the Windows pipe naming scheme
* always disable the JIT
* file logging not possible yet because we can't redirect stderr,
but this will change once env_logger fixes the pipe output target!

* Integrate review feedback

(cherry picked from commit 567f30aa1a)

# Conflicts:
#	validator/src/bin/solana-test-validator.rs
#	validator/src/lib.rs
#	validator/src/main.rs

* Fix merge conflicts

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-09-24 12:59:12 +00:00
mergify[bot]
03da3eaa81 Optimize RPC pubsub for multiple clients with the same subscription (backport #18943) (#19987)
* Optimize RPC pubsub for multiple clients with the same subscription (#18943)

* reimplement rpc pubsub with a broadcast queue

* update tests for new pubsub implementation

* fix: fix review suggestions

* chore(rpc): add additional pubsub metrics

* integrate max subscriptions check into SubscriptionTracker to reduce locking

* separate subscription control from tracker

* limit memory usage of items in pubsub broadcast queue, improve error handling

* add more pubsub metrics

* add final count metrics to pubsub

* add metric for total number of subscriptions

* fix small review suggestions

* remove by_params from SubscriptionTracker and add node_progress_watchers map instead

* add subscription tracker tests

* add metrics for number of pubsub notifications as a counter

* ignore clippy lint in TokenCounter

* fix underflow in token counter

* reduce queue capacity in pubsub tests

* fix(rpc): fix test timeouts

* fix race in account subscription test

* Add RpcSubscriptions::new_for_tests

Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 65227f44dc)

# Conflicts:
#	Cargo.lock
#	core/Cargo.toml
#	core/src/replay_stage.rs
#	core/src/validator.rs
#	replica-node/src/replica_node.rs
#	rpc/Cargo.toml

* Fix conflicts (and standardize naming to make future subscription backports easier

Co-authored-by: Pavel Strakhov <ri@idzaaus.org>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-20 06:00:08 +00:00
sakridge
70d556782b Bump 1.7 version (#19943) 2021-09-16 13:16:09 -06:00
mergify[bot]
8f082a239a Revert "fixup! test-validator: start logging asap (#19655)" (#19740) (#19745)
This reverts commit 64a2d7081b.

(cherry picked from commit 46fc92f392)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-10 21:57:57 +00:00
mergify[bot]
a0e6a7c73b fixup! test-validator: start logging asap (backport #19655) (#19711)
* fixup! test-validator: start logging asap (#19655)

(cherry picked from commit 64a2d7081b)

# Conflicts:
#	validator/src/bin/solana-test-validator.rs

* Resolves conflicts

Co-authored-by: steviez <steven@solana.com>
2021-09-08 10:58:07 +00:00
Tyera Eulberg
f73a61d2ec Bump version to 1.7.12 2021-08-27 16:24:24 +00:00
mergify[bot]
52dfb4a09c Bump jsonrpc crates and remove old tokio (backport #18779) (#19453)
* Bump jsonrpc crates and remove old tokio (#18779)

* Bump jsonrpc crates and replace old tokio

* Bump tokio

* getBlockTime

* getBlocks

* getBlocksWithLimit, getInflationReward

* getBlock

* getFirstAvailableBlock

* getTransaction

* getSignaturesForAddress

* getSignatureStatuses

* Remove superfluous runtime

(cherry picked from commit 8596db8f53)

# Conflicts:
#	Cargo.lock
#	client/Cargo.toml
#	core/Cargo.toml
#	programs/bpf/Cargo.lock
#	rpc/Cargo.toml
#	rpc/src/rpc.rs
#	validator/Cargo.toml

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-27 00:55:02 +00:00
mergify[bot]
caea9c99cd Minimize trust (backport #19279) (#19335)
* docs: Mainnet Beta inflation has been enabled for quite some time

(cherry picked from commit 169ded9a70)

* validator: Trusted validators are now called known validators

(cherry picked from commit e0bc5fa690)

* docs: trust minimize

(cherry picked from commit 40613161a0)

* docs: correct known validator operator

(cherry picked from commit eced50d103)

* docs: Remove decommissioned testnet archetype validator

(cherry picked from commit a587eec20b)

* docs: update devnet start args with new validators

(cherry picked from commit 2a877ae06e)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-08-20 06:24:43 +00:00
Trent Nelson
ab5d032634 Bump version to v1.7.11 2021-08-12 06:55:18 +00:00
mergify[bot]
7584262f47 add validator option --accounts-db-skip-shrink (backport #19028) (#19128)
* add validator option --accounts-db-skip-shrink (#19028)

* add validator option --accounts-db-skip-shrink

* typo

(cherry picked from commit 3280ae3e9f)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	ledger/src/blockstore_processor.rs
#	replica-node/src/replica_node.rs
#	runtime/src/snapshot_utils.rs

* Fix conflicts

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-10 10:07:27 -06:00
NearlyAlwaysThere
bb06502d24 v1.7: Fix broken test validator
Removes duplicated locking logic
2021-08-02 08:08:28 -06:00
Trent Nelson
b7f1f19d8e Bump version to v1.7.10 2021-07-31 01:19:33 -06:00
mergify[bot]
b142ef5f8b Improve target CPU feature checks (backport #18940) (#18958)
* Improve check for Apple M1 silicon under Rosetta

(cherry picked from commit 59641623d1)

# Conflicts:
#	core/src/validator.rs

* test-validator: start logging asap

(cherry picked from commit ee65ffb3c8)

* test-validator: move feature check earlier in startup

(cherry picked from commit e641f257ef)

* validator: add avx2 runtime check

(cherry picked from commit c435f7b3e3)

* validator: start logging asap

(cherry picked from commit ed8285c096)

* validator: check target CPU features earlier

(cherry picked from commit 8ed0cd0fff)

* validator: remove disused cuda config argument

(cherry picked from commit 71f6d839f9)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-07-30 06:47:22 +00:00
mergify[bot]
20ad3005b5 allows private addresses by default in test-validator (backport #18976) (#18980)
* allows private addresses by default in test-validator (#18976)

(cherry picked from commit 1cef6fd4b4)

# Conflicts:
#	validator/src/bin/solana-test-validator.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-30 00:05:19 +00:00
mergify[bot]
eacc69efba adds validator flag to allow private ip addresses (backport #18850) (#18975)
* adds validator flag to allow private ip addresses (#18850)

(cherry picked from commit d2d5f36a3c)

# Conflicts:
#	accounts-cluster-bench/Cargo.toml
#	bench-tps/Cargo.toml
#	cli/Cargo.toml
#	core/benches/cluster_info.rs
#	core/src/banking_stage.rs
#	core/src/broadcast_stage.rs
#	core/src/broadcast_stage/broadcast_duplicates_run.rs
#	core/src/broadcast_stage/fail_entry_verification_broadcast_run.rs
#	core/src/broadcast_stage/standard_broadcast_run.rs
#	core/src/cluster_slots_service.rs
#	core/src/repair_service.rs
#	core/src/tvu.rs
#	core/src/validator.rs
#	dos/Cargo.toml
#	gossip/src/cluster_info.rs
#	gossip/src/crds_gossip_pull.rs
#	gossip/src/crds_gossip_push.rs
#	gossip/src/gossip_service.rs
#	local-cluster/Cargo.toml
#	local-cluster/src/cluster_tests.rs
#	local-cluster/tests/local_cluster.rs
#	rpc/Cargo.toml
#	rpc/src/rpc.rs
#	tokens/Cargo.toml
#	validator/Cargo.toml
#	validator/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-29 21:43:24 +00:00
Michael Vines
548ddff7ed Bump version to v1.7.9 2021-07-24 11:23:44 -06:00
Ryo Onodera
1cc8de0fed Bump version to v1.7.8 (#18866) 2021-07-24 01:14:03 +09:00
Trent Nelson
19049ca91b Bump version to v1.7.7 2021-07-17 08:42:22 +00:00
sakridge
551dc0a74c Bump 1.7 version (#18723) 2021-07-16 09:43:18 -06:00
Trent Nelson
4466aa39c4 Bump version to v1.7.5 2021-06-30 22:55:01 -06:00
mergify[bot]
2fdda2ec1b test-validator: hold rent constant with --slots-per-epoch (#18318)
(cherry picked from commit 02b14caa5f)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-06-30 08:37:54 +00:00
Trent Nelson
597429ab3e Bump version to v1.7.4 2021-06-22 19:57:34 +00:00
mergify[bot]
363b75619f adds shred-version to ip-echo-server response (backport #18066) (#18113)
* adds shred-version to ip-echo-server response

When starting a validator, the node initially joins gossip with
shred_verison = 0, until it adopts the entrypoint's shred-version:
https://github.com/solana-labs/solana/blob/9b182f408/validator/src/main.rs#L417

Depending on the load on the entrypoint, this adopting entrypoint
shred-version through gossip sometimes becomes very slow, and causes
several problems in gossip because we have to partially support
shred_version == 0 which is a source of leaking crds values from one
cluster to another. e.g. see
https://github.com/solana-labs/solana/pull/17899
and the other linked issues there.

In order to remove shred_version == 0 from gossip, this commit adds
shred-version to ip-echo-server response. Once the entrypoints are
updated, on validator start-up, if --expected_shred_version is not
specified we will obtain shred-version from the entrypoint using
ip-echo-server.

(cherry picked from commit 598093b5db)

# Conflicts:
#	Cargo.lock
#	net-utils/Cargo.toml
#	programs/bpf/Cargo.lock

* removes backport merge conflicts

* obtains shred-version from entrypoint's ip-echo-server in validator-main

(cherry picked from commit 58e115275a)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-06-21 23:04:08 +00:00
mergify[bot]
0e7512a225 Fix Nightly Clippy Warnings (backport #18065) (#18070)
* chore: cargo +nightly clippy --fix -Z unstable-options

(cherry picked from commit 6514096a67)

# Conflicts:
#	core/src/banking_stage.rs
#	core/src/cost_model.rs
#	core/src/cost_tracker.rs
#	core/src/execute_cost_table.rs
#	core/src/replay_stage.rs
#	core/src/tvu.rs
#	ledger-tool/src/main.rs
#	programs/bpf_loader/build.rs
#	rbpf-cli/src/main.rs
#	sdk/cargo-build-bpf/src/main.rs
#	sdk/cargo-test-bpf/src/main.rs
#	sdk/src/secp256k1_instruction.rs

* chore: cargo fmt

(cherry picked from commit 789f33e8db)

* Updates BPF program assert_instruction_count tests.

(cherry picked from commit c1e03f3410)

# Conflicts:
#	programs/bpf/tests/programs.rs

* Resolve conflicts

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-18 20:02:48 +00:00
Stephen Akridge
d159ae9342 Bump version to v1.7.3 2021-06-17 15:34:50 -06:00
mergify[bot]
e9c234d89f Make account shrink configurable #17544 (backport #17778) (#18013)
* Make account shrink configurable #17544 (#17778)

1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544

(cherry picked from commit 269d995832)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/accounts_db.rs
#	runtime/src/snapshot_utils.rs

* fix some merge errors

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-17 17:32:03 +00:00
mergify[bot]
b472dac6b3 validator: expose max active pubsub subscriptions to CLI (#18036)
(cherry picked from commit 5efc48fc69)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-06-17 08:02:20 +00:00
mergify[bot]
f633f34e43 Clean up remove_all_authorized_voters() (#18029)
(cherry picked from commit f1ebbbab8f)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-17 00:28:41 +00:00
Ryo Onodera
48e565038a Bump version to v1.7.2 (#17831) 2021-06-08 10:29:39 +00:00
mergify[bot]
cd6e1d921c update dependence version for gag to latest support windows (#17801) (#17825)
* update dependence version for gag to leatest support windows

* fix compile on windows

* add Cargo.lock

(cherry picked from commit e0ab5ee4f8)

Co-authored-by: Govlzkoy <gotope@users.noreply.github.com>
2021-06-08 07:48:39 +00:00
mergify[bot]
d06ca605cf Bump jsonrpc crates (backport #17797) (#17805)
* Bump jsonrpc crates (#17797)

* Bump jsonrpc crates

* Update error text

* gitignore for rpc

(cherry picked from commit 2e998ed11d)

# Conflicts:
#	Cargo.lock
#	validator/Cargo.toml

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-07 20:15:23 +00:00
mergify[bot]
e247625025 Create solana-poh and move remaining rpc modules to solana-rpc (backport #17698) (#17745)
* Create solana-poh and move remaining rpc modules to solana-rpc (#17698)

* Create solana-poh crate

* Move BigTableUploadService to solana-ledger

* Add solana-rpc to workspace

* Move dependencies to solana-rpc

* Move remaining rpc modules to solana-rpc

* Single use statement solana-poh

* Single use statement solana-rpc

(cherry picked from commit 544b3c0d17)

# Conflicts:
#	Cargo.lock
#	banking-bench/Cargo.toml
#	core/Cargo.toml
#	core/benches/banking_stage.rs
#	local-cluster/Cargo.toml
#	rpc/Cargo.toml
#	stake-monitor/Cargo.toml
#	validator/Cargo.toml

* Fix conflicts & versions

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-04 18:19:08 +00:00
mergify[bot]
5e7db52087 Avoid full-range compactions with periodic filtered b.g. ones (backport #16697) (#17741)
* Avoid full-range compactions with periodic filtered b.g. ones (#16697)

* Update rocksdb to v0.16.0

* Promote the infrequent and important log to info!

* Force background compaction by ttl without manual compaction

* Fix test

* Support no compaction mode in test_ledger_cleanup_compaction

* Fix comment

* Make compaction_interval customizable

* Avoid major compaction with periodic filtering...

* Adress lazy_static, special cfs and range check

* Clean up a bit and add comment

* Add comment

* More comments...

* Config code cleanup

* Add comment

* Use .conflicts_with()

* Nullify unneeded delete_range ops for special CFs

* Some clean ups

* Clarify the locking intention

* Ensure special CFs' consistency with PurgeType::CompactionFilter

* Fix comment

* Fix bad copy paste

* Fix various types...

* Don't use tuples

* Add a unit test for compaction_filter

* Fix typo...

* Remove flag and just use new behavior always

* Fix wrong condition negation...

* Doc. about no set_last_purged_slot in purge_slots

* Write a test and fix off-by-one bug....

* Apply suggestions from code review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Follow up to github review suggestions

* Fix line-wrapping

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 1f97b2365f)

# Conflicts:
#	ledger/src/blockstore_db.rs

* Fix conflict

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-06-04 14:38:02 +00:00
mergify[bot]
893df9b277 Rename ValidatorExit and move to sdk (#17728) (#17729)
(cherry picked from commit 3a647c4bea)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-06-04 04:38:49 +00:00
Michael Vines
975c942ea7 Bump version to v1.7.1 2021-06-02 05:21:14 +00:00