Commit Graph

14748 Commits

Author SHA1 Message Date
6b24dd1c6a client: Add retry logic on Pubsub 429 error during connect (backport #19990) (#20002)
* client: Add retry logic on Pubsub 429s (#19990)

(cherry picked from commit e9b066d497)

* Use exponential backoff for older version of tungstenite

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-09-20 12:13:06 +00:00
03da3eaa81 Optimize RPC pubsub for multiple clients with the same subscription (backport #18943) (#19987)
* Optimize RPC pubsub for multiple clients with the same subscription (#18943)

* reimplement rpc pubsub with a broadcast queue

* update tests for new pubsub implementation

* fix: fix review suggestions

* chore(rpc): add additional pubsub metrics

* integrate max subscriptions check into SubscriptionTracker to reduce locking

* separate subscription control from tracker

* limit memory usage of items in pubsub broadcast queue, improve error handling

* add more pubsub metrics

* add final count metrics to pubsub

* add metric for total number of subscriptions

* fix small review suggestions

* remove by_params from SubscriptionTracker and add node_progress_watchers map instead

* add subscription tracker tests

* add metrics for number of pubsub notifications as a counter

* ignore clippy lint in TokenCounter

* fix underflow in token counter

* reduce queue capacity in pubsub tests

* fix(rpc): fix test timeouts

* fix race in account subscription test

* Add RpcSubscriptions::new_for_tests

Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 65227f44dc)

# Conflicts:
#	Cargo.lock
#	core/Cargo.toml
#	core/src/replay_stage.rs
#	core/src/validator.rs
#	replica-node/src/replica_node.rs
#	rpc/Cargo.toml

* Fix conflicts (and standardize naming to make future subscription backports easier

Co-authored-by: Pavel Strakhov <ri@idzaaus.org>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-20 06:00:08 +00:00
07b71329a7 Update feature switch for reduced required deploy balance (backport #19999) (#20012)
* Update feature switch for reduced required deploy balance (#19999)

(cherry picked from commit ea34eb8a4b)

# Conflicts:
#	programs/bpf_loader/src/lib.rs

* fix conflict

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-19 23:17:59 +00:00
53b387f113 Report consumed_buffered_packets_count stat to metrics (backport #19900) (#19915)
* Report consumed_buffered_packets_count stat to metrics (#19900)

(cherry picked from commit 34c1a9ac85)

# Conflicts:
#	core/src/banking_stage.rs

* fix conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-18 20:57:28 +00:00
d0143dad8f Reduce payer balance needed to deploy programs (backport #19645) (#19998)
* Reduce payer balance needed to deploy programs (#19645)

* Reduce payer balance needed to deploy programs

* Fix test and unbalanced ix error

* fix test

* fix up tests

(cherry picked from commit efd024510a)

# Conflicts:
#	programs/bpf_loader/src/lib.rs

* fix conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-18 04:46:42 +00:00
3e870a40f2 Fix broken links in terminology.md (#19978) (#19992)
(cherry picked from commit 073c5359b0)

Co-authored-by: visortelle <visortelle@gmail.com>
2021-09-18 00:16:32 +00:00
707302d9f2 Add delete subcommand to ledger-tool bigtable (#19931) (#19963)
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback

(cherry picked from commit c71fab6cb3)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-17 16:26:57 -05:00
b8ab6a46a8 Fix typos in terminology.md (#19977) (#19979)
(cherry picked from commit 1ec22572f2)

Co-authored-by: visortelle <visortelle@gmail.com>
2021-09-17 16:48:55 +00:00
9ad801a52c stake: Add BorshDeserialize trait to structs (#19958) (#19976)
(cherry picked from commit 9eb98adf97)

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-09-17 11:33:18 +00:00
c85816c44e rpc: performance fix for getProgramAccounts (#19941) (#19950)
* rpc: performance fix for getProgramAccounts

The accounts were gradually pushed into a vector, which produced
significant slowdowns for very large responses.

* rpc: rewrite loops using iterators

Co-authored-by: Christian Kamm <ckamm@delightful-solutions.de>
(cherry picked from commit f1bbf1d8b0)

Co-authored-by: Christian Kamm <mail@ckamm.de>
2021-09-16 23:26:21 +00:00
70d556782b Bump 1.7 version (#19943) 2021-09-16 13:16:09 -06:00
ca83167cfc Only allow votes when root distance gets too high (#19933) v1.7.12 2021-09-16 16:48:51 +02:00
54ad080bf2 Fix native invoke writable privileges (backport #19750) (#19860)
* Fix native invoke writable privileges (#19750)

* Fix native invoke writable privileges

* build downstream spl bpf programs for tests

(cherry picked from commit 00d7981f64)

# Conflicts:
#	program-runtime/src/instruction_processor.rs
#	runtime/src/message_processor.rs
#	sdk/src/feature_set.rs

* resolve conflictds

Co-authored-by: Jack May <jack@solana.com>
2021-09-16 01:42:26 +00:00
80e239550b Add banking metrics for buffered and dropped packets (#19902) (#19927)
(cherry picked from commit ca3f147670)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-16 00:59:27 +00:00
992b313941 Cli: check current authorities before attempting to change them (backport #19853) (#19923)
* Cli: check current authorities before attempting to change them (#19853)

* Stake-authorize: check account current authority

* Stake-set-lockup: check account current custodian

* Make helper fn pub(crate)

* Vote-authorize: check account current authority

(cherry picked from commit 15144fc923)

# Conflicts:
#	cli/src/vote.rs

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-15 22:53:44 +00:00
97bd521725 Add logging after taking a bank snapshot (backport #19891) (#19920)
* Add logging after taking a bank snapshot (#19891)

(cherry picked from commit 8e3c420414)

* Fix conflict

Co-authored-by: Brooks Prumo <brooks@solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-15 14:29:33 -06:00
3ae2917603 build downstream spl bpf programs for tests (backport #19851) (#19887)
Co-authored-by: Justin Starry <justin@solana.com>
2021-09-15 12:20:00 -06:00
e51c2d1a84 Add an info log to indicate the node has reached supermajority and print the active stake percentage (#19893)
(cherry picked from commit 4ff50519ff)
2021-09-15 01:59:01 -07:00
a24b0dc81c Use f64 for stake math in get_stake_percent_in_gossip (#19895)
(cherry picked from commit c91519961c)
2021-09-15 01:58:52 -07:00
6e856cd468 Silence vercel github comments (#19827) (#19833)
(cherry picked from commit 8ac12c29ed)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-13 17:30:05 +00:00
0dde54b95b Add --destake-vote-account <VOTE_ADDRESS>... argument to create-snapshot command (#19749) (#19796)
(cherry picked from commit 0f76077969)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-09-11 23:34:34 +00:00
1fa863e4b2 docs: be less specific about zen3 skus (#19791)
(cherry picked from commit 999ba9e026)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-09-11 05:30:30 +00:00
73e97aab5b Recover from interrupted bpf sdk installs (#19651) 2021-09-10 20:02:52 -07:00
8f082a239a Revert "fixup! test-validator: start logging asap (#19655)" (#19740) (#19745)
This reverts commit 64a2d7081b.

(cherry picked from commit 46fc92f392)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-10 21:57:57 +00:00
7e5026bde2 removes backport merge conflicts 2021-09-09 23:33:29 -07:00
7ac0ea0885 filters for recent contact-infos when checking for live stake (#19204)
Contact-infos are saved to disk:
https://github.com/solana-labs/solana/blob/9dfeee299/gossip/src/cluster_info.rs#L1678-L1683

and restored on validator start-up:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L450

Staked nodes entries will not expire until an epoch after. So when the
validator checks for online stake it is erroneously picking up
contact-infos restored from disk, which breaks the entire
wait-for-supermajority logic:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L1515-L1561

This commit adds an extra check for the age of contact-info entries and
filters out old ones.

(cherry picked from commit 7a789e0763)

# Conflicts:
#	core/src/validator.rs
2021-09-09 23:33:29 -07:00
dc06d3dee5 Reduce wait for supermajority threshold back to 80%
(cherry picked from commit 4386e09710)
2021-09-09 21:45:00 -07:00
0b1aadf446 Add RpcClient::get_transport_stats()
(cherry picked from commit 21f4606212)
2021-09-09 21:17:56 -07:00
b9a0156a93 Return error if Transaction contains writable executable or ProgramData accounts (backport #19629) (#19730)
* Return error if Transaction contains writable executable or ProgramData accounts (#19629)

* Return error if Transaction locks an executable as writable

* Return error if a ProgramData account is writable but the upgradable loader isn't present

* Remove unreachable clause

* Fixup bpf tests

* Review comments

* Add new TransactionError

* Disallow writes to any upgradeable-loader account when loader not present; remove is_upgradeable_loader_present exception for all other executables

(cherry picked from commit 38bbb77989)

# Conflicts:
#	programs/bpf/tests/programs.rs
#	runtime/src/accounts.rs
#	runtime/src/bank.rs
#	sdk/src/transaction.rs
#	storage-proto/proto/transaction_by_addr.proto
#	storage-proto/src/convert.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-09 14:02:07 +00:00
f786d1d0f3 cargo-build-bpf: allow sdk path to be set by environment variables (#19509)
In many Linux distros such as NixOS, the directory in which packages are
installed is assumed to be read-only. To work around this, it is
expected that the filepaths which packaged CLI tools take in are able to
be freely configured through either 1) command-line flags, 2)
environment variables, or 3) a configuration file.

In this commit, environment variables 'BPF_SDK_PATH' and 'BPF_OUT_PATH',
which map respectively to command-line flags '--bpf-sdk-path' and
'--bpf-out-dir', are now handled in cargo-build-bpf.

Additionally, given that arbitrary filepaths may now be set in which the
BPF SDK is located, the requirement in which
'$BPF_SDK_PATH/dependencies/bpf-tools' must strictly be a symbolic link
to the directory '$HOME/.cache/solana/${bpf-tools.version}/bpf-tools has
been relaxed.

Ideally, the directory in which bpf-tools is expected to be downloaded
to and stored should be configurable. Though, this commit serves as a
temporary fix which enables NixOS users to now start being able to build
applications with the Solana SDK.

Co-authored-by: Kenta Iwasaki <kenta@lithdew.net>
2021-09-09 12:29:33 +00:00
b360c90d21 Add cargo-build-bpf integration tests (backport #18255) (#19720)
* Add cargo-build-bpf integration tests

(cherry picked from commit cf25729eaa)

* reduce version number

Co-authored-by: Dmitri Makarov <dmakarov@alumni.stanford.edu>
Co-authored-by: Justin Starry <justin@solana.com>
2021-09-08 23:31:13 +00:00
da801b753b Speed up program deploys by writing larger chunks (backport #19654) (#19680)
* Speed up program deploys by writing larger chunks (#19654)

(cherry picked from commit 24fd47a32d)

# Conflicts:
#	cli/src/program.rs

* resolve conflict

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-08 21:45:30 +00:00
8f168a610e Updating Phantom description, removing invite code (#19004) (#19725)
* Updating Phantom description, removing invite code

* Updated copy

* Update web-wallets.md

(cherry picked from commit 9d2f0e237b)

Co-authored-by: Chris Kalani <chriskalani@gmail.com>
2021-09-08 21:23:37 +00:00
d7e6ab58c4 TpuClient now uses the processed slot to seed its leader schedule cache 2021-09-08 19:15:35 +00:00
a0e6a7c73b fixup! test-validator: start logging asap (backport #19655) (#19711)
* fixup! test-validator: start logging asap (#19655)

(cherry picked from commit 64a2d7081b)

# Conflicts:
#	validator/src/bin/solana-test-validator.rs

* Resolves conflicts

Co-authored-by: steviez <steven@solana.com>
2021-09-08 10:58:07 +00:00
ed4e7c0c87 Check seed length before trying to cal program address (backport #19699) (#19708)
* Check seed length before trying to cal program address (#19699)

(cherry picked from commit 778b2adbea)

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
2021-09-08 08:26:07 +00:00
584f8deae3 cli: show upgradeable program accounts (backport #19431) (#19435)
* cli: show upgradeable program accounts (#19431)

(cherry picked from commit 57bbbb83a4)

* resolve conflicts

* nudge

Co-authored-by: Jack May <jack@solana.com>
2021-09-08 07:48:31 +00:00
e9c3f11d24 Remove native id check in pda creation (backport #19595) (#19689)
* Remove native id check in pda creation (#19595)

(cherry picked from commit 529fefc7cc)

# Conflicts:
#	programs/bpf/rust/invoke/src/lib.rs
#	programs/bpf_loader/src/syscalls.rs
#	sdk/src/feature_set.rs

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
2021-09-07 23:26:08 -07:00
57e87a09c0 Grammar nits (#19703) (#19706)
* Grammar nits

* add Phantom to staking docs

(cherry picked from commit 590e113f16)

Co-authored-by: Ryan M. Shea <8948187+rmshea@users.noreply.github.com>
2021-09-08 04:53:24 +00:00
a2b435916d Fix handling of stale links to bpf-tools in cargo-build-bpf (#17772) (#19667)
(cherry picked from commit 910ac94a8c)

Co-authored-by: Dmitri Makarov <dmakarov@users.noreply.github.com>
2021-09-06 22:23:40 +00:00
ad66189463 Update web-wallets.md (#19648) (#19650)
(cherry picked from commit f0be3e4ea9)

Co-authored-by: bitkeepwallet <85662627+bitkeepwallet@users.noreply.github.com>
2021-09-05 16:21:32 +00:00
9a280426da docs: fix typo (#19646) (#19647)
(cherry picked from commit f68ca3895b)

Co-authored-by: Zahin Mohammad <zahin.dev@gmail.com>
2021-09-04 18:48:36 +00:00
864006a78a docs: fix typo (#19641) (#19644)
(cherry picked from commit 497b6c152f)

Co-authored-by: Zahin Mohammad <zahin.dev@gmail.com>
2021-09-04 17:21:20 +00:00
e37e46cfc2 update Telegram link at docs (#19638) (#19639)
(cherry picked from commit 8352bc48db)

Co-authored-by: Pierre/Long <pierreneter@gmail.com>
2021-09-04 08:30:27 +00:00
376b21c6e7 Tiny dedupe 2021-09-04 01:43:23 -06:00
fcda5d4a7d Demote write locks on transaction program ids (backport #19593) (#19633)
* Demote write locks on transaction program ids (#19593)

* Add feature

* Demote write lock on program ids

* Fixup bpf tests

* Update MappedMessage::is_writable

* Comma nit

* Review comments

(cherry picked from commit decec3cd8b)

# Conflicts:
#	core/src/banking_stage.rs
#	core/src/cost_model.rs
#	core/src/cost_tracker.rs
#	ledger-tool/src/main.rs
#	program-runtime/src/instruction_processor.rs
#	programs/bpf/tests/programs.rs
#	programs/bpf_loader/src/syscalls.rs
#	rpc/src/transaction_status_service.rs
#	runtime/src/accounts.rs
#	runtime/src/bank.rs
#	runtime/src/message_processor.rs
#	sdk/benches/serialize_instructions.rs
#	sdk/program/src/message/mapped.rs
#	sdk/program/src/message/sanitized.rs
#	sdk/src/transaction/sanitized.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-04 06:46:09 +00:00
2e4a2c15be s/authorized_withdrawer_pubkey/authorized_withdrawer/
(cherry picked from commit f1ba759940)
2021-09-03 22:07:58 -07:00
a8ec380c56 Rework authorized_withdrawer usage
(cherry picked from commit fa61b90796)
2021-09-03 22:07:58 -07:00
afb87a386a Removed the --authorized-withdrawer argument from create-vote-account
The parameter is now a required third argument.  This is because authorized
withdrawer should never be the same as vote account keypair or validator
identity keypair for security reasons.

Added a --allow-unsafe-authorized-withdrawer to override this restriction if
necessary.

(cherry picked from commit e288459cf2)
2021-09-03 22:07:58 -07:00
2d060fd2d9 Add RpcClient::get_multiple_accounts_with_config()
(cherry picked from commit ae43ca3bfb)
2021-09-03 20:24:50 -07:00