Commit Graph

14772 Commits

Author SHA1 Message Date
e50a26a493 passes through --allow-private-addr to validators in system perf tests (backport #18876) (#20180)
* passes through --allow-private-addr to validators in system perf tests (#18876)

(cherry picked from commit 81026f9ea5)

# Conflicts:
#	multinode-demo/bootstrap-validator.sh
#	multinode-demo/validator.sh
#	net/net.sh

* removes backport merge conflicts

* ignores RUSTSEC-2021-0115

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-09-25 20:08:32 +00:00
302887da67 Add new logos to README files and docs (#20049) (#20093)
* Add new logos to README files and docs

* Add explorer logos

(cherry picked from commit 8dbed193c2)

Co-authored-by: Ryan M. Shea <8948187+rmshea@users.noreply.github.com>
2021-09-25 15:15:29 -04:00
597c504c27 generate deterministic seeds for shreds (backport #17950) (#20172)
* generate deterministic seeds for shreds (#17950)

* generate shred seed from leader pubkey

* clippy

* clippy

* review

* review 2

* fmt

* review

* check

* review

* cleanup

* fmt

(cherry picked from commit a86ced0bac)

# Conflicts:
#	core/benches/cluster_info.rs
#	core/src/broadcast_stage.rs
#	core/src/broadcast_stage/fail_entry_verification_broadcast_run.rs
#	core/src/broadcast_stage/standard_broadcast_run.rs
#	ledger/src/shred.rs
#	sdk/src/feature_set.rs

* removes backport merge conflicts

Co-authored-by: jbiseda <jbiseda@gmail.com>
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-09-25 19:09:49 +00:00
b7af118091 Fix typo (#20218) (#20219)
(cherry picked from commit b95653331c)

Co-authored-by: Ted Robertson <10043369+tredondo@users.noreply.github.com>
2021-09-25 18:54:59 +00:00
9e392687eb chore: bump tiny-bip39 from 0.8.0 to 0.8.1 (backport #20136) (#20207)
* chore: bump tiny-bip39 from 0.8.0 to 0.8.1 (#20136)

* chore: bump tiny-bip39 from 0.8.0 to 0.8.1

Bumps [tiny-bip39](https://github.com/maciejhirsz/tiny-bip39) from 0.8.0 to 0.8.1.
- [Release notes](https://github.com/maciejhirsz/tiny-bip39/releases)
- [Changelog](https://github.com/maciejhirsz/tiny-bip39/blob/master/CHANGELOG.md)
- [Commits](https://github.com/maciejhirsz/tiny-bip39/compare/v0.8.0...v0.8.1)

---
updated-dependencies:
- dependency-name: tiny-bip39
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
(cherry picked from commit 038d77347a)

# Conflicts:
#	Cargo.lock
#	clap-utils/Cargo.toml
#	cli/Cargo.toml
#	keygen/Cargo.toml
#	sdk/Cargo.toml

* resolve conflicts

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Justin Starry <justin@solana.com>
2021-09-25 16:48:42 +00:00
bee923ca6c gossip.md: simplify Markdown (#20201) (#20205)
No need for \-escaping

(cherry picked from commit e9c839a9e7)

Co-authored-by: Ted Robertson <10043369+tredondo@users.noreply.github.com>
2021-09-25 14:01:47 +00:00
879df38059 Terminology: link BPF (#20199) (#20204)
(cherry picked from commit cb4121da19)

Co-authored-by: Ted Robertson <10043369+tredondo@users.noreply.github.com>
2021-09-25 13:50:12 +00:00
3e776267b5 Resolve zeroize_derive audit warning by bumping version (#20182) (#20190)
* Revert "temporarily disable new audit"

This reverts commit 3dfbd95ddc.

* Bump version of zeroize_derive from v1.0.0 to v1.2.0

(cherry picked from commit 0c62a6fe3f)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-25 09:48:07 -04:00
0bfb466184 Fix blatant md formatting at the very top (#20198) (#20203)
(cherry picked from commit c137c50d15)

Co-authored-by: Ted Robertson <10043369+tredondo@users.noreply.github.com>
2021-09-25 13:43:43 +00:00
71bb3bf6aa Improve grammar in terminology/ledger (#20197) (#20202)
(cherry picked from commit 20fbf09072)

Co-authored-by: Ted Robertson <10043369+tredondo@users.noreply.github.com>
2021-09-25 13:30:39 +00:00
99c74c8902 Limit transaction forwarding from banking_stage (#19940) (#20081)
(cherry picked from commit 013e1d9d49)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-09-24 18:04:22 +00:00
085f5f945d uses tikv_jemallocator::Jemalloc as the global allocator (backport #20149) (#20166)
* uses tikv_jemallocator::Jemalloc as the global allocator (#20149)

https://github.com/solana-labs/solana/pull/16346
switched default allocator from jemalloc to system allocator, but that
has shown regressions in form of higher ram usage causing nodes go OOM:
https://discord.com/channels/428295358100013066/439194979856809985/890413193858539580

This commit sets jemalloc as the default allocator.

(cherry picked from commit 2cf081d863)

# Conflicts:
#	Cargo.lock

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-09-24 17:37:57 +00:00
e757e51ddc Port sigverify to identify and mark simple vote transaction to v1.7 (#20147)
* sigverify to identify and mark simple vote transaction (#20021)

* check vote tx at get_packet_offsets to cover both cpu and gpu paths

* add pubkey_len to PacketOffsets to reduce the redundant bytes counting

* allow vote to have 1 or 2 sigs (#20082)
2021-09-24 11:35:50 -05:00
458ccecb5d Bump bpf-tools version to 1.12
(cherry picked from commit e98a7504f2)
2021-09-24 18:05:13 +02:00
7f21a55a32 Fix public key md links (#20162) (#20167)
(cherry picked from commit 9653f6b28d)

Co-authored-by: Israel Ferrer Camacho <rallat@gmail.com>
2021-09-24 15:28:46 +00:00
b112e4a8aa windows: Make solana-test-validator work (backport #20099) (#20123)
* windows: Make solana-test-validator work (#20099)

* windows: Make solana-test-validator work

The important changes to get this going on Windows:

* ledger lock needs to be done on a file instead of the directory
* IPC service needs to use the Windows pipe naming scheme
* always disable the JIT
* file logging not possible yet because we can't redirect stderr,
but this will change once env_logger fixes the pipe output target!

* Integrate review feedback

(cherry picked from commit 567f30aa1a)

# Conflicts:
#	validator/src/bin/solana-test-validator.rs
#	validator/src/lib.rs
#	validator/src/main.rs

* Fix merge conflicts

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-09-24 12:59:12 +00:00
63b24d9577 Update TransactionError link to docs.rs (#20145) (#20146)
(cherry picked from commit a70fd8e606)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-09-23 21:15:36 +00:00
49a6acffca Add metric for duplicate retransmit (#20129) 2021-09-22 22:51:44 -07:00
7b4638aa0b runtime: remove inactive delegation from stakes cache 2021-09-22 20:43:34 -06:00
f51ee23837 Fix typo in docs/cli/deploy-a-program (#20097) (#20098)
(cherry picked from commit 3d0db28d12)

Co-authored-by: Christoph Michel <MrToph@users.noreply.github.com>
2021-09-22 01:46:58 +00:00
55f27d5c26 Fix memo handling and "blocks" key formatting (#20044) (#20080)
* Fix memo handling and "blocks" key formatting

* Skip memo equality check

* Define slot_to_tx_by_addr_key

Co-authored-by: Justin Starry <justin@solana.com>
(cherry picked from commit 795dde109c)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-09-21 22:22:55 +00:00
f33c651114 Minor ProgramTest improvements (backport #20051) (#20054)
* Add stable_log output when a program is loaded as native code instead of BPF

(cherry picked from commit 34f5020457)

* Add ProgramTest::add_builtin_program()

This permits the unit testing of builtin programs in the ProgramTest environment

(cherry picked from commit 830ca369f1)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-09-21 17:04:16 +00:00
a71ebcc9f3 demote ./run.sh (backport #19679) (#20052)
* move `./run.sh` into `./scripts`

(cherry picked from commit 92e343da26)

* add some guidance in place of `./run.sh`

(cherry picked from commit 33de7b856f)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-09-21 15:59:48 +00:00
cd575945b6 Make bigtable delete-slots actually usable (#20037) (#20042)
(cherry picked from commit 22f45dca80)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-09-21 13:40:21 +00:00
6b24dd1c6a client: Add retry logic on Pubsub 429 error during connect (backport #19990) (#20002)
* client: Add retry logic on Pubsub 429s (#19990)

(cherry picked from commit e9b066d497)

* Use exponential backoff for older version of tungstenite

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-09-20 12:13:06 +00:00
03da3eaa81 Optimize RPC pubsub for multiple clients with the same subscription (backport #18943) (#19987)
* Optimize RPC pubsub for multiple clients with the same subscription (#18943)

* reimplement rpc pubsub with a broadcast queue

* update tests for new pubsub implementation

* fix: fix review suggestions

* chore(rpc): add additional pubsub metrics

* integrate max subscriptions check into SubscriptionTracker to reduce locking

* separate subscription control from tracker

* limit memory usage of items in pubsub broadcast queue, improve error handling

* add more pubsub metrics

* add final count metrics to pubsub

* add metric for total number of subscriptions

* fix small review suggestions

* remove by_params from SubscriptionTracker and add node_progress_watchers map instead

* add subscription tracker tests

* add metrics for number of pubsub notifications as a counter

* ignore clippy lint in TokenCounter

* fix underflow in token counter

* reduce queue capacity in pubsub tests

* fix(rpc): fix test timeouts

* fix race in account subscription test

* Add RpcSubscriptions::new_for_tests

Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 65227f44dc)

# Conflicts:
#	Cargo.lock
#	core/Cargo.toml
#	core/src/replay_stage.rs
#	core/src/validator.rs
#	replica-node/src/replica_node.rs
#	rpc/Cargo.toml

* Fix conflicts (and standardize naming to make future subscription backports easier

Co-authored-by: Pavel Strakhov <ri@idzaaus.org>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-20 06:00:08 +00:00
07b71329a7 Update feature switch for reduced required deploy balance (backport #19999) (#20012)
* Update feature switch for reduced required deploy balance (#19999)

(cherry picked from commit ea34eb8a4b)

# Conflicts:
#	programs/bpf_loader/src/lib.rs

* fix conflict

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-19 23:17:59 +00:00
53b387f113 Report consumed_buffered_packets_count stat to metrics (backport #19900) (#19915)
* Report consumed_buffered_packets_count stat to metrics (#19900)

(cherry picked from commit 34c1a9ac85)

# Conflicts:
#	core/src/banking_stage.rs

* fix conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-18 20:57:28 +00:00
d0143dad8f Reduce payer balance needed to deploy programs (backport #19645) (#19998)
* Reduce payer balance needed to deploy programs (#19645)

* Reduce payer balance needed to deploy programs

* Fix test and unbalanced ix error

* fix test

* fix up tests

(cherry picked from commit efd024510a)

# Conflicts:
#	programs/bpf_loader/src/lib.rs

* fix conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-18 04:46:42 +00:00
3e870a40f2 Fix broken links in terminology.md (#19978) (#19992)
(cherry picked from commit 073c5359b0)

Co-authored-by: visortelle <visortelle@gmail.com>
2021-09-18 00:16:32 +00:00
707302d9f2 Add delete subcommand to ledger-tool bigtable (#19931) (#19963)
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback

(cherry picked from commit c71fab6cb3)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-17 16:26:57 -05:00
b8ab6a46a8 Fix typos in terminology.md (#19977) (#19979)
(cherry picked from commit 1ec22572f2)

Co-authored-by: visortelle <visortelle@gmail.com>
2021-09-17 16:48:55 +00:00
9ad801a52c stake: Add BorshDeserialize trait to structs (#19958) (#19976)
(cherry picked from commit 9eb98adf97)

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-09-17 11:33:18 +00:00
c85816c44e rpc: performance fix for getProgramAccounts (#19941) (#19950)
* rpc: performance fix for getProgramAccounts

The accounts were gradually pushed into a vector, which produced
significant slowdowns for very large responses.

* rpc: rewrite loops using iterators

Co-authored-by: Christian Kamm <ckamm@delightful-solutions.de>
(cherry picked from commit f1bbf1d8b0)

Co-authored-by: Christian Kamm <mail@ckamm.de>
2021-09-16 23:26:21 +00:00
70d556782b Bump 1.7 version (#19943) 2021-09-16 13:16:09 -06:00
ca83167cfc Only allow votes when root distance gets too high (#19933) v1.7.12 2021-09-16 16:48:51 +02:00
54ad080bf2 Fix native invoke writable privileges (backport #19750) (#19860)
* Fix native invoke writable privileges (#19750)

* Fix native invoke writable privileges

* build downstream spl bpf programs for tests

(cherry picked from commit 00d7981f64)

# Conflicts:
#	program-runtime/src/instruction_processor.rs
#	runtime/src/message_processor.rs
#	sdk/src/feature_set.rs

* resolve conflictds

Co-authored-by: Jack May <jack@solana.com>
2021-09-16 01:42:26 +00:00
80e239550b Add banking metrics for buffered and dropped packets (#19902) (#19927)
(cherry picked from commit ca3f147670)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-16 00:59:27 +00:00
992b313941 Cli: check current authorities before attempting to change them (backport #19853) (#19923)
* Cli: check current authorities before attempting to change them (#19853)

* Stake-authorize: check account current authority

* Stake-set-lockup: check account current custodian

* Make helper fn pub(crate)

* Vote-authorize: check account current authority

(cherry picked from commit 15144fc923)

# Conflicts:
#	cli/src/vote.rs

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-15 22:53:44 +00:00
97bd521725 Add logging after taking a bank snapshot (backport #19891) (#19920)
* Add logging after taking a bank snapshot (#19891)

(cherry picked from commit 8e3c420414)

* Fix conflict

Co-authored-by: Brooks Prumo <brooks@solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-15 14:29:33 -06:00
3ae2917603 build downstream spl bpf programs for tests (backport #19851) (#19887)
Co-authored-by: Justin Starry <justin@solana.com>
2021-09-15 12:20:00 -06:00
e51c2d1a84 Add an info log to indicate the node has reached supermajority and print the active stake percentage (#19893)
(cherry picked from commit 4ff50519ff)
2021-09-15 01:59:01 -07:00
a24b0dc81c Use f64 for stake math in get_stake_percent_in_gossip (#19895)
(cherry picked from commit c91519961c)
2021-09-15 01:58:52 -07:00
6e856cd468 Silence vercel github comments (#19827) (#19833)
(cherry picked from commit 8ac12c29ed)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-13 17:30:05 +00:00
0dde54b95b Add --destake-vote-account <VOTE_ADDRESS>... argument to create-snapshot command (#19749) (#19796)
(cherry picked from commit 0f76077969)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-09-11 23:34:34 +00:00
1fa863e4b2 docs: be less specific about zen3 skus (#19791)
(cherry picked from commit 999ba9e026)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-09-11 05:30:30 +00:00
73e97aab5b Recover from interrupted bpf sdk installs (#19651) 2021-09-10 20:02:52 -07:00
8f082a239a Revert "fixup! test-validator: start logging asap (#19655)" (#19740) (#19745)
This reverts commit 64a2d7081b.

(cherry picked from commit 46fc92f392)

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-10 21:57:57 +00:00
7e5026bde2 removes backport merge conflicts 2021-09-09 23:33:29 -07:00
7ac0ea0885 filters for recent contact-infos when checking for live stake (#19204)
Contact-infos are saved to disk:
https://github.com/solana-labs/solana/blob/9dfeee299/gossip/src/cluster_info.rs#L1678-L1683

and restored on validator start-up:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L450

Staked nodes entries will not expire until an epoch after. So when the
validator checks for online stake it is erroneously picking up
contact-infos restored from disk, which breaks the entire
wait-for-supermajority logic:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L1515-L1561

This commit adds an extra check for the age of contact-info entries and
filters out old ones.

(cherry picked from commit 7a789e0763)

# Conflicts:
#	core/src/validator.rs
2021-09-09 23:33:29 -07:00