Commit Graph

14674 Commits

Author SHA1 Message Date
ed18d6d38d Remove tar-rs audit ignore (#19344) (#19352)
(cherry picked from commit 17978c2ff4)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-08-20 22:21:04 +00:00
ef336a44a1 change untar to use unpack instead of unpack_in (backport #19216) (#19340)
* change untar to use unpack instead of unpack_in (#19216)

* change untar to use unpack instead of unpack_in

* hacky, but maybe passes tests

* chore: bump tar from 0.4.35 to 0.4.37

Bumps [tar](https://github.com/alexcrichton/tar-rs) from 0.4.35 to 0.4.37.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases)
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.35...0.4.37)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* cleanup

* cleanup, add validate_inside_dst

* collapse use

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* delete comment line

* add comments

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 89a31ff473)

# Conflicts:
#	Cargo.lock
#	download-utils/Cargo.toml
#	install/Cargo.toml
#	programs/bpf/Cargo.lock
#	runtime/Cargo.toml
#	sdk/cargo-build-bpf/Cargo.toml

* Fix conflicts

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-20 17:13:35 +00:00
de5f503a76 Add note about ASK keyword to paper wallet doc (#19330) (#19343)
* Add note about ASK keyword to paper wallet doc

Per https://github.com/solana-labs/solana/issues/17325#issuecomment-844434585, this change adds a note to the paper wallet docs mentioning the use of the `ASK` keyword.

Also specifes that the `solana-keygen pubkey prompt://` command returns a derived bip44 base address.

(I'm new to Solana development and this was a point of confusion for me, only resolved by finding this issue)

* Update docs/src/wallet-guide/paper-wallet.md

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
(cherry picked from commit 3be5715f45)

Co-authored-by: Brent Fitzgerald <burnto@gmail.com>
2021-08-20 15:20:09 +00:00
b5b1ed2a55 stake: Remove v2 program references (backport #19308) (#19336)
* stake: Remove v2 program references (#19308)

* stake: Remove v2 program references

* Remove stake v2 feature, along with stake rewrite

(cherry picked from commit 73aa004c59)

# Conflicts:
#	runtime/src/bank.rs

* Fix merge conflict

* Fix uses of old `stake` function

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-08-20 07:32:24 +00:00
caea9c99cd Minimize trust (backport #19279) (#19335)
* docs: Mainnet Beta inflation has been enabled for quite some time

(cherry picked from commit 169ded9a70)

* validator: Trusted validators are now called known validators

(cherry picked from commit e0bc5fa690)

* docs: trust minimize

(cherry picked from commit 40613161a0)

* docs: correct known validator operator

(cherry picked from commit eced50d103)

* docs: Remove decommissioned testnet archetype validator

(cherry picked from commit a587eec20b)

* docs: update devnet start args with new validators

(cherry picked from commit 2a877ae06e)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-08-20 06:24:43 +00:00
1495b94f7a discards epoch-slots epochs ahead of the current root (#19256) (#19328)
Cross cluster gossip contamination is causing cluster-slots hash map to
contain a lot of bogus values and consume too much memory:
https://github.com/solana-labs/solana/issues/17789

If a node is using the same identity key across clusters, then these
erroneous values might not be filtered out by shred-versions check,
because one of the variants of the contact-info will have matching
shred-version:
https://github.com/solana-labs/solana/issues/17789#issuecomment-896304969

The cluster-slots hash-map is bounded and trimmed at the lower end by
the current root. This commit also discards slots epochs ahead of the
root.

(cherry picked from commit 563aec0b4d)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-08-19 23:35:34 +00:00
f55bb78307 updates cluster-slots with root-bank instead of root-slot + bank-forks (backport #19058) (#19324)
* removes unused code from cluster-slots

(cherry picked from commit 2fc112edcf)

# Conflicts:
#	core/src/cluster_slots.rs

* updates cluster-slots with root-bank instead of root-slot + bank-forks

ClusterSlots::update is taking both root-slot and bank-forks only to
later lookup root-bank from bank-forks, which is redundant. Also
potentially by the time bank-forks is locked to obtain root-bank,
root-slot may have already changed and so be inconsistent with the
root-slot passed in as the argument.
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L32-L39
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L122

(cherry picked from commit 40914de811)

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-08-19 20:52:41 +00:00
fa5a71dbf0 ci: pin patched solana crates version for downstream project builds (#19322)
(cherry picked from commit b59b557163)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-08-19 15:58:24 +00:00
80a6479c47 Enable booting testnet validators without tower required (backport #19298) (#19306)
* Enable booting testnet validators without tower required (#19298)

(cherry picked from commit 7d135f185c)

# Conflicts:
#	net/net.sh

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-19 06:37:17 +00:00
a492de1bea Plumb accounts-db-skip-shrink through testnet scripts (backport #19290) (#19303)
* Plumb accounts-db-skip-shrink through testnet scripts (#19290)

(cherry picked from commit 1d375ff2da)

# Conflicts:
#	multinode-demo/bootstrap-validator.sh
#	multinode-demo/validator.sh
#	net/net.sh

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-19 01:14:06 +00:00
43b414b0df RPC: add option to exclude accounts from get_supply (backport #19270) (#19283)
* RPC: add option to exclude accounts from get_supply (#19270)

(cherry picked from commit c053df143f)

# Conflicts:
#	rpc/src/rpc.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-08-18 20:22:06 +00:00
6f31882260 Fix memoverlap check (backport #19232) (#19259)
* Fix memoverlap check (#19232)

(cherry picked from commit 9be988db41)

# Conflicts:
#	programs/bpf_loader/src/syscalls.rs
#	sdk/src/feature_set.rs

* Resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
2021-08-18 18:07:36 +00:00
5042808ecf Expose genesis block time via rpc (backport #19267) (#19277)
* Expose genesis block time via rpc (#19267)

* Expose genesis_creation_time from Bank

* Backfill genesis_creation_time for block/block-time requests of slot 0

(cherry picked from commit c167211611)

# Conflicts:
#	rpc/src/rpc.rs

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-18 00:19:59 +00:00
1c9d0521ca Revert TestValidatorGenesis::start() to v1.7.8 signature; add TestValidatorGenesis::start_with_socket_addr_space() (#19241)
(cherry picked from commit 3e5ba594e0)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-16 08:21:48 +00:00
ddda94e486 Add AsRef<AccountInfo> for AccountInfo (backport #19235) (#19238)
* Add AsRef<AccountInfo> for AccountInfo

(cherry picked from commit 930465e67c)

* add test for AsRef

(cherry picked from commit d10e37a829)

Co-authored-by: Kirill Fomichev <fanatid@ya.ru>
2021-08-15 16:21:04 +00:00
73ed9f56d7 Fixing missing pubsub notification for programSubscribe and logsSubscribe (backport #19092) (#19207)
* Fixing missing pubsub notification for programSubscribe and logsSubscribe (#19092)

#18587: programSubscribe is missing notifications randomly. The issue is because of two reasons

Not all rooted slots get OptimisticallyConfirmed notifications
The OptimisticallyConfirmed notifications can be out of order for slots: slot A and B with A < B can see notification for B first before A.
Summary of Changes

Changed OptimisticallyConfirmedBankTracker to send notifications for parent banks if they have not been notified yet. We use a new variable last_notified_slot to track that.

Tests:
With my validator running against testnet, before the fix, it was failing 75% of time, with the fix, it is passing consistently. Using the program mentioned in #18587.

(cherry picked from commit 1a372a792e)

* Use v1.7 api

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-12 19:17:36 +00:00
ab5d032634 Bump version to v1.7.11 2021-08-12 06:55:18 +00:00
03b930515b Handle 0-lamport account in index generation (#19041) (#19160)
* Handle 0-lamport account in index generation

* rename duplicate to dirty keys

Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit 5a4979f25f)

Co-authored-by: sakridge <sakridge@gmail.com>
v1.7.10
2021-08-11 09:49:19 +00:00
c9f763ea6e Use last_valid_block_height in services and client apps (backport #19163) (#19171)
* Use last_valid_block_height in services and client apps (#19163)

* Add deprecated tag to Bank::get_blockhash_last_valid_slot

* Update SendTransactionService to use last_valid_block_height

* Update solana-tokens to use last_valid_block_height

* Remove dangling file

* Update solana program to use last_valid_block_height

* Update Banks crates to use last_valid_block_height

(cherry picked from commit 5970083b4d)

# Conflicts:
#	cli/src/program.rs

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-11 09:07:53 +00:00
422044f375 Updated transaction fee burn description. (#19161) (#19164)
(cherry picked from commit 83f0915e15)

Co-authored-by: bji <bryan@ischo.com>
2021-08-10 23:17:51 +00:00
7584262f47 add validator option --accounts-db-skip-shrink (backport #19028) (#19128)
* add validator option --accounts-db-skip-shrink (#19028)

* add validator option --accounts-db-skip-shrink

* typo

(cherry picked from commit 3280ae3e9f)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	ledger/src/blockstore_processor.rs
#	replica-node/src/replica_node.rs
#	runtime/src/snapshot_utils.rs

* Fix conflicts

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-10 10:07:27 -06:00
894f121d0e Fix deserialization of RPC errors in HttpSender (#19110) (#19127)
Fixes #15576

(cherry picked from commit e4b66a5913)

Co-authored-by: Brian Anderson <andersrb@gmail.com>
2021-08-09 22:36:16 -07:00
4b133509d9 v1.7: Handle new security advisories (#19131)
* Update hyper 0.14

* Add audit ignores

* Bump assert_cmd and remove audit ignore

* Remove pin-project-lite warning
2021-08-09 18:22:28 -06:00
e8040a828d Clean within shrink_all_slots (#19042) (#19109)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit 592013eaf4)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-08-07 06:16:43 +00:00
78086329be Add more API documentation for Rust RpcClient (#19021) (#19091)
* Add doc links to Transaction API docs

* Add more RpcClient API docs

* Reflow some rpc_client docs

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update sdk/src/transaction.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update RpcClient docs per review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit b67ffab370)

Co-authored-by: Brian Anderson <andersrb@gmail.com>
2021-08-05 23:42:56 +00:00
6deeedd886 bump recommended maps/nofiles (#19059)
(cherry picked from commit d60ccf64e1)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-08-04 20:28:34 +00:00
1a0146f21d stake: Allow stakes with unmatched credits observed to merge (backport #18985) (#19055)
* stake: Allow stakes with unmatched credits observed to merge (#18985)

* stake: Allow stakes with unmatched credits observed to merge

* Address feedback

* Remove branch by doing a ceiling in one calc

(cherry picked from commit 2b33c0c165)

# Conflicts:
#	Cargo.lock
#	programs/stake/Cargo.toml
#	sdk/src/feature_set.rs

* Fix merge conflicts

* Fix clippy lint

* Add back whitespace

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-08-04 17:43:20 +00:00
7d0a9e0381 Add some docs for RpcClient and friends (backport #18748) (#19036)
* Add some docs for RpcClient and friends (#18748)

* Add some docs for RpcSender, HttpSender, MockSender

* Support SimulateTransaction in MockSender

* Add docs for RpcClient constructors

* Add some more RpcClient examples

* rustfmt

* Reflow docs in rpc_client and friends

(cherry picked from commit 5dcfd7ce74)

# Conflicts:
#	client/src/mock_sender.rs

* Fix conflicts

Co-authored-by: Brian Anderson <andersrb@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-03 22:18:48 +00:00
0f7b84197f Exit stable-bpf CI runs before localnet-sanity (#19016) (#19018)
(cherry picked from commit 95c2873b3c)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-08-02 19:28:55 +00:00
bb06502d24 v1.7: Fix broken test validator
Removes duplicated locking logic
2021-08-02 08:08:28 -06:00
b7f1f19d8e Bump version to v1.7.10 2021-07-31 01:19:33 -06:00
0707290bbf Fix serialization of parameters in RpcClient::get_block_production_with_config (backport #18998) (#19000)
* Fix serialization of parameters in RpcClient::get_block_production_with_config (#18998)

Params must be an array or null.

(cherry picked from commit 58f395257b)

# Conflicts:
#	client/src/mock_sender.rs

* Fix conflict

Co-authored-by: Brian Anderson <andersrb@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
v1.7.9
2021-07-31 02:00:32 +00:00
8934a3961f Add get_clock to BanksClient (#18591)
(cherry picked from commit d30a36641e)

Co-authored-by: Kirill Fomichev <fanatid@ya.ru>
2021-07-30 18:29:42 +00:00
b142ef5f8b Improve target CPU feature checks (backport #18940) (#18958)
* Improve check for Apple M1 silicon under Rosetta

(cherry picked from commit 59641623d1)

# Conflicts:
#	core/src/validator.rs

* test-validator: start logging asap

(cherry picked from commit ee65ffb3c8)

* test-validator: move feature check earlier in startup

(cherry picked from commit e641f257ef)

* validator: add avx2 runtime check

(cherry picked from commit c435f7b3e3)

* validator: start logging asap

(cherry picked from commit ed8285c096)

* validator: check target CPU features earlier

(cherry picked from commit 8ed0cd0fff)

* validator: remove disused cuda config argument

(cherry picked from commit 71f6d839f9)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-07-30 06:47:22 +00:00
f2dc9dd96e Fix unstable retransmit-num_nodes (#18970) (#18974)
(cherry picked from commit da480bdb5f)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-07-30 10:52:37 +09:00
20ad3005b5 allows private addresses by default in test-validator (backport #18976) (#18980)
* allows private addresses by default in test-validator (#18976)

(cherry picked from commit 1cef6fd4b4)

# Conflicts:
#	validator/src/bin/solana-test-validator.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-30 00:05:19 +00:00
eacc69efba adds validator flag to allow private ip addresses (backport #18850) (#18975)
* adds validator flag to allow private ip addresses (#18850)

(cherry picked from commit d2d5f36a3c)

# Conflicts:
#	accounts-cluster-bench/Cargo.toml
#	bench-tps/Cargo.toml
#	cli/Cargo.toml
#	core/benches/cluster_info.rs
#	core/src/banking_stage.rs
#	core/src/broadcast_stage.rs
#	core/src/broadcast_stage/broadcast_duplicates_run.rs
#	core/src/broadcast_stage/fail_entry_verification_broadcast_run.rs
#	core/src/broadcast_stage/standard_broadcast_run.rs
#	core/src/cluster_slots_service.rs
#	core/src/repair_service.rs
#	core/src/tvu.rs
#	core/src/validator.rs
#	dos/Cargo.toml
#	gossip/src/cluster_info.rs
#	gossip/src/crds_gossip_pull.rs
#	gossip/src/crds_gossip_push.rs
#	gossip/src/gossip_service.rs
#	local-cluster/Cargo.toml
#	local-cluster/src/cluster_tests.rs
#	local-cluster/tests/local_cluster.rs
#	rpc/Cargo.toml
#	rpc/src/rpc.rs
#	tokens/Cargo.toml
#	validator/Cargo.toml
#	validator/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-29 21:43:24 +00:00
3c200ae45a Tiny typo fix (#18978) (#18979)
(cherry picked from commit 46fdf8a4d2)

Co-authored-by: NearlyAlwaysThere <87867628+NearlyAlwaysThere@users.noreply.github.com>
2021-07-29 14:22:10 -06:00
74e7c7cbd0 Cli: app and wallet command reorg (backport #18955) (#18959)
* Cli: app and wallet command reorg (#18955)

* Clean up wallet commands

* Move global args

(cherry picked from commit 662ccfa558)

# Conflicts:
#	cli/src/cli.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-29 09:46:48 -06:00
6bd6d6212c Fix erroneous default start_slot (#18948) (#18952)
(cherry picked from commit 578f2aa22b)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-07-29 00:11:35 +00:00
4d2e66cf9c Added missing method to index and updated token account methods docs. (#18906) (#18954)
(cherry picked from commit 4d0cd9b283)

Co-authored-by: hugo <murlux@protonmail.com>
2021-07-28 22:35:55 +00:00
65fe00b7ad Auto-generate shell completions for solana-cli (issue #8879 and #14005) (#18931) (#18945)
* Auto-generate shell completions for solana-cli (issue #8879 and #14005)

Implement `completion` SubCommand for solana-cli, which outputs
completion script to stdout and exits the process. The script generation
handled completely by clap.

In order to implement the generation, one minor design change was
necessary regarding the creation of clap `App`.

Previously: One part of App initialization was in the `app` function,
and some other arguments and subcommands were added later directly in
the `main` function.

Now: The whole construction of App was moved to `get_clap_app` function.

P.S. I wasn't sure if constructing App separately had visual importance,
so both constructing parts are still separate in `base_clap_app` and
`final_clap_app` functions. But they sure could be in one single
function.

* Dereplicode match expr, fix clippy warning.

* Move clap App construction into separate module

Also join two parts of the construction into a single function

* Fix tests

* Apply rustfmt lints

(cherry picked from commit 9d0a937a05)

Co-authored-by: theonekeyg <34949189+theonekeyg@users.noreply.github.com>
2021-07-28 17:26:19 +00:00
02c509390a Add cli-output helpers (backport #18933) (#18935)
* Add cli-output helpers (#18933)

* Add OutputFormat helper to reduce copy-pasta

* Add CliSignOnlyData constructor

(cherry picked from commit 467c18e0d1)

# Conflicts:
#	Cargo.lock
#	programs/bpf/Cargo.lock

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-28 15:25:56 +00:00
49b0d1792b filters crds values in parallel when responding to gossip pull-requests (backport #18877) (#18901)
* filters crds values in parallel when responding to gossip pull-requests (#18877)

When responding to gossip pull-requests, filter_crds_values takes a lot of time
while holding onto read-lock:
https://github.com/solana-labs/solana/blob/f51d64868/gossip/src/crds_gossip_pull.rs#L509-L566

This commit will filter-crds-values in parallel using rayon thread-pools.

(cherry picked from commit f1198fc6d5)

# Conflicts:
#	gossip/src/cluster_info.rs
#	gossip/src/crds_gossip_pull.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-07-26 21:30:24 +00:00
9511031490 v1.7: backport new column families from master (#18897)
* backport new column families from master to 1.6 (#18743)

* backporting bank_hash and program_costs column families from master to 1.6 for rocksdb backward compatibility

* missed a line to allow dead code

* include code for purge

* Exclude stubbed ProgramCosts column from compaction (#18840)

Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
2021-07-26 13:38:47 -06:00
04ee86e93c rebase 2021-07-24 19:48:12 -07:00
82f25f982e Put is_x86_feature_detected behind a target_arch check
This change adds a target arch check so that cargo build can be run from an
Apple M1 mac.

(cherry picked from commit 7ee5b02bc5)
2021-07-24 19:48:12 -07:00
1cc32c9cda Disable reed-solomon-erasure/simd-accel feature on aarch64 (Apple M1)
This feature requires clang to support -march=native on M1. Before this change,
cargo build would output a build error when building for aarch64.

(cherry picked from commit 8a9b7f5ef2)

# Conflicts:
#	ledger/Cargo.toml
2021-07-24 19:48:12 -07:00
aedcab846c RpcClient::send<T> now supports client-defined RPC methods via RpcRequest::Custom (#18882)
(cherry picked from commit f264511585)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-07-24 19:15:51 +00:00
548ddff7ed Bump version to v1.7.9 2021-07-24 11:23:44 -06:00