Commit Graph

11360 Commits

Author SHA1 Message Date
51a48ae507 Bound ip-echo-server reply read (bp #13543) (#13545)
* ip-echo-server: Name the header length magic number

(cherry picked from commit aab5f24518)

* ip-echo-server: Add helper to compute reply length

(cherry picked from commit 7481ba5618)

* ip-echo-server: Limit socket read to expected reply length

(cherry picked from commit d2cfeb31b9)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-12 07:03:28 +00:00
b34895db37 Fix slow/stuck unstaking due to toggling in epoch (bp #13501) (#13534)
* Fix slow/stuck unstaking due to toggling in epoch (#13501)

* Fix slow/stuck unstaking due to toggling in epoch

* nits

* nits

* Add stake_program_v2 feature status check to cli

Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 89b474e192)

* Fix conflict

* PartialEq<Vec<T>> is not impl for &[T] on rust v1.45.1

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-11-12 01:20:52 +00:00
3e6398caec Validator: Periodically log what we're waiting for during --wait-for-supermajority (#13530)
(cherry picked from commit 38f15e41b5)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-11 21:18:00 +00:00
b9555116ec Increment version to v1.3.21 (#13528) 2020-11-11 19:30:46 +00:00
81a4769de7 Fix parsing CreateAccountWithSeed instructions (#13513) (#13516)
* Reduce required num_system_accounts and handle 2-account instructions properly

* Update CreateAccountWithSeed account docs to be correct

* Add CreateAccountWithSeed test

(cherry picked from commit 91f4e99b4c)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
v1.3.20
2020-11-11 01:01:38 +00:00
9ee741e021 watchtower: Fix all clear duration message (#13509)
(cherry picked from commit 2a96e722b4)

Co-authored-by: Justin Starry <justin@solana.com>
2020-11-10 19:01:13 +00:00
74c228a9d4 Bump SPL Token version fetched for localnet (bp #13490) (#13505)
* Bump token version fetched for localnet (#13490)

(cherry picked from commit 3282334741)

# Conflicts:
#	fetch-spl.sh

* Fix conflict, ie bump loader to be consistent with devnet/mainnet-beta

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-11-10 18:03:53 +00:00
320140fe8e Fix signature access (#13491) (#13502)
(cherry picked from commit 70c4626efe)

Co-authored-by: sakridge <sakridge@gmail.com>
2020-11-10 18:03:10 +00:00
dfcd0c41ec Make testnet section less ambiguous (#13504) (#13507)
(cherry picked from commit 599dae8f09)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-11-10 17:12:52 +00:00
d217ea2afc Clean up Delegation::stake_activating_and_deactivating (#13471) (#13472)
(cherry picked from commit 5306eb93cc)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-09 09:52:34 +00:00
553209f64c Clean up Delegation::stake_and_activating (#13460) (#13468)
(cherry picked from commit 737d3e376d)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-08 09:31:14 +00:00
4fcf19e414 Output more inflation calc details in ledger-tool (bp #13345) (#13466)
* Output more inflation calc details in ledger-tool (#13345)

* Output more inflation calc details in ledger-tool

* Fix broken ci...

* Rename confusing variables

* Fix panic by wrapping PointValue with Opiton...

* Minor modifications

* Remove explict needless flush; Drop already does

* Yet another csv field adjustments

* Add data_size and rename epochs to earned_epochs

* Introduce null_tracer

* Unwrap Option in new_from_parent_with_tracer

* Don't shorten identifiers

* Allow irrefutable_let_patterns temporalily

* More null_tracer

* More field adjustments

(cherry picked from commit a81e7e7749)

# Conflicts:
#	runtime/src/bank.rs

* Fix conflict

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-08 09:03:09 +00:00
e57e632870 Add builtin mem tests (bp #13429) (#13436)
* Add builtin mem tests (#13429)


(cherry picked from commit 84b139cc94)

* resolve crate version

* port to v1.3 conventions

* nudge

Co-authored-by: Jack May <jack@solana.com>
2020-11-07 01:23:14 +00:00
20678cf5ef Fix stake split rent-exempt adjustment (#13357) (#13452)
* Add failing tests

* Fix stake split

* Calculate split rent-exempt-reserve and use

* Add comment in rent.rs

* Add tests for edge cases when splitting to larger accounts, and reject overflow splits

* Reframe InsufficientFunds checks in terms of lamports var

* Test hardening review comments

(cherry picked from commit 4c5f345798)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-11-06 21:55:08 +00:00
0a05bbca2f Fix stake redelegate (bp #13358) (#13449)
* stake: Add redelegation failing test

(cherry picked from commit 491ad59d2e)

* stake: Consider withdraws we redelegating

(cherry picked from commit fe1e08b9ad)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-06 20:27:06 +00:00
ab7dff16a2 Feature-gate stake program (bp #13394) (#13438)
* Feature-gate stake program (#13394)

* Add legacy stake-program handling

* Strip out duplicative legacy code

* Add feature for stake-program-fix

* Feature-deploy new stake program

* Expand comment

(cherry picked from commit 1b1d9f6b0c)

# Conflicts:
#	runtime/src/builtins.rs
#	runtime/src/feature_set.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-11-06 14:00:37 +00:00
5e16e80993 CI: Check monorepo for consistent crate versions (bp #13431) (#13432)
* increment-cargo-version.sh: Add check subcommand

(cherry picked from commit 5d4015358a)

* CI: Check monorepo for consistent crate versions

(cherry picked from commit 7a4e293b3b)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-06 02:27:51 +00:00
a388b5a356 docs: Clarify the commitment levels based on questions (#13387) (#13425)
* Clarify the commitment levels based on questions

Many people have asked about what commitment levels mean, and which to
choose.  This update includes some of the language at
`sdk/src/commitment_config.rs` and a recommendation for different use
cases.

Additionally, the preflight commitment documentation was out of date,
specifying that "max" was always used, and this is no longer the case.

* Update docs/src/developing/clients/jsonrpc-api.md

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update docs/src/developing/clients/jsonrpc-api.md

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update docs/src/developing/clients/jsonrpc-api.md

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Fix typo

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit ede891a6c6)

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2020-11-05 19:38:59 +00:00
ef776c0a0e Allow feature builtins to overwrite existing builtins (bp #13403) (#13419)
* Allow feature builtins to overwrite existing builtins (#13403)

* Allow feature builtins to overwrite existing builtins

* Add feature_builtin ActivationType

* Correctly retain idempotent for replacing case

* Fix test

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
(cherry picked from commit bc62313c66)

# Conflicts:
#	ledger/src/builtins.rs
#	runtime/src/bank.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-11-05 17:53:03 +00:00
a899d75d2d Fix duplicate records of inner instructions (#13380) (#13412)
* Fix duplicate records of inner instructions

* fix tests

* fix clippy

* Remove bad_inner_instructions

(cherry picked from commit c24fbb6f8b)

Co-authored-by: Justin Starry <justin@solana.com>
2020-11-05 08:44:16 +00:00
27733bb4d7 Bump low end validator RAM requirement (#13406) 2020-11-05 15:24:26 +08:00
0b9c87b6ec Docs: Clarify validator disk requirements 2020-11-05 15:24:26 +08:00
df1e62f23f Update docs to latest processors (#11613) 2020-11-05 15:24:26 +08:00
f697a86d1e Comment Stakes::clone_with_epoch (#13388) (#13389)
(cherry picked from commit b0d1ae1d8b)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-04 13:49:48 +00:00
64c76c2f4b Revert "check sysvar id for AccountInfo (#13175) (#13351)"
This reverts commit 290d514051.
2020-11-02 12:14:33 -08:00
290d514051 check sysvar id for AccountInfo (#13175) (#13351)
(cherry picked from commit 322c667655)

# Conflicts:
#	sdk/program/src/sysvar/mod.rs

Co-authored-by: Jack May <jack@solana.com>
2020-11-02 18:18:07 +00:00
958c43f337 Small code cleanup and typo fixes (#13325) (#13340)
* Small code cleanup and typo fixes

* Clean up calculate_points_and_credits

(cherry picked from commit 0e4509c497)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-02 02:38:43 +00:00
4513128c75 check sysvar id for AccountInfo (#13175) 2020-11-01 06:58:02 +00:00
d01968ed21 cargo update -p futures-task / cargo update -p futures-util 2020-10-31 19:00:22 +00:00
f9ac24d1f2 Switch to dirs-next 2020-10-31 19:00:22 +00:00
fe5a09b50a Ignore stdweb 2020-10-31 19:00:22 +00:00
f59c70a836 Print the entry type as well when checking archive (#13312) (#13313)
(cherry picked from commit bc7133d752)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-30 17:46:02 +00:00
74bfb00959 Update AccountInfo comments (#13302)
(cherry picked from commit 72d41e5801)
2020-10-30 08:09:42 -07:00
4296bfc728 clarify comment (#13289) (#13291)
(cherry picked from commit b5c8b86e7c)

Co-authored-by: Jack May <jack@solana.com>
2020-10-29 22:28:09 +00:00
ccec111178 more portable install.sh (bp #13114) (#13219)
* more portable install.sh (#13114)

(cherry picked from commit 4e0d1b1d4a)

# Conflicts:
#	sdk/bpf/scripts/install.sh

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
2020-10-29 21:10:14 +00:00
3a8c6f33a3 adds more parallel processing to gossip packets handling (#12988) (#13287)
(cherry picked from commit 3738611f5c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 18:05:52 +00:00
06067dd823 implements ping-pong packets between nodes (bp #12794) (#13281)
* implements ping-pong packets between nodes (#12794)

https://hackerone.com/reports/991106

> It’s possible to use UDP gossip protocol to amplify DDoS attacks. An attacker
> can spoof IP address in UDP packet when sending PullRequest to the node.
> There's no any validation if provided source IP address is not spoofed and
> the node can send much larger PullResponse to victim's IP. As I checked,
> PullRequest is about 290 bytes, while PullResponse is about 10 kB. It means
> that amplification is about 34x. This way an attacker can easily perform DDoS
> attack both on Solana node and third-party server.
>
> To prevent it, need for example to implement ping-pong mechanism similar as
> in Ethereum: Before accepting requests from remote client needs to validate
> his IP. Local node sends Ping packet to the remote node and it needs to reply
> with Pong packet that contains hash of matching Ping packet. Content of Ping
> packet is unpredictable. If hash from Pong packet matches, local node can
> remember IP where Ping packet was sent as correct and allow further
> communication.
>
> More info:
> https://github.com/ethereum/devp2p/blob/master/discv4.md#endpoint-proof
> https://github.com/ethereum/devp2p/blob/master/discv4.md#wire-protocol

The commit adds a PingCache, which maintains records of remote nodes
which have returned a valid response to a ping message, and on-the-fly
ping messages pending a pong response from the remote node.

When handling pull-requests, those from addresses which have not passed
the ping-pong check are filtered out, and additionally ping packets are
added for addresses which need to be (re)verified.

(cherry picked from commit ae91270961)

# Conflicts:
#	Cargo.lock
#	core/src/cluster_info.rs

* resolves mergify merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 16:33:29 +00:00
9f58a0383c Disable eager rent collection for less noise (#13275) (#13279)
(cherry picked from commit 363c148dbe)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-29 15:38:56 +00:00
428cacff88 scans crds table in parallel for finding old labels (bp #13073) (#13277)
* scans crds table in parallel for finding old labels (#13073)

From runtime profiles, the majority time of ClusterInfo::handle_purge
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1605-L1626
is spent scanning crds table finding old labels:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/crds.rs#L175-L197

This can be done in parallel given that gossip thread-pool:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1637-L1641
is idle when handle_purge is invoked:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1681

(cherry picked from commit 37c8842bcb)

# Conflicts:
#	core/tests/crds_gossip.rs

* resolves mergify merge conflict

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 15:33:59 +00:00
6a4f89b193 excludes origin from prune set (#13204) (#13278)
On the receiving end, prune messages are ignored if the origin points to
the node itself:
https://github.com/solana-labs/solana/blob/631f029fe/core/src/crds_gossip_push.rs#L285-L295
So to avoid sending these over the wire, the requester can exclude
origin from the prune set.

(cherry picked from commit be80f6d5c5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 15:22:47 +00:00
65ad72cd64 improves threads' utilization in processing gossip packets (#12962) (#13251)
ClusterInfo::process_packets handles incoming packets in a thread_pool:
https://github.com/solana-labs/solana/blob/87311cce7/core/src/cluster_info.rs#L2118-L2134

However, profiling runtime shows that threads are not well utilized and
a lot of the processing is done sequentially.

This commit redistributes the work done in parallel. Testing on a gce
cluster shows 20%+ improvement in processing gossip packets with much
smaller variations.

(cherry picked from commit 75d62ca095)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 13:59:12 +00:00
0ddf684376 marks pull request creation time only once per peer (#13113) (#13252)
mark_pull_request_creation time requires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/16944e218/core/src/cluster_info.rs#L1547-L1548
Current code is redundantly marking each peer once for each request.
There are at most only 2 unique peers, whereas there are hundreds of
requests per each. So the lock is acquired hundreds of time longer than
necessary.

(cherry picked from commit 4bfda3e766)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 13:54:39 +00:00
1376c325b6 Improve final report of ledger-tool capitalization (#13232) (#13235)
(cherry picked from commit 4698ee5e4a)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 18:28:23 +00:00
e29a49f107 Use pico inflation for ledger-tool capitalization --enable-inflation (#13215) (#13221)
* Use pico inflation for ledger-tool capitalization --enable-inflation

* rust fmt

(cherry picked from commit 7d2962135d)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 20:54:56 +09:00
4e7c096177 Use zstd for create-snapshot (#13214) (#13217)
(cherry picked from commit 6d4c69b7c3)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 06:37:28 +00:00
b18fa8deac Fix log (#13207) (#13210)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit f96ab5a818)

Co-authored-by: carllin <wumu727@gmail.com>
2020-10-28 03:11:21 +00:00
bba4f3006f Parse vote instructions (#13202) (#13208)
(cherry picked from commit c4962af9eb)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-28 01:53:13 +00:00
802a2cc985 passes through feature-set to gossip requests handling (#12878) (#13205)
* passes through feature-set to down to gossip requests handling
* takes the feature-set from root_bank instead of working_bank

(cherry picked from commit 48283161c3)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-27 23:46:12 +00:00
ad5ef9cc48 Account for forward delay in transaction simulation (#13199) (#13200)
(cherry picked from commit 631f029fe9)

Co-authored-by: Justin Starry <justin@solana.com>
2020-10-27 18:34:41 +00:00
cf1c7c8c00 macos portable rust-bpf (#13176) (#13186)
(cherry picked from commit fc83a666fc)

Co-authored-by: Jack May <jack@solana.com>
2020-10-27 04:45:41 +00:00