Commit Graph

11339 Commits

Author SHA1 Message Date
0b9c87b6ec Docs: Clarify validator disk requirements 2020-11-05 15:24:26 +08:00
df1e62f23f Update docs to latest processors (#11613) 2020-11-05 15:24:26 +08:00
f697a86d1e Comment Stakes::clone_with_epoch (#13388) (#13389)
(cherry picked from commit b0d1ae1d8b)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-04 13:49:48 +00:00
64c76c2f4b Revert "check sysvar id for AccountInfo (#13175) (#13351)"
This reverts commit 290d514051.
2020-11-02 12:14:33 -08:00
290d514051 check sysvar id for AccountInfo (#13175) (#13351)
(cherry picked from commit 322c667655)

# Conflicts:
#	sdk/program/src/sysvar/mod.rs

Co-authored-by: Jack May <jack@solana.com>
2020-11-02 18:18:07 +00:00
958c43f337 Small code cleanup and typo fixes (#13325) (#13340)
* Small code cleanup and typo fixes

* Clean up calculate_points_and_credits

(cherry picked from commit 0e4509c497)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-02 02:38:43 +00:00
4513128c75 check sysvar id for AccountInfo (#13175) 2020-11-01 06:58:02 +00:00
d01968ed21 cargo update -p futures-task / cargo update -p futures-util 2020-10-31 19:00:22 +00:00
f9ac24d1f2 Switch to dirs-next 2020-10-31 19:00:22 +00:00
fe5a09b50a Ignore stdweb 2020-10-31 19:00:22 +00:00
f59c70a836 Print the entry type as well when checking archive (#13312) (#13313)
(cherry picked from commit bc7133d752)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-30 17:46:02 +00:00
74bfb00959 Update AccountInfo comments (#13302)
(cherry picked from commit 72d41e5801)
2020-10-30 08:09:42 -07:00
4296bfc728 clarify comment (#13289) (#13291)
(cherry picked from commit b5c8b86e7c)

Co-authored-by: Jack May <jack@solana.com>
2020-10-29 22:28:09 +00:00
ccec111178 more portable install.sh (bp #13114) (#13219)
* more portable install.sh (#13114)

(cherry picked from commit 4e0d1b1d4a)

# Conflicts:
#	sdk/bpf/scripts/install.sh

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
2020-10-29 21:10:14 +00:00
3a8c6f33a3 adds more parallel processing to gossip packets handling (#12988) (#13287)
(cherry picked from commit 3738611f5c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 18:05:52 +00:00
06067dd823 implements ping-pong packets between nodes (bp #12794) (#13281)
* implements ping-pong packets between nodes (#12794)

https://hackerone.com/reports/991106

> It’s possible to use UDP gossip protocol to amplify DDoS attacks. An attacker
> can spoof IP address in UDP packet when sending PullRequest to the node.
> There's no any validation if provided source IP address is not spoofed and
> the node can send much larger PullResponse to victim's IP. As I checked,
> PullRequest is about 290 bytes, while PullResponse is about 10 kB. It means
> that amplification is about 34x. This way an attacker can easily perform DDoS
> attack both on Solana node and third-party server.
>
> To prevent it, need for example to implement ping-pong mechanism similar as
> in Ethereum: Before accepting requests from remote client needs to validate
> his IP. Local node sends Ping packet to the remote node and it needs to reply
> with Pong packet that contains hash of matching Ping packet. Content of Ping
> packet is unpredictable. If hash from Pong packet matches, local node can
> remember IP where Ping packet was sent as correct and allow further
> communication.
>
> More info:
> https://github.com/ethereum/devp2p/blob/master/discv4.md#endpoint-proof
> https://github.com/ethereum/devp2p/blob/master/discv4.md#wire-protocol

The commit adds a PingCache, which maintains records of remote nodes
which have returned a valid response to a ping message, and on-the-fly
ping messages pending a pong response from the remote node.

When handling pull-requests, those from addresses which have not passed
the ping-pong check are filtered out, and additionally ping packets are
added for addresses which need to be (re)verified.

(cherry picked from commit ae91270961)

# Conflicts:
#	Cargo.lock
#	core/src/cluster_info.rs

* resolves mergify merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 16:33:29 +00:00
9f58a0383c Disable eager rent collection for less noise (#13275) (#13279)
(cherry picked from commit 363c148dbe)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-29 15:38:56 +00:00
428cacff88 scans crds table in parallel for finding old labels (bp #13073) (#13277)
* scans crds table in parallel for finding old labels (#13073)

From runtime profiles, the majority time of ClusterInfo::handle_purge
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1605-L1626
is spent scanning crds table finding old labels:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/crds.rs#L175-L197

This can be done in parallel given that gossip thread-pool:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1637-L1641
is idle when handle_purge is invoked:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1681

(cherry picked from commit 37c8842bcb)

# Conflicts:
#	core/tests/crds_gossip.rs

* resolves mergify merge conflict

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 15:33:59 +00:00
6a4f89b193 excludes origin from prune set (#13204) (#13278)
On the receiving end, prune messages are ignored if the origin points to
the node itself:
https://github.com/solana-labs/solana/blob/631f029fe/core/src/crds_gossip_push.rs#L285-L295
So to avoid sending these over the wire, the requester can exclude
origin from the prune set.

(cherry picked from commit be80f6d5c5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 15:22:47 +00:00
65ad72cd64 improves threads' utilization in processing gossip packets (#12962) (#13251)
ClusterInfo::process_packets handles incoming packets in a thread_pool:
https://github.com/solana-labs/solana/blob/87311cce7/core/src/cluster_info.rs#L2118-L2134

However, profiling runtime shows that threads are not well utilized and
a lot of the processing is done sequentially.

This commit redistributes the work done in parallel. Testing on a gce
cluster shows 20%+ improvement in processing gossip packets with much
smaller variations.

(cherry picked from commit 75d62ca095)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 13:59:12 +00:00
0ddf684376 marks pull request creation time only once per peer (#13113) (#13252)
mark_pull_request_creation time requires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/16944e218/core/src/cluster_info.rs#L1547-L1548
Current code is redundantly marking each peer once for each request.
There are at most only 2 unique peers, whereas there are hundreds of
requests per each. So the lock is acquired hundreds of time longer than
necessary.

(cherry picked from commit 4bfda3e766)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 13:54:39 +00:00
1376c325b6 Improve final report of ledger-tool capitalization (#13232) (#13235)
(cherry picked from commit 4698ee5e4a)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 18:28:23 +00:00
e29a49f107 Use pico inflation for ledger-tool capitalization --enable-inflation (#13215) (#13221)
* Use pico inflation for ledger-tool capitalization --enable-inflation

* rust fmt

(cherry picked from commit 7d2962135d)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 20:54:56 +09:00
4e7c096177 Use zstd for create-snapshot (#13214) (#13217)
(cherry picked from commit 6d4c69b7c3)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 06:37:28 +00:00
b18fa8deac Fix log (#13207) (#13210)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit f96ab5a818)

Co-authored-by: carllin <wumu727@gmail.com>
2020-10-28 03:11:21 +00:00
bba4f3006f Parse vote instructions (#13202) (#13208)
(cherry picked from commit c4962af9eb)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-28 01:53:13 +00:00
802a2cc985 passes through feature-set to gossip requests handling (#12878) (#13205)
* passes through feature-set to down to gossip requests handling
* takes the feature-set from root_bank instead of working_bank

(cherry picked from commit 48283161c3)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-27 23:46:12 +00:00
ad5ef9cc48 Account for forward delay in transaction simulation (#13199) (#13200)
(cherry picked from commit 631f029fe9)

Co-authored-by: Justin Starry <justin@solana.com>
2020-10-27 18:34:41 +00:00
cf1c7c8c00 macos portable rust-bpf (#13176) (#13186)
(cherry picked from commit fc83a666fc)

Co-authored-by: Jack May <jack@solana.com>
2020-10-27 04:45:41 +00:00
a28fb586b0 Add SSH key for buildkite-agent on achille (#13182)
(cherry picked from commit ff4b34202c)

# Conflicts:
#	net/scripts/solana-user-authorized_keys.sh

Co-authored-by: Trent Nelson <trent@solana.com>
2020-10-27 04:25:53 +00:00
5737ea448e CLI: Surface deploy transaction errors (#13169)
(cherry picked from commit a82971879f)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-10-26 22:26:02 +00:00
e54a537015 update call depth docs (#13155) (#13161)
(cherry picked from commit 35f77ccc73)

Co-authored-by: Jack May <jack@solana.com>
2020-10-26 19:45:02 +00:00
39eae50024 implements DataBudget using atomics (#12856) (#13157)
(cherry picked from commit 05cf15a382)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-26 18:27:45 +00:00
736e5c3ec7 Fix BigTable reward type encoding (bp #13142) (#13146)
* Fix reward type encoding

(cherry picked from commit 0a89bb4d3c)

* Don't reuse BPF target build artifacts

(cherry picked from commit 41a56e14fc)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-26 08:18:05 +00:00
f8e310d99d Hide noisy specialization warnings for frozen abi (#13141) (#13143)
(cherry picked from commit 5caf81dbf8)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-26 05:11:12 +00:00
fd7bcfdb4d add precompile verification to simulate_transaction (#13080) (#13125)
(cherry picked from commit 766406fd23)

Co-authored-by: Josh <josh.hundley@gmail.com>
2020-10-24 04:57:12 +00:00
e553c8bb45 Clean up opt conf verifier and vote state tracker (#13081) (#13123)
* Clean up opt conf verifier and vote state tracker

* Update test to follow new message and some knob

* Rename

(cherry picked from commit 0264147d42)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-24 02:39:33 +00:00
1c5f8f51ee Shorten magic install URL (#13121)
(cherry picked from commit b5170b993e)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-23 23:20:03 +00:00
5084871e27 Cli: deploy programs via TPU (#13090) (#13110)
* Deploy: send write transactions to leader tpu

* Less apparent stalling during confirmation

* Add EpochInfo mock

* Only get cluster nodes once

* Send deploy writes to next leader

(cherry picked from commit 16944e218f)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-23 17:14:37 +00:00
184a56aae7 shrink debug (#13089) (#13108)
(cherry picked from commit 7d2729f6bd)

Co-authored-by: sakridge <sakridge@gmail.com>
2020-10-23 16:14:09 +00:00
7d66cba7f4 Remove spammy invalid rpc log (#13100) (#13101)
(cherry picked from commit c95f6c4b83)

Co-authored-by: Justin Starry <justin@solana.com>
2020-10-23 08:21:33 +00:00
8f615278d2 Add deploy err if program-account balance is too high (#13091) (#13097)
* Add deploy err if program-account balance is too high

* Review comments

* Add system-program check

* Rename and unhide flag

(cherry picked from commit 4669fa0f98)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-23 05:50:21 +00:00
884d68ddcf Add Pubkey::new_unique()/Hash::new_unique() 2020-10-23 03:19:38 +00:00
b095a52027 Add programming-faq to sidebar (#12586) (#13087)
* Add programming-faq to sidebar

* nudge

* fix path

(cherry picked from commit 22d16c69b7)

Co-authored-by: Jack May <jack@solana.com>
2020-10-22 12:34:22 +00:00
1f5861e107 Add programming faq (#12545) (#13086)
* Add programming faq

* feedback and new content

* nudge

(cherry picked from commit b51c0f3095)

Co-authored-by: Jack May <jack@solana.com>
2020-10-22 12:33:29 +00:00
b90b46fee1 Allow nodes to advertise a different rpc address over gossip (#13053) (#13077)
* Allow nodes to advertise a different rpc address over gossip

* Feedback

(cherry picked from commit 8b0242a5d8)

Co-authored-by: Justin Starry <justin@solana.com>
2020-10-22 04:44:41 +00:00
bff820d549 Remove unused pubkey::Pubkey imports 2020-10-21 20:27:57 -07:00
5ad0ccdfe1 cargo fmt 2020-10-21 20:27:57 -07:00
d8c7d06737 Run codemod --extensions rs Hash::new_rand solana_sdk:#️⃣:new_rand 2020-10-21 20:27:57 -07:00
422bb3c526 Run codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand 2020-10-21 20:27:57 -07:00