Commit Graph

11875 Commits

Author SHA1 Message Date
98095b6f8d drops older gossip packets when load shedding (#13364) (#13423)
Gossip drops incoming packets when overloaded:
https://github.com/solana-labs/solana/blob/f6a73098a/core/src/cluster_info.rs#L2462-L2475
However newer packets are dropped in favor of the older ones.
This is probably not ideal as newer packets are more likely to contain
more recent data, so dropping them will keep the validator state
lagging.

(cherry picked from commit 7f4debdad5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-11-05 18:30:00 +00:00
a2c32d7d0e shares the lock on gossip when processing prune messages (#13339) (#13422)
Processing prune messages acquires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L1824-L1825
This can be reduced to a shared lock if active-sets are changed to use
atomic bloom filters:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/crds_gossip_push.rs#L50

(cherry picked from commit 8f0796436a)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-11-05 17:18:22 +00:00
b15d826476 Allow feature builtins to overwrite existing builtins (#13403) (#13420)
* Allow feature builtins to overwrite existing builtins

* Add feature_builtin ActivationType

* Correctly retain idempotent for replacing case

* Fix test

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
(cherry picked from commit bc62313c66)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-11-05 16:51:25 +00:00
ed97a2578d measures processing time of each kind of gossip packets (#13366) (#13418)
(cherry picked from commit 118ce47b97)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-11-05 16:46:25 +00:00
89f61f0b41 Bump low end validator RAM requirement (#13406) (#13408) 2020-11-05 15:24:16 +08:00
04cc9c1148 CI: Use branch-versioned cargo throughout (#13411)
(cherry picked from commit 66c3c6c2b3)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-05 06:25:11 +00:00
8314ab4508 Improve invoke_signed() docs (#13405)
(cherry picked from commit 8c4995b22b)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-05 01:07:24 +00:00
3a98042753 Final program ELF is now placed in ./target/deploy/ instead of . (#13404)
(cherry picked from commit 661a935075)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-05 00:19:19 +00:00
60d316c9fd Bump spl-token and spl-memo crate versions (#13400)
Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-04 23:16:41 +00:00
e324c221a6 uses thread-pool when handling push messages (#13338) (#13395)
From runtime profiles, the majority time of solana-listen thread:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L2720
is spent handling push messages. The code here:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L2272-L2364
may utilize the idle gossip thread-pool.

(cherry picked from commit 10fa4f45ab)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-11-04 20:28:56 +00:00
61246999ac Update SPL Token exchange documentation to include associated-token-account workflow (#13397)
(cherry picked from commit 97284adabb)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-04 19:57:02 +00:00
e476dc4eaa Comment Stakes::clone_with_epoch (#13388) (#13390)
(cherry picked from commit b0d1ae1d8b)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-04 12:29:13 +00:00
ee18e7668b Refine transaction log count message (#13378)
(cherry picked from commit b5ef319038)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-04 06:44:49 +00:00
62db7f6562 Surface transaction logs in rpc client (#13376)
(cherry picked from commit 6d9ca0ae15)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-04 02:12:46 +00:00
2e9b501355 more informative feature error message (#13373) (#13375)
(cherry picked from commit 04c5e6cc48)

Co-authored-by: Jack May <jack@solana.com>
2020-11-04 01:35:22 +00:00
089a99f1e3 docs: Mainnet-beta totally has smart contract enabled (#13369)
(cherry picked from commit fefa297877)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-03 19:21:18 +00:00
57961b1d17 Update sol_log_compute_units (#13360) (#13363)
(cherry picked from commit f6a73098a4)

Co-authored-by: Jack May <jack@solana.com>
2020-11-03 17:28:44 +00:00
fe8b2b7850 Include Rent in ProgramTest::start() output (#13356)
(cherry picked from commit c3d2d2134c)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-03 03:12:17 +00:00
0bf45cbab6 Small code cleanup and typo fixes (#13325) (#13341)
* Small code cleanup and typo fixes

* Clean up calculate_points_and_credits

(cherry picked from commit 0e4509c497)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-02 02:42:38 +00:00
5877427389 Bump version to v1.4.5 2020-11-01 17:05:45 +00:00
25141288f4 Fix typos (#13334) (#13335)
(cherry picked from commit af9a3f004e)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
v1.4.4
2020-11-01 07:06:46 +00:00
b28d10d46f Add bank timestamp bounding (bp #13120) (#13331)
* Add bounding feature

(cherry picked from commit 96b8aa8bd1)

* Repurpose unused as Clock::epoch_start_timestamp; add gated update

(cherry picked from commit 0049ab69fb)

* Add bounded timestamp-estimation method

(cherry picked from commit 80db6c0980)

* Use bounded timestamp-correction when feature enabled

(cherry picked from commit 90778615f6)

* Prevent block times from ever going backward

(cherry picked from commit eb2560e782)

* Sample votes from ancestors back to root

(cherry picked from commit 4260b3b416)

* Add Clock sysvar details, update struct docs

(cherry picked from commit 3a1e125ce3)

* Add design proposal and update validator-timestamp-oracle

(cherry picked from commit a3912bc084)

* Adapt to feature::create_account

Co-authored-by: Tyera Eulberg <tyera@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-01 06:02:34 +00:00
b6dc48da75 Add solana-program-test crate (bp #13324) (#13329)
* MockInvokeContext::get_programs() implementation

(cherry picked from commit 8acc47ee1b)

* start_local_server() now works with Banks > 0

(cherry picked from commit fa4bab4608)

* Add solana-program-test crate

(cherry picked from commit 52a292a75b)

* rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-01 05:43:43 +00:00
f2d929c12d Move Feature struct to solana-program (#13321)
(cherry picked from commit 4b65e32f22)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-31 20:33:35 +00:00
c49b89091a cargo update -p futures-task / cargo update -p futures-util 2020-10-31 18:50:51 +00:00
23fe3a86d9 Switch to dirs-next 2020-10-31 18:50:51 +00:00
2f778725d6 Ignore stdweb 2020-10-31 18:50:51 +00:00
93a119a51e Print the entry type as well when checking archive (#13312) (#13314)
(cherry picked from commit bc7133d752)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-30 17:48:23 +00:00
65a7b536c9 Update AccountInfo comments (#13302)
(cherry picked from commit 72d41e5801)
2020-10-30 08:09:37 -07:00
1281483a8c Fix tower/blockstore unsync due to external causes (#12671) (#13310)
* Fix tower/blockstore unsync due to external causes

* Add and clean up long comments

* Clean up test

* Comment about warped_slot_history

* Run test_future_tower with master-only/master-slave

* Update comments about false leader condition

(cherry picked from commit 1df15d85c3)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-30 11:47:22 +00:00
4312841433 de-mut some InvokeContext methods (bp #13301) (#13309)
* de-mut some InvokeContext methods

(cherry picked from commit da9548fd12)

* Simplify CPI interface into MessageProcessor

(cherry picked from commit 9263ae1c60)

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-30 10:35:59 +00:00
b859acbfea Upgrade tarpc and tokio (bp #13293) (#13300)
* Upgrade tarpc and tokio (#13293)

(cherry picked from commit ca00197009)

# Conflicts:
#	banks-client/Cargo.toml
#	banks-interface/Cargo.toml
#	banks-server/Cargo.toml

* rebase

Co-authored-by: Greg Fitzgerald <greg@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-30 08:05:27 +00:00
40a3885d3b Native/builtin programs now receive an InvokeContext (bp #13286) (#13298)
* Native/builtin programs now receive an InvokeContext

(cherry picked from commit df8dab9d2b)

* Remove MessageProcessor::loaders

(cherry picked from commit 2664a1f7ef)

* Remove Entrypoint type

(cherry picked from commit 225bed11c7)

* Remove programs clone()

(cherry picked from commit 33884d847a)

* Add sol_log_compute_units syscall

(cherry picked from commit 66e51a7363)

* Add Bank::set_bpf_compute_budget()

(cherry picked from commit 7d686b72a0)

* Rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-30 07:47:17 +00:00
36b7c2ea97 Refactors the common code of test and bench targets into the solana_runtime::bpf_test_utils module. (#13203)
(cherry picked from commit 65ee3a6bdd)
2020-10-29 22:03:09 -07:00
24bd4ff6d4 clarify comment (#13289) (#13292)
(cherry picked from commit b5c8b86e7c)

Co-authored-by: Jack May <jack@solana.com>
2020-10-29 22:38:26 +00:00
69b3f10207 move Account to solana-sdk (bp #13198) (#13269)
* move Account to solana-sdk (#13198)

(cherry picked from commit c458d4b213)

# Conflicts:
#	programs/bpf/benches/bpf_loader.rs

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
2020-10-29 19:16:52 +00:00
9922f09a1d adds more parallel processing to gossip packets handling (#12988) (#13282)
(cherry picked from commit 3738611f5c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 16:47:37 +00:00
38a99c0c25 Disable eager rent collection for less noise (#13275) (#13280)
(cherry picked from commit 363c148dbe)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-29 15:49:02 +00:00
7031235714 excludes origin from prune set (#13204) (#13276)
On the receiving end, prune messages are ignored if the origin points to
the node itself:
https://github.com/solana-labs/solana/blob/631f029fe/core/src/crds_gossip_push.rs#L285-L295
So to avoid sending these over the wire, the requester can exclude
origin from the prune set.

(cherry picked from commit be80f6d5c5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-29 14:19:06 +00:00
dfb2356a9a Update FeatureSet::active to include slot-activated (#13256) (#13263)
* Update FeatureSet::active to include slot-activated

* Clippy suggestion

(cherry picked from commit c2dbf53d76)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-29 03:42:42 +00:00
010794806a Finer grained AccountsIndex locking (#12787) (#13240)
Co-authored-by: Carl Lin <carl@solana.com>

Co-authored-by: carllin <wumu727@gmail.com>
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-28 23:46:54 +00:00
6f95d5f72a Update links from sdk to program (#13248) (#13249)
(cherry picked from commit db9ddc7e5b)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-28 21:32:12 +00:00
2720b939fd Calculate accounts hash async in accounts background service (#12852) (#13244)
(cherry picked from commit 456eae6ccb)

Co-authored-by: sakridge <sakridge@gmail.com>
2020-10-28 21:20:07 +00:00
a25c3fcf7d Add doc page on sysvar accounts (#13237) (#13246)
* Add doc page on sysvar accounts

* Update with suggestions

(cherry picked from commit 664b6125b6)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-28 21:16:10 +00:00
7cc4810174 docs: Metrics update (bp #13239) (#13241)
* docs: Remove stale metrics steps

(cherry picked from commit 4dc4fefee2)

* docs: Reference metrics envvars for each cluster

(cherry picked from commit eb597cd60f)

Co-authored-by: Trent Nelson <trent@solana.com>
2020-10-28 19:46:59 +00:00
c1a55bf249 Improve final report of ledger-tool capitalization (#13232) (#13236)
(cherry picked from commit 4698ee5e4a)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 18:45:19 +00:00
f19778b7d9 implements ping-pong packets between nodes (#12794) (#13234)
https://hackerone.com/reports/991106

> It’s possible to use UDP gossip protocol to amplify DDoS attacks. An attacker
> can spoof IP address in UDP packet when sending PullRequest to the node.
> There's no any validation if provided source IP address is not spoofed and
> the node can send much larger PullResponse to victim's IP. As I checked,
> PullRequest is about 290 bytes, while PullResponse is about 10 kB. It means
> that amplification is about 34x. This way an attacker can easily perform DDoS
> attack both on Solana node and third-party server.
>
> To prevent it, need for example to implement ping-pong mechanism similar as
> in Ethereum: Before accepting requests from remote client needs to validate
> his IP. Local node sends Ping packet to the remote node and it needs to reply
> with Pong packet that contains hash of matching Ping packet. Content of Ping
> packet is unpredictable. If hash from Pong packet matches, local node can
> remember IP where Ping packet was sent as correct and allow further
> communication.
>
> More info:
> https://github.com/ethereum/devp2p/blob/master/discv4.md#endpoint-proof
> https://github.com/ethereum/devp2p/blob/master/discv4.md#wire-protocol

The commit adds a PingCache, which maintains records of remote nodes
which have returned a valid response to a ping message, and on-the-fly
ping messages pending a pong response from the remote node.

When handling pull-requests, those from addresses which have not passed
the ping-pong check are filtered out, and additionally ping packets are
added for addresses which need to be (re)verified.

(cherry picked from commit ae91270961)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-28 18:36:28 +00:00
eecdacac42 Don't hold dashmap write lock in store create (#13007) (#13230)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit c8fc0a6ba1)

Co-authored-by: carllin <wumu727@gmail.com>
2020-10-28 11:36:28 +00:00
429f130532 Switch accounts storage lock to DashMap (#12126) (#13223)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit f8d338c9cb)

Co-authored-by: carllin <wumu727@gmail.com>
2020-10-28 08:07:28 +00:00
19b9839dfc Use pico inflation for ledger-tool capitalization --enable-inflation (#13215) (#13222)
* Use pico inflation for ledger-tool capitalization --enable-inflation

* rust fmt

(cherry picked from commit 7d2962135d)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-28 07:40:21 +00:00