gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
11,877 Commits 81 Branches 357 Tags
1072bd76401053307b81bf430d715eb219330806
Commit Graph

1944 Commits

Author SHA1 Message Date
mergify[bot]
44fffcbb1c filters out offline nodes from pull options (#13533) (#13551) 
Inactive nodes are still observing incoming gossip traffic:
https://discord.com/channels/428295358100013066/670512312339398668/776140351291260968
likely because of pull-requests.

Previous related issues and commits:
https://github.com/solana-labs/solana/issues/12409
https://github.com/solana-labs/solana/pull/12620
https://github.com/solana-labs/solana/pull/12674

This commit implements same logic as
https://github.com/solana-labs/solana/pull/12674
to exclude inactive nodes from pull options, with the same periodic
retry logic for offline staked nodes in order to mitigate eclipse
attack.

(cherry picked from commit 4e4e12b384)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-11-12 18:43:04 +00:00
Ryo Onodera
e14c2f94f4 Bump version to v1.4.8 (#13552) 2020-11-12 17:43:31 +00:00
mergify[bot]
437c356626 Discard pre hard fork persisted tower if hard-forking (#13536) (#13550) 
* Discard pre hard fork persisted tower if hard-forking

* Relax config.require_tower

* Add cluster test

* nits

* Remove unnecessary check

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit 9821a7754c)

Co-authored-by: carllin <wumu727@gmail.com>
 2020-11-12 15:43:43 +00:00
mergify[bot]
2cb9ca5966 Fix slow/stuck unstaking due to toggling in epoch (#13501) (#13535) 
* Fix slow/stuck unstaking due to toggling in epoch

* nits

* nits

* Add stake_program_v2 feature status check to cli

Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 89b474e192)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-11-11 22:27:10 +00:00
mergify[bot]
4f247a232f Validator: Periodically log what we're waiting for during --wait-for-supermajority (#13531) 
(cherry picked from commit 38f15e41b5)

Co-authored-by: Trent Nelson <trent@solana.com>
 2020-11-11 21:29:50 +00:00
mergify[bot]
e9eba97299 Fix signature access (#13491) (#13503) 
(cherry picked from commit 70c4626efe)

Co-authored-by: sakridge <sakridge@gmail.com>
 2020-11-10 17:53:43 +00:00
mergify[bot]
8104895a07 Send RPC notification when account is deleted (#13440) (#13496) 
* Send RPC notification when account is deleted

* Remove unwrap

(cherry picked from commit a97c04b400)

Co-authored-by: Justin Starry <justin@solana.com>
 2020-11-10 13:07:51 +00:00
Tyera Eulberg
c9e646b86b Bump version to v1.4.7 (#13488) 2020-11-10 05:55:26 +00:00
mergify[bot]
9627bfced3 propagates errors out of Packet::from_data (#13445) (#13470) 
Packet::from_data is ignoring serialization errors:
https://github.com/solana-labs/solana/blob/d08c3232e/sdk/src/packet.rs#L42-L48
This is likely never useful as the packet will be sent over the wire
taking bandwidth but at the receiving end will either fail to
deserialize or it will be invalid.
This commit will propagate the errors out of the function to the
call-site, allowing the call-site to handle the error.

(cherry picked from commit 73ac104df2)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-11-08 16:25:36 +00:00
Michael Vines
e738bf1c9a Bump version to v1.4.6 2020-11-07 02:49:14 +00:00
mergify[bot]
98095b6f8d drops older gossip packets when load shedding (#13364) (#13423) 
Gossip drops incoming packets when overloaded:
https://github.com/solana-labs/solana/blob/f6a73098a/core/src/cluster_info.rs#L2462-L2475
However newer packets are dropped in favor of the older ones.
This is probably not ideal as newer packets are more likely to contain
more recent data, so dropping them will keep the validator state
lagging.

(cherry picked from commit 7f4debdad5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-11-05 18:30:00 +00:00
mergify[bot]
a2c32d7d0e shares the lock on gossip when processing prune messages (#13339) (#13422) 
Processing prune messages acquires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L1824-L1825
This can be reduced to a shared lock if active-sets are changed to use
atomic bloom filters:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/crds_gossip_push.rs#L50

(cherry picked from commit 8f0796436a)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-11-05 17:18:22 +00:00
mergify[bot]
ed97a2578d measures processing time of each kind of gossip packets (#13366) (#13418) 
(cherry picked from commit 118ce47b97)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-11-05 16:46:25 +00:00
mergify[bot]
60d316c9fd Bump spl-token and spl-memo crate versions (#13400) 
Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-11-04 23:16:41 +00:00
mergify[bot]
e324c221a6 uses thread-pool when handling push messages (#13338) (#13395) 
From runtime profiles, the majority time of solana-listen thread:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L2720
is spent handling push messages. The code here:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L2272-L2364
may utilize the idle gossip thread-pool.

(cherry picked from commit 10fa4f45ab)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-11-04 20:28:56 +00:00
Michael Vines
5877427389 Bump version to v1.4.5 2020-11-01 17:05:45 +00:00
mergify[bot]
1281483a8c Fix tower/blockstore unsync due to external causes (#12671) (#13310) 
* Fix tower/blockstore unsync due to external causes

* Add and clean up long comments

* Clean up test

* Comment about warped_slot_history

* Run test_future_tower with master-only/master-slave

* Update comments about false leader condition

(cherry picked from commit 1df15d85c3)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-30 11:47:22 +00:00
mergify[bot]
b859acbfea Upgrade tarpc and tokio (bp #13293) (#13300) 
* Upgrade tarpc and tokio (#13293)

(cherry picked from commit ca00197009)

# Conflicts:
#	banks-client/Cargo.toml
#	banks-interface/Cargo.toml
#	banks-server/Cargo.toml

* rebase

Co-authored-by: Greg Fitzgerald <greg@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-10-30 08:05:27 +00:00
mergify[bot]
40a3885d3b Native/builtin programs now receive an InvokeContext (bp #13286) (#13298) 
* Native/builtin programs now receive an InvokeContext

(cherry picked from commit df8dab9d2b)

* Remove MessageProcessor::loaders

(cherry picked from commit 2664a1f7ef)

* Remove Entrypoint type

(cherry picked from commit 225bed11c7)

* Remove programs clone()

(cherry picked from commit 33884d847a)

* Add sol_log_compute_units syscall

(cherry picked from commit 66e51a7363)

* Add Bank::set_bpf_compute_budget()

(cherry picked from commit 7d686b72a0)

* Rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-10-30 07:47:17 +00:00
mergify[bot]
69b3f10207 move Account to solana-sdk (bp #13198) (#13269) 
* move Account to solana-sdk (#13198)

(cherry picked from commit c458d4b213)

# Conflicts:
#	programs/bpf/benches/bpf_loader.rs

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
 2020-10-29 19:16:52 +00:00
mergify[bot]
9922f09a1d adds more parallel processing to gossip packets handling (#12988) (#13282) 
(cherry picked from commit 3738611f5c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-29 16:47:37 +00:00
mergify[bot]
7031235714 excludes origin from prune set (#13204) (#13276) 
On the receiving end, prune messages are ignored if the origin points to
the node itself:
https://github.com/solana-labs/solana/blob/631f029fe/core/src/crds_gossip_push.rs#L285-L295
So to avoid sending these over the wire, the requester can exclude
origin from the prune set.

(cherry picked from commit be80f6d5c5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-29 14:19:06 +00:00
mergify[bot]
f19778b7d9 implements ping-pong packets between nodes (#12794) (#13234) 
https://hackerone.com/reports/991106

> It’s possible to use UDP gossip protocol to amplify DDoS attacks. An attacker
> can spoof IP address in UDP packet when sending PullRequest to the node.
> There's no any validation if provided source IP address is not spoofed and
> the node can send much larger PullResponse to victim's IP. As I checked,
> PullRequest is about 290 bytes, while PullResponse is about 10 kB. It means
> that amplification is about 34x. This way an attacker can easily perform DDoS
> attack both on Solana node and third-party server.
>
> To prevent it, need for example to implement ping-pong mechanism similar as
> in Ethereum: Before accepting requests from remote client needs to validate
> his IP. Local node sends Ping packet to the remote node and it needs to reply
> with Pong packet that contains hash of matching Ping packet. Content of Ping
> packet is unpredictable. If hash from Pong packet matches, local node can
> remember IP where Ping packet was sent as correct and allow further
> communication.
>
> More info:
> https://github.com/ethereum/devp2p/blob/master/discv4.md#endpoint-proof
> https://github.com/ethereum/devp2p/blob/master/discv4.md#wire-protocol

The commit adds a PingCache, which maintains records of remote nodes
which have returned a valid response to a ping message, and on-the-fly
ping messages pending a pong response from the remote node.

When handling pull-requests, those from addresses which have not passed
the ping-pong check are filtered out, and additionally ping packets are
added for addresses which need to be (re)verified.

(cherry picked from commit ae91270961)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-28 18:36:28 +00:00
mergify[bot]
9fac507606 Fix log (#13207) (#13211) 
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit f96ab5a818)

Co-authored-by: carllin <wumu727@gmail.com>
 2020-10-28 03:21:01 +00:00
mergify[bot]
cf896dbeee Use bank timestamp to populate Blockstore::blocktime_cf when correction active (#13158) (#13160) 
(cherry picked from commit 39686ef098)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-10-26 20:34:15 +00:00
mergify[bot]
f0b74a4ecf marks pull request creation time only once per peer (#13113) (#13156) 
mark_pull_request_creation time requires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/16944e218/core/src/cluster_info.rs#L1547-L1548
Current code is redundantly marking each peer once for each request.
There are at most only 2 unique peers, whereas there are hundreds of
requests per each. So the lock is acquired hundreds of time longer than
necessary.

(cherry picked from commit 4bfda3e766)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-26 18:27:26 +00:00
mergify[bot]
605e767259 Allow existence of vote on root in saved tower (#13135) (#13139) 
(cherry picked from commit 66c7a98009)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-26 12:34:58 +09:00
Michael Vines
597618846b Bump version to v1.4.4 2020-10-24 22:10:32 +00:00
mergify[bot]
eb9cef0cd4 Separate the "program" feature of solana-sdk into a new crate called solana-program (bp #12989) (#13131) 
* Add solana-program-sdk boilerplate

(cherry picked from commit 3718771ffb)

# Conflicts:
#	sdk/Cargo.toml

* Initial population of solana-program-sdk

(cherry picked from commit 63db324204)

# Conflicts:
#	Cargo.lock

* Port programs to solana-program-sdk

(cherry picked from commit fe68f7f786)

# Conflicts:
#	programs/bpf/Cargo.lock
#	programs/bpf/rust/128bit/Cargo.toml
#	programs/bpf/rust/128bit_dep/Cargo.toml
#	programs/bpf/rust/alloc/Cargo.toml
#	programs/bpf/rust/call_depth/Cargo.toml
#	programs/bpf/rust/custom_heap/Cargo.toml
#	programs/bpf/rust/dep_crate/Cargo.toml
#	programs/bpf/rust/deprecated_loader/Cargo.toml
#	programs/bpf/rust/dup_accounts/Cargo.toml
#	programs/bpf/rust/error_handling/Cargo.toml
#	programs/bpf/rust/external_spend/Cargo.toml
#	programs/bpf/rust/instruction_introspection/Cargo.toml
#	programs/bpf/rust/invoke/Cargo.toml
#	programs/bpf/rust/invoked/Cargo.toml
#	programs/bpf/rust/iter/Cargo.toml
#	programs/bpf/rust/many_args/Cargo.toml
#	programs/bpf/rust/many_args_dep/Cargo.toml
#	programs/bpf/rust/noop/Cargo.toml
#	programs/bpf/rust/panic/Cargo.toml
#	programs/bpf/rust/param_passing/Cargo.toml
#	programs/bpf/rust/param_passing_dep/Cargo.toml
#	programs/bpf/rust/rand/Cargo.toml
#	programs/bpf/rust/ristretto/Cargo.toml
#	programs/bpf/rust/sanity/Cargo.toml
#	programs/bpf/rust/sha256/Cargo.toml
#	programs/bpf/rust/sysval/Cargo.toml

* Only activate legacy program feature for the solana-sdk crate

(cherry picked from commit 85c51f5787)

* Run serum-dex unit tests

(cherry picked from commit 92ce381d60)

* Rename solana-program-sdk to solana-program

(cherry picked from commit dd711ab5fb)

# Conflicts:
#	programs/bpf/rust/128bit/Cargo.toml
#	programs/bpf/rust/128bit_dep/Cargo.toml
#	programs/bpf/rust/alloc/Cargo.toml
#	programs/bpf/rust/call_depth/Cargo.toml
#	programs/bpf/rust/custom_heap/Cargo.toml
#	programs/bpf/rust/dep_crate/Cargo.toml
#	programs/bpf/rust/deprecated_loader/Cargo.toml
#	programs/bpf/rust/dup_accounts/Cargo.toml
#	programs/bpf/rust/error_handling/Cargo.toml
#	programs/bpf/rust/external_spend/Cargo.toml
#	programs/bpf/rust/instruction_introspection/Cargo.toml
#	programs/bpf/rust/invoke/Cargo.toml
#	programs/bpf/rust/invoked/Cargo.toml
#	programs/bpf/rust/iter/Cargo.toml
#	programs/bpf/rust/many_args/Cargo.toml
#	programs/bpf/rust/many_args_dep/Cargo.toml
#	programs/bpf/rust/noop/Cargo.toml
#	programs/bpf/rust/panic/Cargo.toml
#	programs/bpf/rust/param_passing/Cargo.toml
#	programs/bpf/rust/param_passing_dep/Cargo.toml
#	programs/bpf/rust/rand/Cargo.toml
#	programs/bpf/rust/ristretto/Cargo.toml
#	programs/bpf/rust/sanity/Cargo.toml
#	programs/bpf/rust/sha256/Cargo.toml
#	programs/bpf/rust/sysval/Cargo.toml

* Update frozen_abi hashes

The movement of files in sdk/ caused ABI hashes to change

(cherry picked from commit a4956844bd)

* Resolve merge conflicts

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-10-24 17:25:22 +00:00
mergify[bot]
62e0e19961 add precompile verification to simulate_transaction (#13080) (#13126) 
(cherry picked from commit 766406fd23)

Co-authored-by: Josh <josh.hundley@gmail.com>
 2020-10-24 05:02:41 +00:00
mergify[bot]
9aee9cb867 Clean up opt conf verifier and vote state tracker (#13081) (#13124) 
* Clean up opt conf verifier and vote state tracker

* Update test to follow new message and some knob

* Rename

(cherry picked from commit 0264147d42)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-24 04:12:02 +00:00
mergify[bot]
a536f779ee scans crds table in parallel for finding old labels (#13073) (#13107) 
From runtime profiles, the majority time of ClusterInfo::handle_purge
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1605-L1626
is spent scanning crds table finding old labels:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/crds.rs#L175-L197

This can be done in parallel given that gossip thread-pool:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1637-L1641
is idle when handle_purge is invoked:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1681

(cherry picked from commit 37c8842bcb)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-23 15:36:06 +00:00
mergify[bot]
84a5e5ec97 Remove spammy invalid rpc log (#13100) (#13102) 
(cherry picked from commit c95f6c4b83)

Co-authored-by: Justin Starry <justin@solana.com>
 2020-10-23 08:32:46 +00:00
Trent Nelson
53b074aa35 Bump version to 1.4.3 2020-10-23 04:20:28 +00:00
mergify[bot]
a4ad2925a2 Allow nodes to advertise a different rpc address over gossip (#13053) (#13078) 
* Allow nodes to advertise a different rpc address over gossip

* Feedback

(cherry picked from commit 8b0242a5d8)

Co-authored-by: Justin Starry <justin@solana.com>
 2020-10-22 07:06:27 +00:00
mergify[bot]
edfbd8d65a Add replacements for Pubkey::new_rand()/Hash::new_rand() (bp #12987) (#13076) 
* Add pubkey_new_rand(), mark Pubkey::new_rand() deprecated

(cherry picked from commit 0e68ed6a8d)

* Add hash_new_rand(), mark Hash::new_rand() as deprecated

(cherry picked from commit 76f11c7dae)

* Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand`

(cherry picked from commit 7bc073defe)

# Conflicts:
#	programs/bpf/benches/bpf_loader.rs
#	runtime/benches/accounts.rs
#	runtime/src/accounts.rs

* Run `codemod --extensions rs Hash::new_rand solana_sdk:#️⃣:new_rand`

(cherry picked from commit 17c391121a)

* Remove unused pubkey::Pubkey imports

(cherry picked from commit 959880db60)

# Conflicts:
#	runtime/src/accounts_index.rs

* Resolve conflicts

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-10-22 05:08:01 +00:00
mergify[bot]
5236acf4b0 Add ledger-tool dead-slots and improve purge a lot (#13065) (#13071) 
* Add ledger-tool dead-slots and improve purge a lot

* Reduce batch size...

* Add --dead-slots-only and fixed purge ordering

(cherry picked from commit 0776fa05c7)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-21 18:57:22 +00:00
mergify[bot]
f76c128f4f Various clean-ups before assert adjustment (#13006) (#13041) 
* Various clean-ups before assert adjustment

* oops

(cherry picked from commit efdb560e97)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-21 02:41:20 +00:00
mergify[bot]
b143b9c3c2 Remove frozen ABI modules from solana-sdk (bp #13008) (#13036) 
* Remove frozen ABI modules from solana-sdk

(cherry picked from commit 6858950f76)

# Conflicts:
#	Cargo.lock
#	core/Cargo.toml
#	frozen-abi/macro/Cargo.toml
#	programs/bpf/Cargo.lock
#	programs/stake/Cargo.toml
#	programs/vote/Cargo.toml
#	runtime/Cargo.toml
#	sdk/Cargo.toml
#	version/Cargo.toml

* rebase

* fix broken ci (#13039)

Co-authored-by: Michael Vines <mvines@gmail.com>
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-21 02:00:45 +00:00
mergify[bot]
d05bfa08c7 improves threads' utilization in processing gossip packets (#12962) (#13023) 
ClusterInfo::process_packets handles incoming packets in a thread_pool:
https://github.com/solana-labs/solana/blob/87311cce7/core/src/cluster_info.rs#L2118-L2134

However, profiling runtime shows that threads are not well utilized and
a lot of the processing is done sequentially.

This commit redistributes the work done in parallel. Testing on a gce
cluster shows 20%+ improvement in processing gossip packets with much
smaller variations.

(cherry picked from commit 75d62ca095)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-20 19:59:35 +00:00
mergify[bot]
9da2ac7a44 passes through feature-set to gossip requests handling (#12878) (#12991) 
* passes through feature-set to down to gossip requests handling
* takes the feature-set from root_bank instead of working_bank

(cherry picked from commit 48283161c3)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-20 18:25:44 +00:00
mergify[bot]
94cad9873c Support Debug Bank (#13017) (#13019) 
(cherry picked from commit c0675968b1)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-20 17:20:40 +00:00
mergify[bot]
aa6406f263 implements DataBudget using atomics (#12856) (#12990) 
(cherry picked from commit 05cf15a382)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-10-20 12:51:42 +00:00
mergify[bot]
b51715d33c validator: Activate RPC before halting on slot (#13002) 
(cherry picked from commit 3b3f7341fa)

Co-authored-by: Trent Nelson <trent@solana.com>
 2020-10-20 03:31:04 +00:00
mergify[bot]
8a526f2f53 Follow up to persistent tower with tests and API cleaning (#12350) (#12972) 
* Follow up to persistent tower

* Ignore for now...

* Hard-code validator identities for easy reasoning

* Add a test for opt. conf violation without tower

* Fix compile with rust < 1.47

* Remove unused method

* More move of assert tweak to the asser pr

* Add comments

* Clean up

* Clean the test addressing various review comments

* Clean up a bit

(cherry picked from commit 54517ea454)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-19 08:47:58 +00:00
mergify[bot]
677c184e47 Another some tower logging improvements (#12940) (#12943) 
(cherry picked from commit fd8ec27fe8)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-16 07:00:27 +00:00
Trent Nelson
f3c986385f Bump version to 1.4.2 2020-10-15 20:26:29 +00:00
Tyera Eulberg
489a7bb576 Bump spl-memo and spl-token versions (#12914) 2020-10-15 18:05:41 +00:00
mergify[bot]
f8413a28b5 Better tower logs for SwitchForkDecision and etc (#12875) (#12905) 
* Better tower logs for SwitchForkDecision and etc

* nits

* Update comment

(cherry picked from commit a44e4d386f)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-10-15 10:41:03 +00:00
mergify[bot]
4b5ac44fc8 RPC: Add metrics for TX encoding (#12880) 
(cherry picked from commit c26512255d)

Co-authored-by: Trent Nelson <trent@solana.com>
 2020-10-14 19:39:08 +00:00