gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
11,883 Commits 81 Branches 357 Tags
f11617c192e139409c70b8a5fa88e8c0d1c2d15a
Commit Graph

1919 Commits

Author SHA1 Message Date
Tyera Eulberg
39686ef098 Use bank timestamp to populate Blockstore::blocktime_cf when correction active (#13158) 2020-10-26 19:23:45 +00:00
behzad nouri
4bfda3e766 marks pull request creation time only once per peer (#13113) 
mark_pull_request_creation time requires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/16944e218/core/src/cluster_info.rs#L1547-L1548
Current code is redundantly marking each peer once for each request.
There are at most only 2 unique peers, whereas there are hundreds of
requests per each. So the lock is acquired hundreds of time longer than
necessary.
 2020-10-26 17:11:31 +00:00
Ryo Onodera
66c7a98009 Allow existence of vote on root in saved tower (#13135) 2020-10-26 11:08:20 +09:00
Michael Vines
a4956844bd Update frozen_abi hashes 
The movement of files in sdk/ caused ABI hashes to change
 2020-10-24 08:37:55 -07:00
Josh
766406fd23 add precompile verification to simulate_transaction (#13080) 2020-10-23 20:47:51 -07:00
Ryo Onodera
0264147d42 Clean up opt conf verifier and vote state tracker (#13081) 
* Clean up opt conf verifier and vote state tracker

* Update test to follow new message and some knob

* Rename
 2020-10-24 10:19:12 +09:00
behzad nouri
37c8842bcb scans crds table in parallel for finding old labels (#13073) 
From runtime profiles, the majority time of ClusterInfo::handle_purge
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1605-L1626
is spent scanning crds table finding old labels:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/crds.rs#L175-L197

This can be done in parallel given that gossip thread-pool:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1637-L1641
is idle when handle_purge is invoked:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1681
 2020-10-23 14:17:37 +00:00
Justin Starry
c95f6c4b83 Remove spammy invalid rpc log (#13100) 2020-10-23 07:05:29 +00:00
Justin Starry
8b0242a5d8 Allow nodes to advertise a different rpc address over gossip (#13053) 
* Allow nodes to advertise a different rpc address over gossip

* Feedback
 2020-10-22 03:31:48 +00:00
Michael Vines
959880db60 Remove unused pubkey::Pubkey imports 2020-10-21 19:08:13 -07:00
Michael Vines
17c391121a Run codemod --extensions rs Hash::new_rand solana_sdk:#️⃣:new_rand 2020-10-21 19:08:13 -07:00
Michael Vines
7bc073defe Run codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand 2020-10-21 19:08:13 -07:00
Ryo Onodera
0776fa05c7 Add ledger-tool dead-slots and improve purge a lot (#13065) 
* Add ledger-tool dead-slots and improve purge a lot

* Reduce batch size...

* Add --dead-slots-only and fixed purge ordering
 2020-10-21 17:45:21 +00:00
Ryo Onodera
efdb560e97 Various clean-ups before assert adjustment (#13006) 
* Various clean-ups before assert adjustment

* oops
 2020-10-21 10:26:20 +09:00
Michael Vines
6858950f76 Remove frozen ABI modules from solana-sdk 2020-10-20 16:11:30 -07:00
Ryo Onodera
c0675968b1 Support Debug Bank (#13017) 2020-10-21 01:05:45 +09:00
Trent Nelson
3b3f7341fa validator: Activate RPC before halting on slot 2020-10-20 02:09:07 +00:00
behzad nouri
75d62ca095 improves threads' utilization in processing gossip packets (#12962) 
ClusterInfo::process_packets handles incoming packets in a thread_pool:
https://github.com/solana-labs/solana/blob/87311cce7/core/src/cluster_info.rs#L2118-L2134

However, profiling runtime shows that threads are not well utilized and
a lot of the processing is done sequentially.

This commit redistributes the work done in parallel. Testing on a gce
cluster shows 20%+ improvement in processing gossip packets with much
smaller variations.
 2020-10-19 19:03:38 +00:00
Ryo Onodera
54517ea454 Follow up to persistent tower with tests and API cleaning (#12350) 
* Follow up to persistent tower

* Ignore for now...

* Hard-code validator identities for easy reasoning

* Add a test for opt. conf violation without tower

* Fix compile with rust < 1.47

* Remove unused method

* More move of assert tweak to the asser pr

* Add comments

* Clean up

* Clean the test addressing various review comments

* Clean up a bit
 2020-10-19 16:37:03 +09:00
Ryo Onodera
fd8ec27fe8 Another some tower logging improvements (#12940) 2020-10-16 14:44:07 +09:00
behzad nouri
48283161c3 passes through feature-set to gossip requests handling (#12878) 
* passes through feature-set to down to gossip requests handling
* takes the feature-set from root_bank instead of working_bank
 2020-10-15 20:54:21 +00:00
Tyera Eulberg
d008dfb7ad Bump spl-memo and spl-token versions (#12917) 2020-10-15 18:23:41 +00:00
behzad nouri
05cf15a382 implements DataBudget using atomics (#12856) 2020-10-15 11:33:58 +00:00
Ryo Onodera
a44e4d386f Better tower logs for SwitchForkDecision and etc (#12875) 
* Better tower logs for SwitchForkDecision and etc

* nits

* Update comment
 2020-10-15 18:30:33 +09:00
Trent Nelson
c26512255d RPC: Add metrics for TX encoding 2020-10-14 12:24:11 -06:00
sakridge
1f1eb9f26e Add separate push queue to reduce push lock contention (#12713) 2020-10-13 18:10:25 -07:00
Michael Vines
247228ee61 Implementation-defined RPC server errors are now accessible to client/ users 2020-10-13 10:05:44 -07:00
Michael Vines
649fe6d3b6 get_vote_accounts: access HashMap directly instead of turning it into an iterator 2020-10-13 04:12:10 +00:00
Michael Vines
8a119c1483 Bump reqwest/rayon to get past cargo audit 2020-10-11 18:51:09 +00:00
Michael Vines
c5c8da1ac0 Expose all rewards (fees, rent, voting and staking) in RPC getConfirmedBlock and the cli 2020-10-09 21:54:13 -07:00
Ryo Onodera
1f4bcf70b0 Fix various ledger-tool error due to no builtins (#12759) 
* Fix various ledger-tool error due to no builtins

* Add missing file...
 2020-10-09 12:19:36 -06:00
carllin
c879e7c1ad Fix fee mismatch on snapshot deserialize (#12697) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-10-08 23:44:41 -07:00
Dan Albert
2c5f83c264 Add new internal accounts (#12740) 
Co-authored-by: publish-docs.sh <maintainers@solana.com>
 2020-10-09 00:48:32 +00:00
Josh
8f5431551e Store program logs in blockstore / bigtable (TransactionWithStatusMeta) (#12678) 
* introduce store program logs in blockstore / bigtable

* fix test, transaction logs created for successful transactions

* fix test for legacy bincode implementation around log_messages

* only api nodes should record logs

* truncate transaction logs to 100KB

* refactor log truncate for improved coverage
 2020-10-08 12:06:15 -07:00
Michael Vines
11df2e2236 Bump version to v1.5.0 2020-10-08 04:51:36 +00:00
behzad nouri
b5faa11f73 removes invalid/outdated pending push messages early (#12555) 
In CrdsGossipPush::new_push_messages:
https://github.com/solana-labs/solana/blob/972619edb/core/src/crds_gossip_push.rs#L211-L228
we already have paid the cost of looking-up the label in crds table and
checking the hash value and wallclock only to find out that in some
cases the value is invalid or is outdated. So might as well remove the
value here rather than wait for the next call to
purge_old_pending_push_messages:
https://github.com/solana-labs/solana/blob/972619edb/core/src/crds_gossip_push.rs#L372
 2020-10-07 18:29:20 +00:00
Jack May
41ad3dd8f0 Remove skip-no-mangle entirely (#12696) 2020-10-07 09:13:48 -07:00
Trent Nelson
e35889542b RPC: Support base64 encoded transactions 
Defaults to base58
 2020-10-06 22:41:06 -06:00
Trent Nelson
7f67d36777 RPC: Check encoded transaction size before decoding 2020-10-06 22:41:06 -06:00
behzad nouri
a5c6a78f6d filters out inactive nodes from push options (#12674) 
* filters out inactive nodes from push options

https://github.com/solana-labs/solana/pull/12620
patched the DDOS issue with nodes which go offline:
https://github.com/solana-labs/solana/issues/12409

However, offline nodes still see (much lesser) traffic spike, likely
because no origins are pruned from their bloom filter in active set:
https://github.com/solana-labs/solana/blob/aaf3790d8/core/src/crds_gossip_push.rs#L276-L286
and so multiple nodes push redundant duplicate messages to them
simultaneously:
https://github.com/solana-labs/solana/blob/aaf3790d8/core/src/crds_gossip_push.rs#L254-L255

This commit will filter out inactive peers from potential push targets
entirely. To mitigate eclipse attacks, staked nodes are retried
periodically.

* uses current timestamp in test/crds_gossip
 2020-10-06 13:48:32 +00:00
Ryo Onodera
026e7de819 Show commit in --version and ledger-tool's log (#12636) 
* Show commit in `--version` and ledger-tool's log

* Another handy hidden env var

* Fix test

* Rename to semver!

* Fix syntax error...
 2020-10-04 00:30:26 +09:00
sakridge
71c469c72b Weight push peers by how long we haven't pushed to them (#12620) 2020-10-02 13:57:26 -07:00
behzad nouri
2c669f65f1 limits number of threads in core/tests/crds_gossip.rs (#12615) 
crds_gossip tests start large networks, which with large thread-pools
will exhaust system resources, causing failures in ci tests:
https://buildkite.com/solana-labs/solana/builds/31953

The commit limits size of thread-pools in the test.
 2020-10-02 16:55:44 +00:00
Michael Vines
75b621160e Add GetConfirmedBlocksWithLimit RPC method 2020-10-01 22:56:17 -07:00
Michael Vines
f41a73d76a Expose validator cli arguments for pubsub buffer tuning 2020-10-01 20:30:40 -07:00
behzad nouri
1866521df6 retains hash value of outdated responses received from pull requests (#12513) 
pull_response_fail_inserts has been increasing:
https://cdn.discordapp.com/attachments/478692221441409024/759096187587657778/pull_response_fail_insert.png
but for outdated values which fail to insert:
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds_gossip_pull.rs#L332-L344
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds.rs#L104-L108
are not recorded anywhere, and so the next pull request may obtain the
same redundant payload again, unnecessary taking bandwidth.

This commit holds on to the hashes of failed-inserts for a while, similar
to purged_values:
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds_gossip_pull.rs#L380
and filter them out for the next pull request:
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds_gossip_pull.rs#L204
 2020-10-01 00:39:22 +00:00
Tyera Eulberg
c31a34fbcb Include post balance information for rewards (#12598) 
* Include post balance information for rewards

* Add post-balance to stored Reward struct

* Handle extended Reward in bigtable

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-30 17:57:06 -06:00
Trent Nelson
3c7b9c2938 Move remaining nonce utils from runtime to SDK 2020-09-30 05:45:42 +00:00
behzad nouri
537bbde22e builds crds filters in parallel (#12360) 
Based on run-time profiles, the majority time of new_pull_requests is
spent building bloom filters, in hashing and bit-vec ops.

This commit builds crds filters in parallel using rayon constructs. The
added benchmark shows ~5x speedup (4-core machine, 8 threads).
 2020-09-29 23:06:02 +00:00
Tyera Eulberg
96a7d4dbd8 Query BigTable for block time if does not exist in blockstore (#12560) 2020-09-29 21:39:36 +00:00