Commit Graph

18284 Commits

Author SHA1 Message Date
1f136de294 (LedgerStore) Report perf metrics for RocksDB deletes (#24138)
#### Summary of Changes
This PR enables perf metrics reporting for RocksDB deletes.
Samples are reported under "blockstore_rocksdb_write_perf" with op=delete
The sampling rate is still controlled by env arg SOLANA_METRICS_ROCKSDB_PERF_SAMPLES_IN_1K
and its default to 10 (meaning we report 10 in 1000 perf samples).
2022-04-08 00:18:05 -07:00
b84521d47d (LedgerStore) Report perf metrics for RocksDB write batch (#24061)
#### Summary of Changes
This PR enables perf metrics reporting for RocksDB write-batches.
Samples are reported under "blockstore_rocksdb_write_perf" with op=write_batch

Its cf_name tag is set to "write_batch" as well as each write-batch could include multiple column families.

The sampling rate is still controlled by env arg SOLANA_METRICS_ROCKSDB_PERF_SAMPLES_IN_1K
and its default to 10 (meaning we report 10 in 1000 perf samples).
2022-04-08 00:17:51 -07:00
1dd63631c0 Add high level overview comments on ledger_cleanup_service (#24184) 2022-04-08 00:49:21 -05:00
e105547c14 tvu and tpu timeout on joining its microservices (#24111)
* panic when test timeout

* nonblocking send when when droping banks

* debug log

* timeout for tvu

* unused varaible

* timeout for tpu

* Revert "debug log"

This reverts commit da780a3301.

* add timeout const

* fix typo

* Revert "nonblocking send when when droping banks".
I will create another pull request for this.

This reverts commit 088c98ec0f.

* Update core/src/tpu.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/tpu.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/tvu.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/tvu.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/validator.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-04-07 20:20:13 -05:00
fbe5e51a16 Move duplicate-block proposal (#24167) 2022-04-07 17:30:31 -06:00
4dd3987451 Reset onLogs subscriptions when websocket disconnects 2022-04-07 15:45:35 -07:00
781094edb2 providing clarity on airdrop amount constraints (#24115)
* providing clarity on airdrop amount constraints

This change is in response to a review of a PR in the `solana-program-library` found here: https://github.com/solana-labs/solana-program-library/pull/3062

* replaced static limits with info on how to find them

* removed trailing whitespace
2022-04-07 16:35:13 -06:00
c27150b1a3 reserialize_bank_fields_with_hash (#23916)
* reserialize_bank_with_new_accounts_hash

* Update runtime/src/serde_snapshot.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/serde_snapshot/tests.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/serde_snapshot/tests.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* pr feedback

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
0c2d9194dd Fix typo in solana-program lib.rs (#24170) 2022-04-07 11:23:54 -06:00
a100b32b37 Add test for GetMinimumDelegation stake instruction (#24158) 2022-04-07 11:54:15 -05:00
48d1af01c8 add metrics around rewards (#24160) 2022-04-07 11:44:26 -05:00
f7b2951c79 move around some index metrics to reduce locks (#24161) 2022-04-07 09:43:19 -05:00
42c094739d add test cfg attribute (#24154) 2022-04-07 09:05:20 -05:00
206c3dd402 (LedgerStore) Enable RocksDB Perf metrics reporting for get_bytes and put_bytes (#24066)
#### Summary of Changes
Enable RocksDB Perf metrics reporting for get_bytes and put_bytes.
2022-04-07 00:24:10 -07:00
4f0e887702 (LedgerStore) Report RocksDB perf metrics for Protobuf Columns (#24065)
This PR enables the reporting of both RocksDB read and write perf metrics for ProtobufColumns,
including TransactionStatus and Rewards.
2022-04-07 00:15:00 -07:00
550ca7bf92 compare contents of serialized banks instead of exact file format (#24141)
* compare contents of serialized banks instead of exact file format

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* pr feedback

* get rid of clone

* pr feedback

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
fddd162645 reserialize bank in ahv by first writing to temp file in abs (#23947) 2022-04-06 21:39:26 -05:00
fb67ff14de Remove replica-node crates (#24152) 2022-04-06 16:52:19 -06:00
7ee1edddd1 add transaction update error to geyser plugin interface. (#24140) 2022-04-06 15:41:23 -07:00
afeb1d3cca Bump lru crate (#24150) 2022-04-06 16:18:42 -06:00
efb9cbd8e7 Refactor: Remove trait from stake keyed account (#24148)
Removes trait from StakeAccount.
2022-04-06 22:58:09 +02:00
25304ce485 Inlines verify_rent_exemption() in vote processor. (#24146) 2022-04-06 22:13:06 +02:00
2d1f27ed8e (LedgerStore) Perf Metric for RocksDB Writes (#23951)
#### Summary of Changes
This PR implements the reporting of RocksDB write perf metrics to blockstore_rocksdb_write_perf
based on RocksDB's PerfContext.  The default sample rate is 10 in 1000, and the env arg SOLANA_METRICS_ROCKSDB_PERF_SAMPLES_IN_1K can control the sample rate.
2022-04-06 12:12:38 -07:00
559ee5a843 Explorer: Add Anchor Decoding to Programs/Accounts/Transactions (#23972)
* Add program idl to the Program page
* Add instruction decoding to the Tx page
* Add account decoding to the Account page
2022-04-06 10:22:49 -07:00
cd09390367 reduces gossip crds stats (#24132) 2022-04-06 15:35:25 +00:00
22224127e0 Refactor thin_client::create_client (#24067)
Refactor the thin_client::create_client to take addresses separately instead of as a tuple

Co-authored-by: Bijie Zhu <bijiezhu@Bijies-MBP.cable.rcn.com>
2022-04-06 11:03:38 -04:00
a38bd4acc8 Use LRU in connection-cache (#24109)
Switch to using LRU for connection-cache
2022-04-06 10:58:32 -04:00
c322842257 Replace channel with Mutex<Option> for AccountsPackage (#24013) 2022-04-06 05:47:19 -05:00
07f4a9040a Removes KeyedAccount from tests in stake instruction. (Part 4) (#24124)
* Moves tests from stake state to stake instruction.

* Migrates test_merge.

* Migrates test_merge_self_fails.

* Migrates test_merge_incorrect_authorized_staker.

* Migrates test_merge_invalid_account_data.

* Migrates test_merge_fake_stake_source.

* Migrates test_merge_active_stake.
2022-04-06 12:04:35 +02:00
24cc6c33de (LedgerStore)(Refactor) Move metric reporting functions to a dedicate mod (#24060)
Previously, the metric reporting functions are implemented under LedgerColumnMetric.
However, there're operations like write batch which is issued by the function inside Rocks.

This PR moves reporting functions to its own dedicate mod so that both LedgerColumn and
Rocks can report column perf metrics.
2022-04-05 15:06:17 -07:00
302142bb25 fix typo (#24123) 2022-04-05 15:55:47 -05:00
db23295e1c removes legacy weighted_shuffle and weighted_best methods (#24125)
Older weighted_shuffle is based on a heuristic which results in biased
samples as shown in:
https://github.com/solana-labs/solana/pull/18343
and can be replaced with WeightedShuffle.

Also, as described in:
https://github.com/solana-labs/solana/pull/13919
weighted_best can be replaced with rand::distributions::WeightedIndex,
or WeightdShuffle::first.
2022-04-05 19:19:22 +00:00
4ea59d8cb4 Set drop callback on first root bank (#23999) 2022-04-05 13:02:33 -05:00
2282571493 removes outdated and flaky test_skip_repair from retransmit-stage (#24121)
test_skip_repair in retransmit-stage is no longer relevant because
following: https://github.com/solana-labs/solana/pull/19233
repair packets are filtered out earlier in window-service and so
retransmit stage does not know if a shred is repaired or not.
Also, following turbine peer shuffle changes:
https://github.com/solana-labs/solana/pull/24080
the test has become flaky since it does not take into account how peers
are shuffled for each shred.
2022-04-05 16:02:53 +00:00
eb65ffb779 Optimize replay waking up (#24051)
* timeout for validator exits

* clippy

* print backtrace when panic

* add backtrace package

* increase time out to 30s

* debug logging

* make rpc complete service non blocking

* reduce log level

* remove logging

* recv_timeout

* remove backtrace

* remove sleep

* wip

* remove unused variable

* add comments

* Update core/src/validator.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/validator.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* whitespace

* more whitespace

* fix build

* clean up import

* add mutex for signal senders in blockstore

* remove mut

* refactor: extract add signal functions

* make blockstore signal private

* increase replay wake up channel bounds

* reduce replay wakeup signal bound to 1

* reduce log level

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-04-05 08:57:12 -05:00
4a11fa072f hash_account_with_rent_epoch (#24104) 2022-04-05 08:10:31 -05:00
ba92ba0e06 Zk instructions check length (#24103)
* zk-token-sdk: add a length check before decoding proof instruction

* zk-token-sdk: fix minor spelling

* zk-token-sdk: one-liner for length check

* zk-token-sdk: one-liner fix
2022-04-05 08:40:45 -04:00
2b718d00b0 removes legacy compatibility turbine peers shuffle code 2022-04-05 12:04:12 +00:00
d0b850cdd9 removes turbine peers shuffle patch feature 2022-04-05 12:04:12 +00:00
855801cc95 removes deterministic-shred-seed feature 2022-04-05 12:04:12 +00:00
e051c7c162 Removes KeyedAccount from tests in stake instruction. (Part 3) (#24110)
* Moves test from stake state to stake instruction.

* Migrates test_split_source_uninitialized.

* Migrates test_split_split_not_uninitialized.

* Migrates test_split_more_than_staked.

* Migrates test_split_with_rent.

* Migrates test_split_to_account_with_rent_exempt_reserve.

* Migrates test_split_from_larger_sized_account.

* Migrates test_split_from_smaller_sized_account.

* Migrates test_split_100_percent_of_source.

* Migrates test_split_100_percent_of_source_to_account_with_lamports.

* Migrates test_split_rent_exemptness.
2022-04-05 12:36:01 +02:00
6fb99891f2 stopped "autolock_bot_PR.yml" action file. 2022-04-05 11:41:23 +05:30
41f2fd7fca Implement get_account_with_config (#23997). (#24095) 2022-04-04 22:58:58 +00:00
ee6bb0d5d3 track fec set turbine stats (#23989) 2022-04-04 14:44:21 -07:00
6a7f6585ce persist historical_roots (#24029) 2022-04-04 13:13:11 -05:00
132f08486a remove basically duplicate function (#24107) 2022-04-04 12:55:05 -05:00
997db7637c do simple math without floats 2022-04-04 12:32:00 -05:00
6ba4e870c4 Blockstore should drop signals before validator exit (#24025)
* timeout for validator exits

* clippy

* print backtrace when panic

* add backtrace package

* increase time out to 30s

* debug logging

* make rpc complete service non blocking

* reduce log level

* remove logging

* recv_timeout

* remove backtrace

* remove sleep

* wip

* remove unused variable

* add comments

* Update core/src/validator.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/validator.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* whitespace

* more whitespace

* fix build

* clean up import

* add mutex for signal senders in blockstore

* remove mut

* refactor: extract add signal functions

* make blockstore signal private

* let compiler infer mutex type

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-04-04 11:38:05 -05:00
f8f3edac3c update comment (#24108) 2022-04-04 11:06:01 -05:00
2820b64eb3 roots_original -> historical_roots (#24063) 2022-04-04 09:12:12 -05:00