Commit Graph

2578 Commits

Author SHA1 Message Date
9b74988fc6 calculate_accounts_hash does not throttle threads on startup (#17338) 2021-05-20 10:25:54 -05:00
a9aa533684 replace Ancestors HashMap for performance (#16477)
* replace Ancestors HashMap for performance

* add ancestors module
2021-05-20 10:11:56 -05:00
a544010b03 don't log shrink metrics on first call (#17328)
* don't log shrink metrics on first call

* simplify logic
2021-05-20 00:15:35 +00:00
32ec8341f9 generate_index inserts ideal initial data (#17247)
* improve insert into map initially

* rework towards single code path

* rename

* update test
2021-05-19 16:21:24 -05:00
b5302e7619 add calc_stored_meta_us metric (#17318) 2021-05-19 12:29:41 -05:00
c20b27bc8f shink all in parallel on startup (#17308) 2021-05-19 12:15:24 -05:00
ed9cbd50f0 move Ancestors to its own module (#17316) 2021-05-19 11:50:34 -05:00
0781fe1b4f Upgrade Rust to 1.52.0 (#17096)
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script

* upgrade to Rust 1.52.1

* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
4788976517 rework slot list update (#17232) 2021-05-19 08:48:46 -05:00
7fe24c455c simplify generate_index (#17292) 2021-05-19 08:11:41 -05:00
827355a6b1 Create solana-rpc crate and move subscriptions (#17320)
* Move non_circulating_supply to runtime

* Add solana-rpc crate and move max_slots

* Move subscriptions to solana-rpc

* Single use statements
2021-05-19 00:54:28 -06:00
a3c0833a1c tests: cleanup test (#17305)
* tests: cleanup test

* 2 more replacements
2021-05-18 18:08:37 -05:00
75335b4f58 datapoint for verify_snapshot_bank (#17306) 2021-05-18 12:14:43 -05:00
2175fc0625 calculate account infos before lock (#17245) 2021-05-18 08:06:19 -05:00
3a56a56d69 StoredMetaWriteVersion (#17293) 2021-05-17 23:58:36 +00:00
bcbe155575 Minor test cleanup and comments (#17283) 2021-05-18 07:22:43 +09:00
d461aba6a4 don't try to build secondary indexes if we don't need them (#17246) 2021-05-17 17:21:15 -05:00
9d6837c904 fix comment (#17286) 2021-05-17 14:06:25 -05:00
d1db5448b9 hold lock to speed up insert (#17194)
* hold lock to speed up insert

* add tests
2021-05-17 11:58:33 -05:00
3745e0babc minor refactoring of AccountsIndex in preparation of bigger things (#17237) 2021-05-14 15:27:10 -05:00
fd88db7339 ancestors::new -> default (#17195) 2021-05-14 15:20:59 -05:00
c6f7867aa1 log mismatch (#17209) 2021-05-14 09:50:11 -05:00
76dfacca61 Add documentation for BankForks::prune_non_rooted() (#17213) 2021-05-14 07:47:48 -05:00
27004f1b76 Return error for excluded secondary-index keys (#17193)
* Add runtime helpers to check secondary indexes for key

* Add custom rpc error

* Check secondary-index key inclusion in rpc

* Clone complete AccountSecondaryIndexes into rpc to avoid bank query
2021-05-13 21:04:21 +00:00
3dbc7744ab metrics for generating index time (#17192)
* metrics for generating index time

* update metrics to include scan time
2021-05-13 14:32:19 -05:00
239ab8799c Remove bloat from secondary indexes (#17048) 2021-05-12 15:29:30 -07:00
9c42a89a43 Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
7d96f78821 include/exclude keys on account secondary index (#17110)
* AccountSecondaryIndexes.include/exclude

* use normal scan if key is not indexed

* add a test to ask for a scan for an excluded secondary index

* fix up cli args
2021-05-11 22:06:22 +00:00
f39dda00e0 type AccountSecondaryIndexes = HashSet (#17108) 2021-05-10 14:22:48 +00:00
a6a1355b80 fix secondary index test (#17111) 2021-05-10 08:54:30 -05:00
c616c34825 Add comment for another unspoken eager collection subtlety (#17137) 2021-05-10 22:44:46 +09:00
225ec00ec4 add comment (#17059) 2021-05-06 15:17:41 -05:00
dc0429f5e6 add metric for assumption (#17061) 2021-05-06 15:04:13 -05:00
edc2ab3e48 Test account index and store alignment (#17038)
* Use ReclaimResult::Default() instead of building subtypes

* Add test to ensure account_db store and index are aligned
2021-05-06 14:13:44 -05:00
d19526e6c2 Dump rent_collector/inflation with ledger-tool cap (#17069) 2021-05-06 19:29:46 +09:00
ffbe8906ed get root lock once (#16829) 2021-05-05 15:17:45 -05:00
e6f49a3e79 Clean unrooted unfrozen banks (#16580) (#17003)
Cleanup pubkeys when an unrooted, unfrozen bank is dropped.  This is a
continuation of PR #16911.
2021-05-05 15:02:02 -05:00
ab7c96aa81 insert accounts in parallel when building initial index (#17040)
* insert accounts in parallel when building initial index

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review
2021-05-05 17:08:45 +00:00
3e0fed48e7 Don't recognize temp snapshots as possible snapshots to open 2021-05-05 08:45:03 -07:00
5786be13a4 flatten_hash_intermediate sets capacity first (#17013)
* flatten_hash_intermediate sets capacity first

* use iterator instead of for
2021-05-05 09:07:05 -05:00
d94e4ef7c9 More docs for bank.rs (#17006) 2021-05-03 17:37:18 +00:00
541aa5ad85 tests: lamports -> lamports() (#16982) 2021-05-03 10:45:54 -05:00
744ac1535f tests: lamports -> ReadableAccount (#16983) 2021-05-03 10:45:15 -05:00
1fe131a5bb Readable account trait for stored account meta (#16690)
* ReadableAccount for StoredAccountMeta

* add test
2021-04-30 16:25:02 -05:00
1a9954f85b bank deposit checked_add error (#16917)
* bank deposit checked_add error

* add id

* rename variables

* rename error and metric
2021-04-30 16:22:17 -05:00
01308cd890 distribute_rent_to_validators checked_add_lamports unwrap (#16847)
* distribute_rent_to_validators checked_add_lamports unwrap

* make rent disappear on add failure

* add pubkey to message

* update message text

* don't store account that we failed to transfer to

* format
2021-04-30 16:19:20 -05:00
763c04adf3 lamports = -> .set_lamports() (#16980) 2021-04-30 21:17:05 +00:00
17e6bd579f Clean unrooted dropped banks (#16580) (#16911)
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.

This work was originally based on PR #15106.
2021-04-30 15:34:38 -05:00
6f88aeac56 tests: .lamports= -> .set_lamports() (#16977)
* .lamports= -> .set_lamports()

* another

* fix

* another
2021-04-30 18:44:15 +00:00
0af84bb91e lamports= -> .set_lamports() (#16978) 2021-04-30 18:20:54 +00:00