Commit Graph

1464 Commits

Author SHA1 Message Date
Jeff Washington (jwash)
3a56a56d69 StoredMetaWriteVersion (#17293) 2021-05-17 23:58:36 +00:00
Ryo Onodera
bcbe155575 Minor test cleanup and comments (#17283) 2021-05-18 07:22:43 +09:00
Jeff Washington (jwash)
d461aba6a4 don't try to build secondary indexes if we don't need them (#17246) 2021-05-17 17:21:15 -05:00
Jeff Washington (jwash)
9d6837c904 fix comment (#17286) 2021-05-17 14:06:25 -05:00
Jeff Washington (jwash)
d1db5448b9 hold lock to speed up insert (#17194)
* hold lock to speed up insert

* add tests
2021-05-17 11:58:33 -05:00
Jeff Washington (jwash)
3745e0babc minor refactoring of AccountsIndex in preparation of bigger things (#17237) 2021-05-14 15:27:10 -05:00
Jeff Washington (jwash)
fd88db7339 ancestors::new -> default (#17195) 2021-05-14 15:20:59 -05:00
Jeff Washington (jwash)
c6f7867aa1 log mismatch (#17209) 2021-05-14 09:50:11 -05:00
Brooks Prumo
76dfacca61 Add documentation for BankForks::prune_non_rooted() (#17213) 2021-05-14 07:47:48 -05:00
Tyera Eulberg
27004f1b76 Return error for excluded secondary-index keys (#17193)
* Add runtime helpers to check secondary indexes for key

* Add custom rpc error

* Check secondary-index key inclusion in rpc

* Clone complete AccountSecondaryIndexes into rpc to avoid bank query
2021-05-13 21:04:21 +00:00
Jeff Washington (jwash)
3dbc7744ab metrics for generating index time (#17192)
* metrics for generating index time

* update metrics to include scan time
2021-05-13 14:32:19 -05:00
carllin
239ab8799c Remove bloat from secondary indexes (#17048) 2021-05-12 15:29:30 -07:00
Lijun Wang
9c42a89a43 Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Jeff Washington (jwash)
7d96f78821 include/exclude keys on account secondary index (#17110)
* AccountSecondaryIndexes.include/exclude

* use normal scan if key is not indexed

* add a test to ask for a scan for an excluded secondary index

* fix up cli args
2021-05-11 22:06:22 +00:00
Jeff Washington (jwash)
f39dda00e0 type AccountSecondaryIndexes = HashSet (#17108) 2021-05-10 14:22:48 +00:00
Jeff Washington (jwash)
a6a1355b80 fix secondary index test (#17111) 2021-05-10 08:54:30 -05:00
Ryo Onodera
c616c34825 Add comment for another unspoken eager collection subtlety (#17137) 2021-05-10 22:44:46 +09:00
Jeff Washington (jwash)
225ec00ec4 add comment (#17059) 2021-05-06 15:17:41 -05:00
Jeff Washington (jwash)
dc0429f5e6 add metric for assumption (#17061) 2021-05-06 15:04:13 -05:00
steviez
edc2ab3e48 Test account index and store alignment (#17038)
* Use ReclaimResult::Default() instead of building subtypes

* Add test to ensure account_db store and index are aligned
2021-05-06 14:13:44 -05:00
Ryo Onodera
d19526e6c2 Dump rent_collector/inflation with ledger-tool cap (#17069) 2021-05-06 19:29:46 +09:00
Jeff Washington (jwash)
ffbe8906ed get root lock once (#16829) 2021-05-05 15:17:45 -05:00
Brooks Prumo
e6f49a3e79 Clean unrooted unfrozen banks (#16580) (#17003)
Cleanup pubkeys when an unrooted, unfrozen bank is dropped.  This is a
continuation of PR #16911.
2021-05-05 15:02:02 -05:00
Jeff Washington (jwash)
ab7c96aa81 insert accounts in parallel when building initial index (#17040)
* insert accounts in parallel when building initial index

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review
2021-05-05 17:08:45 +00:00
Stephen Akridge
3e0fed48e7 Don't recognize temp snapshots as possible snapshots to open 2021-05-05 08:45:03 -07:00
Jeff Washington (jwash)
5786be13a4 flatten_hash_intermediate sets capacity first (#17013)
* flatten_hash_intermediate sets capacity first

* use iterator instead of for
2021-05-05 09:07:05 -05:00
sakridge
d94e4ef7c9 More docs for bank.rs (#17006) 2021-05-03 17:37:18 +00:00
Jeff Washington (jwash)
541aa5ad85 tests: lamports -> lamports() (#16982) 2021-05-03 10:45:54 -05:00
Jeff Washington (jwash)
744ac1535f tests: lamports -> ReadableAccount (#16983) 2021-05-03 10:45:15 -05:00
Jeff Washington (jwash)
1fe131a5bb Readable account trait for stored account meta (#16690)
* ReadableAccount for StoredAccountMeta

* add test
2021-04-30 16:25:02 -05:00
Jeff Washington (jwash)
1a9954f85b bank deposit checked_add error (#16917)
* bank deposit checked_add error

* add id

* rename variables

* rename error and metric
2021-04-30 16:22:17 -05:00
Jeff Washington (jwash)
01308cd890 distribute_rent_to_validators checked_add_lamports unwrap (#16847)
* distribute_rent_to_validators checked_add_lamports unwrap

* make rent disappear on add failure

* add pubkey to message

* update message text

* don't store account that we failed to transfer to

* format
2021-04-30 16:19:20 -05:00
Jeff Washington (jwash)
763c04adf3 lamports = -> .set_lamports() (#16980) 2021-04-30 21:17:05 +00:00
Brooks Prumo
17e6bd579f Clean unrooted dropped banks (#16580) (#16911)
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.

This work was originally based on PR #15106.
2021-04-30 15:34:38 -05:00
Jeff Washington (jwash)
6f88aeac56 tests: .lamports= -> .set_lamports() (#16977)
* .lamports= -> .set_lamports()

* another

* fix

* another
2021-04-30 18:44:15 +00:00
Jeff Washington (jwash)
0af84bb91e lamports= -> .set_lamports() (#16978) 2021-04-30 18:20:54 +00:00
Jeff Washington (jwash)
c4943f3cb0 checked_sub_lamports (#16932) 2021-04-30 09:56:34 -05:00
Jeff Washington (jwash)
63c56b57a9 checked_sub_lamports with unwrap (#16922)
* checked_sub_lamports with unwrap

* avoid unwrap
2021-04-29 15:04:28 -05:00
Jeff Washington (jwash)
c9c94a7ef2 checked_sub_lamports (#16923) 2021-04-29 12:38:21 -05:00
Jeff Washington (jwash)
ca7b36ad8f lamports -> lamports() (#16920) 2021-04-29 10:44:46 -05:00
Jeff Washington (jwash)
23d67e4ac7 stretchy roots tracker (#16830)
* stretchy roots tracker

* rename hash to hash_set in tests

* update comment

* try 2 widths in test

* bool iter

* add assert

* helper function for bitfield insert/remove

* introduce RollingBitFieldTester

* another bool iter replacement

* map cleanup

* map to cloned
2021-04-29 09:11:28 -05:00
Jeff Washington (jwash)
a7070a5ca7 set_lamports() (#16921) 2021-04-28 21:30:06 +00:00
Jeff Washington (jwash)
21c75d9d29 lamports = -> set_lamports (#16919) 2021-04-28 15:39:54 -05:00
Jeff Washington (jwash)
f533d3be77 Write account path impl ReadableAccount (#16779) 2021-04-28 15:29:22 -05:00
Jeff Washington (jwash)
9070191b8b stats on clean for removing roots (#16849)
* stats on clean for removing roots

* rename

* accumulate and swap metrics
2021-04-28 13:24:01 -05:00
Brooks Prumo
77f1ffd84b Collect uncleaned pubkeys from all slots (#16786)
While working on another issue (#16580), the list of uncleaned pubkeys
returned from `remove_uncleaned_slots_and_collect_pubkeys_up_to_slot()`
did not include unrooted slots.  This meant that during cleaning,
unrooted slots would not have their pubkeys cleaned up properly.

Now, return all uncleaned pubkeys, regardless if the slot is rooted or
not.  Additionally, update the tests to have unrooted slots to ensure
this behavior.

This is part two of PR #16879, and originally based on PR #15106.
2021-04-28 10:57:42 -05:00
Jeff Washington (jwash)
a7a671b3aa AccountSharedData.set_executable() (#16881) 2021-04-28 14:07:43 +00:00
Jeff Washington (jwash)
2021255f91 executable() (#16880) 2021-04-28 08:53:54 -05:00
Jeff Washington (jwash)
da3342759b private AccountSharedData.rent_epoch (#16877) 2021-04-28 08:52:20 -05:00
Jeff Washington (jwash)
6381ee38eb reclaims unref accounts from index (#16838) 2021-04-28 08:50:38 -05:00