Commit Graph

147 Commits

Author SHA1 Message Date
Jeff Washington (jwash)
2df96cd81e remove type specifics to make funciton more flexible (#19536) 2021-08-31 20:03:42 -05:00
Jeff Washington (jwash)
a655b01700 some basic accounts index refactoring (#19510) 2021-08-30 18:40:10 -05:00
Jeff Washington (jwash)
02b050e0f5 replace AccountsIndex btree with hashmap of 8k bins (#19212) 2021-08-26 18:12:43 -05:00
Jeff Washington (jwash)
7c70f2158b accounts_index_bins to AccountsIndexConfig (#19257)
* accounts_index_bins to AccountsIndexConfig

* rename param bins -> config

* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
Jeff Washington (jwash)
651343688d Use option for account index bucket parameter (#19150) 2021-08-10 11:32:25 -05:00
Jeff Washington (jwash)
47e0d9aa95 plumb more accounts_index bins (#19123) 2021-08-10 05:45:46 -05:00
Jeff Washington (jwash)
db14bcc43b AccountsIndex RefCount() returns u64 instead of atomic (#19088) 2021-08-09 08:59:56 -05:00
Jeff Washington (jwash)
9616ae0f2c signal to upsert whether reclaims are important (#18900)
* signal to upsert whether reclaims are important

* reclaims_must_be_empty -> previous_slot_entry_was_cached

* UPSERT_RECLAIMS_MUST_BE_EMPTY_FALSE -> UPSERT_PREVIOUS_SLOT_ENTRY_WAS_CACHED_FALSE
2021-08-09 08:58:59 -05:00
Jeff Washington (jwash)
8878f526ce rework AccountsIndex traits (#19089) 2021-08-06 08:39:34 -05:00
Jeff Washington (jwash)
24207a09ac remove AccountsIndex::default (#19082)
* accounts_db calls AccountsDb::new(bins)

* remove AccountsIndex::default
2021-08-05 11:38:53 -05:00
Jeff Washington (jwash)
5cf28689e6 accounts_db calls AccountsDb::new(bins) (#19068) 2021-08-05 11:15:26 -05:00
Jeff Washington (jwash)
67788ad206 move AccountsIndex upsert into static WriteAccountMapEntry (#18899)
* rework accounts index to push upsert deeper

* clean up return value of upsert_existing_key

* upsert_existing_key -> update_key_if_exists

* upsert_new_key -> upsert

* upsert_item -> lock_and_update_slot_list

* update_static -> update_slot_list
2021-08-05 08:45:08 -05:00
Jeff Washington (jwash)
087db70df6 add traits required by IsCached (#19066) 2021-08-05 08:43:00 -05:00
Jeff Washington (jwash)
dfe99efa7c introduce AccountsIndex::default_for_tests() (#19067) 2021-08-04 21:58:53 -05:00
sakridge
5a4979f25f Handle 0-lamport account in index generation (#19041)
* Handle 0-lamport account in index generation

* rename duplicate to dirty keys

Co-authored-by: Carl Lin <carl@solana.com>
2021-08-04 23:33:47 +00:00
Jeff Washington (jwash)
0b8d14b0fc move towards account index being dynamically allocated (#19034) 2021-08-04 15:28:35 -05:00
Jeff Washington (jwash)
31a620c42b move towards accounts index being dynamic (#19032) 2021-08-04 09:18:05 -05:00
Jeff Washington (jwash)
6dadc75a6f during range, stop scanning earlier than currently scanning (#18896)
* during range, stop scanning earlier than currently scanning

* refactor for better testing

* clean up a different edge case
2021-07-27 13:40:45 -05:00
Jeff Washington (jwash)
53d8cad206 remove unused return value from account index upsert (#18895) 2021-07-27 08:46:27 -05:00
Jeff Washington (jwash)
181f21529d add Debug trait to T in AccountsIndex (#18702) 2021-07-16 17:05:23 -05:00
Jeff Washington (jwash)
958d27bb0e accounts index scan starts looking in bin where bound would exist (#18613) 2021-07-15 10:26:50 -05:00
Jeff Washington (jwash)
da9fdd785c measure and speed up 2nd half of generate_index (#18503)
* add metrics and speedup storage calcs in generate_index

* add tests, refactor

* rename map variable

* rename another index variable
2021-07-15 10:26:00 -05:00
Jeff Washington (jwash)
bb41cf3461 remove unnecessary generic args to make refactoring easier (#18619) 2021-07-13 12:05:23 -05:00
Jeff Washington (jwash)
0bd8710d34 prevent excess allocation with AccountsIndexIterator (#18605) 2021-07-13 11:11:17 -05:00
Jeff Washington (jwash)
f5ff4b2058 serial insertion of bins into accounts index (#18469) 2021-07-12 12:00:45 -05:00
Jeff Washington (jwash)
f04340b125 move allocation out of critical section and estimate capacity better (#18450) 2021-07-07 15:36:40 -05:00
Jeff Washington (jwash)
49c4e54b28 accounts index insert uses pubkey by value (#18460) 2021-07-07 15:35:35 -05:00
Jeff Washington (jwash)
afd64d27c9 bin accounts index (#18111) 2021-06-28 15:03:57 -05:00
Jeff Washington (jwash)
1f1e54b9d8 reduce pubkey copies on insert path (#18240) 2021-06-28 12:04:29 -05:00
Jeff Washington (jwash)
e06376664b replace account index bulk insert with iterator (#18198) 2021-06-25 14:31:55 -05:00
Jeff Washington (jwash)
f2a2581259 metric for accounts index insertion time (#18202) 2021-06-25 08:36:55 -05:00
dependabot[bot]
9429d0463c chore: bump ouroboros from 0.5.1 to 0.9.3 (#18189)
* chore: bump ouroboros from 0.5.1 to 0.9.3

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.5.1 to 0.9.3.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Api changes

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-25 00:53:54 -06:00
Jeff Washington (jwash)
f9fccdee85 remove unnecessary copies from accounts index code paths (#18196) 2021-06-24 17:56:25 -05:00
Jeff Washington (jwash)
4b314be5bd rework dirty_pubkeys from insert_new_if_missing_into_primary_index (#18200) 2021-06-24 14:52:11 -05:00
sakridge
3b1738c000 Keep track of dirty stores on remove accounts to clean (#17601)
* Keep track of dirty stores on remove accounts to clean

and not zero_lamport key set

* Only dirty when count==0?

* Add another clean
2021-06-23 10:28:35 +02:00
Lijun Wang
ccf6b21bf8 Removed unused purged_slots in purge_older_root_entries (#18102)
Removed unused purged_slots in purge_older_root_entries
2021-06-22 15:27:37 -07:00
Alexander Meißner
6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Jeff Washington (jwash)
0a81c37fce introduce LockMapType for accounts_index (#18021) 2021-06-17 14:41:52 -05:00
carllin
ccc013e134 Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
sakridge
c2191d885d Fix accounts index panic in purge_exact (#17757) 2021-06-14 16:10:26 +02:00
Jeff Washington (jwash)
d4cc975fe9 calculate_capitalization uses hash calculation (#17443)
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math
2021-06-14 08:53:07 -05:00
Jeff Washington (jwash)
58fe1d0764 add min to roots_tracker (#17806) 2021-06-08 08:32:16 -06:00
sakridge
14c52ab018 Skip shrink when it doesn't save anything (#17405) 2021-06-02 09:51:46 +02:00
Jeff Washington (jwash)
8924fbf6a0 RollingBitField: bug fixes and add tests (#17525)
* RollingBitField: bug fixes and add tests

* respond to feedback
2021-05-28 16:33:07 +00:00
sakridge
49402b7d82 Only add keys with multiple index entries to the clean set (#17473) 2021-05-27 16:31:24 +02:00
carllin
d8bc56fa51 Refactor purge_slots_from_cache_and_store() and handle_reclaims() (#17319) 2021-05-24 13:51:17 -07:00
Jeff Washington (jwash)
3f3324231d Hold read lock during startup shrink (#17309)
* hold read lock during account scan of shrink

* rename and improve rusty
2021-05-21 13:59:32 -05:00
Jeff Washington (jwash)
8143ee5502 add AccountsIndex AccountMapsReadLock type (#17365) 2021-05-21 10:02:14 -05:00
Jeff Washington (jwash)
33ab9c4e8d batch insert account_index items in generate_index (#17290) 2021-05-20 10:29:13 -05:00
Jeff Washington (jwash)
32ec8341f9 generate_index inserts ideal initial data (#17247)
* improve insert into map initially

* rework towards single code path

* rename

* update test
2021-05-19 16:21:24 -05:00