Commit Graph

2196 Commits

Author SHA1 Message Date
Jeff Washington (jwash)
f9fccdee85 remove unnecessary copies from accounts index code paths (#18196) 2021-06-24 17:56:25 -05:00
Jeff Washington (jwash)
31ec986ea1 untar in parallel (#18184)
* untar in parallel

* make enum for 'ignore' return value
2021-06-24 17:29:49 -05:00
Jeff Washington (jwash)
4b314be5bd rework dirty_pubkeys from insert_new_if_missing_into_primary_index (#18200) 2021-06-24 14:52:11 -05:00
Jeff Washington (jwash)
77c3ffe4cf add min/max bin size metrics to hash calculation (#18155) 2021-06-23 12:57:05 -05:00
sakridge
3b1738c000 Keep track of dirty stores on remove accounts to clean (#17601)
* Keep track of dirty stores on remove accounts to clean

and not zero_lamport key set

* Only dirty when count==0?

* Add another clean
2021-06-23 10:28:35 +02:00
Lijun Wang
ccf6b21bf8 Removed unused purged_slots in purge_older_root_entries (#18102)
Removed unused purged_slots in purge_older_root_entries
2021-06-22 15:27:37 -07:00
Jeff Washington (jwash)
d3ee73e151 capture sort time in hash calculation (#18118) 2021-06-22 16:02:21 -05:00
Jeff Washington (jwash)
fbc94d84c8 apply 'debug_do_not_add_builtins' to feature activations (#18110) 2021-06-22 14:21:49 -05:00
Jeff Washington (jwash)
d5f9f3b7ce correctly calculate hash_total for hash/lamport calculation (#18144) 2021-06-22 11:36:43 -05:00
sakridge
e808f34b0b Add batch stats (#18096) 2021-06-22 15:23:26 +02:00
Tyera Eulberg
0ab361b828 Use newly stabilized Duration methods/consts (#18101) 2021-06-22 00:11:16 +00:00
Tyera Eulberg
19fe1dd463 Move stake_weighted_timestamp module (#18114)
* Move timestamp module into runtime

* Less public

* Remove unused enum
2021-06-21 16:14:54 -06:00
Jeff Washington (jwash)
2087f5da94 refactor generate_index_for_slot (#17984) 2021-06-21 15:32:50 -05:00
Jeff Washington (jwash)
bf97627021 eliminate flatten and sort in hash calculation (#17802)
* eliminate flatten and sort in hash calculation

* reduce critical section time

* remove now no-longer necessary test code

* conflict with reset bins to 0 pr
2021-06-21 15:32:03 -05:00
Tyera Eulberg
4d46cb686a Fix build: clippy lint 2021-06-21 10:18:21 -07:00
Jeff Washington (jwash)
ae82e4e25b refactor reconstruct_accountsdb_from_fields (#17987) 2021-06-21 10:47:58 -05:00
Alexander Meißner
789f33e8db chore: cargo fmt 2021-06-18 10:42:46 -07:00
Alexander Meißner
6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Jeff Washington (jwash)
0a81c37fce introduce LockMapType for accounts_index (#18021) 2021-06-17 14:41:52 -05:00
Jeff Washington (jwash)
1feaaf009d generate_index timings (#17983) 2021-06-17 13:35:25 -05:00
Jeff Washington (jwash)
20e714b3d0 remove unused code (#17981) 2021-06-17 13:30:46 -05:00
Jeff Washington (jwash)
71796f4951 break out generate index reporting (#17980) 2021-06-16 12:09:17 -05:00
Jeff Washington (jwash)
7ca04d6a86 refactor generate_index process_storage_slot (#17982) 2021-06-16 11:02:34 -05:00
Jeff Washington (jwash)
bb7413c9f9 refactor rebuild_bank_from_snapshots (#17988) 2021-06-16 11:01:52 -05:00
Jeff Washington (jwash)
eee5414c64 avoid unnecessary empty arrays when binning (#17944) 2021-06-16 09:24:59 -05:00
Jeff Washington (jwash)
adc683956f do not use index for verify_bank_hash_and_lamports (#17812) 2021-06-16 09:10:38 -05:00
Jeff Washington (jwash)
55ee3b5f2f set hash bins to 65k (#17912) 2021-06-16 09:09:55 -05:00
Jeff Washington (jwash)
7de79425ce parallel storage -> accounts folder (#17955) 2021-06-16 08:48:24 -05:00
Jeff Washington (jwash)
dbd4dc04b0 ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974) 2021-06-15 15:39:22 -05:00
Jeff Washington (jwash)
f558b9b6bf verify bank hash on startup with ledger tool option (#17939) 2021-06-15 11:52:12 -05:00
Jon Cinque
1b1d34da59 Refactor stake program into solana_program (#17906)
* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt
2021-06-15 18:04:00 +02:00
Jeff Washington (jwash)
5ecb30ff58 nit: rename measure (#17946) 2021-06-15 09:01:11 -05:00
Jeff Washington (jwash)
4d8ffdcc11 remove unused parameters from CalculateHashIntermediate (#17949) 2021-06-15 09:00:43 -05:00
carllin
ccc013e134 Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
Jeff Washington (jwash)
471b34132e add metrics for startup (#17913)
* add metrics for startup

* roll timings up higher

* fix test

* fix duplicate
2021-06-14 17:46:49 -05:00
Jeff Washington (jwash)
7fde9b6ff0 name arguments to help with confusion (#17942) 2021-06-14 16:21:48 -05:00
Jeff Washington (jwash)
62e1b20e56 refactor so hash verify can be done by more callers (#17941) 2021-06-14 16:20:03 -05:00
Jeff Washington (jwash)
e6bbd4b3f0 add metrics to handle_snapshot_requests (#17937) 2021-06-14 15:46:19 -05:00
sakridge
c2191d885d Fix accounts index panic in purge_exact (#17757) 2021-06-14 16:10:26 +02:00
Jeff Washington (jwash)
d4cc975fe9 calculate_capitalization uses hash calculation (#17443)
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math
2021-06-14 08:53:07 -05:00
Jeff Washington (jwash)
2dc6969858 use trait to simplify and consolidate cumulative code (#17852) 2021-06-14 08:43:59 -05:00
Lijun Wang
269d995832 Make account shrink configurable #17544 (#17778)
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544
2021-06-09 21:21:32 -07:00
Jack May
a1fab0c5ca Log more info on runtime account verification errors (#17861) 2021-06-09 16:27:00 -07:00
Justin Starry
050bb5446d Add local cluster tests that broadcast duplicate slots (#13995)
* Add duplicate node local cluster test

* fix clippy

* remove dupe test
2021-06-09 15:01:48 -07:00
Jeff Washington (jwash)
576e3d95f7 refactor: create type (#17818) 2021-06-08 14:49:08 -06:00
Jeff Washington (jwash)
2a9b127029 Pass iterator for SortedStorages::new_with_slots (#17811)
Co-authored-by: Carl Lin <carl@solana.com>
2021-06-08 13:59:50 -06:00
Jeff Washington (jwash)
01ef2a5c4a test: simple CalculateHashIntermediate refactoring (#17813) 2021-06-08 13:54:10 -06:00
Jeff Washington (jwash)
58fe1d0764 add min to roots_tracker (#17806) 2021-06-08 08:32:16 -06:00
Jeff Washington (jwash)
817d48be21 binary search in slices for hashing (#17755) 2021-06-07 12:09:17 -05:00
Jeff Washington (jwash)
b5bb91b50f rework hash calculation to not keep slot and write version (#17685)
* rework hash calculation to not keep slot and write version

* refactor functions and add tests

* always use multiple slot code path
2021-06-07 09:01:16 -05:00