gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
13,972 Commits 81 Branches 357 Tags
9218b51de705fb75fc0146670a5911e32d68119c
Commit Graph

266 Commits

Author SHA1 Message Date
Greg Fitzgerald
2463cc1e6a Fix typos (#15610) 2021-03-02 06:36:49 -08:00
carllin
97eaf3c334 Fix finalize_dead_slot_removal() of cached slots decrementing refcount (#15534) 2021-02-26 17:49:37 -08:00
Ryo Onodera
21b43009f6 Introduce ttl eviction for RecycleStore (#15513) 2021-02-25 17:27:27 +09:00
Michael Vines
5df36aec7d Pacify clippy 2021-02-19 20:08:41 -08:00
Jeff Washington (jwash)
7fee0bc69c move hashing functionality to accounts_hash (#15353) 2021-02-17 00:29:50 +00:00
Jeff Washington (jwash)
8367740ff9 compute merkle root on chunks of fanout^3 (#15344) 
* compute merkle root on chunks of fanout^3

* improve test_accountsdb_compute_merkle_root_large
 2021-02-16 17:03:35 -06:00
Jeff Washington (jwash)
d83aeebcca change 2d cumulative calculation (#15346) 2021-02-16 14:24:45 -06:00
Jeff Washington (jwash)
708b8380b7 assert 2 items in stores don't have same slot and version with different values (#15250) 
* assert 2 items in stores don't have same slot and version with different values

* add message to assert
 2021-02-16 18:12:59 +00:00
Jeff Washington (jwash)
53959b4bbc divide account scan into bins for more efficient downstream processing (#15303) 2021-02-15 21:27:39 -06:00
Jeff Washington (jwash)
78a176fc00 rename CumulativeOffset in preparation of 2d support (#15338) 2021-02-15 20:57:05 -06:00
Ryo Onodera
30f18319f2 Track RecycleStore basic stats with needed refactor (#15291) 
* Track RecycleStore basic stats with needed refactor

* Fix another wrong metrics def
 2021-02-15 16:09:17 +09:00
Jeff Washington (jwash)
b8448f4189 merkle root code no longer adds lamports (#15298) 2021-02-12 17:30:14 -06:00
Jeff Washington (jwash)
8944efddf7 combine flatten and hash (#15269) 2021-02-12 11:35:54 -06:00
Jeff Washington (jwash)
eced3776b0 add metrics to hash calculation and consolidate (#15165) 
* add metrics to hash calculation

* add metrics to hash calculation

* create HashStats struct to consolidate hash timing

* formatting
 2021-02-11 20:06:04 -06:00
Jeff Washington (jwash)
eb54e544c2 add div_ceil (#15267) 2021-02-11 20:05:41 -06:00
Jeff Washington (jwash)
59e55f0371 remove hash accumulation with lamports (#15275) 2021-02-11 20:05:22 -06:00
Jeff Washington (jwash)
e59a24d9f9 pass expected capitalization to hash calculation to improve assert msg (#15191) 
* cleanup if

* pass expected capitalization to hash calculation to improve assert message

* fix bank function

* one more level

* calculate_accounts_hash_helper

* add slot to error message

* success
 2021-02-10 14:38:00 -06:00
Jeff Washington (jwash)
600cea274d change store account scan to not use dashmap (#15104) 
* change store account scan to not use dashmap

* add test_accountsdb_de_dup_accounts_from_stores

* add tests

* add test_accountsdb_flatten_hash_intermediate

* add tests

* add sort test

* add test

* clippy

* first_slice -> is_first_slice

* comment

* use partial_cmp
 2021-02-05 21:16:30 -06:00
Jeff Washington (jwash)
fbf9dc47e9 Threadpool2 (#15151) 
* rework thread pool for hash calculation

* rename
 2021-02-05 18:48:16 -06:00
Jeff Washington (jwash)
fabecdc86c use thread pool for non-index hash calculations (#15149) 2021-02-05 19:48:55 +00:00
Jeff Washington (jwash)
f85be6259b sentinel value for zero lamport accounts in hash scanning (#15097) 
* sentinel value for zero lamport accounts in hash scanning

* fix test
 2021-02-05 11:23:48 -06:00
Jeff Washington (jwash)
7d9f5ad525 remove unused arg from function (#15096) 2021-02-04 20:46:30 -06:00
carllin
e4d0d4bfae Fix pubkey refcount for shrink + clean (#14987) 2021-02-04 12:44:19 -08:00
Jeff Washington (jwash)
f49a70e626 borrow storages (#15088) 2021-02-04 11:29:03 -06:00
sakridge
e5225b7e68 Add ref count from storage (#15078) 2021-02-04 09:11:05 -08:00
Jeff Washington (jwash)
e7f53182e2 fix merge conflict (#15085) 2021-02-04 09:20:50 -06:00
Jeff Washington (jwash)
600ff0d915 calculate hash from store instead of index (#15034) 
* calculate hash from store instead of index

* restore update hash in abs
 2021-02-04 09:00:33 -06:00
carllin
2970b59853 Don't load all accounts into memory for capitalization check (#14957) 
* Don't load all accounts into memory for capitalization check
 2021-02-03 15:00:42 -08:00
sakridge
2b9a0311d3 Don't reset accounts if the remove_account comes from a clean (#15022) 
Store may be in-use with a snapshot creation, so don't disturb
it's state.
 2021-02-03 13:06:28 -08:00
sakridge
37aac5a12d Cleanup v1 shrink path (#15009) 
move legacy functions to another impl block
 2021-02-02 13:29:46 -08:00
Jeff Washington (jwash)
22800054e6 streamline calculate_accounts_hash (#14980) 2021-02-02 10:58:19 -06:00
sakridge
63c44bd690 Use helper for count() in accountsDB (#14953) 2021-01-31 09:32:02 -08:00
sakridge
65315fa4c2 Use already-generated key set to populate dirty keys for clean (#14905) 
Don't need to scan the stores again when we already found the key
set of updates per slot. Just insert it earlier.
 2021-01-29 11:44:43 -08:00
sakridge
a53b8558cd Only mmap file from snapshot once (#14815) 2021-01-28 08:15:33 -08:00
carllin
72f10f5f29 Aggregate purge and shrink metrics (#14763) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-27 01:39:47 -08:00
carllin
daddcd361a Snapshots missing slots from accounts cache clean optimization (#14852) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-26 20:08:55 -08:00
Jeff Washington (jwash)
1b85114a9c remove legacy merkle root (#14772) 
* remove legacy merkle root
f78197a

* clippy

* compile error

* borrow error

* derministic results

* clippy

* borrow
 2021-01-25 15:26:08 -06:00
sakridge
0d32a0e0f4 Partial clean (#14800) 
* Revert "Revert "Partial accounts clean (#14652)" (#14777)"

This reverts commit ad2e10e17b.

* Remove squashed uncleaned keys
 2021-01-24 09:50:19 -08:00
Michael Vines
cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
sakridge
424bb797a6 Speed up generate_index (#14792) 2021-01-23 08:05:05 -08:00
carllin
c77461e428 Remove unnecesary flushes in previous roots (#14596) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-23 04:02:44 -08:00
sakridge
ad2e10e17b Revert "Partial accounts clean (#14652)" (#14777) 
This reverts commit ca7914aa3d.
 2021-01-22 18:05:49 -08:00
Jeff Washington (jwash)
18bd0c9a5b speed up merkle calculation (#14710) 2021-01-22 09:21:50 -06:00
sakridge
ca7914aa3d Partial accounts clean (#14652) 
Clean less keys by tracking the two cases:
* Touched keys per slot in uncleaned_keys derived from
accounts delta hash operation.
* Set of keys with any zero-lamport updates.
 2021-01-21 11:01:53 -08:00
carllin
2745b79b74 Parallel cache scan (#14544) 
* Parallel cache scan

* PR comments

* PR comments

Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-20 08:50:17 +00:00
carllin
5f14f45282 More generic accounts purge functions (#14595) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-17 20:31:03 -08:00
carllin
6dfad0652f Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
sakridge
53d65009a0 Lower recycle store count (#14429) 
Too many stores can cause swap usage which
is detrimental to account store times.
 2021-01-05 12:21:17 -08:00
Michael Vines
b6dcdb90e8 Remove dead code 2021-01-02 17:26:33 -08:00
carllin
5affd8aa72 Add secondary indexes (#14212) 2020-12-31 18:06:03 -08:00