gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
13,726 Commits 81 Branches 357 Tags
105a6bfb469e2d91486173e7e3ccecd8a678f407
Commit Graph

76 Commits

Author SHA1 Message Date
Jeff Washington (jwash)
105a6bfb46 Replace RootsTracker HashSet (#16310) 
* Replace RootsTracker HashSet

* use bitvec

* cleanup, add brenchmark test

* test cleanup

* add lots of tests

* get rid of demo

* change warp test constant

* get rid of unused function

* pr feedback

* reorder use

* rework get_all to remove range checks

* add tests, fix bugs
 2021-04-12 12:11:33 -05:00
Jeff Washington (jwash)
7ba3e710d3 latest_slot uses > current_max (#16374) 2021-04-12 10:52:24 -05:00
Jeff Washington (jwash)
6930a77a0f prepare replace Ancestors HashMap for performance (#16476) 2021-04-12 10:51:57 -05:00
Jeff Washington (jwash)
95dc7b5449 reduce lock contention on latest_root (#16306) 2021-04-05 11:35:14 -05:00
Jeff Washington (jwash)
7a997759fa reduce locking on get_rooted_entries (#16304) 2021-04-05 11:26:53 -05:00
Jeff Washington (jwash)
8a3135d17b Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
Greg Fitzgerald
edd159e7d7 Fix typos (#15721) 2021-03-05 07:16:53 +00:00
Greg Fitzgerald
2463cc1e6a Fix typos (#15610) 2021-03-02 06:36:49 -08:00
carllin
e4d0d4bfae Fix pubkey refcount for shrink + clean (#14987) 2021-02-04 12:44:19 -08:00
sakridge
e5225b7e68 Add ref count from storage (#15078) 2021-02-04 09:11:05 -08:00
carllin
2970b59853 Don't load all accounts into memory for capitalization check (#14957) 
* Don't load all accounts into memory for capitalization check
 2021-02-03 15:00:42 -08:00
carllin
72f10f5f29 Aggregate purge and shrink metrics (#14763) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-27 01:39:47 -08:00
sakridge
0d32a0e0f4 Partial clean (#14800) 
* Revert "Revert "Partial accounts clean (#14652)" (#14777)"

This reverts commit ad2e10e17b.

* Remove squashed uncleaned keys
 2021-01-24 09:50:19 -08:00
sakridge
96b27ca0cb Remove zero_lamport_pubkeys reference (#14808) 2021-01-23 18:31:55 +00:00
sakridge
424bb797a6 Speed up generate_index (#14792) 2021-01-23 08:05:05 -08:00
carllin
c77461e428 Remove unnecesary flushes in previous roots (#14596) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-23 04:02:44 -08:00
sakridge
ad2e10e17b Revert "Partial accounts clean (#14652)" (#14777) 
This reverts commit ca7914aa3d.
 2021-01-22 18:05:49 -08:00
sakridge
ca7914aa3d Partial accounts clean (#14652) 
Clean less keys by tracking the two cases:
* Touched keys per slot in uncleaned_keys derived from
accounts delta hash operation.
* Set of keys with any zero-lamport updates.
 2021-01-21 11:01:53 -08:00
carllin
5f14f45282 More generic accounts purge functions (#14595) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-17 20:31:03 -08:00
carllin
6dfad0652f Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Ryo Onodera
c9df6134fa Save 7G mem on mainnet fixing AccIndex overalloc. (#14435) 2021-01-06 09:48:59 +09:00
carllin
5affd8aa72 Add secondary indexes (#14212) 2020-12-31 18:06:03 -08:00
sakridge
baa9602411 Add shrink paths (#14238) 2020-12-21 21:33:37 -08:00
Michael Vines
7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
carllin
955b99cf69 Fix roots_and_ref_count (#13827) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-11-26 13:09:20 -08:00
carllin
791fb17437 Prevent scans on unrooted slots from seeing partial clean (#13628) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-11-20 13:01:04 -08:00
carllin
6276360468 Prevent scans from seeing root updates/clean (#13464) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-11-16 17:23:11 -08:00
joshua-maros
a8a77614fe Fix potential undefined behavior (#13555) 
* Switch to ouroboros 0.5.1

* Update other lock files
 2020-11-13 18:12:41 +09:00
Michael Vines
959880db60 Remove unused pubkey::Pubkey imports 2020-10-21 19:08:13 -07:00
carllin
e6b821c392 Finer grained AccountsIndex locking (#12787) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-10-21 17:05:27 -07:00
carllin
64c4861e04 Fix error in max root calculation (#12661) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-10-03 15:18:58 -07:00
carllin
06f84c65f1 Fix rooted accounts cleanup, simplify locking (#12194) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-09-28 16:04:46 -07:00
carllin
00b36e6db2 Fix ref-count for multiple stores to the same pubkey in a slot, fixes zero lamport purge detection (#12462) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-09-25 18:54:48 -07:00
sakridge
832d47317e Move clean accounts to background service (#10898) 2020-07-02 22:25:17 -07:00
Ryo Onodera
50f7ed80c8 Dont skip eager rent collect across gapped epochs (#10206) 
* Dont skip eager rent collect across gapped epochs

* Adjust style and comment

* Adjust ascii chart and comment a bit

* Moar assert

* Relax the partition_count assert for completeness

* Tweak comment...

* tweak a bit

* Add gating logic

* Address reviews

* small formatting

* Clarify the code by replacing auto_generated...

* small formatting

* small formatting

* small formatting

* small formatting

* Narrow down conditional compilation scope
 2020-06-25 23:11:33 +09:00
Ryo Onodera
dfe72d5242 Optimize stale slot shrinking for previously cleaned roots (#10099) 
* Prioritize shrinking of previously cleaned roots

* measure time of stale slot shrinking

* Disable shrink for test

* shrink: budgeting, store count, force for snapshot

* Polish implementation and fix tests

* Fix ci..

* Clean up a bit

* Further polish implementation and fix/add tests

* Rebase fixes

* Remove unneeded Default for AccountStorageEntry

* Address review comments

* More cleanup

* More cleanup
 2020-06-12 14:51:43 +09:00
Kristofer Peterson
e23340d89e Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445) 
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
 2020-06-09 09:38:14 +09:00
Ryo Onodera
1eb40c3fe0 Introduce eager rent collection (#9527) 
* Switch AccountsIndex.account_maps from HashMap to BTreeMap

* Introduce eager rent collection

* Start to add tests

* Avoid too short eager rent collection cycles

* Add more tests

* Add more tests...

* Refacotr!!!!!!

* Refactoring follow up

* More tiny cleanups

* Don't rewrite 0-lamport accounts to be deterministic

* Refactor a bit

* Do hard fork, restore tests, and perf. mitigation

* Fix build...

* Refactor and add switch over for testnet (TdS)

* Use to_be_bytes

* cleanup

* More tiny cleanup

* Rebase cleanup

* Set Bank::genesis_hash when resuming from snapshot

* Reorder fns and clean ups

* Better naming and commenting

* Yet more naming clarifications

* Make prefix width strictly uniform for 2-base partition_count

* Fix typo...

* Revert cluster-dependent gate

* kick ci?

* kick ci?

* kick ci?
 2020-05-13 16:22:14 +09:00
carllin
3442f36f8a Repair alternate versions of dead slots (#9805) 
Co-authored-by: Carl <carl@solana.com>
 2020-05-05 14:07:21 -07:00
Ryo Onodera
9918539229 Introduce type alias Ancestors (#9699) 
* Introduce type alias AncestorList

* Rename AncestorList => Ancestors
 2020-04-27 11:07:03 +09:00
Ryo Onodera
2623c71ed3 Use type aliases/resulting var names consistently (#9060) 2020-03-26 13:08:56 +09:00
sakridge
4b397d15b3 Accounts cleanup service and perf improvements (#8799) 
* Use atomic for ref count instead of taking rwlock

* Accounts cleanup service

* Review comments
 2020-03-23 08:50:23 -07:00
Ryo Onodera
952cd38b7b Avoid early clean and bad snapshot by ref-counting (#8724) 
* Avoid early clean and bad snapshot by ref-counting

* Add measure

* Clean ups

* clean ups
 2020-03-13 14:14:37 +09:00
Ryo Onodera
d86103383a Do periodic inbound cleaning for rooted slots (#8436) 
* Do periodic inbound compaction for rooted slots

* Add comment

* nits

* Consider not_compacted_roots in cleanup_dead_slot

* Renames in AccountsIndex

* Rename to reflect expansion of removed accounts

* Fix a comment

* rename

* Parallelize clean over AccountsIndex

* Some niceties

* Reduce locks and real chunked parallelism

* Measure each step for sampling opportunities

* Just noticed par iter is maybe lazy

* Replace storage scan with optimized index scan

* Various clean-ups

* Clear uncleared_roots even if no updates
 2020-03-03 14:57:25 +09:00
Tyera Eulberg
ab361a8073 Rename KeypairUtil to Signer (#8360) 
automerge
 2020-02-20 13:28:55 -08:00
Ryo Onodera
58727463e1 Remove needless last_root for better reclaims (#8148) 
* Restore last_root to fix unintended storage delete

* Remove last_root thing altogether

* Remove unneeded test...
 2020-02-13 08:19:53 +09:00
François Garillot
f016c9a669 Maintenance : simplify a few patterns, remove unneeded dependencies (#8137) 
* Simplify a few pattern matches

* Removing unneeded dependencies, upgrading internal version #s

 net-shaper: Removing log, semver, serde_derive
 bench-tps: Removing serde, serde_derive
 banking-bench: Removing solana
 ledger-tool: Removing bincode, serde, serde_derive
 librapay: Removing solana, language_e2e_tests
 log-analyzer: Removing log, semver, serde_derive
 exchange: Removing solana
 core: Removing crc, memmap, symlink, untrusted
 perf: Removing serde_derive
 genesis: Removing hex, serde_derive
 sdk-c: Removing sha2
 sys-tuner: Removing semver
 bench-exchange: Removing bincode, bs58, env_logger, serde, serde_derive, untrusted, ws
 btc_spv_bin: Removing serde_json
 btc_spv: Removing chrono
 bpf_loader: Removing serde
 ledger: Removing dlopen, dlopen_derive, serde_derive
 move_loader: Removing byteorder, libc, language_e2e_tests
 ownable: Removing serde, serde_derive
 client: Removing rand
 archiver-utils: Removing rand_chacha
 validator: Removing serde_json, tempfile
 param_passing_dep: Removing solana
 failure: Removing log
 vest: Removing log
 vote-signer: Removing bs58, serde
 local-cluster: Removing symlink
 keygen: Removing rpassword
 install: Removing bs58, log
 upload-perf: Removing log
 runtime: Removing serde_json
 stake: Removing rand

* Add modified Cargo.lock

* fixup! Simplify a few pattern matches

* fixup! Simplify a few pattern matches
 2020-02-06 10:02:38 -07:00
Jack May
83718a3b3e Cleanup runtime use syntax (#8002) 2020-01-28 17:03:20 -08:00
Jack May
7c92bf15e2 Only return accounts that have changed since the bank's parent (#7520) 
* Only return accounts that have changed since the bank's parent

* add test
 2019-12-16 23:24:18 -08:00
sakridge
0aa4dc904e Accounts index updates, remove pubkey when dead account (#7408) 2019-12-11 11:11:31 -08:00