Commit Graph

44 Commits

Author SHA1 Message Date
c1d788880d Limit Rayon threadpool threads (#5871) 2019-09-12 11:39:39 -07:00
4e827af392 Remove unnecessary trailing semicolons (#5636) 2019-08-23 22:47:54 -07:00
0ffe7a9c8f plumb some rent (#5610)
* plumb some rent

* nits

* fixups

* fixups

* fixups
2019-08-23 14:04:53 -07:00
3fc5009ef2 Snapshot pipefitting through the validator cli (#5617)
* Handle 404 errors better

* Snapshot pipefitting through the validator cli

* Add download progress bar

* Log the current entrypoint slot
2019-08-23 13:02:07 -07:00
087c43b9ef Add snapshotting integration test (#5519)
* Add snapshotting integration test

* Update ContactInfo on restart in local cluster nodes
2019-08-21 23:59:11 -07:00
a8b82a0b68 optimize store_accounts (#5557) 2019-08-19 13:00:37 -07:00
58d4e32c97 Remove serialization of future AppendVecs and serialize AccountStorage correctly (#5510) 2019-08-13 16:05:37 -07:00
e30ca01999 Only create more append_vecs when the account number grows (#5454)
We only need many append_vecs if the number of accounts is high,
so only create opportunistic ones as accounts are created.
2019-08-07 16:43:52 -07:00
1dbb5c8647 Deserialize snapshots (#5417)
* Deserialize snapshots
2019-08-05 22:53:19 -07:00
6cb2040a1b Snapshot Packaging Service (#5262)
* Snapshot serialization and packaging
2019-07-31 17:58:10 -07:00
8537da19bb groom accounts_db (#5283) 2019-07-25 22:59:28 -07:00
8a12ed029c make accounts_db own the directory paths (#5230)
* change paths to something accounts_db (the singleton) owns, fixes SIGILL

* fail deserialize if paths don't work
serialize paths, too

* test that paths are populated from a bank snapshot
2019-07-23 13:47:48 -07:00
595017499e accounts_index: RwLock per-account (#5198)
* accounts_index: RwLock per-account

Lots of lock contention on the accounts_index lock,
only take write-lock on accounts_index if we need to insert/remove an
account.
For updates, take a read-lock and then write-lock on the individual
account.

* Remove unneeded enumerate and add comments.
2019-07-20 17:58:39 -07:00
027ebb6670 no more OUT_DIR (#5139)
* no more OUT_DIR

* no more OUT_DIR

* more information about failure
2019-07-17 14:27:58 -07:00
b505a0df22 Throw more threads at hash_internal_state (#5023) 2019-07-16 16:58:30 -07:00
0d27515d09 tmp dirs target to farf (#5079) 2019-07-12 18:28:42 -07:00
08f6de0acd Plumb scan_accounts into accounts_db, adding load from storage (#5029) 2019-07-11 12:16:02 -06:00
727c15ef8a start from random point in fork stores (#5010) 2019-07-10 18:44:49 -07:00
a89589a1d5 Add Measure abstraction over measuring time intervals (#4851)
Allows one to swap in different implementations. This provides
the normal Insant::now() -> .elapsed() path.
2019-06-29 15:34:49 +02:00
66552d7047 Credit-Only Accounts: Cache account balance for thread-safe load/store (#4691)
* Implement CreditOnlyLocks

* Update credit-only atomic on account load

* Update credit-only atomic after bank.freeze_lock; store credits if all credit-only lock references are dropped

* Commit credit-only credits on bank freeze

* Update core to CreditAccountLocks

* Impl credit-only in System Transfer

* Rework CreditAccountLocks, test, and fix bugs

* Review comments: Pass CreditAccountLocks by reference; Tighten up insert block

* Only store credits on completed slot

* Check balance in bench_exchange funding to ensure commit_credits has completed

* Add is_debitable info to KeyedAccount meta to pass into programs

* Reinstate CreditOnlyLocks check on lock_account

* Rework CreditAccountLocks to remove strong_count usage

* Add multi-threaded credit-only locks test

* Improve RwLocks usage

* Review comments: panic if bad things happen; tighter code

* Assert lock_accounts race does not happen

* Revert panic if bad things happen; not a bad thing
2019-06-27 17:25:10 -04:00
9e7f618cff Set proper count value for account stores (#4797)
* set count values for store accounts

* Use AppendVecId type
2019-06-25 07:21:45 -07:00
2562e48b9d Check for non zero count value 2019-06-19 08:47:12 -07:00
44967abd1c update storage len 2019-06-17 22:48:27 -07:00
8b41a5d725 periodically save config in separate folders 2019-06-17 22:48:27 -07:00
7fd879b417 Restart validator nodes from snapshots 2019-06-17 22:48:27 -07:00
9e3758983d Find max root and purge roots below it. (#4645)
* Test for forking accounts

* Find max root and purge roots below it.
2019-06-13 17:35:16 -07:00
0da9ac1a47 Remove unnecessary parameter element (#4666)
* Stop passing pubkey ref unnecessarily

* Cargo.lock
2019-06-12 16:18:27 -06:00
807c69d97c Slimmer implementation of credit-only accounts (#4592)
* Add credit-only debit/data check to verify_instruction

* Store credits and pass to accounts_db

* Add InstructionErrors and tests

* Relax account locks for credit-only accounts

* Collect credit-only account credits before passing to accounts_db to store properly

* Convert System Transfer  accounts to credit-only, and fixup test

* Functionalize collect_accounts to unit test

* Review comments

* Rebase
2019-06-10 20:50:02 -06:00
966b6999d1 Accounts index opt (#4621)
* Add accounts_index bench

* Don't take the accounts index lock unless needed

* Accounts_index remove insert return vec and add capacity stats

* Use hashbrown hashmap for accounts_index
2019-06-10 18:15:39 -07:00
e7129757c9 store_accounts to use try_available (#4523)
* store_accounts to use try_available

* tighter

* clippy
2019-06-04 11:21:12 -07:00
dea663d509 Storage arranged by fork (#4518) 2019-06-03 15:34:32 -07:00
182096dc1a Create bank snapshots (#4244)
* Revert "Revert "Create bank snapshots (#3671)" (#4243)"

This reverts commit 81fa69d347.

* keep saved and unsaved copies of status cache

* fix format check

* bench for status cache serialize

* misc cleanup

* remove appendvec storage on purge

* fix accounts restore

* cleanup

* Pass snapshot path as args

* Fix clippy
2019-05-30 21:31:35 -07:00
b5324063f1 Use thread pools for rayon par_iter (#4473)
* Use thread pools for rayon par_iter

* address review comments

* cleanup
2019-05-29 17:16:36 -07:00
9843c3a5cb Restrict transaction fee payers to system accounts (#4198)
automerge
2019-05-24 13:06:55 -07:00
cfe5afd34c _id => _pubkey variable renaming (#4419)
* wallet: rename *_account_id to *_account_pubkey

* s/from_id/from_pubkey/g

* s/node_id/node_pubkey/g

* s/stake_id/stake_pubkey/g

* s/voter_id/voter_pubkey/g

* s/vote_id/vote_pubkey/g

* s/delegate_id/delegate_pubkey/g

* s/account_id/account_pubkey/g

* s/to_id/to_pubkey/g

* s/my_id/my_pubkey/g

* cargo fmt

* s/staker_id/staker_pubkey/g

* s/mining_pool_id/mining_pool_pubkey/g

* s/leader_id/leader_pubkey/g

* cargo fmt

* s/funding_id/funding_pubkey/g
2019-05-23 23:20:04 -07:00
81fa69d347 Revert "Create bank snapshots (#3671)" (#4243)
This reverts commit abf2b300da.
2019-05-09 19:27:27 -07:00
abf2b300da Create bank snapshots (#3671)
* Be able to create bank snapshots

* fix clippy

* load snapshot on start

* regenerate account index from the storage

* Remove rc feature dependency

* cleanup

* save snapshot for slot 0
2019-05-09 19:27:06 -07:00
71f9b44687 Add Confirmations parameter to RPC Subscriptions (#4154)
* Add optional depth parameter to pubsub, and store in subscriptions

* Pass bank_forks into rpc_subscription; add method to check depth before notify and impl for account subscriptions

* Impl check-depth for signature subscriptions

* Impl check-depth for program subscriptions

* Plumb fork id through accounts

* Use fork id and root to prevent repeated account notifications; also s/Depth/Confirmations

* Write tests in terms of bank_forks

* Fixup accounts tests

* Add pubsub-confirmations tests

* Update pubsub documentation
2019-05-06 08:31:50 -06:00
cb528af4e2 fix accounts_db storage.reset() (#4094)
* fix accounts_db storage.reset()

* fix compilation errors, remove unused, fix test_accounts_grow() failure
2019-05-01 09:27:13 -07:00
c969975fde lockfree storage (#3963) 2019-04-24 11:51:57 -05:00
5406d82d89 fix race in get_exclusive_storage() and load() (#3941)
* Revert "Revert "account storage is not in sync with the index after gc (#3914)" (#3936)"

This reverts commit 4f47fc00bc.

* fix get_exclusive_storage

* clippy
2019-04-23 09:56:36 -07:00
4f47fc00bc Revert "account storage is not in sync with the index after gc (#3914)" (#3936)
This reverts commit 101d6b92ee.
2019-04-22 17:14:41 -07:00
101d6b92ee account storage is not in sync with the index after gc (#3914)
* account storage is not in sync with the index after gc

* builds

* clippy fmt

* test

* purge dead forks on store

* rm println

* also fixed count_stores

* comments
2019-04-20 20:50:50 -05:00
f73d38739a Split AccountsDB from Accounts (#3808)
Split AccountsDB from Accounts
2019-04-16 13:32:22 -07:00