186 Commits

Author SHA1 Message Date
mergify[bot]
14baa511f0 Add --warp-epoch and --enable-inflation to ledger-tool cap. (bp #11107) (#11132)
* Add --warp-epoch and --force-inflation to ledger-tool cap. (#11107)

* Add --warp-epoch and --force-inflation to ledger-tool cap.

* Add more arguments

* Address review comments

* Fix message

* Fix various typos...

(cherry picked from commit 3db246f596)

# Conflicts:
#	ledger-tool/src/main.rs

* Fix conflicts..

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-07-20 12:00:56 +00:00
Tyera Eulberg
b0addba2a9 Gate nonce-overwrite change (#11079) 2020-07-15 21:14:28 +00:00
mergify[bot]
d718ab2491 accounts_clean: Convert stack dependency calculation with iterative (#11067) (#11076)
* accounts_clean: Convert stack dependency calculation with iterative

* optimize clean with by creating a reverse-lookup hashset of the affected
keys

* Add dependency bench

reduce bench

* Huge clean

(cherry picked from commit 8bf3a0aa05)

Co-authored-by: sakridge <sakridge@gmail.com>
2020-07-15 15:13:33 +00:00
mergify[bot]
94b2565969 Fix nonce fee_calculator overwrite (#10973) (#10976)
* Add failing test

* Pass fee_calculator to prepare_if_nonce_account; only overwrite in error case

(cherry picked from commit 25228ca957)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-07-09 16:57:34 -06:00
mergify[bot]
94ba700e58 Permit paying oneself (#10337) (#10342)
automerge
2020-05-31 08:50:42 -07:00
Kristofer Peterson
b7a32f01c0 Multi-version snapshot support (#9980)
* Multi-version snapshot support

* rustfmt

* Remove CLI options and runtime support for selection output snapshot version.
Address some clippy complaints.

* Muzzle clippy type complexity warning.

Despite clippy's suggestion, it is not currently possible to create type aliases
for traits and so everything within the 'Box<...>' cannot be type aliased.

This then leaves creating full blown traits, and either implementing
said traits by closure (somehow) or moving the closures into new structs
implementing said traits which seems a bit of a palaver.

Alternatively it is possible to define and use the type alias 'type ResultBox<T> = Result<Box<T>>'
which does seems rather pointless and not a great reduction in complexity but is enough to keep clippy happy.

In the end I simply went with squelching the clippy warning.

* Remove now unused Serialize/Deserialize trait implementations for AccountStorageEntry and AppendVec

* refactor versioned de/serialisers

* rename serde_utils to serde_snapshot

* move call to accounts_db.generate_index() back down to context_accountsdb_from_stream()

* update version 1.1.1 to 1.2.0
remove nested use of serialize_bytes

* cleanups

* Add back measurement of account storage entry serialization.
Remove construction of Vec and HashMap temporaries during serialization.

* consolidate serialisation test cases into serde_snapshot.
clean up leakage of implementation details in serde_snapshot.

* move short term / legacy snapshot code into child module

* add serialize_iter_as_tuple

* preliminary integration of following commit

commit 6d58b73c47294bfb93465d5a83cd2175660b6e6d
Author: Ryo Onodera <ryoqun@gmail.com>
Date:   Wed May 20 14:02:02 2020 +0900

    Confine snapshot 1.1 relic to versioned codepath

* refactored serde_snapshot, rustfmt
legacy accounts_db format now "owns" both leading u64s, legacy bank_rc format has none

* reduce type complexity (clippy)
2020-05-23 02:54:24 +09:00
Kristofer Peterson
58ef02f02b 9951 clippy errors in the test suite (#10030)
automerge
2020-05-15 09:35:43 -07:00
Jack May
eb1acaf927 Remove archiver and storage program (#9992)
automerge
2020-05-14 18:22:47 -07:00
Ryo Onodera
1eb40c3fe0 Introduce eager rent collection (#9527)
* Switch AccountsIndex.account_maps from HashMap to BTreeMap

* Introduce eager rent collection

* Start to add tests

* Avoid too short eager rent collection cycles

* Add more tests

* Add more tests...

* Refacotr!!!!!!

* Refactoring follow up

* More tiny cleanups

* Don't rewrite 0-lamport accounts to be deterministic

* Refactor a bit

* Do hard fork, restore tests, and perf. mitigation

* Fix build...

* Refactor and add switch over for testnet (TdS)

* Use to_be_bytes

* cleanup

* More tiny cleanup

* Rebase cleanup

* Set Bank::genesis_hash when resuming from snapshot

* Reorder fns and clean ups

* Better naming and commenting

* Yet more naming clarifications

* Make prefix width strictly uniform for 2-base partition_count

* Fix typo...

* Revert cluster-dependent gate

* kick ci?

* kick ci?

* kick ci?
2020-05-13 16:22:14 +09:00
Tyera Eulberg
f5b0d13f08 Rpc: add getLargestAccounts endpoint (#9869)
automerge
2020-05-04 16:46:10 -07:00
sakridge
f37f83fd12 Fuzzer test and fixes (#9853) 2020-05-02 08:07:52 -07:00
sakridge
894549f002 Put empty accounts in the accounts list on load (#9840)
Indexing into accounts array does not match account_keys otherwise.
Also enforce program accounts not at index 0
Enforce at least 1 Read-write signing fee-payer account.
2020-05-01 17:23:33 -07:00
Michael Vines
8dfe0affd4 Add incinerator sysvar (#9815) 2020-04-30 22:04:08 -07:00
Ryo Onodera
9918539229 Introduce type alias Ancestors (#9699)
* Introduce type alias AncestorList

* Rename AncestorList => Ancestors
2020-04-27 11:07:03 +09:00
sakridge
e907c0e650 Filter program ids to store (#9721)
automerge
2020-04-26 00:11:37 -07:00
Ryo Onodera
2623c71ed3 Use type aliases/resulting var names consistently (#9060) 2020-03-26 13:08:56 +09:00
Justin Starry
ef3af104ae Return appropriate error for invalid program account (#9047)
automerge
2020-03-25 13:23:05 -07:00
Michael Vines
88ba8439fc Add frozen account support (#8989)
automerge
2020-03-22 11:10:04 -07:00
sakridge
4dd0367136 Rwlock storage opt (#9006)
* Remove unecessary account paths rwlock

* Remove path rwlock in accounts_db and optimize storage critical section
2020-03-22 10:04:03 -07:00
sakridge
2435c3ce0c Add accounts-bench, a benchmark to test the accounts store speed (#8866) 2020-03-17 11:02:07 -07:00
Ryo Onodera
4bbf09f582 Enable conservative out-of-bound snapshot cleaning (#8811)
* Enable conservative out-of-bound snapshot cleaning

* Add tests
2020-03-13 14:44:00 +09:00
Jack May
97c5fb8141 Allow passing of program_ids to programs (#8639) 2020-03-05 10:57:12 -08:00
Trent Nelson
fd00e5cb35 Store FeeCalculator with blockhash in nonce accounts (#8650)
* Copy current state version to v0

* Add `FeeCalculator` to nonce state

* fixup compile

* Dump v0 handling...

Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...

* Create more representative test `RecentBlockhashes`

* Improve CLI nonce account display

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix that last bank test...

* clippy/fmt

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-03-05 07:40:26 -07:00
Jack May
b85d7c1f70 Fix account tests (#8615) 2020-03-04 10:40:41 -08:00
Trent Nelson
1cc7131bb7 Consolidate Nonce state under one struct (#8624)
automerge
2020-03-04 08:51:48 -08:00
Trent Nelson
be0cc0273f SDK: Re-org nonce state module to facilitate versioning (#8603)
automerge
2020-03-03 17:00:39 -08:00
Trent Nelson
90bedd7e06 Split signature throughput tracking out of FeeCalculator (#8447)
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`

Leaving `FeeCalculator` to *only* calculate transaction fees

* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate

* Expose recent `FeeRateGovernor` to clients

* Move `burn()` back into `FeeCalculator`

Appease BPF tests

* Revert "Move `burn()` back into `FeeCalculator`"

This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.

* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`

* Make `FeeRateGovernor`'s `lamports_per_signature` private

* rebase artifacts

* fmt

* Drop 'Recent'

* Drop _with_commitment variant

* Use a more portable integer for `target_signatures_per_slot`

* Add docs for `getReeRateCalculator` JSON RPC method

* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
2020-02-28 13:27:01 -07:00
sakridge
947a339714 Add snapshot hash of full accounts state (#8295)
* Add snapshot hash of full accounts state

* Use normal hashing for the accounts delta state

* Add merkle
2020-02-22 13:46:40 -08:00
Ryo Onodera
d238371b0c Correct missing entry handling to avoid bad warns (#8339)
* Correct missing entry handling to avoid bad warns

* Pass storage entries to AccountStorageSerialize

* Fix CI.....

* Add tests and reorder condition for cheapest first

* Remove unneeded reference
2020-02-21 15:27:55 +09:00
Tyera Eulberg
ab361a8073 Rename KeypairUtil to Signer (#8360)
automerge
2020-02-20 13:28:55 -08:00
Greg Fitzgerald
e8124324ff Support transaction signing by heterogenous lists of keypairs (#8342)
automerge
2020-02-20 12:13:23 -08:00
Michael Vines
1bf2285fa2 ledger-tool: Add print-accounts command 2020-02-14 19:59:48 -07:00
Ryo Onodera
7614af2a45 Verify frozen bank from snapshot by hashing (#8184) 2020-02-11 16:46:33 +09:00
Michael Vines
3ac0192d40 Better surface bank hash verification failures 2020-02-05 11:39:47 -07:00
Jack May
83718a3b3e Cleanup runtime use syntax (#8002) 2020-01-28 17:03:20 -08:00
Ryo Onodera
2c7447b73e Secure sysvars under hash by freezing all strictly (#7892)
* Secure sysvars under hash by freezing all strictly

* Fix hash's non-idempotnet and add new test

* Clean up

* More cleanups
2020-01-24 16:10:32 +09:00
Jack May
023074650f Allow the same account to be passed multiple times to a single instruction (#7795) 2020-01-22 09:11:56 -08:00
Justin Starry
87598c7612 Consolidate tx error counters and update metrics dashboard (#7724)
automerge
2020-01-16 23:26:50 -08:00
sakridge
3b78be83cf Add hash stats information to check hashes between validators (#7780)
automerge
2020-01-14 11:57:29 -08:00
Trent Nelson
9754fc789e Manage durable nonce stored value in runtime (#7684)
* Bank: Return nonce pubkey/account from `check_tx_durable_nonce`

* Forward account with HashAgeKind::DurableNonce

* Add durable nonce helper for HashAgeKind

* Add nonce util for advancing stored nonce in runtime

* Advance nonce in runtime

* Store rolled back nonce account on TX InstructionError

* nonce: Add test for replayed InstErr fee theft
2020-01-10 16:57:31 -07:00
Trent Nelson
7e94cc2cc3 Move nonce into system program (#7645)
automerge
2020-01-03 16:34:58 -08:00
Ryo Onodera
c8fe4043b6 Rename slot_hash => bank_hash in AcoountsDB (#7579)
* Rename slot_hash => bank_hash in AcoountsDB
2019-12-23 10:50:31 +09:00
sakridge
7dec934bb3 Optimize lock_accounts mutex use (#7593)
Use the lock for the whole batch instead of per-tx
Optimize the critical section to pre-generate the keys necessary
before taking the lock.
2019-12-21 10:43:22 -08:00
Ryo Onodera
3c361eb759 Check account hashes in snapshot (#7559)
* Check for incorrect hash value

* Finish up checking for incorrect hash value

* Fix comment a bit

Co-authored-by: sakridge <sakridge@gmail.com>
2019-12-20 09:39:30 +09:00
Trent Nelson
1ffd6b4b4d Add program and runtime support for Durable Transaction Nonces (#6845)
* Rework transaction processing result forwarding

Durable nonce prereq

* Add Durable Nonce program API

* Add runtime changes for Durable Nonce program

* Register Durable Nonce program

* Concise comments and bad math

* Fix c/p error

* Add rent sysvar to withdraw ix

* Remove rent exempt required balance from Meta struct

* Use the helper
2019-12-07 12:54:10 -07:00
Justin Starry
b7d4330dd4 Fail fast if account paths cannot be canonicalized (#7300)
* Canonicalize account paths to avoid symlink issues

* fixes
2019-12-05 21:41:29 -05:00
Sagar Dhawan
4485b978c1 Clean up accounts hash internal state api (#7090) 2019-11-22 08:56:00 -08:00
Parth
9f6a2e51b2 add credit-debit rent handling (#6947)
* add credit-debit rent handling

* add tests

* charge rent for validator account for fee credit

* rent is stored per tx instead of account
2019-11-21 01:57:02 +05:30
Sagar Dhawan
42da1ce4e2 Fix bank hash not changing when no internal state has changed (#7052)
* Fix bank hash not changing when no internal state has changed

* Fix unnecessary call to hash_internal_state

* Add blockhash into the bank_hash

* Add blockhash into the bank_hash and update tests

* Refactor accounts_db slot_hashes

* More clarity in comments

* Add clippy suggestion

* Grammar

* Fix compile after clippy made me break it

* Schooled by clippy
2019-11-19 20:19:43 -08:00
Michael Vines
e7f63cd336 Upgrade to rust 1.39.0 (#6939)
* Upgrade to rust 1.39.0

* 1.39.0 clippy
2019-11-14 12:27:01 -07:00