2497 Commits

Author SHA1 Message Date
be29568318 runtime: checked math for Bank::withdraw 2021-04-23 20:13:35 +00:00
dcf2d84b24 refactor complicated 'len' (#16777) 2021-04-23 18:36:49 +00:00
04e6aebf35 .owner = X -> .set_owner(X) (#16759) 2021-04-23 18:26:33 +00:00
fb0b76c1f3 account.owner = X -> account.set_owner(X) (#16754) 2021-04-23 12:34:22 -05:00
7602911f9a read_only_cache remove updates cache size (#16742) 2021-04-23 12:00:17 -05:00
1cc9a1c0eb log roots range metric (#16636)
* log roots range metric

* rename
2021-04-23 16:09:39 +00:00
96d21335e1 .owner -> .owner() (#16758) 2021-04-23 09:35:55 -05:00
5591cd6ef3 .owner -> .owner() (#16757)
* .owner -> .get_owner()

* use
2021-04-23 09:35:09 -05:00
6c5b7ecf90 account.owner = X -> account.set_owner(X) (#16756) 2021-04-23 09:34:28 -05:00
91be2903da Reduce account index lookups during clean (#16689)
* reduce account index lookups in clean

* rename

* rename enum

* hold locks during removal from zero pubkey list

* merge with zero lamport fix

* tests
2021-04-23 09:33:14 -05:00
fc12841d95 Readonlyaccounts (#16743)
* lamports -> lamports()

* format
2021-04-22 20:04:55 +00:00
739f0698ab clean read_only_cache in the bg (#16722)
* clean read_only_cache in the bg

* reset lru_index

* cleanup

* cleanup

* drop for read only cache

* join bg thread

* reuse existing lru vec

* simplify reserve logic
2021-04-22 13:59:58 -05:00
333998d008 .lamports = <number> -> .set_lamports(<number>) (#16746) 2021-04-22 18:56:47 +00:00
8a6b80095e Set lamports (#16747)
* lamports = -> set_lamports()

* .lamports = X -> .set_lamports(X)
2021-04-22 13:53:06 -05:00
8d9d6b62d9 use cheaper account get (#16682) 2021-04-22 12:54:08 -05:00
123e0bdba7 don't iterate slot_list if zero-lamport (#16678) 2021-04-21 20:38:48 -05:00
69cbad0869 some ReadableAccount changes (#16688)
* some ReadableAccount changes

* deref
2021-04-21 12:20:37 -05:00
189d2121e6 simplify do_shrink_slot_stores, delay/reduce account clone (#16691) 2021-04-21 11:17:38 -05:00
9852572eb9 increase rolling slot width to 4M (#16583) 2021-04-20 21:18:15 -05:00
03f7b251b8 ReadableAccount.data returns slice (#16686)
* ReadAbleAccount.data returns slice

* more slice fixup

* more slice

* slice
2021-04-20 16:41:16 -05:00
36e11998c7 read only accounts cache uses accurate size representation (#16610)
* read ony accounts cache uses accurate size representation

* add comment and test
2021-04-20 09:58:01 -05:00
01786f684e Remove unwrap (#16652) 2021-04-19 20:17:56 -07:00
4aa753ff01 rename threads: 15 char limit (#16625) 2021-04-19 12:16:58 -05:00
9dfcb921cf Refactoring: Move KeyedAccounts to InvokeContext (#15410)
Collects all parametric occurrences and the construction of keyed_accounts and puts them into InvokeContext.
2021-04-19 18:48:48 +02:00
015bc034a5 improve failing assert (#16581) 2021-04-19 08:55:01 -05:00
a911ae00ba clippy 2021-04-18 20:55:02 -07:00
285f3c9d56 Feature-gate hash-based duplicate transaction check 2021-04-16 18:51:18 +00:00
d747614b27 Account for possibility of cache flush in load() (#15454)
* Account for possibility of cache flush in load()

* More cleaning

* More cleaning

* Remove unused method and some comment cleaning

* Fix typo

* Make the detected impossible purge race panic()!

* Finally revert to original .expect()

* Fix typos...

* Add assertion for max_root for easier reasoning

* Reframe races with LoadHint as possible opt.

* Fix test

* Make race bug tests run longer for less flaky

* Delay the clone-in-lock slow path even for RPC

* Make get_account panic-free & add its onchain ver.

* Fix rebase conflicts...

* Clean up

* Clean up comment

* Revert fn name change

* Fix flaky test...

* fmt...

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-04-17 00:23:32 +09:00
17aa45fad1 Remove old way of account hashing
Account data hashing used to use different ways of hashing on different
clusters.  That is no longer the case, but the old code still existed.
This commit removes that old, now used code.

**NOTE** The golden hash values in bank.rs needed to be updated.  Since
the original code that selected the hash algorithm used `if >` instead
of `if >=`, this meant that the genesis block's hash _always_ used the
old hashing method, which is no longer valid.

Validated by running `cargo test` successfully.
2021-04-13 14:42:21 -05:00
f641429056 Remove blake3 from bpf program dependencies (#16506) 2021-04-13 10:06:05 +00:00
85eb37fab0 Merge pull request from GHSA-8v47-8c53-wwrc
* Track transaction check time separately from account loads

* banking packet process metrics

* Remove signature clone in status cache lookup

* Reduce allocations when converting packets to transactions

* Add blake3 hash of transaction messages in status cache

* Bug fixes

* fix tests and run fmt

* Address feedback

* fix simd tx entry verification

* Fix rebase

* Feedback

* clean up

* Add tests

* Remove feature switch and fall back to signature check

* Bump programs/bpf Cargo.lock

* clippy

* nudge benches

* Bump `BankSlotDelta` frozen ABI hash`

* Add blake3 to sdk/programs/Cargo.lock

* nudge bpf tests

* short circuit status cache checks

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
fa83f3bd73 Return sysvars via syscalls (#16422) 2021-04-12 16:04:57 -07:00
54ef065cc8 Use slot 1 for tests' sample store and accounts
Anticipating the removal of the old account hashing way, the Account DB
tests use slot 0 for the sample storages and accounts data.  Slot 0 is a
special case, and so to make future changes not break any tests, update
just the tests here now to use any other slot.

The golden hash values were obtained by running `cargo test` and copying
in the "actual" values from the test output into the `raw_expected`
vector of the test code.

Validated by running `cargo test` successfully.
2021-04-12 17:09:07 -05:00
105a6bfb46 Replace RootsTracker HashSet (#16310)
* Replace RootsTracker HashSet

* use bitvec

* cleanup, add brenchmark test

* test cleanup

* add lots of tests

* get rid of demo

* change warp test constant

* get rid of unused function

* pr feedback

* reorder use

* rework get_all to remove range checks

* add tests, fix bugs
2021-04-12 12:11:33 -05:00
7ba3e710d3 latest_slot uses > current_max (#16374) 2021-04-12 10:52:24 -05:00
6930a77a0f prepare replace Ancestors HashMap for performance (#16476) 2021-04-12 10:51:57 -05:00
dc7030ffaa Allow fork choice to support multiple versions of a slot (#16266) 2021-04-12 01:00:59 -07:00
ac5462e7a2 Merge pull request from GHSA-fmvj-vqp5-qqh9
* Sanitize permissions

* Forbid creating directories under ledger/rocksdb/

* hardened_unpack: Disallow dirs under rocksdb/ in genesis

* hardened_unpack: expand valid genesis entry test coverage

* hardened_unpack: rework old-style bsd directory entry rejection

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
2021-04-10 00:57:32 -06:00
b08cff9e77 Simplify some pattern-matches (#16402)
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
2021-04-08 12:40:37 -06:00
e69f37a464 Remove old function: account_balance_for_capitalization (#16383)
This function currently returns one of its' parameters and is thus useless.
2021-04-07 23:57:40 -05:00
476fd40948 Add program-test tests to bpf programs (#16407) 2021-04-07 01:47:15 -07:00
92f4018b07 Cleanup unsupported sysvars (#16390)
* Cleanup unsupported sysvars

* fix ser description
2021-04-06 00:08:03 -07:00
95dc7b5449 reduce lock contention on latest_root (#16306) 2021-04-05 11:35:14 -05:00
7a997759fa reduce locking on get_rooted_entries (#16304) 2021-04-05 11:26:53 -05:00
0596cf5405 Remove unprocessed transactions from log notifications (#16349) 2021-04-04 23:44:59 +08:00
3996b699dc read only account cache for executable accounts - improve replay (#16150)
* read only account cache

* tests

* clippy

* cleanup

* new file, add tests

* remove copy/paste code from test

* remove dead code

* all loads use cache

* remove stale comments

* add metrics logging for read only cache size

* report read only cache hits and misses

* consistency

* formatting

* rename, add comment

* u64

* better interaction with existing cache

* lru list saved between cleans
2021-04-01 07:16:34 -05:00
3f63ed9a72 removes OrderedIterator and transaction batch iteration order (#16153)
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817

However load_and_execute_transactions is iterating over
  lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.

If iteratorion_order is [1, 2, 0], and i is 0, then:
  lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
  transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).

This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
416312b30b update timings only on outer call (#16275)
* update timings only on outer call

* test & format
2021-03-31 18:02:59 -05:00
f374b35944 Hash stored accounts in bg (#16157)
* lazy calculate account hash

* push to bg thread

* remove deadlock

* logs

* format

* some cleanup on aisle 9

* format, fix up some metrics

* fix test, remove legacy function only there for tests

* cleanup

* remove unused store_hasher

* Switch to crossbeam

* clippy

* format

* use iter()

* rework from feedback

* hash_slot -> slot

* hash(cluster_type)

Co-authored-by: Carl Lin <carl@solana.com>
2021-03-31 15:39:34 -05:00
a5dcee254d get rid of unnecessary data copy (#16214)
* get rid of unnecessary data copy

* preserve rent_epoch
2021-03-31 09:11:39 -05:00