155 Commits

Author SHA1 Message Date
0ad4757159 plumbing for 'other_slot' in 'update_index' (#23330) 2022-02-25 12:58:08 -06:00
cafc18c3f9 update_index uses ReadableAccount to reduce params (#23305) 2022-02-23 17:01:23 -06:00
6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
d343713f61 Optimize packet dedup (#22571)
* Use bloom filter to dedup packets

* dedup first

* Update bloom/src/bloom.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* fixup

* fixup

* fixup

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-01-19 13:58:20 -08:00
93a7b94507 Add benchmark for vote processing (#22486) 2022-01-14 17:10:17 +08:00
2370e61431 Perf: Store deserialized sysvars in the sysvars cache (#22455)
* Perf: Store deserialized sysvars in sysvars cache

* add bench
2022-01-13 05:36:21 +00:00
ef7f49a21d move AccountInfo to its own file (#21784) 2021-12-11 11:47:05 -06:00
b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
bfdb775ffc Unifies ThisInvokeContext and dyn trait InvokeContext. (#21563) 2021-12-02 18:47:16 +01:00
ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
79d21d6805 add --accounts-index-scan-results-limit-mb to allow scans to abort (#21327)
* ScanConfig -> &ScanConfig

* add --accounts-index-scan-results-limit-mb to allow scans to abort

* feedback
2021-11-19 09:00:19 -06:00
0f69a14247 Add ability to abort scan (#21314) 2021-11-17 13:10:29 -06:00
e540b1cf3c Refactor: Move sdk::process_instruction in program-runtime-crate (#21180)
* Moves the Executor dyn Trait to instruction_processor.rs

* Moves the Logger dyn Trait as well as the ic_msg and ic_logger_msg macros to log_collector.rs,
and moves the stable_log to stable_log.rs

* Moves the ComputeMeter dyn Trait to invoke_context.rs

* Moves the InvokeContext dyn Trait and the ProcessInstructionWithContext type to invoke_context.rs

* Updates cargo files.

* Re-export InvokeContext in program-test

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-11-17 19:35:07 +01:00
f30f3bddbb Refactor: Remove program_id from process_instruction() (#20540)
* Replaces usage of program_id parameter by invoke_context.get_caller()?.

* Removes "pubkey: &Pubkey" parameter from "process_instruction()".
2021-10-10 22:29:18 +02:00
4e65487d2f Refactor: process_instruction() (#20448)
* Adds first_instruction_account parameter to process_instruction().

* Removes InvokeContext::remove_first_keyed_account() from all BPF loaders.

* Removes InvokeContext::remove_first_keyed_account() from all builtin programs.

* Removes InvokeContext::remove_first_keyed_account() from all mock ups.

* Deprecates InvokeContext::remove_first_keyed_account().

* Documents index base of keyed_account_at_index().

* Adds dynamic offset to call sites of "keyed_account_at_index()".
2021-10-08 11:41:07 +02:00
8fee9a2e1a Dont call precompiled programs (#19930) 2021-09-29 06:25:08 +00:00
2d7f036afd Add solana-program-runtime crate (#19438) 2021-08-27 00:30:36 +00:00
7c70f2158b accounts_index_bins to AccountsIndexConfig (#19257)
* accounts_index_bins to AccountsIndexConfig

* rename param bins -> config

* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
0b50bb2b20 Deprecate FeeCalculator returning APIs (#19120) 2021-08-13 09:08:20 -07:00
5fb6b341c4 Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots (#18870)
* Handle cleaning zero-lamport accounts

Handle cleaning zero-lamport accounts in slots higher than the last full
snapshot slot.  This is part of the Incremental Snapshot work.

Fixes #18825
2021-08-12 15:56:08 -05:00
651343688d Use option for account index bucket parameter (#19150) 2021-08-10 11:32:25 -05:00
9616ae0f2c signal to upsert whether reclaims are important (#18900)
* signal to upsert whether reclaims are important

* reclaims_must_be_empty -> previous_slot_entry_was_cached

* UPSERT_RECLAIMS_MUST_BE_EMPTY_FALSE -> UPSERT_PREVIOUS_SLOT_ENTRY_WAS_CACHED_FALSE
2021-08-09 08:58:59 -05:00
41f4973f0d accounts default refactoring stragglers (#19097) 2021-08-06 12:36:42 -05:00
ca37873e16 rework bank::new_with_paths (#19087)
* rework bank::new_with_paths

* missing 1 bench
2021-08-06 09:30:40 -05:00
24207a09ac remove AccountsIndex::default (#19082)
* accounts_db calls AccountsDb::new(bins)

* remove AccountsIndex::default
2021-08-05 11:38:53 -05:00
bde9b4de94 Bank::new -> Bank::new_for_benches (#19063) 2021-08-04 17:30:43 -05:00
d166b9856a Move transaction sanitization earlier in the pipeline (#18655)
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
ccdf93e2b8 featurize_policy_update (#18492) 2021-07-07 13:19:17 -07:00
6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
dbd4dc04b0 ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974) 2021-06-15 15:39:22 -05:00
f558b9b6bf verify bank hash on startup with ledger tool option (#17939) 2021-06-15 11:52:12 -05:00
ccc013e134 Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
269d995832 Make account shrink configurable #17544 (#17778)
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544
2021-06-09 21:21:32 -07:00
98f1b11edd Ancestors to HashSet: From(Vec) (#17447)
* ancestors.large_range_slots becomes HashSet

* Ancestors to HashSet: From(Vec)
2021-05-25 11:01:11 -05:00
0486df02ba don't limit to thread pool when cleaning on startup (#17317) 2021-05-20 14:36:35 -05:00
a9aa533684 replace Ancestors HashMap for performance (#16477)
* replace Ancestors HashMap for performance

* add ancestors module
2021-05-20 10:11:56 -05:00
ed9cbd50f0 move Ancestors to its own module (#17316) 2021-05-19 11:50:34 -05:00
fd88db7339 ancestors::new -> default (#17195) 2021-05-14 15:20:59 -05:00
f39dda00e0 type AccountSecondaryIndexes = HashSet (#17108) 2021-05-10 14:22:48 +00:00
1a9954f85b bank deposit checked_add error (#16917)
* bank deposit checked_add error

* add id

* rename variables

* rename error and metric
2021-04-30 16:22:17 -05:00
5591cd6ef3 .owner -> .owner() (#16757)
* .owner -> .get_owner()

* use
2021-04-23 09:35:09 -05:00
03f7b251b8 ReadableAccount.data returns slice (#16686)
* ReadAbleAccount.data returns slice

* more slice fixup

* more slice

* slice
2021-04-20 16:41:16 -05:00
9dfcb921cf Refactoring: Move KeyedAccounts to InvokeContext (#15410)
Collects all parametric occurrences and the construction of keyed_accounts and puts them into InvokeContext.
2021-04-19 18:48:48 +02:00
d747614b27 Account for possibility of cache flush in load() (#15454)
* Account for possibility of cache flush in load()

* More cleaning

* More cleaning

* Remove unused method and some comment cleaning

* Fix typo

* Make the detected impossible purge race panic()!

* Finally revert to original .expect()

* Fix typos...

* Add assertion for max_root for easier reasoning

* Reframe races with LoadHint as possible opt.

* Fix test

* Make race bug tests run longer for less flaky

* Delay the clone-in-lock slow path even for RPC

* Make get_account panic-free & add its onchain ver.

* Fix rebase conflicts...

* Clean up

* Clean up comment

* Revert fn name change

* Fix flaky test...

* fmt...

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-04-17 00:23:32 +09:00
85eb37fab0 Merge pull request from GHSA-8v47-8c53-wwrc
* Track transaction check time separately from account loads

* banking packet process metrics

* Remove signature clone in status cache lookup

* Reduce allocations when converting packets to transactions

* Add blake3 hash of transaction messages in status cache

* Bug fixes

* fix tests and run fmt

* Address feedback

* fix simd tx entry verification

* Fix rebase

* Feedback

* clean up

* Add tests

* Remove feature switch and fall back to signature check

* Bump programs/bpf Cargo.lock

* clippy

* nudge benches

* Bump `BankSlotDelta` frozen ABI hash`

* Add blake3 to sdk/programs/Cargo.lock

* nudge bpf tests

* short circuit status cache checks

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
6930a77a0f prepare replace Ancestors HashMap for performance (#16476) 2021-04-12 10:51:57 -05:00
3f63ed9a72 removes OrderedIterator and transaction batch iteration order (#16153)
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817

However load_and_execute_transactions is iterating over
  lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.

If iteratorion_order is [1, 2, 0], and i is 0, then:
  lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
  transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).

This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
416312b30b update timings only on outer call (#16275)
* update timings only on outer call

* test & format
2021-03-31 18:02:59 -05:00
5e5b63712b Status cache improvements (#16174) 2021-03-28 19:10:14 -07:00
4bbeb9c033 Remove old feature: simple_capitalization (#15763)
* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit
2021-03-12 11:12:40 +09:00