2497 Commits

Author SHA1 Message Date
7711cd74c3 partition_from_pubkey (#22430)
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)"

This reverts commit 8aa3d690b5.

* partition_from_pubkey
2022-01-13 17:02:42 -06:00
e291342c4a Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)" (#22488)
This reverts commit 8aa3d690b5.
2022-01-13 13:06:39 -06:00
b82d71d22a Inline DEFAULT_SNAPSHOT_VERSION constant string (#22487) 2022-01-13 18:19:15 +00:00
2756abce39 More serde snapshot cleanup (#22449) 2022-01-13 09:20:20 -06:00
9c3144e286 Refactor serde snapshot's "future" to "newer" (#22431) 2022-01-13 07:12:09 -06:00
2370e61431 Perf: Store deserialized sysvars in the sysvars cache (#22455)
* Perf: Store deserialized sysvars in sysvars cache

* add bench
2022-01-13 05:36:21 +00:00
7171c95bdd Refactor: move sysvar cache to new module 2022-01-12 12:35:28 -07:00
3ca16de851 Only examine explicit tx accounts for rent state (#22441)
* Add failing test

* Fix: only examine accounts explicitly included in a tx
2022-01-11 20:23:51 -07:00
c45dde6164 Handle accounts data size changes due to rent-collected accounts (#22412) 2022-01-11 17:20:28 -06:00
157f165a3d Bump memmap2 from 0.5.1 to 0.5.2 (#22414)
* Bump memmap2 from 0.5.1 to 0.5.2

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.1 to 0.5.2.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.1...v0.5.2)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-11 15:08:13 -07:00
637e366b18 Prevent rent-paying account creation (#22292)
* Fixup typo

* Add new feature

* Add new TransactionError

* Add framework for checking account state before and after transaction processing

* Fail transactions that leave new rent-paying accounts

* Only check rent-state of writable tx accounts

* Review comments: combine process_result success behavior; log and metrics before feature activation

* Fix tests that assume rent-exempt accounts are okay

* Remove test no longer relevant

* Remove native/sysvar special case

* Move metrics submission to report legacy->legacy rent paying transitions as well
2022-01-11 11:32:25 -07:00
a49ef49f87 optimizes ReadOnlyAccountsCache LRU eviction implementation (#22403)
ReadOnlyAccountsCache is using a background thread, table scan and sort
to implement LRU eviction policy:
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L66-L73
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L186-L191
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L222

DashMap internally locks each shard when accessed; so a table scan in
the background thread can create a lot of lock contention.

This commit adds an index-list queue containing cached keys in the order
that they are accessed. Each hash-map entry also includes its index into
this queue.
When an item is first entered into the cache, it is added to the end of
the queue. Also each time an entry is looked up from the cache it is
moved to the end of queue. As a result, items in the queue are always
sorted in the order that they have last been accessed. When doing LRU
eviction, cache entries are evicted from the front of the queue.
Using index-list, all queue operations above are O(1) with low overhead
and so above achieves an efficient implementation of LRU cache eviction
policy.
2022-01-11 17:25:28 +00:00
8b66625c95 convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
04a2c19c87 Bump memmap2 from 0.5.0 to 0.5.1 (#22409)
* Bump memmap2 from 0.5.0 to 0.5.1

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.0 to 0.5.1.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.0...v0.5.1)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-10 11:14:56 -07:00
9bc2592da1 Use lazy_rent_collection directly (#22410) 2022-01-10 10:47:08 -06:00
28275a33d6 Bump tempfile from 3.2.0 to 3.3.0 (#22401)
* Bump tempfile from 3.2.0 to 3.3.0

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-09 12:26:08 -07:00
3b4aad9df1 bank: fix executor cache metrics 2022-01-09 04:22:39 +00:00
eaa52bc935 Bump zstd from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1 (#22388)
* Bump zstd from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/compare/0.9.1...v0.9.2)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-08 22:21:56 +00:00
d8ec8ca2e6 Bump blake3 from 1.2.0 to 1.3.0 (#22373)
* Bump blake3 from 1.2.0 to 1.3.0

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.2.0...1.3.0)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-08 20:28:04 +00:00
d90d5ee9b6 Refactor Rent::due() with RentDue enum (#22346) 2022-01-08 09:03:46 -06:00
4a9f4e2505 improve multi executor cache addition 2022-01-08 03:47:23 -07:00
ad3cb0bc93 --amend 2022-01-08 08:19:27 +00:00
4ce48307bb bank: prime new executor cache entry use-counts 2022-01-08 08:19:27 +00:00
6d76db1de5 bank: Add executors cache metrics 2022-01-07 16:53:27 -07:00
c2389fc209 removes CowCachedExecutors (#22343)
Copy-on-write semantics for cached executors can be implemented by a
simple Arc<CachedExecutors> as opposed to CowCachedExecutors:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L244-L247

This will also avoid the need for double locking as in:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3490-L3491
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3525-L3526
2022-01-07 14:06:29 +00:00
52d12cc802 Add runtime support for address table lookups (#22223)
* Add support for address table lookups in runtime

* feedback

* feedback
2022-01-07 11:59:09 +08:00
f1e2598baa Retain executor cache counts (#22322) 2022-01-06 08:56:00 -08:00
e0c091a9f4 factor out code in do_shrink_slot_stores (#22306) 2022-01-06 10:49:24 -06:00
100293c4b5 refactor get_store_for_shrink (#22307) 2022-01-06 10:49:01 -06:00
390ef0fbcd Consolidate process instruction execution timings to own struct 2022-01-06 03:56:46 -07:00
72fc6096a0 Use saturating_add_assign macro 2022-01-06 03:56:46 -07:00
b25e4a200b Add execute metrics 2022-01-06 03:56:46 -07:00
7d32909e17 move ExecuteTimings from runtime::bank to program_runtime::timings 2022-01-06 03:56:46 -07:00
12e160269e cache executors on failed transactions (#22308) 2022-01-05 22:09:03 -08:00
ab13e39518 Only sum accounts data len from non-zero lamport accounts (#22309) 2022-01-05 23:23:37 +00:00
f2ed6f09ee Skip updating already cached executors if unmodified 2022-01-05 16:08:10 -07:00
bb3a1b6b31 Add zk_token_sdk_enabled feature to gate Zk Token proof program and sol_zk_token_elgamal_op syscalls 2022-01-05 11:57:37 -08:00
9f63493789 Refactor: Remove KeyedAccounts (2) (#22274)
* Adds InstructionContext::get_signers().
Improves error messages when modifying borrowed accounts.

* Removes keyed_accounts from InvokeContext tests.

* Removes keyed_accounts from message_processor.rs

* Removes keyed_accounts from bank.rs

* Removes keyed_accounts from bpf serialization.
2022-01-05 09:39:37 +01:00
45458e7139 Refactor: Improve type safety and readability of transaction execution (#22215)
* Refactor Bank::load_and_execute_transactions

* Refactor: improve type safety of TransactionExecutionResult

* Add enum for extra type safety in execution results

* feedback
2022-01-05 10:15:15 +08:00
1460f00e0f Consume from AccountsDataMeter (#21994) 2022-01-04 10:00:21 -06:00
2b5e00d36d Limit number of accounts that a transaction can lock (#22201) 2022-01-04 14:25:23 +08:00
73e6038986 Refactor: Remove KeyedAccount from program runtime (#22226)
* Makes error handling in BorrowedAccount optional.
Adds BorrowedAccount ::get_rent_epoch().
Exposes InstructionContext::get_index_in_transaction().
Turns accounts and account_keys into pinned boxed slices.

* Introduces "unsafe" to InvokeContext::push().

* Turns &TransactionContext into &mut TransactionContext in InvokeContext.

* Push and pop InstructionContext in InvokeContext.
Makes test_process_cross_program and test_native_invoke symmetric.
Removes the borrow check from test_invoke_context_verify.

* Removes keyed_accounts from prepare_instruction()

* Removes usage of invoke_stack.

* Removes keyed_accounts from program-test.

* Removes caller_write_privileges.

* Removes keyed_accounts from BPF parameter (de-)serialization.
2022-01-03 23:30:56 +01:00
672fed04cb Bump serde from 1.0.132 to 1.0.133 (#22233)
* Bump serde from 1.0.132 to 1.0.133

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.132 to 1.0.133.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.132...v1.0.133)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-03 22:19:39 +00:00
04b76eb066 Use AtomicUsize for next_bucket_to_flush (#22095) 2022-01-03 10:35:35 -06:00
fb62407232 AcctIdx: appendvecid: u32 (#21842) 2022-01-03 10:35:17 -06:00
a2a7e91ad6 set secp256k1 cost similar to sigverify 2021-12-31 17:45:50 -06:00
d743c2917c re-calibrate limit based on mainnet data, see issue #21917 2021-12-31 17:45:50 -06:00
e529d03c11 Cleanup #22182 (#22205)
* Turns compute_units_consumed of ProcessInstructionResult into a &mut parameter.

* Removes second nesting level from test_process_instruction_compute_budget().

* Makes test_process_cross_program and test_native_invoke symmetric.

* Unifies test_process_cross_program(), test_native_invoke() and test_process_instruction_compute_budget() into test_process_instruction().
2021-12-31 17:55:27 +01:00
d06e6c7425 Count compute units even when transaction errors (#22182) 2021-12-30 21:21:42 -05:00
edb20d6909 Splits index of InstructionAccount into index_in_transaction and index_in_caller. (#22165) 2021-12-30 15:46:36 +01:00