2583 Commits

Author SHA1 Message Date
Jeff Washington (jwash)
c7b986220e Accounts::new_with_config_for_tests (#19104) 2021-08-09 09:01:12 -05:00
Jeff Washington (jwash)
1951fe4e80 accounts default stragglers (#19101) 2021-08-09 09:00:30 -05:00
Jeff Washington (jwash)
db14bcc43b AccountsIndex RefCount() returns u64 instead of atomic (#19088) 2021-08-09 08:59:56 -05:00
Jeff Washington (jwash)
9616ae0f2c signal to upsert whether reclaims are important (#18900)
* signal to upsert whether reclaims are important

* reclaims_must_be_empty -> previous_slot_entry_was_cached

* UPSERT_RECLAIMS_MUST_BE_EMPTY_FALSE -> UPSERT_PREVIOUS_SLOT_ENTRY_WAS_CACHED_FALSE
2021-08-09 08:58:59 -05:00
Brooks Prumo
fd937548a0 Move SnapshotArchiveInfo and friends into its own module (#19114) 2021-08-08 07:57:06 -05:00
Brooks Prumo
00890957ee Add snapshot_utils::bank_from_latest_snapshot_archives() (#18983)
While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots.  A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify.  Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled.  Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`.  And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats.  This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
Jeff Washington (jwash)
7923c26939 AccountsDb::new_with_config_for_tests (#19103) 2021-08-06 17:18:16 -05:00
Jeff Washington (jwash)
f771063275 get rid of Accounts new and default (#19100) 2021-08-06 15:52:27 -05:00
Jeff Washington (jwash)
216a1b3d74 remove Bank::default() (#19099) 2021-08-06 13:10:13 -05:00
Jeff Washington (jwash)
c12289fd1b remove AccountsDb::new() from public api (#19098) 2021-08-06 13:07:50 -05:00
Jeff Washington (jwash)
41f4973f0d accounts default refactoring stragglers (#19097) 2021-08-06 12:36:42 -05:00
Jeff Washington (jwash)
ca37873e16 rework bank::new_with_paths (#19087)
* rework bank::new_with_paths

* missing 1 bench
2021-08-06 09:30:40 -05:00
Jeff Washington (jwash)
8878f526ce rework AccountsIndex traits (#19089) 2021-08-06 08:39:34 -05:00
sakridge
592013eaf4 Clean within shrink_all_slots (#19042)
Co-authored-by: Carl Lin <carl@solana.com>
2021-08-05 23:26:38 -07:00
Jeff Washington (jwash)
0028442e14 rework defaults and construction of accountsdb, accounts, bank (#19083)
* rework defaults and construction of accountsdb, accounts, bank

* merge issues
2021-08-05 17:27:13 -05:00
Jeff Washington (jwash)
e368f10973 add _for_tests to new_no_wallclock_throttle (#19086) 2021-08-05 14:50:25 -05:00
Jeff Washington (jwash)
a9014ceceb Bank::default_for_tests() (#19084) 2021-08-05 11:53:29 -05:00
Jeff Washington (jwash)
24207a09ac remove AccountsIndex::default (#19082)
* accounts_db calls AccountsDb::new(bins)

* remove AccountsIndex::default
2021-08-05 11:38:53 -05:00
Jeff Washington (jwash)
5cf28689e6 accounts_db calls AccountsDb::new(bins) (#19068) 2021-08-05 11:15:26 -05:00
Jeff Washington (jwash)
67788ad206 move AccountsIndex upsert into static WriteAccountMapEntry (#18899)
* rework accounts index to push upsert deeper

* clean up return value of upsert_existing_key

* upsert_existing_key -> update_key_if_exists

* upsert_new_key -> upsert

* upsert_item -> lock_and_update_slot_list

* update_static -> update_slot_list
2021-08-05 08:45:08 -05:00
Jeff Washington (jwash)
bf16b0517c add _for_tests to setup_bank_and_vote_pubkeys (#19060) 2021-08-05 08:43:35 -05:00
Jeff Washington (jwash)
087db70df6 add traits required by IsCached (#19066) 2021-08-05 08:43:00 -05:00
Jeff Washington (jwash)
14361906ca for all tests, bank::new -> bank::new_for_tests (#19064) 2021-08-05 08:42:38 -05:00
Jeff Washington (jwash)
dfe99efa7c introduce AccountsIndex::default_for_tests() (#19067) 2021-08-04 21:58:53 -05:00
sakridge
5a4979f25f Handle 0-lamport account in index generation (#19041)
* Handle 0-lamport account in index generation

* rename duplicate to dirty keys

Co-authored-by: Carl Lin <carl@solana.com>
2021-08-04 23:33:47 +00:00
Jeff Washington (jwash)
bde9b4de94 Bank::new -> Bank::new_for_benches (#19063) 2021-08-04 17:30:43 -05:00
Jeff Washington (jwash)
3280ae3e9f add validator option --accounts-db-skip-shrink (#19028)
* add validator option --accounts-db-skip-shrink

* typo
2021-08-04 17:28:33 -05:00
Brooks Prumo
68cc71409e Do not shell out for tar (#19043)
When making a snapshot archive, we used to shell out and call `tar -S`
for sparse file support.  The tar crate supports sparse files, so no
need to do this anymore.

Fixes #10860
2021-08-04 17:07:55 -05:00
Brooks Prumo
a1112254a5 Fix wrong old snapshot archives getting purged (#19061)
I introduced a bug where old snapshot archives were incorrectly purged.
Instead of purged to oldest, I was purged the newest...

The fix is to add a `reverse()` in the purge logic, and I've added a
test to catch this bug in the future.

Fixes #19057
2021-08-04 16:42:42 -05:00
Jeff Washington (jwash)
0b8d14b0fc move towards account index being dynamically allocated (#19034) 2021-08-04 15:28:35 -05:00
Jeff Washington (jwash)
1ed12a07ab introduce Bank::new_for_tests (#19062) 2021-08-04 15:06:57 -05:00
Brooks Prumo
ca14475085 Add incremental_snapshot_archive_interval_slots to SnapshotConfig (#19026)
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
2021-08-04 14:40:20 -05:00
Jeff Washington (jwash)
6a995f5dfd rename to AccountsDb::new_single_for_tests (#19039) 2021-08-04 11:47:11 -05:00
Jeff Washington (jwash)
31a620c42b move towards accounts index being dynamic (#19032) 2021-08-04 09:18:05 -05:00
Oliver
06e08c4840 move package_snapshots to AccountsPackagePre ctors (#18997)
This PR solves #18815. Note that I had to make the snapshot prefix
constants inside `snapshot_utils.rs` public at the crate level in order
to make this work. I'm not sure whether or not introducing this
dependency is entirely good, either way the `snapshot_utils.rs` file
needs a lot of rework so things will move around, I believe this does
the work in the meantime. Any feedback will be greatly appreciated.
2021-08-04 09:03:03 -05:00
Trent Nelson
06a7a9e544 remove superfluous collect()s 2021-08-04 07:21:55 +00:00
Trent Nelson
79b4b83d3c runtime: remove unused allow attrs 2021-08-04 07:21:55 +00:00
dependabot[bot]
c16bf02448 chore: bump serde from 1.0.126 to 1.0.127 (#19010)
* chore: bump serde from 1.0.126 to 1.0.127

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-02 21:16:34 +00:00
Jack May
77861e2d40 Feature disable fees sysvar (#18981)
* Feature disable fees sysvar

* nudge
2021-08-02 00:31:11 +00:00
Trent Nelson
21bc43ed58 nonce: Unify NonceError with SystemError 2021-07-30 14:47:14 -06:00
Brooks Prumo
b05fb87f22 Add test_bank_forks_incremental_snapshot() (#18565)
This commit builds on PR #18504 by adding a test to core/tests/snapshot.rs for Incremental Snapshots. The test adds banks to bank forks in a loop and takes both full snapshots and incremental snapshots at intervals, and validates they are rebuild-able.

For background info about Incremental Snapshots, see #17088.

Fixes #18829 and #18972
2021-07-29 16:46:54 -05:00
Jack May
dfbb0c559b deprecate fees sysvar (#18960) 2021-07-29 10:48:14 -07:00
Jack May
9529284194 Nonce gets blockhash from invoke_context (#18950) 2021-07-29 01:50:20 -07:00
sakridge
84e78316b1 Write helper for multithread update (#18808) 2021-07-29 03:16:36 +02:00
Jack May
0167daa116 cleanup feature consistent_recent_blockhashes_sysvar (#18932) 2021-07-28 10:25:33 -07:00
dependabot[bot]
14f0ce850d chore: bump blake3 from 0.3.8 to 1.0.0 (#18908)
* chore: bump blake3 from 0.3.8 to 1.0.0

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 0.3.8 to 1.0.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/0.3.8...1.0.0)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make versions consistent

* Enable new feature

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-27 17:55:21 -06:00
Jack May
72e374d0f3 Deprecate SysvarRecentBlockhashes (#18875) 2021-07-27 16:34:21 -07:00
Jeff Washington (jwash)
6dadc75a6f during range, stop scanning earlier than currently scanning (#18896)
* during range, stop scanning earlier than currently scanning

* refactor for better testing

* clean up a different edge case
2021-07-27 13:40:45 -05:00
Jeff Washington (jwash)
53d8cad206 remove unused return value from account index upsert (#18895) 2021-07-27 08:46:27 -05:00
carllin
5cabb5bb11 Fix race in remove_unrooted_race and flush_slot_cache (#18785) 2021-07-22 13:49:08 -07:00