2166 Commits

Author SHA1 Message Date
Justin Starry
5ca975383c
Remove feature switch for using message hash for already processed check (#18340) 2021-07-01 09:33:55 -05:00
Brooks Prumo
89a3e4f91e
Move SnapshotConfig into its own module (#18331)
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
2021-07-01 08:55:26 -05:00
Trent Nelson
d3a7e22112 sdk: refactor pda generation 2021-06-30 19:01:51 -06:00
Jeff Washington (jwash)
531f1bce78
report on min/max bin stats for accounts index (#18338) 2021-07-01 00:46:59 +00:00
Justin Starry
b08f8bd1b0
Fail simulation if transaction contains duplicate accounts (#18304) 2021-06-30 13:50:55 -05:00
dependabot[bot]
78968d132f
chore: bump log from 0.4.11 to 0.4.14 (#18323)
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-30 18:32:01 +00:00
dependabot[bot]
00d1125e98
chore: bump ouroboros from 0.9.3 to 0.9.5 (#18324)
* chore: bump ouroboros from 0.9.3 to 0.9.5

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.9.3 to 0.9.5.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-30 09:37:55 -06:00
Jeff Washington (jwash)
ce53b84cdc
refactor untar_snapshot_in to push parallelism deeper for further refactoring (#18310) 2021-06-29 18:26:15 -05:00
dependabot[bot]
52fd10ce03
chore: bump byteorder from 1.3.4 to 1.4.3 (#18300)
* chore: bump byteorder from 1.3.4 to 1.4.3

Bumps [byteorder](https://github.com/BurntSushi/byteorder) from 1.3.4 to 1.4.3.
- [Release notes](https://github.com/BurntSushi/byteorder/releases)
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.3.4...1.4.3)

---
updated-dependencies:
- dependency-name: byteorder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Bump expected instruction count

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-29 15:08:04 -06:00
Brooks Prumo
c1b9d40a64
Add snapshot_runtime_info module (#18199)
The `snapshot_runtime_info` module adds the `SnapshotRuntimeInfo` and
`SyncSnapshotRuntimeInfo` types to handle sharing runtime snapshot information
between threads; in particular: AccountsDb, Bank, SnapshotPackagerService, and
snapshot_utils.
2021-06-29 15:13:34 -05:00
dependabot[bot]
9bd02c9570
chore: bump zstd from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0 (#18299)
* chore: bump zstd from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/commits/0.9.0)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-29 11:12:57 -06:00
carllin
47cafb70da
test (#18159) 2021-06-28 18:27:26 -07:00
Jeff Washington (jwash)
afd64d27c9
bin accounts index (#18111) 2021-06-28 15:03:57 -05:00
Jeff Washington (jwash)
1f1e54b9d8
reduce pubkey copies on insert path (#18240) 2021-06-28 12:04:29 -05:00
Jack May
2fbedd834f Don't update if already an executable 2021-06-26 19:19:26 -06:00
Jeff Washington (jwash)
e06376664b
replace account index bulk insert with iterator (#18198) 2021-06-25 14:31:55 -05:00
dependabot[bot]
91576fdd9b
chore: bump blake3 from 0.3.7 to 0.3.8 (#18223)
* chore: bump blake3 from 0.3.7 to 0.3.8

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/0.3.7...0.3.8)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-06-25 11:44:52 -06:00
Jeff Washington (jwash)
f2a2581259
metric for accounts index insertion time (#18202) 2021-06-25 08:36:55 -05:00
dependabot[bot]
9429d0463c
chore: bump ouroboros from 0.5.1 to 0.9.3 (#18189)
* chore: bump ouroboros from 0.5.1 to 0.9.3

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.5.1 to 0.9.3.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Api changes

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-25 00:53:54 -06:00
Jeff Washington (jwash)
f9fccdee85
remove unnecessary copies from accounts index code paths (#18196) 2021-06-24 17:56:25 -05:00
Jeff Washington (jwash)
31ec986ea1
untar in parallel (#18184)
* untar in parallel

* make enum for 'ignore' return value
2021-06-24 17:29:49 -05:00
Jeff Washington (jwash)
4b314be5bd
rework dirty_pubkeys from insert_new_if_missing_into_primary_index (#18200) 2021-06-24 14:52:11 -05:00
Jeff Washington (jwash)
77c3ffe4cf
add min/max bin size metrics to hash calculation (#18155) 2021-06-23 12:57:05 -05:00
sakridge
3b1738c000
Keep track of dirty stores on remove accounts to clean (#17601)
* Keep track of dirty stores on remove accounts to clean

and not zero_lamport key set

* Only dirty when count==0?

* Add another clean
2021-06-23 10:28:35 +02:00
Lijun Wang
ccf6b21bf8
Removed unused purged_slots in purge_older_root_entries (#18102)
Removed unused purged_slots in purge_older_root_entries
2021-06-22 15:27:37 -07:00
Jeff Washington (jwash)
d3ee73e151
capture sort time in hash calculation (#18118) 2021-06-22 16:02:21 -05:00
Jeff Washington (jwash)
fbc94d84c8
apply 'debug_do_not_add_builtins' to feature activations (#18110) 2021-06-22 14:21:49 -05:00
Jeff Washington (jwash)
d5f9f3b7ce
correctly calculate hash_total for hash/lamport calculation (#18144) 2021-06-22 11:36:43 -05:00
sakridge
e808f34b0b
Add batch stats (#18096) 2021-06-22 15:23:26 +02:00
Tyera Eulberg
0ab361b828
Use newly stabilized Duration methods/consts (#18101) 2021-06-22 00:11:16 +00:00
Tyera Eulberg
19fe1dd463
Move stake_weighted_timestamp module (#18114)
* Move timestamp module into runtime

* Less public

* Remove unused enum
2021-06-21 16:14:54 -06:00
Jeff Washington (jwash)
2087f5da94
refactor generate_index_for_slot (#17984) 2021-06-21 15:32:50 -05:00
Jeff Washington (jwash)
bf97627021
eliminate flatten and sort in hash calculation (#17802)
* eliminate flatten and sort in hash calculation

* reduce critical section time

* remove now no-longer necessary test code

* conflict with reset bins to 0 pr
2021-06-21 15:32:03 -05:00
Tyera Eulberg
4d46cb686a Fix build: clippy lint 2021-06-21 10:18:21 -07:00
dependabot[bot]
d458fac2ff
chore: bump bincode from 1.3.1 to 1.3.3 (#18087)
* chore: bump bincode from 1.3.1 to 1.3.3

Bumps [bincode](https://github.com/servo/bincode) from 1.3.1 to 1.3.3.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/compare/v1.3.1...v1.3.3)

---
updated-dependencies:
- dependency-name: bincode
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-21 16:06:31 +00:00
Jeff Washington (jwash)
ae82e4e25b
refactor reconstruct_accountsdb_from_fields (#17987) 2021-06-21 10:47:58 -05:00
dependabot[bot]
491e4450cc
chore: bump libc from 0.2.96 to 0.2.97 (#18060)
* chore: bump libc from 0.2.96 to 0.2.97

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.96 to 0.2.97.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.96...0.2.97)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-18 11:51:06 -06:00
Alexander Meißner
789f33e8db chore: cargo fmt 2021-06-18 10:42:46 -07:00
Alexander Meißner
6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
dependabot[bot]
abc9839d86
chore: bump flate2 from 1.0.14 to 1.0.20 (#18062)
* chore: bump flate2 from 1.0.14 to 1.0.20

Bumps [flate2](https://github.com/rust-lang/flate2-rs) from 1.0.14 to 1.0.20.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.14...1.0.20)

---
updated-dependencies:
- dependency-name: flate2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-18 15:55:48 +00:00
Jeff Washington (jwash)
0a81c37fce
introduce LockMapType for accounts_index (#18021) 2021-06-17 14:41:52 -05:00
Jeff Washington (jwash)
1feaaf009d
generate_index timings (#17983) 2021-06-17 13:35:25 -05:00
Jeff Washington (jwash)
20e714b3d0
remove unused code (#17981) 2021-06-17 13:30:46 -05:00
Jeff Washington (jwash)
71796f4951
break out generate index reporting (#17980) 2021-06-16 12:09:17 -05:00
Jeff Washington (jwash)
7ca04d6a86
refactor generate_index process_storage_slot (#17982) 2021-06-16 11:02:34 -05:00
Jeff Washington (jwash)
bb7413c9f9
refactor rebuild_bank_from_snapshots (#17988) 2021-06-16 11:01:52 -05:00
Jeff Washington (jwash)
eee5414c64
avoid unnecessary empty arrays when binning (#17944) 2021-06-16 09:24:59 -05:00
Jeff Washington (jwash)
adc683956f
do not use index for verify_bank_hash_and_lamports (#17812) 2021-06-16 09:10:38 -05:00
Jeff Washington (jwash)
55ee3b5f2f
set hash bins to 65k (#17912) 2021-06-16 09:09:55 -05:00
Jeff Washington (jwash)
7de79425ce
parallel storage -> accounts folder (#17955) 2021-06-16 08:48:24 -05:00