Commit Graph

1959 Commits

Author SHA1 Message Date
025261c05e AcctIdx: when no disk index, always wait for stats (#20248) 2021-09-27 11:24:36 -05:00
833f7761d2 AcctIdx: upsert uses get from mem first (#20186) 2021-09-27 11:23:53 -05:00
d8f98b3dcb AcctIdx: avoid crazy #s on ms_per_age stat (#20184) 2021-09-27 08:25:23 -05:00
df6905c3a6 Refactor: Merge MessageProcessor into InvokeContext (#20165)
* Inlines MessageProcessor::execute_instruction() in MessageProcessor::process_message().

* Moves MessageProcessor::create_pre_accounts() into ThisInvokeContext::push().

* Hoists ThisInvokeContext::new() out of loop inside MessageProcessor::process_message().

* Moves MessageProcessor::verify_account_references() and MessageProcessor::verify() into InvokeContext::verify().
2021-09-27 08:28:45 +02:00
c6d189dac2 bg threads wait until next expected event (#20126) 2021-09-24 16:00:41 -05:00
122206dbb1 AcctIdx: bg loop sleeps when all buckets flushed at current age (#20168) 2021-09-24 13:19:06 -05:00
1d13594c1c AcctIdx: store # threads in BucketHolder. Used later. (#20151) 2021-09-23 19:56:44 -05:00
b3bb079d9f AcctIdx: condense upsert to always use read lock (#20148) 2021-09-23 19:19:27 -05:00
1347b507da Handle duplicate AppendVec IDs (#20096)
When reconstructing the AccountsDb, if the storages came from full and
incremental snapshots generated on different nodes, it's possible that
the AppendVec IDs could overlap/have duplicates, which would cause the
reconstruction to fail.

This commit handles this issue by unconditionally remapping the
AppendVec ID for every AppendVec.

Fixes #17088
2021-09-23 17:34:32 -05:00
2ae1e80013 AcctIdx: items needs to load from disk (#20138) 2021-09-23 17:33:54 -05:00
334d38d200 AcctIdx: generation per thread starts at a random bucket (#20122) 2021-09-23 19:59:37 +00:00
254d9c8903 range takes a ref (#20140) 2021-09-23 13:57:56 -05:00
093dd68214 AcctIdx: stats for how much time bg threads wait (#20127) 2021-09-23 18:37:14 +00:00
fe28f17718 AcctIdx: startup causes us to flush remove everything asap (#20121) 2021-09-23 12:17:52 -05:00
5dae615208 AcctIdx: set age of cache accesses to future (#20125) 2021-09-23 11:43:48 -05:00
fb55e9a9b6 AcctIdx: when stop_flush is disabled, prepare to age (#20113) 2021-09-23 11:42:32 -05:00
5bbb0da7b8 AcctIdx: stats for in-mem (#20114) 2021-09-23 08:14:53 -05:00
cb9d93525b AcctIdx: metrics for loading from disk (#20124) 2021-09-23 08:14:08 -05:00
4d3e32803f AcctIdx: PreAllocatedAccountMapEntry (#20117) 2021-09-23 08:11:32 -05:00
7b365c564f runtime: remove inactive delegation from stakes cache 2021-09-22 20:50:58 -06:00
476124de50 AcctIdx: report disk bucket stats (#20120) 2021-09-22 18:17:20 -05:00
f7eebadbed AcctIdx: remove old items from cache (#20076) 2021-09-22 21:37:15 +00:00
0eb0d7f73b AcctIdx: consolidate lock_and_update_slot_list (#20090) 2021-09-22 18:40:19 +00:00
4fb77183ef AcctIdx: move background() to bucket holder (#20085) 2021-09-22 17:40:30 +00:00
681e8728a1 get rid of logs left in test (#20116) 2021-09-22 12:39:48 -05:00
bbd7bc2985 AcctIdx: Fix bad test (#20115) 2021-09-22 12:39:31 -05:00
1eeddf2e23 orderings on AcctIdx dirty (#20087) 2021-09-22 11:24:16 -05:00
681d6d121e AcctIdx: 2 upserts use disk (#20072) 2021-09-22 11:14:08 -05:00
deab944b30 AcctIdx: flush inserts to disk (#20074) 2021-09-22 09:55:40 -05:00
804ed825f7 AcctIdx: rework timing of bg loop (#20077) 2021-09-22 09:39:49 -05:00
a6aef76f52 AcctIdx: put_range_in_cache loads from disk (#20073) 2021-09-22 09:37:34 -05:00
c944bfb1d9 AcctIdx tests use disk buckets (#20070) 2021-09-22 08:55:58 -05:00
9e9417b474 AcctIdx: initialze stats with # bins (#20088) 2021-09-22 08:55:07 -05:00
b918c2f4cd AcctIdx: remove_if_slot_list_empty uses disk backing (#20069) 2021-09-22 08:48:44 -05:00
6c518102dd AcctIdx: fix some missing dirty calls (#20089) 2021-09-22 08:45:08 -05:00
1b58166e9d chore: bump memmap2 from 0.4.0 to 0.5.0 (#20057)
* chore: bump memmap2 from 0.4.0 to 0.5.0

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-21 17:20:49 -06:00
e6795e6fdd AcctIdx: get can load from disk (#20067) 2021-09-21 12:40:07 -05:00
7f3d445af5 AcctIdx: Orderings and cleanup (#20046) 2021-09-21 10:52:39 -05:00
bbe151e594 AcctIdx: only flush if disk index being used (#20045) 2021-09-21 08:42:13 -05:00
38ea3aa31a AcctIdx: keys impl when items are on disk (#20039) 2021-09-21 08:41:39 -05:00
fb71e80962 AcctIdx: accessors for age (#20023) 2021-09-21 08:41:17 -05:00
c3679ab9bd AcctIdx: first pass at aging buckets (#20008) 2021-09-21 08:40:55 -05:00
b507715d44 CPI without Account Refs (#20034)
* Removes search for accounts and unsafe lifetime transmute in InvokeContext::push().

* Replaces accounts by account_indices in verify_and_update() and process_cross_program_instruction().

Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
2021-09-21 14:41:02 +02:00
94330de843 AcctIdx: delete from disk index (#20035) 2021-09-20 16:19:21 -05:00
6399a87485 AcctIdx: report age in stats (#20025) 2021-09-20 14:30:12 -05:00
e9ee90a121 AcctIdx: age adds time component (#20024) 2021-09-20 14:29:34 -05:00
d842705622 AcctIdx: when bg processing in multiple threads, avoid bin collisions (#20019) 2021-09-20 11:50:22 -05:00
e6934e7247 AcctIdx: consolidate to correct CondVar (#20017) 2021-09-20 09:58:20 -05:00
55d9ff9899 WaitableCondVar supports notify_one (#20013) 2021-09-20 09:57:57 -05:00
bc7983b13e AcctIdx: metric for active bg threads (#20026) 2021-09-20 09:44:40 -05:00