Commit Graph

2578 Commits

Author SHA1 Message Date
bd9b2f6f39 AcctIdx: throttling bg flushing to spread out work over interval (#20176)
* AcctIdx: throttling bg flushing to spread out work over interval

* prevent hang
2021-09-29 07:42:54 -05:00
8fee9a2e1a Dont call precompiled programs (#19930) 2021-09-29 06:25:08 +00:00
491877de3d Revert "Refactor: Merge MessageProcessor into InvokeContext (#20165)" (#20301)
This reverts commit df6905c3a6.
2021-09-28 16:59:01 -07:00
fa5b091b4c prior to panicing with cap mismatch, try other calculation (#20292) 2021-09-28 16:09:36 -05:00
320beb76f6 AcctIdx: implement wait_for_idle (#20256) 2021-09-28 11:07:19 -05:00
2cd9dc99b6 Restore ability for programs to upgrade themselves (#20265)
* Make helper associated fn

* Add feature definition

* Add handling to preserve program-id write lock when upgradeable loader is present; restore bpf upgrade-self test

* Use single feature
2021-09-28 09:59:08 -06:00
4e27543415 Allow programs to realloc their accounts within limits (#19475) 2021-09-28 01:13:03 -07:00
e50a26e6cf AcctIdx: add age_interval_ms() to remove constant use (#20264) 2021-09-28 01:22:07 +00:00
af309c126c AcctIdx: refactor get to use more refs (#20268) 2021-09-28 00:18:36 +00:00
34406d4815 chore: bump ouroboros from 0.10.1 to 0.11.1 (#20244)
* chore: bump ouroboros from 0.10.1 to 0.11.1

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.10.1 to 0.11.1.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-27 23:26:08 +00:00
f6812523cd handle race condition with flush and delete (#20255) 2021-09-27 16:55:15 -05:00
bbd2f9672d clone -> Arc::clone to make cost clear (#20262) 2021-09-27 16:49:53 -05:00
4f83818fdb get rid of WriteAccountMapEntry instance (#19093) 2021-09-27 16:24:19 -05:00
e4795ae7ed AcctIdx: keep cached or multiple slot lists in cache (#20257) 2021-09-27 16:09:26 -05:00
9d91a6f1b7 shrink slots at startup uses more threads (#20251) 2021-09-27 14:14:55 -05:00
8266864adc AcctIdx: maybe advance age when a bg thread wakes up (#20250)
* fix math error with stats calc

* AcctIdx: maybe report stats when a bg thread wakes up
2021-09-27 18:39:41 +00:00
95fbc29500 Add stats when getting snapshot storages (#20234) 2021-09-27 13:07:00 -05:00
46263d405e fix math error with stats calc (#20253) 2021-09-27 11:55:19 -05:00
bc50fd4ad6 allocate threads for accounts idx based on cpus (#20118) 2021-09-27 11:25:06 -05:00
025261c05e AcctIdx: when no disk index, always wait for stats (#20248) 2021-09-27 11:24:36 -05:00
833f7761d2 AcctIdx: upsert uses get from mem first (#20186) 2021-09-27 11:23:53 -05:00
d8f98b3dcb AcctIdx: avoid crazy #s on ms_per_age stat (#20184) 2021-09-27 08:25:23 -05:00
df6905c3a6 Refactor: Merge MessageProcessor into InvokeContext (#20165)
* Inlines MessageProcessor::execute_instruction() in MessageProcessor::process_message().

* Moves MessageProcessor::create_pre_accounts() into ThisInvokeContext::push().

* Hoists ThisInvokeContext::new() out of loop inside MessageProcessor::process_message().

* Moves MessageProcessor::verify_account_references() and MessageProcessor::verify() into InvokeContext::verify().
2021-09-27 08:28:45 +02:00
c6d189dac2 bg threads wait until next expected event (#20126) 2021-09-24 16:00:41 -05:00
122206dbb1 AcctIdx: bg loop sleeps when all buckets flushed at current age (#20168) 2021-09-24 13:19:06 -05:00
1d13594c1c AcctIdx: store # threads in BucketHolder. Used later. (#20151) 2021-09-23 19:56:44 -05:00
b3bb079d9f AcctIdx: condense upsert to always use read lock (#20148) 2021-09-23 19:19:27 -05:00
1347b507da Handle duplicate AppendVec IDs (#20096)
When reconstructing the AccountsDb, if the storages came from full and
incremental snapshots generated on different nodes, it's possible that
the AppendVec IDs could overlap/have duplicates, which would cause the
reconstruction to fail.

This commit handles this issue by unconditionally remapping the
AppendVec ID for every AppendVec.

Fixes #17088
2021-09-23 17:34:32 -05:00
2ae1e80013 AcctIdx: items needs to load from disk (#20138) 2021-09-23 17:33:54 -05:00
334d38d200 AcctIdx: generation per thread starts at a random bucket (#20122) 2021-09-23 19:59:37 +00:00
254d9c8903 range takes a ref (#20140) 2021-09-23 13:57:56 -05:00
093dd68214 AcctIdx: stats for how much time bg threads wait (#20127) 2021-09-23 18:37:14 +00:00
fe28f17718 AcctIdx: startup causes us to flush remove everything asap (#20121) 2021-09-23 12:17:52 -05:00
5dae615208 AcctIdx: set age of cache accesses to future (#20125) 2021-09-23 11:43:48 -05:00
fb55e9a9b6 AcctIdx: when stop_flush is disabled, prepare to age (#20113) 2021-09-23 11:42:32 -05:00
5bbb0da7b8 AcctIdx: stats for in-mem (#20114) 2021-09-23 08:14:53 -05:00
cb9d93525b AcctIdx: metrics for loading from disk (#20124) 2021-09-23 08:14:08 -05:00
4d3e32803f AcctIdx: PreAllocatedAccountMapEntry (#20117) 2021-09-23 08:11:32 -05:00
7b365c564f runtime: remove inactive delegation from stakes cache 2021-09-22 20:50:58 -06:00
476124de50 AcctIdx: report disk bucket stats (#20120) 2021-09-22 18:17:20 -05:00
f7eebadbed AcctIdx: remove old items from cache (#20076) 2021-09-22 21:37:15 +00:00
0eb0d7f73b AcctIdx: consolidate lock_and_update_slot_list (#20090) 2021-09-22 18:40:19 +00:00
4fb77183ef AcctIdx: move background() to bucket holder (#20085) 2021-09-22 17:40:30 +00:00
681e8728a1 get rid of logs left in test (#20116) 2021-09-22 12:39:48 -05:00
bbd7bc2985 AcctIdx: Fix bad test (#20115) 2021-09-22 12:39:31 -05:00
1eeddf2e23 orderings on AcctIdx dirty (#20087) 2021-09-22 11:24:16 -05:00
681d6d121e AcctIdx: 2 upserts use disk (#20072) 2021-09-22 11:14:08 -05:00
deab944b30 AcctIdx: flush inserts to disk (#20074) 2021-09-22 09:55:40 -05:00
804ed825f7 AcctIdx: rework timing of bg loop (#20077) 2021-09-22 09:39:49 -05:00
a6aef76f52 AcctIdx: put_range_in_cache loads from disk (#20073) 2021-09-22 09:37:34 -05:00