Commit Graph

41 Commits

Author SHA1 Message Date
7b7b327ff2 core/state/snapshot: update generator marker in sync with flushes 2020-11-09 16:03:58 +02:00
e6402677c2 core/state/snapshot: fix journal recovery from generating old journal (#21775)
* core/state/snapshot: print warning if failed to resolve journal

* core/state/snapshot: fix snapshot recovery

When we meet the snapshot journal consisted with:
- disk layer generator with new-format
- diff layer journal with old-format

The base layer should be returned without error.
The broken diff layer can be reconstructed later
but we definitely don't want to reconstruct the
huge diff layer.

* core: add tests
2020-11-04 13:41:46 +02:00
b63e3c37a6 core: improve snapshot journal recovery (#21594)
* core/state/snapshot: introduce snapshot journal version

* core: update the disk layer in an atomic way

* core: persist the disk layer generator periodically

* core/state/snapshot: improve logging

* core/state/snapshot: forcibly ensure the legacy snapshot is matched

* core/state/snapshot: add debug logs

* core, tests: fix tests and special recovery case

* core: polish

* core: add more blockchain tests for snapshot recovery

* core/state: fix comment

* core: add recovery flag for snapshot

* core: add restart after start-after-crash tests

* core/rawdb: fix imports

* core: fix tests

* core: remove log

* core/state/snapshot: fix snapshot

* core: avoid callbacks in SetHead

* core: fix setHead cornercase where the threshold root has state

* core: small docs for the test cases

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-10-29 21:01:58 +02:00
43c278cdf9 core/state: disable snapshot iteration if it's not fully constructed (#21682)
* core/state/snapshot: add diskRoot function

* core/state/snapshot: disable iteration if the snapshot is generating

* core/state/snapshot: simplify the function

* core/state: panic for undefined layer
2020-10-28 14:27:37 +02:00
a15d71a255 core/state/snapshot: stop generator if it hits missing trie nodes (#21649)
* core/state/snapshot: exit Geth if generator hits missing trie nodes

* core/state/snapshot: error instead of hard die on generator fault

* core/state/snapshot: don't enable logging on the tests
2020-10-05 11:52:36 +03:00
3ee91b9f2e core/state/snapshot: reduce disk layer depth during generation 2020-08-24 13:22:36 +03:00
4f2784b38f all: fix typos in comments (#21118) 2020-05-25 10:21:28 +02:00
85944c2561 core/state/snapshot: fix typo (#21037) 2020-05-07 10:07:59 +03:00
90af6dae6e core/state/snapshot: don't create storage list for non-existing accounts 2020-05-06 17:22:38 +03:00
da59147014 core/state/snapshot: release iterator after verification 2020-05-04 15:14:08 +03:00
2337aa64eb core/state/snapshot: fix trie generator reporter (#21004) 2020-04-30 10:43:50 +03:00
4b6f6ffe23 core/state/snapshot: fix journal nil deserialziation 2020-04-29 18:00:29 +03:00
26d271dfbb core/state/snapshot: implement storage iterator (#20971)
* core/state/snapshot: implement storage iterator

* core/state/snapshot, tests: implement helper function

* core/state/snapshot: fix storage issue

If an account is deleted in the tx_1 but recreated in the tx_2,
the it can happen that in this diff layer, both destructedSet
and storageData records this account. In this case, the storage
iterator should be able to iterate the slots belong to new account
but disable further iteration in deeper layers(belong to old account)

* core/state/snapshot: address peter and martin's comment

* core/state: address comments

* core/state/snapshot: fix test
2020-04-29 12:53:08 +03:00
8a2e8faadd core/state/snapshot: fix binary iterator (#20970) 2020-04-24 14:43:49 +03:00
b9df7ecdc3 all: seperate consensus error and evm internal error (#20830)
* all: seperate consensus error and evm internal error

There are actually two types of error will be returned when
a tranaction/message call is executed: (a) consensus error
(b) evm internal error. The former should be converted to
a consensus issue, e.g. The sender doesn't enough asset to
purchase the gas it specifies. The latter is allowed since
evm itself is a blackbox and internal error is allowed to happen.

This PR emphasizes the difference by introducing a executionResult
structure. The evm error is embedded inside. So if any error
returned, it indicates consensus issue happens.

And also this PR improve the `EstimateGas` API to return the concrete
revert reason if the transaction always fails

* all: polish

* accounts/abi/bind/backends: add tests

* accounts/abi/bind/backends, internal: cleanup error message

* all: address comments

* core: fix lint

* accounts, core, eth, internal: address comments

* accounts, internal: resolve revert reason if possible

* accounts, internal: address comments
2020-04-22 11:25:36 +03:00
c036fe35a8 core/state/snapshot: make difflayer account iterator seek operation inclusive 2020-04-21 16:26:02 +02:00
7599999dcd snapshot: add Unlock before return (#20948)
* Forget Unlock in snapshot

* Remove Unlock before panic
2020-04-21 11:11:38 +03:00
6402c42b67 all: simplify and fix database iteration with prefix/start (#20808)
* core/state/snapshot: start fixing disk iterator seek

* ethdb, rawdb, leveldb, memorydb: implement iterators with prefix and start

* les, core/state/snapshot: iterator fixes

* all: remove two iterator methods

* all: rename Iteratee.NewIteratorWith -> NewIterator

* ethdb: fix review concerns
2020-04-15 14:08:53 +03:00
76eed9e50d snapshotter/tests: verify snapdb post-state against trie (#20812)
* core/state/snapshot: basic trie-to-hash implementation

* tests: validate snapshot after test

* core/state/snapshot: fix review concerns
2020-03-31 10:25:41 +02:00
074efe6c8d core: fix two snapshot iterator flaws, decollide snap storage prefix
* core/state/snapshot/iterator: fix two disk iterator flaws

* core/rawdb: change SnapshotStoragePrefix to avoid prefix collision with preimagePrefix
2020-03-23 12:34:27 +02:00
fab0ee3bfa core/state/snapshot: fix various iteration issues due to destruct set 2020-03-04 15:06:04 +02:00
eff7cfbb03 core/state/snapshot: handle deleted accounts in fast iterator 2020-03-04 14:38:55 +02:00
a4cf279494 core/state: extend snapshotter to handle account resurrections 2020-03-03 15:52:00 +02:00
6e05ccd845 core/state/snapshot, tests: sync snap gen + snaps in consensus tests 2020-03-03 09:17:13 +02:00
06d4470b41 core: fix broken tests due to API changes + linter 2020-02-25 12:51:16 +02:00
19099421dc core/state/snapshot: faster account iteration, CLI integration 2020-02-25 12:51:15 +02:00
6ddb92a089 core/state/snapshot: full featured account iteration 2020-02-25 12:51:14 +02:00
e570835356 core/state/snapshot: implement iterator priority for fast direct data lookup 2020-02-25 12:51:14 +02:00
e567675473 core/state/snapshot: move iterator out into its own files 2020-02-25 12:51:13 +02:00
7e38996301 core/state/snapshot: implement snapshot layer iteration 2020-02-25 12:51:12 +02:00
22c494d399 core/state/snapshot: bloom, metrics and prefetcher fixes 2020-02-25 12:51:11 +02:00
3ad4335acc core/state/snapshot: node behavioural difference on bloom content 2020-02-25 12:51:11 +02:00
fd39f722a3 core: journal the snapshot inside leveldb, not a flat file 2020-02-25 12:51:10 +02:00
d5d7c0c24b core/state/snapshot: fix difflayer origin-initalization after flatten 2020-02-25 12:51:09 +02:00
351a5903b0 core/rawdb, core/state/snapshot: runtime snapshot generation 2020-02-25 12:51:08 +02:00
f300c0df01 core/state/snapshot: replace bigcache with fastcache 2020-02-25 12:51:08 +02:00
d754091a87 core/state/snapshot: unlink snapshots from blocks, quad->linear cleanup 2020-02-25 12:51:07 +02:00
cdf3f016df snapshot: iteration and buffering optimizations 2020-02-25 12:51:06 +02:00
d7d81d7c12 core/state/snapshot: extract and split cap method, cover corners 2020-02-25 12:51:05 +02:00
e146fbe4e7 core/state: lazy sorting, snapshot invalidation 2020-02-25 12:51:05 +02:00
542df8898e core: initial version of state snapshots 2020-02-25 12:51:04 +02:00