gaspersic/go-ethereum
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
12,569 Commits 29 Branches 198 Tags
e69130d9f15d85c5955e1fa94abc74d606accba7
Commit Graph

15 Commits

Author SHA1 Message Date
Péter Szilágyi
745757ac6b core, eth: abort snapshot generation on snap sync and resume later 2021-04-30 17:03:10 +03:00
gary rong
d6ffa14035 core: nuke legacy snapshot supporting (#22663) 2021-04-20 08:27:46 +03:00
gary rong
7088f1e814 core, eth: faster snapshot generation (#22504) 
* eth/protocols: persist received state segments

* core: initial implementation

* core/state/snapshot: add tests

* core, eth: updates

* eth/protocols/snapshot: count flat state size

* core/state: add metrics

* core/state/snapshot: skip unnecessary deletion

* core/state/snapshot: rename

* core/state/snapshot: use the global batch

* core/state/snapshot: add logs and fix wiping

* core/state/snapshot: fix

* core/state/snapshot: save generation progress even if the batch is empty

* core/state/snapshot: fixes

* core/state/snapshot: fix initial account range length

* core/state/snapshot: fix initial account range

* eth/protocols/snap: store flat states during the healing

* eth/protocols/snap: print logs

* core/state/snapshot: refactor (#4)

* core/state/snapshot: refactor

* core/state/snapshot: tiny fix and polish

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* core, eth: fixes

* core, eth: fix healing writer

* core, trie, eth: fix paths

* eth/protocols/snap: fix encoding

* eth, core: add debug log

* core/state/generate: release iterator asap (#5)

core/state/snapshot: less copy

core/state/snapshot: revert split loop

core/state/snapshot: handle storage becoming empty, improve test robustness

core/state: test modified codehash

core/state/snapshot: polish

* core/state/snapshot: optimize stats counter

* core, eth: add metric

* core/state/snapshot: update comments

* core/state/snapshot: improve tests

* core/state/snapshot: replace secure trie with standard trie

* core/state/snapshot: wrap return as the struct

* core/state/snapshot: skip wiping correct states

* core/state/snapshot: updates

* core/state/snapshot: fixes

* core/state/snapshot: fix panic due to reference flaw in closure

* core/state/snapshot: fix errors in state generation logic + fix log output

* core/state/snapshot: remove an error case

* core/state/snapshot: fix condition-check for exhausted snap state

* core/state/snapshot: use stackTrie for small tries

* core/state/snapshot: don't resolve small storage tries in vain

* core/state/snapshot: properly clean up storage of deleted accounts

* core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks

* core/state/snapshot: fix error (+testcase)

* core/state/snapshot: clean up tests a bit

* core/state/snapshot: work in progress on better tests

* core/state/snapshot: polish code

* core/state/snapshot: fix trie iteration abortion trigger

* core/state/snapshot: fixes flaws

* core/state/snapshot: remove panic

* core/state/snapshot: fix abort

* core/state/snapshot: more tests (plus failing testcase)

* core/state/snapshot: more testcases + fix for failing test

* core/state/snapshot: testcase for malformed data

* core/state/snapshot: some test nitpicks

* core/state/snapshot: improvements to logging

* core/state/snapshot: testcase to demo error in abortion

* core/state/snapshot: fix abortion

* cmd/geth: make verify-state report the root

* trie: fix failing test

* core/state/snapshot: add timer metrics

* core/state/snapshot: fix metrics

* core/state/snapshot: udpate tests

* eth/protocols/snap: write snapshot account even if code or state is needed

* core/state/snapshot: fix diskmore check

* core/state/snapshot: review fixes

* core/state/snapshot: improve error message

* cmd/geth: rename 'error' to 'err' in logs

* core/state/snapshot: fix some review concerns

* core/state/snapshot, eth/protocols/snap: clear snapshot marker when starting/resuming snap sync

* core: add error log

* core/state/snapshot: use proper timers for metrics collection

* core/state/snapshot: address some review concerns

* eth/protocols/snap: improved log message

* eth/protocols/snap: fix heal logs to condense infos

* core/state/snapshot: wait for generator termination before restarting

* core/state/snapshot: revert timers to counters to track total time

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
 2021-04-14 23:23:11 +03:00
gary rong
5e9f5ca5d3 core/state/snapshot: write snapshot generator in batch (#22163) 
* core/state/snapshot: write snapshot generator in batch

* core: refactor the tests

* core: update tests

* core: update tests
 2021-01-18 14:39:43 +01:00
Péter Szilágyi
7b7b327ff2 core/state/snapshot: update generator marker in sync with flushes 2020-11-09 16:03:58 +02:00
gary rong
e6402677c2 core/state/snapshot: fix journal recovery from generating old journal (#21775) 
* core/state/snapshot: print warning if failed to resolve journal

* core/state/snapshot: fix snapshot recovery

When we meet the snapshot journal consisted with:
- disk layer generator with new-format
- diff layer journal with old-format

The base layer should be returned without error.
The broken diff layer can be reconstructed later
but we definitely don't want to reconstruct the
huge diff layer.

* core: add tests
 2020-11-04 13:41:46 +02:00
gary rong
b63e3c37a6 core: improve snapshot journal recovery (#21594) 
* core/state/snapshot: introduce snapshot journal version

* core: update the disk layer in an atomic way

* core: persist the disk layer generator periodically

* core/state/snapshot: improve logging

* core/state/snapshot: forcibly ensure the legacy snapshot is matched

* core/state/snapshot: add debug logs

* core, tests: fix tests and special recovery case

* core: polish

* core: add more blockchain tests for snapshot recovery

* core/state: fix comment

* core: add recovery flag for snapshot

* core: add restart after start-after-crash tests

* core/rawdb: fix imports

* core: fix tests

* core: remove log

* core/state/snapshot: fix snapshot

* core: avoid callbacks in SetHead

* core: fix setHead cornercase where the threshold root has state

* core: small docs for the test cases

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
 2020-10-29 21:01:58 +02:00
Péter Szilágyi
3ee91b9f2e core/state/snapshot: reduce disk layer depth during generation 2020-08-24 13:22:36 +03:00
Péter Szilágyi
4b6f6ffe23 core/state/snapshot: fix journal nil deserialziation 2020-04-29 18:00:29 +03:00
Péter Szilágyi
a4cf279494 core/state: extend snapshotter to handle account resurrections 2020-03-03 15:52:00 +02:00
Péter Szilágyi
6e05ccd845 core/state/snapshot, tests: sync snap gen + snaps in consensus tests 2020-03-03 09:17:13 +02:00
Péter Szilágyi
06d4470b41 core: fix broken tests due to API changes + linter 2020-02-25 12:51:16 +02:00
Martin Holst Swende
19099421dc core/state/snapshot: faster account iteration, CLI integration 2020-02-25 12:51:15 +02:00
Péter Szilágyi
fd39f722a3 core: journal the snapshot inside leveldb, not a flat file 2020-02-25 12:51:10 +02:00
Péter Szilágyi
351a5903b0 core/rawdb, core/state/snapshot: runtime snapshot generation 2020-02-25 12:51:08 +02:00