verkle: Implement Trie, NodeIterator and Database ifs
Fix crash in TestDump
Fix TestDump
Fix TrieCopy
remove unnecessary traces
fix: Error() returned errIteratorEnd in verkle node iterator
rewrite the iterator and change the signature of OpenStorageTrie
add the adapter to reuse the account trie for storage
don't try to deserialize a storage leaf into an account
Fix statedb unit tests (#14)
* debug code
* Fix more unit tests
* remove traces
* Go back to the full range
One tree to rule them all
remove updateRoot, there is no root to update
store code inside the account leaf
fix build
save current state for Sina
Update go-verkle to latest
Charge WITNESS_*_COST gas on storage loads
Add witness costs for SSTORE as well
Charge witness gas in the case of code execution
corresponding code deletion
add a --verkle flag to separate verkle experiments from regular geth operations
use the snapshot to get data
stateless execution from block witness
AccessWitness functions
Add block generation test + genesis snapshot generation
test stateless block execution (#18)
* test stateless block execution
* Force tree resolution before generating the proof
increased coverage in stateless test execution (#19)
* test stateless block execution
* Force tree resolution before generating the proof
* increase coverage in stateless test execution
ensure geth compiles
fix issues in tests with verkle trees deactivated
Ensure stateless data is available when executing statelessly (#20)
* Ensure stateless data is available when executing statelessly
* Actual execution of a statless block
* bugfixes in stateless block execution
* code cleanup
- Reduce PR footprint by reverting NewEVM to its original signature
- Move the access witness to the block context
- prepare for a change in AW semantics
Need to store the initial values.
- Use the touch helper function, DRY
* revert the signature of MustCommit to its original form (#21)
fix leaf proofs in stateless execution (#22)
* Fixes in witness pre-state
* Add the recipient's nonce to the witness
* reduce PR footprint and investigate issue in root state calculation
* quick build fix
cleanup: Remove extra parameter in ToBlock
revert ToBlock to its older signature
fix import cycle in vm tests
fix linter issue
fix appveyor build
fix nil pointers in tests
Add indices, yis and Cis to the block's Verkle proof
upgrade geth dependency to drop geth's common dep
fix cmd/devp2p tests
fix rebase issues
quell an appveyor warning
fix address touching in SLOAD and SSTORE
fix access witness for code size
touch target account data before calling
make sure the proper locations get touched in (ext)codecopy
touch all code pages in execution
add pushdata to witness
remove useless code in genesis snapshot generation
testnet: fix some of the rebase/drift issues
Fix verkle proof generation in block
fix an issue occuring when chunking past the code size
fix: ensure the code copy doesn't extend past the code size
This commit fixes a flaw in two testcases, and brings down the exec-time from ~40s to ~8s for trie/TestIncompleteSync.
The checkConsistency was performed over and over again on the complete set of nodes, not just the recently added, turning it into a quadratic runtime.
When we flush a batch of trie nodes into database during the state
sync, we should guarantee that all children should be flushed before
parent.
Actually the trie nodes commit order is strict by: children -> parent.
But when we flush all ready nodes into db, we don't need the order
anymore since
(1) they are all ready nodes (no more dependency)
(2) underlying database provides write atomicity
* core, eth, trie: bloom filter for trie node dedup during fast sync
* eth/downloader, trie: address review comments
* core, ethdb, trie: restart fast-sync bloom construction now and again
* eth/downloader: initialize fast sync bloom on startup
* eth: reenable eth/62 until we properly remove it
With this commit, core/state's access to the underlying key/value database is
mediated through an interface. Database errors are tracked in StateDB and
returned by CommitTo or the new Error method.
Motivation for this change: We can remove the light client's duplicated copy of
core/state. The light client now supports node iteration, so tracing and storage
enumeration can work with the light client (not implemented in this commit).
* eth/downloader: separate state sync from queue
Scheduling of state node downloads hogged the downloader queue lock when
new requests were scheduled. This caused timeouts for other requests.
With this change, state sync is fully independent of all other downloads
and doesn't involve the queue at all.
State sync is started and checked on in processContent. This is slightly
awkward because processContent doesn't have a select loop. Instead, the
queue is closed by an auxiliary goroutine when state sync fails. We
tried several alternatives to this but settled on the current approach
because it's the least amount of change overall.
Handling of the pivot block has changed slightly: the queue previously
prevented import of pivot block receipts before the state of the pivot
block was available. In this commit, the receipt will be imported before
the state. This causes an annoyance where the pivot block is committed
as fast block head even when state downloads fail. Stay tuned for more
updates in this area ;)
* eth/downloader: remove cancelTimeout channel
* eth/downloader: retry state requests on timeout
* eth/downloader: improve comment
* eth/downloader: mark peers idle when state sync is done
* eth/downloader: move pivot block splitting to processContent
This change also ensures that pivot block receipts aren't imported
before the pivot block itself.
* eth/downloader: limit state node retries
* eth/downloader: improve state node error handling and retry check
* eth/downloader: remove maxStateNodeRetries
It fails the sync too much.
* eth/downloader: remove last use of cancelCh in statesync.go
Fixes TestDeliverHeadersHang*Fast and (hopefully)
the weird cancellation behaviour at the end of fast sync.
* eth/downloader: fix leak in runStateSync
* eth/downloader: don't run processFullSyncContent in LightSync mode
* eth/downloader: improve comments
* eth/downloader: fix vet, megacheck
* eth/downloader: remove unrequested tasks anyway
* eth/downloader, trie: various polishes around duplicate items
This commit explicitly tracks duplicate and unexpected state
delieveries done against a trie Sync structure, also adding
there to import info logs.
The commit moves the db batch used to commit trie changes one
level deeper so its flushed after every node insertion. This
is needed to avoid a lot of duplicate retrievals caused by
inconsistencies between Sync internals and database. A better
approach is to track not-yet-written states in trie.Sync and
flush on commit, but I'm focuing on correctness first now.
The commit fixes a regression around pivot block fail count.
The counter previously was reset to 1 if and only if a sync
cycle progressed (inserted at least 1 entry to the database).
The current code reset it already if a node was delivered,
which is not stong enough, because unless it ends up written
to disk, an attacker can just loop and attack ad infinitum.
The commit also fixes a regression around state deliveries
and timeouts. The old downloader tracked if a delivery is
stale (none of the deliveries were requestedt), in which
case it didn't mark the node idle and did not send further
requests, since it signals a past timeout. The current code
did mark it idle even on stale deliveries, which eventually
caused two requests to be in flight at the same time, making
the deliveries always stale and mass duplicating retrievals
between multiple peers.
* eth/downloader: fix state request leak
This commit fixes the hang seen sometimes while doing the state
sync. The cause of the hang was a rare combination of events:
request state data from peer, peer drops and reconnects almost
immediately. This caused a new download task to be assigned to
the peer, overwriting the old one still waiting for a timeout,
which in turned leaked the requests out, never to be retried.
The fix is to ensure that a task assignment moves any pending
one back into the retry queue.
The commit also fixes a regression with peer dropping due to
stalls. The current code considered a peer stalling if they
timed out delivering 1 item. However, the downloader never
requests only one, the minimum is 2 (attempt to fine tune
estimated latency/bandwidth). The fix is simply to drop if
a timeout is detected at 2 items.
Apart from the above bugfixes, the commit contains some code
polishes I made while debugging the hang.
* core, eth, trie: support batched trie sync db writes
* trie: rename SyncMemCache to syncMemBatch
This commit implements EIP158 part 1, 2, 3 & 4
1. If an account is empty it's no longer written to the trie. An empty
account is defined as (balance=0, nonce=0, storage=0, code=0).
2. Delete an empty account if it's touched
3. An empty account is redefined as either non-existent or empty.
4. Zero value calls and zero value suicides no longer consume the 25k
reation costs.
params: moved core/config to params
Signed-off-by: Jeffrey Wilcke <jeffrey@ethereum.org>
This commit replaces the deep-copy based state revert mechanism with a
linear complexity journal. This commit also hides several internal
StateDB methods to limit the number of ways in which calling code can
use the journal incorrectly.
As usual consultation and bug fixes to the initial implementation were
provided by @karalabe, @obscuren and @Arachnid. Thank you!
