Commit Graph

1920 Commits

Author SHA1 Message Date
b63e3c37a6 core: improve snapshot journal recovery (#21594)
* core/state/snapshot: introduce snapshot journal version

* core: update the disk layer in an atomic way

* core: persist the disk layer generator periodically

* core/state/snapshot: improve logging

* core/state/snapshot: forcibly ensure the legacy snapshot is matched

* core/state/snapshot: add debug logs

* core, tests: fix tests and special recovery case

* core: polish

* core: add more blockchain tests for snapshot recovery

* core/state: fix comment

* core: add recovery flag for snapshot

* core: add restart after start-after-crash tests

* core/rawdb: fix imports

* core: fix tests

* core: remove log

* core/state/snapshot: fix snapshot

* core: avoid callbacks in SetHead

* core: fix setHead cornercase where the threshold root has state

* core: small docs for the test cases

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-10-29 21:01:58 +02:00
43c278cdf9 core/state: disable snapshot iteration if it's not fully constructed (#21682)
* core/state/snapshot: add diskRoot function

* core/state/snapshot: disable iteration if the snapshot is generating

* core/state/snapshot: simplify the function

* core/state: panic for undefined layer
2020-10-28 14:27:37 +02:00
18145adf08 core/state: maintain one more diff layer (#21730)
* core/state: maintain one more diff layer

* core/state: address comment
2020-10-28 14:00:22 +02:00
6487c002f6 all: implement EIP-2929 (gas cost increases for state access opcodes) + yolo-v2 (#21509)
* core/vm, core/state: implement EIP-2929 + YOLOv2

* core/state, core/vm: fix some review concerns

* core/state, core/vm: address review concerns

* core/vm: address review concerns

* core/vm: better documentation

* core/vm: unify sload cost as fully dynamic

* core/vm: fix typo

* core/vm/runtime: fix compilation flaw

* core/vm/runtime: fix renaming-err leftovers

* core/vm: renaming

* params/config: use correct yolov2 chainid for config

* core, params: use a proper new genesis for yolov2

* core/state/tests: golinter nitpicks
2020-10-23 08:26:57 +02:00
91c4607979 core: fix blockchain insert report time interval calculation (#21723) 2020-10-21 16:53:30 +02:00
9d25f34263 core: track and improve tx indexing/unindexing (#21331)
* core: add background indexer to waitgroup

* core: make indexer stopable

* core/rawdb: add unit tests

* core/rawdb: fix lint

* core/rawdb: fix tests

* core/rawdb: fix linter
2020-10-20 16:34:50 +02:00
b305591e14 core/vm: marshall returnData as hexstring in trace logs (#21715)
* core/vm: marshall returnData as hexstring in trace logs

* core/vm: marshall returnData as hexstring in trace logs
2020-10-16 11:28:03 +02:00
c37e68e7c1 all: replace RWMutex with Mutex in places where RLock is not used (#21622) 2020-10-13 10:58:41 +02:00
66c3eb2f1a accouts, consensus, core: fix some comments (#21617) 2020-10-12 15:02:38 +02:00
706f5e3b98 core: fix txpool off-by-one error (#21683) 2020-10-09 12:23:46 +03:00
43cd31ea9f core/vm: dedup config check in markdown logger (#21655)
* core/vm: dedup config check

* review feedback: reuse buffer
2020-10-08 14:03:24 +02:00
e43d827a19 core/types: optimize bloom filters (#21624)
* core/types: tests for bloom

* core/types: refactored bloom filter for receipts, added tests

core/types: replaced old bloom implementation

core/types: change interface of bloom add+test

* core/types: refactor bloom

* core/types: minor tweak on LogsBloom

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2020-10-06 16:57:00 +03:00
eb87121300 core/bloombits: faster generator (#21625)
* core/bloombits: add benchmark

* core/bloombits: optimize inserts
2020-10-06 16:34:29 +03:00
a15d71a255 core/state/snapshot: stop generator if it hits missing trie nodes (#21649)
* core/state/snapshot: exit Geth if generator hits missing trie nodes

* core/state/snapshot: error instead of hard die on generator fault

* core/state/snapshot: don't enable logging on the tests
2020-10-05 11:52:36 +03:00
6c8310ebb4 trie: use stacktrie for Derivesha operation (#21407)
core/types: use stacktrie for derivesha

trie: add stacktrie file

trie: fix linter

core/types: use stacktrie for derivesha

rebased: adapt stacktrie to the newer version of DeriveSha

Co-authored-by: Martin Holst Swende <martin@swende.se>

More linter fixes

review feedback: no key offset for nodes converted to hashes

trie: use EncodeRLP for full nodes

core/types: insert txs in order in derivesha

trie: tests for derivesha with stacktrie

trie: make stacktrie use pooled hashers

trie: make stacktrie reuse tmp slice space

trie: minor polishes on stacktrie

trie/stacktrie: less rlp dancing

core/types: explain the contorsions in DeriveSha

ci: fix goimport errors

trie: clear mem on subtrie hashing

squashme: linter fix

stracktrie: use pooling, less allocs (#3)

trie: in-place hex prefix, reduce allocs and add rawNode.EncodeRLP

Reintroduce the `[]node` method, add the missing `EncodeRLP` implementation for `rawNode` and calculate the hex prefix in place.

Co-authored-by: Martin Holst Swende <martin@swende.se>

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-09-29 17:38:13 +02:00
eebfb13053 core: free pointer from slice after popping element from price heap (#21572)
* Fix potential memory leak in price heap

* core: nil free pointer slice (alternative version)

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-09-28 14:24:01 +02:00
0ddd4612b7 core/vm, params: make 2200 in line with spec (#21605) 2020-09-28 14:14:45 +02:00
a25899f3dc cmd/devp2p: add eth protocol test suite (#21598)
This change adds a test framework for the "eth" protocol and some basic
tests. The tests can be run using the './devp2p rlpx eth-test' command.
2020-09-23 15:18:17 +02:00
f354c622ca core: fix a typo in comment (#21439) 2020-09-18 14:26:19 +02:00
07751c3d26 cmd/geth: added counters to the geth inspect report (#21495)
* database: added counters

* Improved stats for ancient db

* Small improvement

* Better message and added percentage while counting receipts

* Fast counting for receipts

* added info message

* Show both receips itemscount  from ancient db and counted receipts

* Fixed default case

* Removed counter for receipts in ancient store

* Removed counting of receipts present in leveldb
2020-09-17 10:23:56 +02:00
0185ee0993 core/rawdb: single point of maintenance for writing and deleting tx lookup indexes (#21480) 2020-09-15 10:37:01 +02:00
a3cd8a040a core/vm: fix benchmark overflow + prep for precompile repricings (#21530)
* core/vm/testdata: add gascost expectations to testcases

* core/vm: verify expected gas in tests for precompiles

* core/vm: fix overflow flaw in gas/s calculation
2020-09-10 09:19:30 +02:00
eeaf191633 core, eth, trie: prepare trie sync for path based operation 2020-09-02 13:21:32 +03:00
d8da0b3d81 core/state, eth, trie: stabilize memory use, fix memory leak 2020-08-26 13:05:06 +03:00
5655dce3b8 core/rawdb: only complain loudly if truncating many items 2020-08-25 09:03:14 +03:00
3ee91b9f2e core/state/snapshot: reduce disk layer depth during generation 2020-08-24 13:22:36 +03:00
1b5a867eec core: do less lookups when writing fast-sync block bodies (#21468) 2020-08-22 18:12:04 +02:00
87c0ba9213 core, eth, les, trie: add a prefix to contract code (#21080) 2020-08-21 15:10:40 +03:00
8cbdc8638f core: define and test chain rewind corner cases (#21409)
* core: define and test chain reparation cornercases

* core: write up a variety of set-head tests

* core, eth: unify chain rollbacks, handle all the cases

* core: make linter smile

* core: remove commented out legacy code

* core, eth/downloader: fix review comments

* core: revert a removed recovery mechanism
2020-08-20 13:01:24 +03:00
0bdd295cc0 core: more detailed metering for reorgs (#21420) 2020-08-20 09:49:35 +02:00
2ff464b29d core/state: fixed some comments (#21450) 2020-08-19 09:54:21 +03:00
4fde0cabc1 Merge pull request #21411 from holiman/fix_codelookup
core/vm: avoid map lookups for accessing jumpdest analysis
2020-08-06 08:09:15 +03:00
5a88a7cf5b core: use errors.Is for consensus errors check (#21095) 2020-08-05 09:52:54 +02:00
8ead45c20b core/vm: avoid map lookups for accessing jumpdest analysis 2020-08-04 15:45:35 +02:00
b35e4fce99 core: avoid modification of accountSet cache in tx_pool (#21159)
* core: avoid modification of accountSet cache in tx_pool

when runReorg, we may copy the dirtyAccounts' accountSet cache to promoteAddrs
in which accounts will be promoted, however, if we have reset request at the
same time, we may reuse promoteAddrs and modify the cache content which is
against the original intention of accountSet cache. So, we need to make a new
slice here to avoid modify accountSet cache.

* core: fix flatten condition + comment

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-08-04 11:51:53 +02:00
c0c01612e9 node: refactor package node (#21105)
This PR significantly changes the APIs for instantiating Ethereum nodes in
a Go program. The new APIs are not backwards-compatible, but we feel that
this is made up for by the much simpler way of registering services on
node.Node. You can find more information and rationale in the design
document: https://gist.github.com/renaynay/5bec2de19fde66f4d04c535fd24f0775.

There is also a new feature in Node's Go API: it is now possible to
register arbitrary handlers on the user-facing HTTP server. In geth, this
facility is used to enable GraphQL.

There is a single minor change relevant for geth users in this PR: The
GraphQL API is no longer available separately from the JSON-RPC HTTP
server. If you want GraphQL, you need to enable it using the
./geth --http --graphql flag combination.

The --graphql.port and --graphql.addr flags are no longer available.
2020-08-03 19:40:46 +02:00
a00dc5095b Merge pull request #21358 from hendrikhofstadt/fix/tx-sort-time
core: sort txs at the same gas price by received time
2020-07-30 10:23:36 +03:00
ff90894636 core/rawdb: convert some comments to godoc convention (#21384) 2020-07-29 21:53:59 +02:00
9e04c5ec83 core/bloombits: use single channel for shutdown (#20878)
This replaces the two-stage shutdown scheme with the one we
use almost everywhere else: a single quit channel signalling
termination.

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-07-29 13:49:12 +02:00
298a19bbc6 core: API-less transaction time sorting 2020-07-28 17:13:40 +03:00
c47052a580 core: sort txs at the same gas price by received time 2020-07-28 17:13:39 +03:00
93da0cf8a1 cmd, core, eth, light, trie: dump clean cache periodically (#20391)
* cmd, core, eth, light, trie: dump clean cache periodically

* eth: update config

* trie: minor fix

* core, trie: address comments

* eth: remove useless

* trie: print clean cache dump start too

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-07-28 16:30:31 +03:00
6793ffa12b Merge pull request #21300 from rjl493456442/txpool-fix-queued-evictions
core: fix queued transaction eviction
2020-07-24 11:14:42 +03:00
5413df1dfa core: fix heartbeat in txpool
core: address comment
2020-07-24 11:12:59 +03:00
c374447401 core: fix queued transaction eviction
Solves issue#20582. Non-executable transactions should not be evicted on each tick if there are no promote transactions or if a pending/reset empties the pending list. Tests and logging expanded to handle these cases in the future.

core/tx_pool: use a ts for each tx in the queue, but only update the heartbeat on promotion or pending replaced

queuedTs proper naming
2020-07-24 11:11:57 +03:00
105922180f eth/downloader: refactor downloader + queue (#21263)
* eth/downloader: refactor downloader + queue

downloader, fetcher: throttle-metrics, fetcher filter improvements, standalone resultcache

downloader: more accurate deliverytime calculation, less mem overhead in state requests

downloader/queue: increase underlying buffer of results, new throttle mechanism

eth/downloader: updates to tests

eth/downloader: fix up some review concerns

eth/downloader/queue: minor fixes

eth/downloader: minor fixes after review call

eth/downloader: testcases for queue.go

eth/downloader: minor change, don't set progress unless progress...

eth/downloader: fix flaw which prevented useless peers from being dropped

eth/downloader: try to fix tests

eth/downloader: verify non-deliveries against advertised remote head

eth/downloader: fix flaw with checking closed-status causing hang

eth/downloader: hashing avoidance

eth/downloader: review concerns + simplify resultcache and queue

eth/downloader: add back some locks, address review concerns

downloader/queue: fix remaining lock flaw

* eth/downloader: nitpick fixes

* eth/downloader: remove the *2*3/4 throttling threshold dance

* eth/downloader: print correct throttle threshold in stats

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-07-24 10:46:26 +03:00
7163a6664e ethclient: serialize negative block number as "pending" (#21177)
Fixes #21175

Co-authored-by: sammy007 <sammy007@users.noreply.github.com>
Co-authored-by: Adam Schmideg <adamschmideg@users.noreply.github.com>
2020-07-21 10:51:15 +02:00
508891e64b core/vm: use pointers to operations vs. copy by value 2020-07-16 15:32:01 +03:00
9e88224eb8 core: raise gas limit in --dev mode, seed blake precompile (#21323)
* Set gasLimit in --dev mode to be 9m.

* core: Set gasLimit to 11.5 milion and add 1 wei allocation for BLAKE2b
2020-07-16 15:08:38 +03:00
295693759e core/vm: less allocations for various call variants (#21222)
* core/vm/runtime/tests: add more benchmarks

* core/vm: initial work on improving alloc count for calls to precompiles

name                                  old time/op    new time/op    delta
SimpleLoop/identity-precompile-10M-6     117ms ±75%      43ms ± 1%  -63.09%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   79.6ms ± 4%    70.5ms ± 1%  -11.42%  (p=0.008 n=5+5)

name                                  old alloc/op   new alloc/op   delta
SimpleLoop/identity-precompile-10M-6    24.4MB ± 0%     4.9MB ± 0%  -79.94%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   13.2kB ± 0%    13.2kB ± 0%     ~     (p=0.357 n=5+5)

name                                  old allocs/op  new allocs/op  delta
SimpleLoop/identity-precompile-10M-6      382k ± 0%      153k ± 0%  -59.99%  (p=0.000 n=5+4)
SimpleLoop/loop-10M-6                     40.0 ± 0%      40.0 ± 0%     ~     (all equal)

* core/vm: don't allocate big.int for touch

name                                  old time/op    new time/op    delta
SimpleLoop/identity-precompile-10M-6    43.3ms ± 1%    42.4ms ± 7%     ~     (p=0.151 n=5+5)
SimpleLoop/loop-10M-6                   70.5ms ± 1%    76.7ms ± 1%   +8.67%  (p=0.008 n=5+5)

name                                  old alloc/op   new alloc/op   delta
SimpleLoop/identity-precompile-10M-6    4.90MB ± 0%    2.46MB ± 0%  -49.83%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   13.2kB ± 0%    13.2kB ± 1%     ~     (p=0.571 n=5+5)

name                                  old allocs/op  new allocs/op  delta
SimpleLoop/identity-precompile-10M-6      153k ± 0%       76k ± 0%  -49.98%  (p=0.029 n=4+4)
SimpleLoop/loop-10M-6                     40.0 ± 0%      40.0 ± 0%     ~     (all equal)

* core/vm: reduce allocs in staticcall

name                                  old time/op    new time/op    delta
SimpleLoop/identity-precompile-10M-6    42.4ms ± 7%    37.5ms ± 6%  -11.68%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   76.7ms ± 1%    69.1ms ± 1%   -9.82%  (p=0.008 n=5+5)

name                                  old alloc/op   new alloc/op   delta
SimpleLoop/identity-precompile-10M-6    2.46MB ± 0%    0.02MB ± 0%  -99.35%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   13.2kB ± 1%    13.2kB ± 0%     ~     (p=0.143 n=5+5)

name                                  old allocs/op  new allocs/op  delta
SimpleLoop/identity-precompile-10M-6     76.4k ± 0%      0.1k ± 0%     ~     (p=0.079 n=4+5)
SimpleLoop/loop-10M-6                     40.0 ± 0%      40.0 ± 0%     ~     (all equal)

* trie: better use of hasher keccakState

* core/state/statedb: reduce allocations in getDeletedStateObject

* core/vm: reduce allocations in all call derivates

* core/vm: reduce allocations in call variants

- Make returnstack `uint32`
- Use a `sync.Pool` of `stack`s

* core/vm: fix tests

* core/vm: goimports

* core/vm: tracer fix + staticcall gas fix

* core/vm: add back snapshot to staticcall

* core/vm: review concerns + make returnstack pooled + enable returndata in traces

* core/vm: fix some test tracer method signatures

* core/vm: run gencodec, minor comment polish

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-07-16 15:06:19 +03:00