Commit Graph

12578 Commits

Author SHA1 Message Date
d5e57948d1 core/types: drop some relice data types 2021-04-14 23:39:42 +03:00
7088f1e814 core, eth: faster snapshot generation (#22504)
* eth/protocols: persist received state segments

* core: initial implementation

* core/state/snapshot: add tests

* core, eth: updates

* eth/protocols/snapshot: count flat state size

* core/state: add metrics

* core/state/snapshot: skip unnecessary deletion

* core/state/snapshot: rename

* core/state/snapshot: use the global batch

* core/state/snapshot: add logs and fix wiping

* core/state/snapshot: fix

* core/state/snapshot: save generation progress even if the batch is empty

* core/state/snapshot: fixes

* core/state/snapshot: fix initial account range length

* core/state/snapshot: fix initial account range

* eth/protocols/snap: store flat states during the healing

* eth/protocols/snap: print logs

* core/state/snapshot: refactor (#4)

* core/state/snapshot: refactor

* core/state/snapshot: tiny fix and polish

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* core, eth: fixes

* core, eth: fix healing writer

* core, trie, eth: fix paths

* eth/protocols/snap: fix encoding

* eth, core: add debug log

* core/state/generate: release iterator asap (#5)

core/state/snapshot: less copy

core/state/snapshot: revert split loop

core/state/snapshot: handle storage becoming empty, improve test robustness

core/state: test modified codehash

core/state/snapshot: polish

* core/state/snapshot: optimize stats counter

* core, eth: add metric

* core/state/snapshot: update comments

* core/state/snapshot: improve tests

* core/state/snapshot: replace secure trie with standard trie

* core/state/snapshot: wrap return as the struct

* core/state/snapshot: skip wiping correct states

* core/state/snapshot: updates

* core/state/snapshot: fixes

* core/state/snapshot: fix panic due to reference flaw in closure

* core/state/snapshot: fix errors in state generation logic + fix log output

* core/state/snapshot: remove an error case

* core/state/snapshot: fix condition-check for exhausted snap state

* core/state/snapshot: use stackTrie for small tries

* core/state/snapshot: don't resolve small storage tries in vain

* core/state/snapshot: properly clean up storage of deleted accounts

* core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks

* core/state/snapshot: fix error (+testcase)

* core/state/snapshot: clean up tests a bit

* core/state/snapshot: work in progress on better tests

* core/state/snapshot: polish code

* core/state/snapshot: fix trie iteration abortion trigger

* core/state/snapshot: fixes flaws

* core/state/snapshot: remove panic

* core/state/snapshot: fix abort

* core/state/snapshot: more tests (plus failing testcase)

* core/state/snapshot: more testcases + fix for failing test

* core/state/snapshot: testcase for malformed data

* core/state/snapshot: some test nitpicks

* core/state/snapshot: improvements to logging

* core/state/snapshot: testcase to demo error in abortion

* core/state/snapshot: fix abortion

* cmd/geth: make verify-state report the root

* trie: fix failing test

* core/state/snapshot: add timer metrics

* core/state/snapshot: fix metrics

* core/state/snapshot: udpate tests

* eth/protocols/snap: write snapshot account even if code or state is needed

* core/state/snapshot: fix diskmore check

* core/state/snapshot: review fixes

* core/state/snapshot: improve error message

* cmd/geth: rename 'error' to 'err' in logs

* core/state/snapshot: fix some review concerns

* core/state/snapshot, eth/protocols/snap: clear snapshot marker when starting/resuming snap sync

* core: add error log

* core/state/snapshot: use proper timers for metrics collection

* core/state/snapshot: address some review concerns

* eth/protocols/snap: improved log message

* eth/protocols/snap: fix heal logs to condense infos

* core/state/snapshot: wait for generator termination before restarting

* core/state/snapshot: revert timers to counters to track total time

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-04-14 23:23:11 +03:00
a50251e6cb eth/fetcher: avoid spurious timer events at startup (#22652)
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-14 12:44:32 +02:00
72e37942f3 cmd/faucet: support testnet flags in the faucet (#22545)
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-04-13 23:51:46 +02:00
271e5b7fc9 cmd/geth: add db-command to inspect freezer index (#22633)
This PR makes it easier to inspect the freezer index, which could be useful to investigate things like #22111
2021-04-13 15:45:30 +02:00
6c27d8f996 accounts: documentation fixes (#22645)
* replaces `an chance` with `a chance`
* replaces `SignHashWithPassphrase` with `SignTextWithPassphrase` as there was no SignHashWithPasspharse function in the file
2021-04-13 10:00:48 +02:00
9c653ff662 Merge pull request #22636 from karalabe/drop-eth64
eth, les: drop support for eth/64
2021-04-09 13:52:21 +03:00
fe1586b094 eth, les: drop support for eth/64, fix eth/66 tests 2021-04-09 10:39:45 +03:00
04dcc9378d params: begin v1.10.3 release cycle 2021-04-08 13:04:30 +02:00
97d11b0187 params: release go-ethereum v1.10.2 stable v1.10.2 2021-04-08 13:02:25 +02:00
a665f5d2d6 Merge pull request #22632 from karalabe/cht-1.10.2
params: update CHTs for v1.10.2
2021-04-08 13:27:01 +03:00
e3ff37c47a params: update CHTs for v1.10.2 2021-04-08 13:23:34 +03:00
264fff7dd4 Merge pull request #22629 from karalabe/fix-chain-tracing-from-0
eth: fix tracing state retrieval if requesting the non-dirty genesis
2021-04-07 18:52:29 +03:00
2a7c4b62a9 eth: fix tracing state retrieval if requesting the non-dirty genesis 2021-04-07 18:35:25 +03:00
9d10856e84 core, eth, internal/ethapi: create access list RPC API (#22550)
* core/vm: implement AccessListTracer

* eth: implement debug.createAccessList

* core/vm: fixed nil panics in accessListTracer

* eth: better error messages for createAccessList

* eth: some fixes on CreateAccessList

* eth: allow for provided accesslists

* eth: pass accesslist by value

* eth: remove created acocunt from accesslist

* core/vm: simplify access list tracer

* core/vm: unexport accessListTracer

* eth: return best guess if al iteration times out

* eth: return best guess if al iteration times out

* core: docstring, unexport methods

* eth: typo

* internal/ethapi: move createAccessList to eth package

* internal/ethapi: remove reexec from createAccessList

* internal/ethapi: break if al is equal to last run, not if gas is equal

* internal/web3ext: fixed arguments

* core/types: fixed equality check for accesslist

* core/types: no hardcoded vals

* core, internal: simplify access list generation, make it precise

* core/vm: fix typo

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-04-07 17:54:31 +03:00
a600dab7e5 eth, les: fix tracers (#22473)
* eth, les: fix tracer

* eth: isolate live trie database in tracer

* eth: fix nil

* eth: fix

* eth, les: add checkLive param

* eth/tracer: fix
2021-04-07 10:30:26 +03:00
2d89fe0883 les: move client pool to les/vflux/server (#22495)
* les: move client pool to les/vflux/server

* les/vflux/server: un-expose NodeBalance, remove unused fn, fix bugs

* tests/fuzzers/vflux: add ClientPool fuzzer

* les/vflux/server: fixed balance tests

* les: rebase fix

* les/vflux/server: fixed more bugs

* les/vflux/server: unexported NodeStateMachine fields and flags

* les/vflux/server: unexport all internal components and functions

* les/vflux/server: fixed priorityPool test

* les/vflux/server: polish balance

* les/vflux/server: fixed mutex locking error

* les/vflux/server: priorityPool bug fixed

* common/prque: make Prque wrap-around priority handling optional

* les/vflux/server: rename funcs, small optimizations

* les/vflux/server: fixed timeUntil

* les/vflux/server: separated balance.posValue and negValue

* les/vflux/server: polish setup

* les/vflux/server: enforce capacity curve monotonicity

* les/vflux/server: simplified requestCapacity

* les/vflux/server: requestCapacity with target range, no iterations in SetCapacity

* les/vflux/server: minor changes

* les/vflux/server: moved default factors to balanceTracker

* les/vflux/server: set inactiveFlag in priorityPool

* les/vflux/server: moved related metrics to vfs package

* les/vflux/client: make priorityPool temp state logic cleaner

* les/vflux/server: changed log.Crit to log.Error

* add vflux fuzzer to oss-fuzz

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2021-04-06 20:42:50 +02:00
e275b1a293 consensus/ethash: replace a magic number with it's constant (#22618) 2021-04-06 21:02:52 +03:00
95219ae62d cmd/utils: move cache sanity check to SetEthConfig (#22510)
Move the cache sanity check to the SetEthConfig function
to allow the config file to load.
2021-04-06 16:23:35 +02:00
5338ce4447 internal/debug: add JSON log format and rename logging flags (#22341)
This change adds support for logging JSON records when the --log.json flag is
given. The --debug and --backtrace flags are deprecated and replaced by
--log.debug and --log.backtrace.

While changing this, it was noticed that the --memprofilerate and
--blockprofilerate were ineffective (they were always overridden even if
--pprof.memprofilerate was not set). This is also fixed.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-06 16:09:51 +02:00
adf09aeab1 graphql: add support for tx types and tx access lists (#22491)
This adds support for EIP-2718 access list transactions in the GraphQL API.

Co-authored-by: Amit Shah <amitshah0t7@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-06 15:58:36 +02:00
706683ea72 internal/ethapi: fix eth_chainId method (#22243)
This removes the duplicated definition of eth_chainID
in package eth and updates the definition in internal/ethapi
to treat chain ID as a bigint.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-04-06 12:24:39 +02:00
c79fc209cd core/state/snapshot: fix data race in diff layer (#22540) 2021-04-06 11:57:00 +03:00
73ed68988f Merge pull request #22604 from karalabe/hashrate-capitalization-fix
all: fix miner hashRate -> hashrate on API calls
2021-03-31 19:27:23 +03:00
55300d4fdb all: fix miner hashRate -> hashrate on API calls 2021-03-31 10:56:51 +03:00
4a37ae510e rpc: tighter shutdown synchronization in client subscription (#22597)
This fixes a rare issue where the client subscription forwarding loop
would attempt send on the subscription's channel after Unsubscribe has
returned, leading to a panic if the subscription channel was already
closed by the user. Example:

    sub, _ := client.Subscribe(..., channel, ...)
    sub.Unsubscribe()
    close(channel)

The race occurred because Unsubscribe called quitWithServer to tell the
forwarding loop to stop sending on sub.channel, but did not wait for the
loop to actually come down. This is fixed by adding an additional channel
to track the shutdown, on which Unsubscribe now waits.

Fixes #22322
2021-03-30 20:09:30 +02:00
61ff3e86b2 core/state/snapshot, ethdb: track deletions more accurately (#22582)
* core/state/snapshot, ethdb: track deletions more accurately

* core/state/snapshot: don't reset the iterator, leveldb's screwy

* ethdb: don't mess with the insert batches for now
2021-03-30 19:04:22 +03:00
3faae5defc ethstats: avoid creating subscriptions on background goroutine (#22587)
This fixes an issue where the ethstats service could crash if geth was
started and then immediately stopped due to an internal error. The
cause of the crash was a nil subscription being returned by the backend,
because the background goroutine creating them was scheduled after
the backend had already shut down.

Moving the creation of subscriptions into the Start method, which runs
synchronously during startup of the node, means the returned subscriptions
can never be 'nil'.

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-03-30 15:52:03 +02:00
44fe466999 core/vm: fix Byzantium address list (#22603) 2021-03-30 16:38:53 +03:00
59ac3c9fd3 cmd/geth: add db dumptrie command (#22563)
Adds the command "geth db dumptrie <root> <seek> <max>", to better help investigate the trie data
2021-03-30 13:57:21 +02:00
43a3768066 Merge pull request #22596 from holiman/snap_timelimits
eth/protocols/snap: try to prevent requests timing out
2021-03-30 14:49:59 +03:00
24588bacfd cmd/puppeth: specify working directory for nodejs 15 (#22549) 2021-03-29 20:58:58 +02:00
b6912c1047 core: add BlockGen.GetBalance method (#22589) 2021-03-29 20:54:49 +02:00
7644795950 eth/protocols/snap: try to prevent requests timing out 2021-03-29 14:17:35 +02:00
76700ac892 core/state/pruner: move the compaction out of the pruning procedure (#22579)
The main idea behind it is: the range compaction is very expensive
which can take a few hours to finish. During this long procedure,
a lot of exceptions can occur, e.g.
- Geth is killed manually
- Geth is killed because of machine crash
- etc

In order to minimize the effect of the exceptions, the compaction
is moved out of the pruning. So that even the compaction is not
finished, the pruning is regarded as done.
2021-03-29 11:09:29 +02:00
27056f62e5 cmd/devp2p: update to newer cloudflare API client (#22588)
This upgrades the cloudflare client dependency to v0.14.0. The new
version changes the API because all methods now require a context
parameter. This change also reduces the log level of the 'Skipping...'
message to debug, following a similar change in the AWS deployer.
2021-03-26 23:15:20 +02:00
e7f3962612 Merge pull request #22591 from karalabe/snap-detailed-timeout-logs
eth/protocols/snap: add peer id and req id to the timeout logs
2021-03-26 23:14:16 +02:00
099be04100 eth/protocols/snap: add peer id and req id to the timeout logs 2021-03-26 22:29:22 +02:00
063f78c6ba Merge pull request #22590 from karalabe/nil-histogram
metrics/influxdb: don't push empty histograms, no measurement != 0
2021-03-26 21:43:09 +02:00
62379f02c6 metrics/influxdb: don't push empty histograms, no measurement != 0 2021-03-26 21:13:52 +02:00
cae6b5527e cmd/geth, consensus/ethash: add support for --miner.notify.full flag (#22558)
The PR implements the --miner.notify.full flag that enables full pending block
notifications. When this flag is used, the block notifications sent to mining
endpoints contain the complete block header JSON instead of a work package
array.

Co-authored-by: AlexSSD7 <alexandersadovskyi7@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-03-26 18:30:10 +01:00
955727181b eth: fix corner case in sync head determination (#21695)
This avoids synchronisation failures when the local header is ahead of
the local full block.
2021-03-26 18:06:25 +01:00
6477ea6502 Merge pull request #22586 from karalabe/resettting-histogram
eth/protocols, metrics: use resetting histograms for rare packets
2021-03-26 16:49:37 +02:00
2550e46269 eth/protocols, metrics: use resetting histograms for rare packets 2021-03-26 16:14:12 +02:00
efe0229768 Merge pull request #22581 from karalabe/network-latency-monitoring
eth/protocols, metrics, p2p: add handler performance metrics
2021-03-26 14:26:51 +02:00
6d7ff6acea eth/protocols, metrics, p2p: add handler performance metrics 2021-03-26 14:00:06 +02:00
54c0d573d7 eth: dump rpc gas cap and tx fee cap (#22574) 2021-03-25 16:37:51 +02:00
4adf0b9187 Merge pull request #22573 from holiman/checkfreq
core: fix condition on header verification
2021-03-25 13:33:53 +02:00
bed74b38d9 cmd/devp2p: fix comparison of TXT record value (#22572)
* cmd/devp2p: fix comparison of TXT record value

The AWS API returns quoted DNS strings, so we must encode the new value
before comparing it against the existing record content.

* cmd/devp2p: add test

* cmd/devp2p: fix typo and rename val -> newValue
2021-03-25 13:32:32 +02:00
497448bf90 core: fix condition on header verification 2021-03-25 11:50:14 +01:00