This updates the consensus tests to commit 31d6630 and
adds support for access list transactions in the test runner.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR implements the first one of the "lespay" UDP queries which
is already useful in itself: the capacity query. The server pool is making
use of this query by doing a cheap UDP query to determine whether it is
worth starting the more expensive TCP connection process.
This replaces the github.com/pborman/uuid dependency with
github.com/google/uuid because the former is only a wrapper for
the latter (since v1.0.0).
Co-authored-by: Felix Lange <fjl@twurst.com>
* les: move serverPool to les/vflux/client
* les: add metrics
* les: moved ValueTracker inside ServerPool
* les: protect against node registration before server pool is started
* les/vflux/client: fixed tests
* les: make peer registration safe
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.
There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.
The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID.
This change also adds support for the YoloV3 testnet.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
Transaction unindexing will be enabled by default as of 1.10, which causes tx status retrieval will be broken without this PR.
This PR introduces a retry mechanism in TxStatus retrieval.
This PR adds a more CLI flag, so that the les-server can serve light clients even the local node is not synced yet.
This functionality is needed in some testing environments(e.g. hive). After launching the les server, no more blocks will be imported so the node is always marked as "non-synced".
This PR prevents users from submitting transactions without EIP-155 enabled. This behaviour can be overridden by specifying the flag --rpc.allow-unprotected-txs=true.
This PR introduces:
- db.put to put a value into the database
- db.get to read a value from the database
- db.delete to delete a value from the database
- db.stats to check compaction info from the database
- db.compact to trigger a db compaction
It also moves inspectdb to db.inspect.
* les: refactored server handler
* tests/fuzzers/les: add fuzzer for les server handler
* tests, les: update les fuzzer
tests: update les fuzzer
tests/fuzzer/les: release resources
tests/fuzzer/les: pre-initialize all resources
* les: refactored server handler and fuzzer
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
In the random sync algorithm used by the DNS node iterator, we first pick a random
tree and then perform one sync action on that tree. This happens in a loop until any
node is found. If no trees contain any nodes, the iterator will enter a hot loop spinning
at 100% CPU.
The fix is complicated. The iterator now checks if a meaningful sync action can
be performed on any tree. If there is nothing to do, it waits for the next root record
recheck time to arrive and then tries again.
Fixes#22306
io.Reader may return n > 0 and io.EOF at the end of the input stream.
readFull did not handle this correctly, looking only at the error. This fixes
it to check for n == len(buf) as well.
This PR optimizes the broadcast loop. Instead of iterating twice through a given set of transactions to weed out which peers have and which do not have a tx, to send/announce transactions, we do it only once.
There was a dormant error with structured inputs that failed unpacking.
This commit fixes the error by switching casting to the better abi.ConvertType function.
It also adds a test for calling a view function that returns a struct
Prevents a situation where we (not running snap) connects with a peer running snap, and get stalled waiting for snap registration to succeed (which will never happen), which cause a waitgroup wait to halt shutdown
This moves the eth config definition into a separate package, eth/ethconfig.
Packages eth and les can now import this common package instead of
importing eth from les, reducing dependencies.
Co-authored-by: Felix Lange <fjl@twurst.com>
fixes an issue where local transactions that were included in the chain before a SetHead were rejected if resubmitted, since the txpool had not reset the state to the current (older) state.
The PR makes use of the stacktrie, which is is more lenient on resource consumption, than the regular trie, in cases where we only need it for DeriveSha
* remove uneeded convertion type
* remove redundant type in composite literal
* omit explicit type where implicit
* remove unused redundant parenthesis
* remove redundant import alias duktape
This change allows users to set a custom path prefix on which to mount the http-rpc
or ws-rpc handlers via the new flags --http.rpcprefix and --ws.rpcprefix.
Fixes#21826
Co-authored-by: Felix Lange <fjl@twurst.com>
Removes the yolov2 definition, adds yolov3, including EIP-2565. This PR also disables some of the erroneously generated blockchain and statetests, and adds the new genesis hash + alloc for yolov3.
This PR disables the CLI switches for yolo, since it's not complete until we merge support for 2930.
This PR enables running the new discv5 protocol in both LES client
and server mode. In client mode it mixes discv5 and dnsdisc iterators
(if both are enabled) and filters incoming ENRs for "les" tag and fork ID.
The old p2p/discv5 package and all references to it are removed.
Co-authored-by: Felix Lange <fjl@twurst.com>
* accounts/scwallet: use go-ethereum crypto instead of go-ecdh
github.com/wsddn/go-ecdh is a wrapper package for ECDH functionality
with any elliptic curve.
Since 'generic' ECDH is not required in accounts/scwallet (the curve is
always secp256k1), we can just use the standard library functionality
and our own crypto libraries to perform ECDH and save a dependency.
* Update accounts/scwallet/securechannel.go
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
* Use the correct key
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
This PR implements the following modifications
- Don't shortcut check if block is present, thus avoid disk lookup
- Don't check hash ancestry in early-check (it's still done in parallel checker)
- Don't check time.Now for every single header
Charts and background info can be found here: https://github.com/holiman/headerimport/blob/main/README.md
With these changes, writing 1M headers goes down to from 80s to 62s.
This moves the tracing RPC API implementation to package eth/tracers.
By doing so, package eth no longer depends on tracing and the duktape JS engine.
The change also enables tracing using the light client. All tracing methods work with the
light client, but it's a lot slower compared to using a full node.
This PR fixes the receipt status field to be decimal instead of a hex string,
as called for by the spec.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Squashed from the following commits:
core/state: lazily init snapshot storage map
core/state: fix flawed meter on storage reads
core/state: make statedb/stateobjects reuse a hasher
core/blockchain, core/state: implement new trie prefetcher
core: make trie prefetcher deliver tries to statedb
core/state: refactor trie_prefetcher, export storage tries
blockchain: re-enable the next-block-prefetcher
state: remove panics in trie prefetcher
core/state/trie_prefetcher: address some review concerns
sq
This PR introduces a new config field SyncFromCheckpoint for light client.
In some special scenarios, it's required to start synchronization from some
arbitrary checkpoint or even from the scratch. So this PR offers this
flexibility to users so that the synchronization start point can be configured.
There are two relevant configs: SyncFromCheckpoint and Checkpoint.
- If the SyncFromCheckpoint is true, the light client will try to sync from the
specified checkpoint.
- If the Checkpoint is not configured, then the light client will sync from the
scratch(from the latest header if the database is not empty)
Additional notes: these two configs are not visible in the CLI flags but only
accessable in the config file.
Example Usage:
[Eth]
SyncFromCheckpoint = true
[Eth.Checkpoint]
SectionIndex = 100
SectionHead = "0xabc"
CHTRoot = "0xabc"
BloomRoot = "0xabc"
PS. Historical checkpoint can be retrieved from the synced full node or light
client via les_getCheckpoint API.
Adding warnings of free disk space left and graceful shutdown when there is not enough space left.
This also adds a flag datadir.minfreedisk which can be used to set the trigger for low disk space, and setting it to zero disables the check.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR has two changes in the les protocol:
- the auxRoot is not supported. See ethereum/devp2p#171 for more information
- the empty response will be returned in GetHelperTrieProofsMsg request if the merkle
proving is failed. note, for backward compatibility, the empty merkle proof as well as
the request auxiliary data will still be returned in les2/3 protocol no matter the proving
is successful or not. the proving failure can happen e.g. request the proving for a
non-included entry in helper trie (unstable header).
The tests sometimes failed with certain go versions because
the behavior of http.Server.Shutdown changed over time. A bug
that was fixed in Go 1.15 could cause active connections on unrelated
servers to close unexpectedly. This is fixed by avoiding use of the
same port number in all tests.
USB enumeration still occured. Make sure it will only occur if --usb is set.
This also deprecates the 'NoUSB' config file option in favor of a new option 'USB'.
Resolves https://github.com/etclabscore/core-geth/issues/273
jsre.JSRE already handles establishing preload
file paths relative to the 'assets' path (aka docroot),
where it joins the assets dir and the file path if relative,
or uses the file path only if absolute.
The duplication of this logic by MakeConsolePreloads
caused preloaded files to have paths which contained
duplicate references to the assets dir path.
Date: 2020-12-30 08:25:01-06:00
Signed-off-by: meows <b5c6@protonmail.com>
This changes the chainID RPC method to return an error when EIP-155 is not yet
active at the current block height. It used to simply return zero in this case, but
that's confusing.
This commit extends the ethclient test suite and increases code coverage of the ethclient
package from ~15% to >55%. These tests act as early smoke tests to signal issues in the
RPC-interface. E.g. if a functionality like eth_chainId or eth_call breaks, the test
will break.
This changes the JSON encoding of blocks returned by the API
to have decimal instead of hexadecimal numbers. The spec wants
it this way.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR removes a logic in the miner, which was originally intended to help temporary testnets based on ethash from "running off into the future". If the difficulty was low, and a few computers started mining several blocks per second, the ethash rules (which demand 1s delay between blocks) would push the blocktimes further and further away.
The solution was to make the miner sleep while this happened.
Nowadays, this problem is solved instead by PoA chains, and it's recommended to let testnets and devnets be based on clique instead. The existing logic is problematic, since it can cause stalls within the miner making it difficult for remote workers to submit work if the channel is blocked on a sleep.
Credits to Saar Tochner for reporting this via the bug bounty
The legacy dot-org URL was displaying a message about the repository
having migrated to the dot-com service, which now covers open-source
projects as well.
During the snap and eth refactor, the net_version rpc call was falsely deprecated.
This restores the net_version RPC handler as most eth2 nodes and other software
depend on it.
* les: allow tx unindexing in les/4 light server mode
* les: minor fixes
* les: more small fixes
* les: add meaningful constants for recentTxIndex handshake field
This PR implements unclean shutdown marker. Every time geth boots, it adds a timestamp to a list of timestamps in the database. This list is capped at 10. At a clean shutdown, the timestamp is removed again.
Thus, when geth exits unclean, the marker remains, and at boot up we show the most recent unclean shutdowns to the user, which makes it easier to diagnose root-causes to certain problems.
Co-authored-by: Nagy Salem <me@muhnagy.com>
This PR adds re-written difficulty calculators, which are based on uint256. It also adds a fuzzer + oss-fuzz integration for the new fuzzer. It does differential fuzzing between the new and old calculators.
Note: this PR does not actually enable the new calculators.
This PR adds support for using Twitter API to query the tweet and author details. There are two reasons behind this change:
- Twitter will be deprecating the legacy website on 15th December. The current method is expected to stop working then.
- More importantly, the current system uses Twitter handle for spam protection but the Twitter handle can be changed via automated calls. This allows bots to use the same tweet to withdraw funds infinite times as long as they keep changing their handle between every request. The Rinkeby as well as the Goerli faucet are being actively drained via this method. This PR changes the spam protection to be based on Twitter IDs instead of usernames. A user can not change their Twitter ID.
In miner/worker.go, there are two goroutine using channel w.newWorkCh: newWorkerLoop() sends to this channel, and mainLoop() receives from this channel. Only the receive operation is in a select.
However, w.exitCh may be closed by another goroutine. This is fine for the receive since receive is in select, but if the send operation is blocking, then it will block forever. This commit puts the send in a select, so it won't block even if w.exitCh is closed.
Similarly, there are two goroutines using channel errc: the parent that runs the test receives from it, and the child created at line 573 sends to it. If the parent goroutine exits too early by calling t.Fatalf() at line 614, then the child goroutine will be blocked at line 574 forever. This commit adds 1 buffer to errc. Now send will not block, and receive is not influenced because receive still needs to wait for the send.
The previous fix#21960 converted the float to an intermediate signed int, before attempting the uint conversion. Although this works, this doesn't guarantee that other architectures will work the same.
This commit fixes a flaw in two testcases, and brings down the exec-time from ~40s to ~8s for trie/TestIncompleteSync.
The checkConsistency was performed over and over again on the complete set of nodes, not just the recently added, turning it into a quadratic runtime.
The database panicked for invalid IPs. This is usually no problem
because all code paths leading to node DB access verify the IP, but it's
dangerous because improper validation can turn this panic into a DoS
vulnerability. The quick fix here is to just turn database accesses
using invalid IP into a noop. This isn't great, but I'm planning to
remove the node DB for discv5 long-term, so it should be fine to have
this quick fix for half a year.
Fixes#21849
This fixes some issues in crypto/signify and makes release signing work.
The archive signing step in ci.go used getenvBase64, which decodes the key data.
This is incorrect here because crypto/signify already base64-decodes the key.
* core: add test for headerchain inserts
* core, light: write headerchains in batches
* core: change to one callback per batch of inserted headers + review concerns
* core: error-check on batch write
* core: unexport writeHeaders
* core: remove callback parameter in InsertHeaderChain
The semantics of InsertHeaderChain are now much simpler: it is now an
all-or-nothing operation. The new WriteStatus return value allows
callers to check for the canonicality of the insertion. This change
simplifies use of HeaderChain in package les, where the callback was
previously used to post chain events.
* core: skip some hashing when writing headers
* core: less hashing in header validation
* core: fix headerchain flaw regarding blacklisted hashes
Co-authored-by: Felix Lange <fjl@twurst.com>
Both Hash and Address have a String method, which returns the value as
hex with 0x prefix. They also had a Format method which tried to print
the value using printf of []byte. The way Format worked was at odds with
String though, leading to a situation where fmt.Sprintf("%v", hash)
returned the decimal notation and hash.String() returned a hex string.
This commit makes it consistent again. Both types now support the %v,
%s, %q format verbs for 0x-prefixed hex output. %x, %X creates
unprefixed hex output. %d is also supported and returns the decimal
notation "[1 2 3...]".
For Address, the case of hex characters in %v, %s, %q output is
determined using the EIP-55 checksum. Using %x, %X with Address
disables checksumming.
Co-authored-by: Felix Lange <fjl@twurst.com>
This commit enables users to specify which signer they want to use while creating their transactOpts.
Previously all contract interactions used the homestead signer. Now a user can specify whether they
want to sign with homestead or EIP155 and specify the chainID which adds another layer of security.
Closes#16484
* cmd/geth: implement vulnerability check
* cmd/geth: use minisign to verify vulnerability feed
* cmd/geth: add the test too
* cmd/geth: more minisig/signify testing
* cmd/geth: support multiple pubfiles for signing
* cmd/geth: add @holiman minisig pubkey
* cmd/geth: polishes on vulnerability check
* cmd/geth: fix ineffassign linter nit
* cmd/geth: add CVE to version check struct
* cmd/geth/testdata: add missing testfile
* cmd/geth: add more keys to versionchecker
* cmd/geth: support file:// URLs in version check
* cmd/geth: improve key ID printing when signature check fails
Co-authored-by: Felix Lange <fjl@twurst.com>
A lot of times when we hit 'core' errors, example: invalid tx, the information provided is
insufficient. We miss several pieces of information: what account has nonce too high,
and what transaction in that block was offending?
This PR adds that information, using the new type of wrapped errors.
It also adds a testcase which (partly) verifies the output from the errors.
The first commit changes all usage of direct equality-checks on core errors, into
using errors.Is. The second commit adds contextual information. This wraps most
of the core errors with more information, and also wraps it one more time in
stateprocessor, to further provide tx index and tx hash, if such a tx is encoutered in
a block. The third commit uses the chainmaker to try to generate chains with such
errors in them, thus triggering the errors and checking that the generated string meets
expectations.
This PR fixes a deadlock reported here: #21925
The cause is that many operations may be pending, but if the close happens, only one of them gets awoken and exits, the others remain waiting for a signal that never comes.
* signer/core/api: fix derivation of ledger live accounts
For ledger hardware wallets, change account iteration as follows:
- ledger legacy: m/44'/60'/0'/X; for 0<=X<5
- ledger live: m/44'/60'/0'/0/X; for 0<=X<5
- ledger legacy: m/44'/60'/0'/X; for 0<=X<10
- ledger live: m/44'/60'/X'/0/0; for 0<=X<10
Non-ledger derivation is unchanged and remains as:
- non-ledger: m/44'/60'/0'/0/X; for 0<=X<10
* signer/core/api: derive ten default paths for all hardware wallets, plus ten legacy and ten live paths for ledger wallets
* signer/core/api: as .../0'/0/0 already included by default paths, do not include it again with ledger live paths
* accounts, signer: implement path iterators for hd wallets
Co-authored-by: Martin Holst Swende <martin@swende.se>
* internal/build: implement signify's signing func
* Add signify to the ci utility
* fix output file format
* Add unit test for signify
* holiman's + travis' feedback
* internal/build: verify signify's output
* crypto: move signify to common dir
* use go-minisign to verify binaries
* more holiman feedback
* crypto, ci: support minisign output
* only accept one-line trusted comments
* configurable untrusted comments
* code cleanup in tests
* revert to use ed25519 from the stdlib
* bug: fix for empty untrusted comments
* write timestamp as comment if trusted comment isn't present
* rename line checker to commentHasManyLines
* crypto: added signify fuzzer (#6)
* crypto: added signify fuzzer
* stuff
* crypto: updated signify fuzzer to fuzz comments
* crypto: repro signify crashes
* rebased fuzzer on build-signify branch
* hide fuzzer behind gofuzz build flag
* extract key data inside a single function
* don't treat \r as a newline
* travis: fix signing command line
* do not use an external binary in tests
* crypto: move signify to crypto/signify
* travis: fix formatting issue
* ci: fix linter build after package move
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This fixes a deadlock that could occur when a response packet arrived
after a call had already received enough responses and was about to
signal completion to the dispatch loop.
Co-authored-by: Felix Lange <fjl@twurst.com>
* Make sure to return 400 when errors are present in the response
* graphql: use less memory in chainconfig for tests
Co-authored-by: Martin Holst Swende <martin@swende.se>
* added bls fuzzer
* crypto/bls12381: revert bls-changes, fixup fuzzer tests
* fuzzers: split bls fuzzing into 8 different units
* fuzzers/bls: remove (now stale) corpus
* crypto/bls12381: added blsfuzz corpus
* fuzzers/bls12381: fix the bls corpus
* fuzzers: fix oss-fuzz script
* tests/fuzzers: fixups on bls corpus
* test/fuzzers: remove leftover corpus
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
* crypto/bn256: fix bn256Mul fuzzer to not hang on large input
* Update crypto/bn256/bn256_fuzz.go
Co-authored-by: ligi <ligi@ligi.de>
Co-authored-by: ligi <ligi@ligi.de>
* trie: fix tests to work on 32-bit systems
* les: make test work on 32-bit platform
* cmd/geth: fix windows-issues on tests
* trie: improve balance
* cmd/geth: make account tests less verbose + less mem intense
* rpc: make debug-level log output less verbose
* cmd/geth: lint
* Only compare hostnames in ws.origins
Also using a helper function for ToLower consolidates all preparation steps in one function for more maintainable consistency.
Spaces => tabs
Remove a semicolon
Add space at start of comment
Remove parens around conditional
Handle case wehre parsed hostname is empty
When passing a single word like "localhost" the parsed hostname is an empty string. Handle this and the error-parsing case together as default, and the nonempty hostname case in the conditional.
Refactor with new originIsAllowed functions
Adds originIsAllowed() & ruleAllowsOrigin(); removes prepOriginForComparison
Remove blank line
Added tests for simple allowed-orign rule
which does not specify a protocol or port, just a hostname
Fix copy-paste: `:=` => `=`
Remove parens around conditional
Remove autoadded whitespace on blank lines
Compare scheme, hostname, and port with rule
if the rule specifies those portions.
Remove one autoadded trailing whitespace
Better handle case where only origin host is given
e.g. "localhost"
Remove parens around conditional
Refactor: attemptWebsocketConnectionFromOrigin DRY
Include return type on helper function
Provide srv obj in helper fn
Provide srv to helper fn
Remove stray underscore
Remove blank line
parent 93e666b4c1e7e49b8406dc83ed93f4a02ea49ac1
author wbt <wbt@users.noreply.github.com> 1598559718 -0400
committer Martin Holst Swende <martin@swende.se> 1605602257 +0100
gpgsig -----BEGIN PGP SIGNATURE-----
iQFFBAABCAAvFiEEypmrtbNuJK1doP1AaDtDjAWl3fAFAl+zi9ARHG1hcnRpbkBz
d2VuZGUuc2UACgkQaDtDjAWl3fDRiwgAoMtzU8dwRV7Q9xkCwWEx9Wz2f3n6jUr2
VWBycDKGKwRkPPOER3oc9kzjGU/P1tFlK07PjfnAKZ9KWzxpDcJZwYM3xCBurG7A
16y4YsQnzgPNONv3xIkdi3RZtDBIiPFFEmdZFFvZ/jKexfI6JIYPngCAoqdTIFb9
On/aPvvVWQn1ExfmarsvvJ7kUDUG77tZipuacEH5FfFsfelBWOEYPe+I9ToUHskv
+qO6rOkV1Ojk8eBc6o0R1PnApwCAlEhJs7aM/SEOg4B4ZJJneiFuEXBIG9+0yS2I
NOicuDPLGucOB5nBsfIKI3USPeE+3jxdT8go2lN5Nrhm6MimoILDsQ==
=sgUp
-----END PGP SIGNATURE-----
Refactor: drop err var for more concise test lines
Add several tests for new WebSocket origin checks
Remove autoadded whitespace on blank lines
Restore TestWebsocketOrigins originally-named test
and rename the others to be helpers rather than full tests
Remove autoadded whitespace on blank line
Temporarily comment out new test sets
Uncomment test around origin rule with scheme
Remove tests without scheme on browser origin
per https://github.com/ethereum/go-ethereum/pull/21481/files#r479371498
Uncomment tests with port; remove some blank lines
Handle when browser does not specify scheme/port
Uncomment test for including scheme & port in rule
Add IP tests
* node: more tests + table-driven, ws origin changes
Co-authored-by: Martin Holst Swende <martin@swende.se>
The z == 0 check is hit whenever we Add two points with the same x1/x2
coordinate. crypto/elliptic uses the same check in their affineFromJacobian
function. This change does not affect block processing or tx signature verification
in any way, because it does not use the Add or Double methods.
* consensus/ethash: only use *reflect.SliceHeader, not reflect.SliceHeader. See comment here: https://github.com/golang/go/issues/40397\#issuecomment-663748689
* consensus/ethash: pr feedback from @mdempsky, makes a copy of dest such that is not mutated
* consensus/ethash: remove noop assign
* consensus/ethash: apply same fix to another location
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
* all: core: split vm.Config into BlockConfig and TxConfig
* core: core/vm: reset EVM between tx in block instead of creating new
* core/vm: added docs
This fixes cross-build and mobile framework failures.
It also disables the mac test builder because it was failing
all the time in hard to understand ways and we can't afford
it anymore under Travis CI's new pricing.
This new flag downloads a known version of Go and builds with it. This
is meant for environments where we can't easily upgrade the installed Go
version.
* .travis.yml: remove install step for PR test builders
We added this step originally to avoid re-building everything
for every test. go test has become much smarter in recent go
releases, so we no longer need to install anything here.
- Remove the ws:// prefix from the status endpoint since
the ws:// is already included in the stack.WSEndpoint().
- Don't register the services again in the node start.
Registration is already done in the initialization stage.
- Expose admin namespace via websocket.
This namespace is necessary for connecting the peers via websocket.
- Offer logging relevant options for exec adapter.
It's really painful to mix all log output in the single console. So
this PR offers two additional options for exec adapter in this case
testers can config the log output(e.g. file output) and log level
for each p2p node.
* trie: fix error in stacktrie not committing small roots
* fuzzers: make trie-fuzzer use correct returnvalues
* trie: improved tests
* tests/fuzzers: fuzzer for stacktrie vs regular trie
* test/fuzzers: make stacktrie fuzzer use 32-byte keys
* trie: fix error in stacktrie with small nodes
* trie: add (skipped) testcase for stacktrie
* tests/fuzzers: address review comments for stacktrie fuzzer
* trie: fix docs in stacktrie
This PR contains a minor optimization in derivesha, by exposing the RLP
int-encoding and making use of it to write integers directly to a
buffer (an RLP integer is known to never require more than 9 bytes
total). rlp.AppendUint64 might be useful in other places too.
The code assumes, just as before, that the hasher (a trie) will copy the
key internally, which it does when doing keybytesToHex(key).
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR replaces the old test genesis.json and chain.rlp files in the testdata
directory for the eth protocol test suite, and also adds documentation for
running the eth test suite locally.
It also improves the test output text and adds more timeouts.
Co-authored-by: Felix Lange <fjl@twurst.com>
TAP is a text format for test results. Parsers for it are available in many languages,
making it easy to consume. I want TAP output from our protocol tests because the
Hive wrapper around them needs to know about the test names and their individual
results and logs. It would also be possible to just write this info as JSON, but I don't
want to invent a new format.
This also improves the normal console output for tests (when running without --tap).
It now prints -- RUN lines before any output from the test, and indents the log output
by one space.
* core/state/snapshot: print warning if failed to resolve journal
* core/state/snapshot: fix snapshot recovery
When we meet the snapshot journal consisted with:
- disk layer generator with new-format
- diff layer journal with old-format
The base layer should be returned without error.
The broken diff layer can be reconstructed later
but we definitely don't want to reconstruct the
huge diff layer.
* core: add tests
ToHex was deprecated a couple years ago. The last remaining use
was in ToHexArray, which itself only had a single call site.
This just moves ToHexArray near its only remaining call site and
implements it using hexutil.Encode. This changes the default behaviour
of ToHexArray and with it the behaviour of eth_getProof. Previously we
encoded an empty slice as 0, now the empty slice is encoded as 0x.
This adds a few tiny fixes for les and the p2p simulation framework:
LES Parts
- Keep the LES-SERVER connection even it's non-synced
We had this idea to reject the connections in LES protocol if the les-server itself is
not synced. However, in LES protocol we will also receive the connection from another
les-server. In this case even the local node is not synced yet, we should keep the tcp
connection for other protocols(e.g. eth protocol).
- Don't count "invalid message" for non-existing GetBlockHeadersMsg request
In the eth syncing mechanism (full sync, fast sync, light sync), it will try to fetch
some non-existent blocks or headers(to ensure we indeed download all the missing chain).
In this case, it's possible that the les-server will receive the request for
non-existent headers. So don't count it as the "invalid message" for scheduling
dropping.
- Copy the announce object in the closure
Before the les-server pushes the latest headers to all connected clients, it will create
a closure and queue it in the underlying request scheduler. In some scenarios it's
problematic. E.g, in private networks, the block can be mined very fast. So before the
first closure is executed, we may already update the latest_announce object. So actually
the "announce" object we want to send is replaced.
The downsize is the client will receive two announces with the same td and then drop the
server.
P2P Simulation Framework
- Don't double register the protocol services in p2p-simulation "Start".
The protocols upon the devp2p are registered in the "New node stage". So don't reigster
them again when starting a node in the p2p simulation framework
- Add one more new config field "ExternalSigner", in order to use clef service in the
framework.
* core/state/snapshot: introduce snapshot journal version
* core: update the disk layer in an atomic way
* core: persist the disk layer generator periodically
* core/state/snapshot: improve logging
* core/state/snapshot: forcibly ensure the legacy snapshot is matched
* core/state/snapshot: add debug logs
* core, tests: fix tests and special recovery case
* core: polish
* core: add more blockchain tests for snapshot recovery
* core/state: fix comment
* core: add recovery flag for snapshot
* core: add restart after start-after-crash tests
* core/rawdb: fix imports
* core: fix tests
* core: remove log
* core/state/snapshot: fix snapshot
* core: avoid callbacks in SetHead
* core: fix setHead cornercase where the threshold root has state
* core: small docs for the test cases
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* core/state/snapshot: add diskRoot function
* core/state/snapshot: disable iteration if the snapshot is generating
* core/state/snapshot: simplify the function
* core/state: panic for undefined layer
The old one was wrong in two ways: the first block in chain.rlp was the
genesis block, and the genesis difficulty was below minimum difficulty.
This also contains some other fixes to the test.
* accounts/keystore: add timeout to test to prevent failure on travis
The TestWalletNotifications test sporadically fails on travis.
This is because we shutdown the event collection before all events are received.
Adding a small timeout (10 milliseconds) allows the collector to be scheduled
and to consume all pending events before we shut it down.
* accounts/keystore: added newlines back in
* accounts/keystore: properly fix the walletNotifications test
* miner: exit loop when downloader Done or Failed
Following the logic of the comment at the method,
this fixes a regression introduced at 7cf56d6f06
, which would allow external parties to DoS with
blocks, preventing mining progress.
Signed-off-by: meows <b5c6@protonmail.com>
* miner: remove ineff assign (lint)
Signed-off-by: meows <b5c6@protonmail.com>
* miner: update test re downloader events
Signed-off-by: meows <b5c6@protonmail.com>
* Revert "miner: remove ineff assign (lint)"
This reverts commit eaefcd34ab4862ebc936fb8a07578aa2744bc058.
* Revert "miner: exit loop when downloader Done or Failed"
This reverts commit 23abd34265aa246c38fc390bb72572ad6ae9fe3b.
* miner: add test showing imprecise TestMiner
Signed-off-by: meows <b5c6@protonmail.com>
* miner: fix waitForMiningState precision
This helper function would return an affirmation
on the first positive match on a desired bool.
This was imprecise; it return false positives
by not waiting initially for an 'updated' value.
This fix causes TestMiner_2 to fail, which is
expected.
Signed-off-by: meows <b5c6@protonmail.com>
* miner: remove TestMiner_2 demonstrating broken test
This test demonstrated the imprecision of the test
helper function waitForMiningState. This function
has been fixed with 6d365c2851, and this test test
may now be removed.
Signed-off-by: meows <b5c6@protonmail.com>
* miner: fix test regarding downloader event/mining expectations
See comment for logic.
Signed-off-by: meows <b5c6@protonmail.com>
* miner: add test describing expectations for downloader/mining events
We expect that once the downloader emits a DoneEvent,
signaling a successful sync, that subsequent StartEvents
are not longer permitted to stop the miner.
This prevents a security vulnerability where forced syncs via
fake high blocks would stall mining operation.
Signed-off-by: meows <b5c6@protonmail.com>
* miner: use 'canStop' state to fix downloader event handling
- Break downloader event handling into event
separating Done and Failed events. We need to
treat these cases differently since a DoneEvent
should prevent the miner from being stopped on
subsequent downloader Start events.
- Use canStop state to handle the one-off
case when a downloader first succeeds.
Signed-off-by: meows <b5c6@protonmail.com>
* miner: improve comment wording
Signed-off-by: meows <b5c6@protonmail.com>
* miner: start mining on downloader events iff not already mining
Signed-off-by: meows <b5c6@protonmail.com>
* miner: refactor miner update logic w/r/t downloader events
This makes mining pause/start logic regarding downloader
events more explicit. Instead of eternally handling downloader
events after the first done event, the subscription is closed
when downloader events are no longer actionable.
Signed-off-by: meows <b5c6@protonmail.com>
* miner: fix handling downloader events on subcription closed
Signed-off-by: meows <b5c6@protonmail.com>
* miner: (lint:gosimple) use range over chan instead of for/select
Signed-off-by: meows <b5c6@protonmail.com>
* miner: refactor update loop to remove race condition
The go routine handling the downloader events handling
vars in parallel with the parent routine, causing a
race condition.
This change, though ugly, remove the condition while
still allowing the downloader event subscription to be
closed when the miner has no further use for it (ie DoneEvent).
* miner: alternate fix for miner-flaw
Co-authored-by: meows <b5c6@protonmail.com>
* peer: return localAddr instead of name to prevent spam
We currently use the name (which can be freely set by the peer) in several log messages.
This enables malicious actors to write spam into your geth log.
This commit returns the localAddr instead of the freely settable name.
* p2p: reduce usage of peer.Name in warn messages
* eth, p2p: use truncated names
* Update peer.go
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
For some reason, using the shared hash causes a cryptographic incompatibility
when using Go 1.15. I noticed this during the development of Discovery v5.1
when I added test vector verification.
The go library commit that broke this is golang/go@97240d5, but the
way we used HKDF is slightly dodgy anyway and it's not a regression.
* core/types: tests for bloom
* core/types: refactored bloom filter for receipts, added tests
core/types: replaced old bloom implementation
core/types: change interface of bloom add+test
* core/types: refactor bloom
* core/types: minor tweak on LogsBloom
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
* core/state/snapshot: exit Geth if generator hits missing trie nodes
* core/state/snapshot: error instead of hard die on generator fault
* core/state/snapshot: don't enable logging on the tests
* trie: update tests to check commit integrity
* trie: polish committer
* trie: fix typo
* trie: remove hasvalue notion
According to the benchmarks, type assertion between the pointer and
interface is extremely fast.
BenchmarkIntmethod-12 1000000000 1.91 ns/op
BenchmarkInterface-12 1000000000 2.13 ns/op
BenchmarkTypeSwitch-12 1000000000 1.81 ns/op
BenchmarkTypeAssertion-12 2000000000 1.78 ns/op
So the overhead for asserting whether the shortnode has "valuenode"
child is super tiny. No necessary to have another field.
* trie: linter nitpicks
Co-authored-by: Martin Holst Swende <martin@swende.se>
* accounts, signer: implement gnosis safe support
* common/math: add type for marshalling big to dec
* accounts, signer: properly sign gnosis requests
* signer, clef: implement account_signGnosisTx
* signer: fix auditlog print, change rpc-name (signGnosisTx to signGnosisSafeTx)
* signer: pass validation-messages/warnings to the UI for gnonsis-safe txs
* signer/core: minor change to validationmessages of typed data
core/types: use stacktrie for derivesha
trie: add stacktrie file
trie: fix linter
core/types: use stacktrie for derivesha
rebased: adapt stacktrie to the newer version of DeriveSha
Co-authored-by: Martin Holst Swende <martin@swende.se>
More linter fixes
review feedback: no key offset for nodes converted to hashes
trie: use EncodeRLP for full nodes
core/types: insert txs in order in derivesha
trie: tests for derivesha with stacktrie
trie: make stacktrie use pooled hashers
trie: make stacktrie reuse tmp slice space
trie: minor polishes on stacktrie
trie/stacktrie: less rlp dancing
core/types: explain the contorsions in DeriveSha
ci: fix goimport errors
trie: clear mem on subtrie hashing
squashme: linter fix
stracktrie: use pooling, less allocs (#3)
trie: in-place hex prefix, reduce allocs and add rawNode.EncodeRLP
Reintroduce the `[]node` method, add the missing `EncodeRLP` implementation for `rawNode` and calculate the hex prefix in place.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Martin Holst Swende <martin@swende.se>
This allows users to estimate gas on top of arbitrary blocks as well as pending and latest.
Tracing on pending is useful for most users as it takes into account the current txpool while
tracing on latest might be useful for users that have little to know knowledge of the current
transactions in the network.
If blockNrOrHash is not specified, estimateGas defaults to pending
This change moves the RLPx protocol implementation into a separate package,
p2p/rlpx. The new package can be used to establish RLPx connections for
protocol testing purposes.
Co-authored-by: Felix Lange <fjl@twurst.com>
* database: added counters
* Improved stats for ancient db
* Small improvement
* Better message and added percentage while counting receipts
* Fast counting for receipts
* added info message
* Show both receips itemscount from ancient db and counted receipts
* Fixed default case
* Removed counter for receipts in ancient store
* Removed counting of receipts present in leveldb
When the license was added to the repository, its text was changed (some
sections at the end removed) and, worse, the authors of go-ethereum
tried to claim copyright on the license text.
The correct way to apply GPL to a project is to copy it verbatim.
This change reverts the text of the GPL to the original.
This PR adds an extra guarantee to NodeStateMachine: it ensures that all
immediate effects of a certain change are processed before any subsequent
effects of any of the immediate effects on the same node. In the original
version, if a cascaded change caused a subscription callback to be called
multiple times for the same node then these calls might have happened in a
wrong chronological order.
For example:
- a subscription to flag0 changes flag1 and flag2
- a subscription to flag1 changes flag3
- a subscription to flag1, flag2 and flag3 was called in the following order:
[flag1] -> [flag1, flag3]
[] -> [flag1]
[flag1, flag3] -> [flag1, flag2, flag3]
This happened because the tree of changes was traversed in a "depth-first
order". Now it is traversed in a "breadth-first order"; each node has a
FIFO queue for pending callbacks and each triggered subscription callback
is added to the end of the list. The already existing guarantees are
retained; no SetState or SetField returns until the callback queue of the
node is empty again. Just like before, it is the responsibility of the
state machine design to ensure that infinite state loops are not possible.
Multiple changes affecting the same node can still happen simultaneously;
in this case the changes can be interleaved in the FIFO of the node but the
correct order is still guaranteed.
A new unit test is also added to verify callback order in the above scenario.
This PR changes several different things:
- Adds test cases for the miner loop
- Stops the worker if it wasn't already stopped in worker.Close()
- Uses channels instead of atomics in the miner.update() loop
Co-authored-by: Felix Lange <fjl@twurst.com>
* core/vm/testdata: add gascost expectations to testcases
* core/vm: verify expected gas in tests for precompiles
* core/vm: fix overflow flaw in gas/s calculation
* whisper: remove whisper
* Update cmd/geth/config.go
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
* cmd/geth: warn on enabling whisper + remove more whisper deps
* mobile: remove all whisper references
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
This changes how the downloader works, a little bit. Previously, when block sync started,
we immediately started filling up to 8192 blocks. Usually this is fine, blocks are small
in the early numbers. The threshold then is lowered as we measure the size of the blocks
that are filled.
However, if the node is shut down and restarts syncing while we're in a heavy segment,
that might be bad. This PR introduces a more conservative initial threshold of 2K blocks
instead.
* "Downloader queue stats" is now a DEBUG information
I think this info is more a DEBUG related information then an INFO. If it must remains an INFO, maybe it can be slow down to one time every 5 minutes or so.
* Update queue.go
"Downloader queue stats" information is now provided once every minute instead of once every 10 seconds.
* accounts/abi: fix a bug in getTypeSize method
e.g. for "Tuple[2]" type, the element of the array is a tuple type and the size of the tuple may not be 32.
* accounts/abi: add unit test of getTypeSize method
This change improves discovery behavior in small networks. Very small
networks would often fail to bootstrap because all member nodes were
dropping table content due to findnode failure. The check is now changed
to avoid dropping nodes on findnode failure when their bucket is almost
empty. It also relaxes the liveness check requirement for FINDNODE/v4
response nodes, returning unverified nodes as results when there aren't
any verified nodes yet.
The "findnode failed" log now reports whether the node was dropped
instead of the number of results. The value of the "results" was
always zero by definition.
Co-authored-by: Felix Lange <fjl@twurst.com>
* metrics: zero temp variable in updateMeter
Previously the temp variable was not updated properly after summing it to count.
This meant we had astronomically high metrics, now we zero out the temp whenever we
sum it onto the snapshot count
* metrics: move temp variable to be aligned, unit tests
Moves the temp variable in MeterSnapshot to be 64-bit aligned because of the atomic bug.
Adds a unit test, that catches the previous bug.
* chaincmd should make config nodes instead of full nodes
* add documentation for using makeConfigNode instead of makeFullNode;
* add documentation to functions
* code style
This adds a lock around requests because some routers can't handle
concurrent requests. Requests are also rate-limited.
The Map function request a new mapping exactly when the map timeout
occurs instead of 5 minutes earlier. This should prevent duplicate mappings.
* core: avoid modification of accountSet cache in tx_pool
when runReorg, we may copy the dirtyAccounts' accountSet cache to promoteAddrs
in which accounts will be promoted, however, if we have reset request at the
same time, we may reuse promoteAddrs and modify the cache content which is
against the original intention of accountSet cache. So, we need to make a new
slice here to avoid modify accountSet cache.
* core: fix flatten condition + comment
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR significantly changes the APIs for instantiating Ethereum nodes in
a Go program. The new APIs are not backwards-compatible, but we feel that
this is made up for by the much simpler way of registering services on
node.Node. You can find more information and rationale in the design
document: https://gist.github.com/renaynay/5bec2de19fde66f4d04c535fd24f0775.
There is also a new feature in Node's Go API: it is now possible to
register arbitrary handlers on the user-facing HTTP server. In geth, this
facility is used to enable GraphQL.
There is a single minor change relevant for geth users in this PR: The
GraphQL API is no longer available separately from the JSON-RPC HTTP
server. If you want GraphQL, you need to enable it using the
./geth --http --graphql flag combination.
The --graphql.port and --graphql.addr flags are no longer available.
This replaces the two-stage shutdown scheme with the one we
use almost everywhere else: a single quit channel signalling
termination.
Co-authored-by: Felix Lange <fjl@twurst.com>
* Disable symbol table and DWARF generation by default.
Trimpath if compiling with Go >= 1.13
* Set Go to minimum version 1.13. Revert debug symbol changes.
* init
notes
removed some mentions of eth62, bumped protocol err too old to >=63
* remove sanity checks and bump supported protocol version up to 63
* remove 62 tests, still need to add 65
* remove 65 tests
Solves issue#20582. Non-executable transactions should not be evicted on each tick if there are no promote transactions or if a pending/reset empties the pending list. Tests and logging expanded to handle these cases in the future.
core/tx_pool: use a ts for each tx in the queue, but only update the heartbeat on promotion or pending replaced
queuedTs proper naming
* eth/downloader: refactor downloader + queue
downloader, fetcher: throttle-metrics, fetcher filter improvements, standalone resultcache
downloader: more accurate deliverytime calculation, less mem overhead in state requests
downloader/queue: increase underlying buffer of results, new throttle mechanism
eth/downloader: updates to tests
eth/downloader: fix up some review concerns
eth/downloader/queue: minor fixes
eth/downloader: minor fixes after review call
eth/downloader: testcases for queue.go
eth/downloader: minor change, don't set progress unless progress...
eth/downloader: fix flaw which prevented useless peers from being dropped
eth/downloader: try to fix tests
eth/downloader: verify non-deliveries against advertised remote head
eth/downloader: fix flaw with checking closed-status causing hang
eth/downloader: hashing avoidance
eth/downloader: review concerns + simplify resultcache and queue
eth/downloader: add back some locks, address review concerns
downloader/queue: fix remaining lock flaw
* eth/downloader: nitpick fixes
* eth/downloader: remove the *2*3/4 throttling threshold dance
* eth/downloader: print correct throttle threshold in stats
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
This fixes the iOS framework build by naming the second parameter of the
Signer interface method. The name is important because it becomes part
of the objc method signature.
Fixes#21340
* geth,utils: implement configurable developer account options
Prior to this change --dev (developer) mode
generated one account with an empty password,
irrespective of existing --password and --miner.etherbase
options.
This change makes --dev mode compatible with these
existing flags.
--dev mode may now be used in conjunction with
--password and --miner.etherbase flags to configure
the developer faucet using an existing keystore or
in creating a new account.
Signed-off-by: meows <b5c6@protonmail.com>
* main: remove key/pass flags from usage developer section
These flags are included already in other sections,
and it is not desired to duplicate them.
They were originally included in this section
along with added support for these flags in the
developer mode.
Signed-off-by: meows <b5c6@protonmail.com>
* core: added local tx pool test case
* core, crypto: various allocation savings regarding tx handling
* core/txlist, txpool: save a reheap operation, avoid some bigint allocs
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This change introduces garbage collection for the light client. Historical
chain data is deleted periodically. If you want to disable the GC, use
the --light.nopruning flag.
This fixes two issues with state sync restarts:
When sync restarts with a new root, some peers can have in-flight requests.
Since all peers with active requests were marked idle when exiting sync,
the new sync would schedule more requests for those peers. When the
response for the earlier request arrived, the new sync would reject it and
mark the peer idle again, rendering the peer useless until it disconnected.
The other issue was that peers would not be marked idle when they had
delivered a response, but the response hadn't been processed before
restarting the state sync. This also made the peer useless because it
would be permanently marked busy.
Co-authored-by: Felix Lange <fjl@twurst.com>
This adds a test suite for discovery v4. The test suite is a port of the Hive suite for
discovery, and will replace the current suite on Hive soon-ish. The tests can be
run locally with this command:
devp2p discv4 test -remote enode//...
Co-authored-by: Felix Lange <fjl@twurst.com>
This change further improves the performance of RLP encoding by removing
allocations for big.Int and [...]byte types. I have added a new benchmark
that measures RLP encoding of types.Block to verify that performance is
improved.
Exposing /debug/metrics and /debug/metrics/prometheus was dependent
on --pprof, which also exposes other HTTP APIs. This change makes it possible
to run the metrics server on an independent endpoint without enabling pprof.
This PR reduces the bandwidth used by the light client to compute the
recommended gas price. The current mechanism for suggesting the price is:
- retrieve recent 20 blocks
- get the lowest gas price of these blocks
- sort the price array and return the middle(60%) one
This works for full nodes, which have all blocks available locally.
However, this is very expensive for the light client because the light
client needs to retrieve block bodies from the network.
The PR changes the default options for light client. With the new config,
the light client only retrieves the two latest blocks, but in order to
collect more sample transactions, the 3 lowest prices are collected from
each block.
This PR also changes the behavior for empty blocks. If the block is empty,
the lastest price is reused for sampling.
* core: use uint64 for total tx costs instead of big.Int
* core: added local tx pool test case
* core, crypto: various allocation savings regarding tx handling
* Update core/tx_list.go
* core: added tx.GasPriceIntCmp for comparison without allocation
adds a method to remove unneeded allocation in comparison to tx.gasPrice
* core: handle pools full of locals better
* core/tests: benchmark for tx_list
* core/txlist, txpool: save a reheap operation, avoid some bigint allocs
Co-authored-by: Martin Holst Swende <martin@swende.se>
List headers made up 11% of all allocations during sync. This change
removes most of those allocations by keeping the list header values
cached in the encoder buffer instead. Since encoder buffers are pooled,
list headers are no longer allocated in the common case where an
encoder buffer is available for reuse.
Co-authored-by: Felix Lange <fjl@twurst.com>
* eth/downloaded: fixed datarace between synchronize and Progress
There was a race condition between `downloader.synchronize()` and `Progress` `syncWithPeer` `fetchHeight` `findAncestors` and `processHeaders`
This PR changes the behavior of the downloader a bit.
Previously the functions `Progress` `syncWithPeer` `fetchHeight` `findAncestors` and `processHeaders` read the syncMode anew within their loops. Now they read the syncMode at the start of their function and don't change it during their runtime.
* eth/downloaded: comment
* eth/downloader: added comment
* core, crypto: various allocation savings regarding tx handling
* core: reduce allocs for gas price comparison
This change reduces the allocations needed for comparing different transactions to each other.
A call to `tx.GasPrice()` copies the gas price as it has to be safe against modifications and
also needs to be threadsafe. For comparing and ordering different transactions we don't need
these guarantees
* core: added tx.GasPriceIntCmp for comparison without allocation
adds a method to remove unneeded allocation in comparison to tx.gasPrice
* core/types: pool legacykeccak256 objects in rlpHash
rlpHash is by far the most used function in core that allocates a legacyKeccak256 object on each call.
Since it is so widely used it makes sense to add pooling here so we relieve the GC.
On my machine these changes result in > 100 MILLION less allocations and > 30 GB less allocated memory.
* reverted some changes
* reverted some changes
* trie: use crypto.KeccakState instead of replicating code
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR implements the EVM state transition tool, which is intended
to be the replacement for our retesteth client implementation.
Documentation is present in the cmd/evm/README.md file.
Co-authored-by: Felix Lange <fjl@twurst.com>
Without use of global, these flags didn't actually modify
miner configuration, since we weren't grabbing from the
proper context scope, which should be global (vs. subcommand).
Signed-off-by: meows <b5c6@protonmail.com>
The import command should not return a 0 status
code if the import finishes prematurely becaues
of an import error.
Returning the error causes the program to exit with 1
if the err is non nil.
Signed-off-by: meows <b5c6@protonmail.com>
* core/vm: fix incorrect computation of discount
During testing on Yolov1 we found that the way geth calculates the discount
is not in line with the specification. Basically what we did is calculate
128 * Bls12381GXMulGas * discount / 1000 whenever we received more than 128 pairs
of values. Correct would be to calculate k * Bls12381... for k > 128.
* core/vm: better logic for discount calculation
* core/vm: better calculation logic, added worstcase benchmarks
* core/vm: better benchmarking logic
The ancients variable in the freezer is a list of hashes, which
identifies all of the hashes to be frozen. The slice is being allocated
with a capacity of `limit`, which is the number of the last block
this batch will attempt to add to the freezer. That means we are
allocating memory for all of the blocks in the freezer, not just
the ones to be added.
If instead we allocate `limit - f.frozen`, we will only allocate
enough space for the blocks we're about to add to the freezer. On
mainnet this reduces usage by about 320 MB.
* eth/downloader tests: fix spurious failing test due to race between receipts/headers
* miner tests: fix travis failure on arm64
* eth/downloader: tests - store td in ancients too
* core/vm: use fixed uint256 library instead of big
* core/vm: remove intpools
* core/vm: upgrade uint256, fixes uint256.NewFromBig
* core/vm: use uint256.Int by value in Stack
* core/vm: upgrade uint256 to v1.0.0
* core/vm: don't preallocate space for 1024 stack items (only 16)
Co-authored-by: Martin Holst Swende <martin@swende.se>
* internal/ethapi: return revert reason for eth_call
* internal/ethapi: moved revert reason logic to doCall
* accounts/abi/bind/backends: added revert reason logic to simulated backend
* internal/ethapi: fixed linting error
* internal/ethapi: check if require reason can be unpacked
* internal/ethapi: better error logic
* internal/ethapi: simplify logic
* internal/ethapi: return vmError()
* internal/ethapi: move handling of revert out of docall
* graphql: removed revert logic until spec change
* rpc: internal/ethapi: added custom error types
* graphql: use returndata instead of return
Return() checks if there is an error. If an error is found, we return nil.
For most use cases it can be beneficial to return the output even if there
was an error. This code should be changed anyway once the spec supports
error reasons in graphql responses
* accounts/abi/bind/backends: added tests for revert reason
* internal/ethapi: add errorCode to revert error
* internal/ethapi: add errorCode of 3 to revertError
* internal/ethapi: unified estimateGasErrors, simplified logic
* internal/ethapi: unified handling of errors in DoEstimateGas
* rpc: print error data field
* accounts/abi/bind/backends: unify simulatedBackend and RPC
* internal/ethapi: added binary data to revertError data
* internal/ethapi: refactored unpacking logic into newRevertError
* accounts/abi/bind/backends: fix EstimateGas
* accounts, console, internal, rpc: minor error interface cleanups
* Revert "accounts, console, internal, rpc: minor error interface cleanups"
This reverts commit 2d3ef53c53.
* re-apply the good parts of 2d3ef53c53
* rpc: add test for returning server error data from client
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Once we detect an invalid transaction during recovering signatures, we should
directly exclude this transaction to avoid validating the signatures hereafter.
This should optimize the validations times of transactions with invalid signatures
to only one time.
* replace gosigar with gopsutil
* removed check for whether GOOS is openbsd
* removed accidental import of runtime
* potential fix for difference in units between gosig and gopsutil
* fixed lint error
* remove multiplication factor
* uses cpu.ClocksPerSec as the multiplication factor
* changed dependency from shirou to renaynay (#20)
* updated dep
* switching back from using renaynay fork to using upstream as PRs were merged on upstream
* removed empty line
* optimized imports
* tidied go mod
This PR makes use of go 1.13 error handling, wrapping errors and using
errors.Is to check a wrapped root-cause. It also removes the travis
builders for go 1.11 and go 1.12.
This adds a new API method on core.BlockChain to allow interrupting
running data inserts, and calls the method before shutting down the
downloader.
The BlockChain interrupt checks are now done through a method instead
of inlining the atomic load everywhere. There is no loss of efficiency from
this and it makes the interrupt protocol a lot clearer because the check is
defined next to the method that sets the flag.
This PR reimplements the light client server pool. It is also a first step
to move certain logic into a new lespay package. This package will contain
the implementation of the lespay token sale functions, the token buying and
selling logic and other components related to peer selection/prioritization
and service quality evaluation. Over the long term this package will be
reusable for incentivizing future protocols.
Since the LES peer logic is now based on enode.Iterator, it can now use
DNS-based fallback discovery to find servers.
This document describes the function of the new components:
https://gist.github.com/zsfelfoldi/3c7ace895234b7b345ab4f71dab102d4
* abi/bind/backends: testcase for double-lock
* accounts: add blockByNumberNoLock to avoid double-lock
* backend/simulated: use stateroot, not blockhash for retrieveing state
Co-authored-by: Martin Holst Swende <martin@swende.se>
* cmd/clef, signer/core: use better terminal input for passwords, make it possible to avoid boot-up warning
* all: move commonly used prompter to isolated (small) package
* cmd/clef: Add new --acceptWarn to clef README
* cmd/clef: rename flag 'acceptWarn' to 'suppress-bootwarn'
Co-authored-by: ligi <ligi@ligi.de>
* accounts/abi: simplified reflection logic
* accounts/abi: simplified reflection logic
* accounts/abi: removed unpack
* accounts/abi: removed comments
* accounts/abi: removed uneccessary complications
* accounts/abi: minor changes in error messages
* accounts/abi: removed unnused code
* accounts/abi: fixed indexed argument unpacking
* accounts/abi: removed superfluous test cases
This commit removes two test cases. The first one is trivially invalid as we have the same
test cases as passing in packing_test.go L375. The second one passes now,
because we don't need the mapArgNamesToStructFields in unpack_atomic anymore.
Checking for purely underscored arg names generally should not be something we do
as the abi/contract is generally out of the control of the user.
* accounts/abi: removed comments, debug println
* accounts/abi: added commented out code
* accounts/abi: addressed comments
* accounts/abi: remove unnecessary dst.CanSet check
* accounts/abi: added dst.CanSet checks
* accounts/abi: allow overloaded argument names
In solidity it is possible to create the following contract:
```
contract Overloader {
struct F { uint _f; uint __f; uint f; }
function f(F memory f) public {}
}
```
This however resulted in a panic in the abi package.
* accounts/abi fixed error handling
* accounts/abi/bind: added test cases for waitDeployed
* accounts/abi/bind: added test case for boundContract
* accounts/abi/bind: removed unnecessary resolve methods
* accounts/abi: moved topics from /bind to /abi
* accounts/abi/bind: cleaned up format... functions
* accounts/abi: improved log message
* accounts/abi: added type tests
* accounts/abi/bind: remove superfluous template methods
* cmd, miner: add noempty-precommit flag
* cmd, miner: get rid of external flag
* miner: change bool to atomic int
* miner: fix tiny typo
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* cmd, core, eth: init tx lookup in background
* core/rawdb: tiny log fixes to make it clearer what's happening
* core, eth: fix rebase errors
* core/rawdb: make reindexing less generic, but more optimal
* rlp: implement rlp list iterator
* core/rawdb: new implementation of tx indexing/unindex using generic tx iterator and hashing rlp-data
* core/rawdb, cmd/utils: fix review concerns
* cmd/utils: fix merge issue
* core/rawdb: add some log formatting polishes
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* p2p: add low port check in dialer
We already have a check like this for UDP ports, add a similar one in
the dialer. This prevents dials to port zero and it's also an extra
layer of protection against spamming HTTP servers.
* p2p/discover: use errLowPort in v4 code
* p2p: change port check
* p2p: add comment
* p2p/simulations/adapters: ensure assigned port is in all node records
* core/rawdb: Stop freezer process as part of freezer.Close()
When you call db.Close(), it was closing the leveldb database first,
then closing the freezer, but never stopping the freezer process.
This could cause the freezer to attempt to write to leveldb after
leveldb had been closed, leading to a crash with a non-zero exit code.
This change adds a quit channel to the freezer, and freezer.Close()
will not return until the freezer process has stopped.
Additionally, when you call freezerdb.Close(), it will close the
AncientStore before closing leveldb, to ensure that the freezer goroutine
will be stopped before leveldb is closed.
* core/rawdb: Fix formatting for golint
* core/rawdb: Use backoff flag to avoid repeating select
* core/rawdb: Include accidentally omitted backoff
* accounts/abi/bind/backend, internal/ethapi: recap gas limit with balance
* accounts, internal: address comment and fix lint
* accounts, internal: extend log message
* tiny nits to format hexutil.Big and nil properly
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* core/state: more verbose stateb errors
* core/state: fix flaw
* core/state: fixed lint
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
This finally adds the error check that the documentation of StateDB.dbErr
promises to do. dbErr was added in 9e5f03b6c (June 2017), and the check was
already missing in that commit. We somehow survived without it for three years.
* accounts/abi/bind: add void if no return args specified
Currently the java generator generates invalid input on pure/view functions
that have no return type. e.g. `function f(uint u) view public {}`
This is not a problem in practice as people rarely ever write functions like this.
* accounts/abi/bind: use elseif instead of nested if
* rpc flags related to starting http server renamed to http
* old rpc flags aliased and still functional
* pprof flags fixed
* renames gpo related flags
* linted
* renamed rpc flags for consistency and clarity
* added warn logs
* added more warn logs for all deprecated flags for consistency
* moves legacy flags to separate file, hides older flags under show-deprecated-flags command
* legacy prefix and moved some more legacy flags to legacy file
* fixed circular import
* added docs
* fixed imports lint error
* added notes about when flags were deprecated
* cmd/utils: group flags by deprecation date + reorder by date,
* modified deprecated comments for consistency, added warn log for --rpc
* making sure deprecated flags are still functional
* show-deprecated-flags command cleaned up
* fixed lint errors
* corrected merge conflict
* IsSet --> GlobalIsSet
* uncategorized flags, if not deprecated, displayed under misc
Co-authored-by: Martin Holst Swende <martin@swende.se>
* core/state/snapshot: implement storage iterator
* core/state/snapshot, tests: implement helper function
* core/state/snapshot: fix storage issue
If an account is deleted in the tx_1 but recreated in the tx_2,
the it can happen that in this diff layer, both destructedSet
and storageData records this account. In this case, the storage
iterator should be able to iterate the slots belong to new account
but disable further iteration in deeper layers(belong to old account)
* core/state/snapshot: address peter and martin's comment
* core/state: address comments
* core/state/snapshot: fix test
golang-lru is now a go module, and the upgrade corrects a couple
of minor issues. In particular, the library could crash if you inserted
nil into an LRU cache.
This is a resubmit of #20668 which rewrites the problematic test
without any additional goroutines. It also documents the test better.
The purpose of this test is checking whether log events are sent
correctly when importing blocks. The test was written at a time when
blockchain events were delivered asynchronously, making the check hard
to pull off. Now that core.BlockChain delivers events synchronously
during the call to InsertChain, the test can be simplified.
Co-authored-by: BurtonQin <bobbqqin@gmail.com>
This corrects the call to eth_getBlockByNumber, which previously
returned this error:
can't get latest block: missing value for required argument 1
Co-authored-by: Felix Lange <fjl@twurst.com>
This fixes a theoretical double lock condition which could occur in
indexer.AddChildIndexer(indexer)
Nobody would ever do that though.
Co-authored-by: Felix Lange <fjl@twurst.com>
* accounts/abi: added documentation
* accounts/abi: reduced usage of arguments.LengthNonIndexed
* accounts/abi: simplified reflection logic
* accounts/abi: moved testjson data into global declaration
* accounts/abi: removed duplicate test cases
* accounts/abi: reworked abi tests
* accounts/abi: added more tests for abi packing
* accounts/abi/bind: refactored base tests
* accounts/abi: run pack tests as subtests
* accounts/abi: removed duplicate tests
* accounts/abi: removed unnused arguments.LengthNonIndexed
Due to refactors to the code, we do not need the arguments.LengthNonIndexed function anymore.
You can still get the length by calling len(arguments.NonIndexed())
* accounts/abi: added type test
* accounts/abi: modified unpack test to pack test
* accounts/abi: length check on arrayTy
* accounts/abi: test invalid abi
* accounts/abi: fixed rebase error
* accounts/abi: fixed rebase errors
* accounts/abi: removed unused definition
* accounts/abi: merged packing/unpacking tests
* accounts/abi: fixed [][][32]bytes encoding
* accounts/abi: added tuple test cases
* accounts/abi: renamed getMockLog -> newMockLog
* accounts/abi: removed duplicate test
* accounts/abi: bools -> booleans
Rather than just closing the underlying network listener to stop our
HTTP servers, use the graceful shutdown procedure, waiting for any
in-process requests to finish.
It is possible to specify enode URLs using domain name since
commit b90cdbaa79, but the code comment still said that only
IP addresses are allowed.
Co-authored-by: admin@komgo.io <KomgoRocks2018!>
* trie: initial implementation for range proof
* trie: add benchmark
* trie: fix lint
* trie: fix minor issue
* trie: unset the edge valuenode as well
* trie: unset the edge valuenode as nilValuenode
* core: fix the condition of reorg
* core: fix nitpick to only retrieve head once
* core: don't reorg if received chain is longer at same diff
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* all: seperate consensus error and evm internal error
There are actually two types of error will be returned when
a tranaction/message call is executed: (a) consensus error
(b) evm internal error. The former should be converted to
a consensus issue, e.g. The sender doesn't enough asset to
purchase the gas it specifies. The latter is allowed since
evm itself is a blackbox and internal error is allowed to happen.
This PR emphasizes the difference by introducing a executionResult
structure. The evm error is embedded inside. So if any error
returned, it indicates consensus issue happens.
And also this PR improve the `EstimateGas` API to return the concrete
revert reason if the transaction always fails
* all: polish
* accounts/abi/bind/backends: add tests
* accounts/abi/bind/backends, internal: cleanup error message
* all: address comments
* core: fix lint
* accounts, core, eth, internal: address comments
* accounts, internal: resolve revert reason if possible
* accounts, internal: address comments
* accounts/abi: prevent recalculation of ID, Sig and String
* accounts/abi: fixed unpacking of no values
* accounts/abi: multiple fixes to arguments
* accounts/abi: refactored methodName and eventName
This commit moves the complicated logic of how we assign method names
and event names if they already exist into their own functions for
better readability.
* accounts/abi: prevent recalculation of internal
In this commit, I changed the way we calculate the string
representations, sig representations and the id's of methods. Before
that these fields would be recalculated everytime someone called .Sig()
.String() or .ID() on a method or an event.
Additionally this commit fixes issue #20856 as we assign names to inputs
with no name (input with name "" becomes "arg0")
* accounts/abi: added unnamed event params test
* accounts/abi: fixed rebasing errors in method sig
* accounts/abi: fixed rebasing errors in method sig
* accounts/abi: addressed comments
* accounts/abi: added FunctionType enumeration
* accounts/abi/bind: added test for unnamed arguments
* accounts/abi: improved readability in NewMethod, nitpicks
* accounts/abi: method/eventName -> overloadedMethodName
This adds two new methods to UDPv5, AllNodes and LocalNode.
AllNodes returns all the nodes stored in the local table; this is
useful for the purposes of metrics collection and also debugging any
potential issues with other discovery v5 implementations.
LocalNode returns the local node object. The reason for exposing this
is so that users can modify and set/delete new key-value entries in
the local record.
* accounts/abi: implement new fackball functions
In Solidity v0.6.0, the original fallback is separated
into two different sub types: fallback and receive.
This PR addes the support for parsing new format abi
and the relevant abigen functionalities.
* accounts/abi: fix unit tests
* accounts/abi: minor fixes
* accounts/abi, mobile: support jave binding
* accounts/abi: address marius's comment
* accounts/abi: Work around the uin64 conversion issue
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
This PR adds service value measurement statistics to the light client. It
also adds a private API that makes these statistics accessible. A follow-up
PR will add the new server pool which uses these statistics to select
servers with good performance.
This document describes the function of the new components:
https://gist.github.com/zsfelfoldi/3c7ace895234b7b345ab4f71dab102d4
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
* cmd/utils: make goerli the default testnet
* cmd/geth: explicitly rename testnet to ropsten
* core: explicitly rename testnet to ropsten
* params: explicitly rename testnet to ropsten
* cmd: explicitly rename testnet to ropsten
* miner: explicitly rename testnet to ropsten
* mobile: allow for returning the goerli spec
* tests: explicitly rename testnet to ropsten
* docs: update readme to reflect changes to the default testnet
* mobile: allow for configuring goerli and rinkeby nodes
* cmd/geth: revert --testnet back to ropsten and mark as legacy
* cmd/util: mark --testnet flag as deprecated
* docs: update readme to properly reflect the 3 testnets
* cmd/utils: add an explicit deprecation warning on startup
* cmd/utils: swap goerli and ropsten in usage
* cmd/geth: swap goerli and ropsten in usage
* cmd/geth: if running a known preset, log it for convenience
* docs: improve readme on usage of ropsten's testnet datadir
* cmd/utils: check if legacy `testnet` datadir exists for ropsten
* cmd/geth: check for legacy testnet path in console command
* cmd/geth: use switch statement for complex conditions in main
* cmd/geth: move known preset log statement to the very top
* cmd/utils: create new ropsten configurations in the ropsten datadir
* cmd/utils: makedatadir should check for existing testnet dir
* cmd/geth: add legacy testnet flag to the copy db command
* cmd/geth: add legacy testnet flag to the inspect command
* accounts/abi/bind: fixed erroneous packing of negative ints
* accounts/abi/bind: added test cases for negative ints in topics
* accounts/abi/bind: fixed genIntType for go 1.12
* accounts/abi: minor nitpick
This improves error messages when the file is too short or too long.
Also rewrite the test for SaveECDSA because LoadECDSA has its own
test now.
Co-authored-by: Felix Lange <fjl@twurst.com>
This change makes it possible to run geth with JSON-RPC over HTTP and
WebSocket on the same TCP port. The default port for WebSocket
is still 8546.
geth --rpc --rpcport 8545 --ws --wsport 8545
This also removes a lot of deprecated API surface from package rpc.
The rpc package is now purely about serving JSON-RPC and no longer
provides a way to start an HTTP server.
* accounts/abi/bind: refactored topics
* accounts/abi/bind: use store function to remove code duplication
* accounts/abi/bind: removed unused type defs
* accounts/abi/bind: error on tuples in topics
* Cosmetic changes to restart travis build
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
This adds an implementation of the current discovery v5 spec.
There is full integration with cmd/devp2p and enode.Iterator in this
version. In theory we could enable the new protocol as a replacement of
discovery v4 at any time. In practice, there will likely be a few more
changes to the spec and implementation before this can happen.
ToMessage is used to convert between ethapi.CallArgs and types.Message.
It reduces the length of the DoCall method by about half by abstracting out
the conversion between the CallArgs and the Message. This should improve the
code's maintainability and reusability.
The leaks were mostly in unit tests, and could all be resolved by
adding suitably-sized channel buffers or by restructuring the test
to not send on a channel after an error has occurred.
There is an unavoidable goroutine leak in Console.Interactive: when
we receive a signal, the line reader cannot be unblocked and will get
stuck. This leak is now documented and I've tried to make it slightly
less bad by adding a one-element buffer to the output channels of
the line-reading loop. Should the reader eventually awake from its
blocked state (i.e. when stdin is closed), at least it won't get stuck
trying to send to the interpreter loop which has quit long ago.
Co-authored-by: Felix Lange <fjl@twurst.com>
This adds a couple of metrics for tracking the timing
and frequency of method calls:
- rpc/requests gauge counts all requests
- rpc/success gauge counts requests which return err == nil
- rpc/failure gauge counts requests which return err != nil
- rpc/duration/all timer tracks timing of all requests
- rpc/duration/<method>/<success/failure> tracks per-method timing
This removes a bunch of weird code around the counter overflow check in
concatKDF and makes it actually work for different hash output sizes.
The overflow check worked as follows: concatKDF applies the hash function N
times, where N is roundup(kdLen, hashsize) / hashsize. N should not
overflow 32 bits because that would lead to a repetition in the KDF output.
A couple issues with the overflow check:
- It used the hash.BlockSize, which is wrong because the
block size is about the input of the hash function. Luckily, all standard
hash functions have a block size that's greater than the output size, so
concatKDF didn't crash, it just generated too much key material.
- The check used big.Int to compare against 2^32-1.
- The calculation could still overflow before reaching the check.
The new code in concatKDF doesn't check for overflow. Instead, there is a
new check on ECIESParams which ensures that params.KeyLen is < 512. This
removes any possibility of overflow.
There are a couple of miscellaneous improvements bundled in with this
change:
- The key buffer is pre-allocated instead of appending the hash output
to an initially empty slice.
- The code that uses concatKDF to derive keys is now shared between Encrypt
and Decrypt.
- There was a redundant invocation of IsOnCurve in Decrypt. This is now removed
because elliptic.Unmarshal already checks whether the input is a valid curve
point since Go 1.5.
Co-authored-by: Felix Lange <fjl@twurst.com>
- Dump stats also for --bench flag.
- From memory stats only show number and size of allocations. This is what `test -bench` shows. I doubt others like number of GC runs are any useful, but can be added if requested.
- Now the mem stats are for single execution in case of --bench.
* les: move execqueue into utilities package
execqueue is a util for executing queued functions
in a serial order which is used by both les server
and les client. Move it to common package.
* les: move randselect to utilities package
weighted_random_selector is a helpful tool for randomly select
items maintained in a set but based on the item weight.
It's used anywhere is LES package, mainly by les client but will
be used in les server with very high chance. So move it into a
common package as the second step for les separation.
* les: rename to utils
The test failed due to what appears to be fluctuations in time.Sleep, which is
not the actual method under test. This change modifies it so we compare the
metered Max to the actual time instead of the desired time.
This new API allows reading accounts and their content by address range.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
* eth: improve shutdown synchronization
Most goroutines started by eth.Ethereum didn't have any shutdown sync at
all, which lead to weird error messages when quitting the client.
This change improves the clean shutdown path by stopping all internal
components in dependency order and waiting for them to actually be
stopped before shutdown is considered done. In particular, we now stop
everything related to peers before stopping 'resident' parts such as
core.BlockChain.
* eth: rewrite sync controller
* eth: remove sync start debug message
* eth: notify chainSyncer about new peers after handshake
* eth: move downloader.Cancel call into chainSyncer
* eth: make post-sync block broadcast synchronous
* eth: add comments
* core: change blockchain stop message
* eth: change closeBloomHandler channel type
Turns out the way RDATA limits work is documented after all,
I just didn't search right. The trick to make it work is to
count UPSERTs twice.
This also adds an additional check to ensure TTL changes are
applied on existing records.
Prior to this change, eth_call changed the balance of the sender account in the
EVM environment to 2^256 wei to cover the gas cost of the call execution.
We've had this behavior for a long time even though it's super confusing.
This commit sets the default call gasprice to zero instead of updating the balance,
which is better because it makes eth_call semantics less surprising. Removing
the built-in balance assignment also makes balance overrides work as expected.
This just prevents a false negative ERROR warning when, for some unknown
reason, a user attempts to turn on the module rpc even though it's already going
to be on.
This PR fixes issues in TableDatabase.
TableDatabase is a wrapper of underlying ethdb.Database with an additional prefix.
The prefix is applied to all entries it maintains. However when we try to retrieve entries
from it we don't handle the key properly. In theory the prefix should be truncated and
only user key is returned. But we don't do it in some cases, e.g. the iterator and batch
replayer created from it. So this PR is the fix to these issues.
This revision of go-duktype fixes the following warning
```
duk_logging.c: In function ‘duk__logger_prototype_log_shared’:
duk_logging.c:184:64: warning: ‘Z’ directive writing 1 byte into a region of size between 0 and 9 [-Wformat-overflow=]
184 | sprintf((char *) date_buf, "%04d-%02d-%02dT%02d:%02d:%02d.%03dZ",
| ^
In file included from /usr/include/stdio.h:867,
from duk_logging.c:5:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: ‘__builtin___sprintf_chk’ output between 25 and 85 bytes into a destination of size 32
36 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
37 | __bos (__s), __fmt, __va_arg_pack ());
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
This makes eth_call and eth_estimateGas use the zero address
as sender when the "from" parameter is not supplied.
Co-authored-by: Felix Lange <fjl@twurst.com>
This removes the error added in #20597 in favor of a log message at
error level. Failing to start broke a bunch of people's setups and is
probably not the right thing to do for this check.
This adds additional logic to re-resolve the root name of a tree when a
couple of leaf requests have failed. We need this change to avoid
getting into a failure state where leaf requests keep failing for half
an hour when the tree has been updated.
* node: expose config in service context
* eth: integrate p2p/dnsdisc
* cmd/geth: add some DNS flags
* eth: remove DNS URLs
* cmd/utils: configure DNS names for testnets
* params: update DNS URLs
* cmd/geth: configure mainnet DNS
* cmd/utils: rename DNS flag and fix flag processing
* cmd/utils: remove debug print
* node: fix test
* p2p: new dial scheduler
This change replaces the peer-to-peer dial scheduler with a new and
improved implementation. The new code is better than the previous
implementation in two key aspects:
- The time between discovery of a node and dialing that node is
significantly lower in the new version. The old dialState kept
a buffer of nodes and launched a task to refill it whenever the buffer
became empty. This worked well with the discovery interface we used to
have, but doesn't really work with the new iterator-based discovery
API.
- Selection of static dial candidates (created by Server.AddPeer or
through static-nodes.json) performs much better for large amounts of
static peers. Connections to static nodes are now limited like dynanic
dials and can no longer overstep MaxPeers or the dial ratio.
* p2p/simulations/adapters: adapt to new NodeDialer interface
* p2p: re-add check for self in checkDial
* p2p: remove peersetCh
* p2p: allow static dials when discovery is disabled
* p2p: add test for dialScheduler.removeStatic
* p2p: remove blank line
* p2p: fix documentation of maxDialPeers
* p2p: change "ok" to "added" in static node log
* p2p: improve dialTask docs
Also increase log level for "Can't resolve node"
* p2p: ensure dial resolver is truly nil without discovery
* p2p: add "looking for peers" log message
* p2p: clean up Server.run comments
* p2p: fix maxDialedConns for maxpeers < dialRatio
Always allocate at least one dial slot unless dialing is disabled using
NoDial or MaxPeers == 0. Most importantly, this fixes MaxPeers == 1 to
dedicate the sole slot to dialing instead of listening.
* p2p: fix RemovePeer to disconnect the peer again
Also make RemovePeer synchronous and add a test.
* p2p: remove "Connection set up" log message
* p2p: clean up connection logging
We previously logged outgoing connection failures up to three times.
- in SetupConn() as "Setting up connection failed addr=..."
- in setupConn() with an error-specific message and "id=... addr=..."
- in dial() as "Dial error task=..."
This commit ensures a single log message is emitted per failure and adds
"id=... addr=... conn=..." everywhere (id= omitted when the ID isn't
known yet).
Also avoid printing a log message when a static dial fails but can't be
resolved because discv4 is disabled. The light client hit this case all
the time, increasing the message count to four lines per failed
connection.
* p2p: document that RemovePeer blocks
The feature update allows the GraphQL API endpoint to retrieve
transaction signature R,S,V parameters.
Co-authored-by: amitshah <amitshah0t7@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
For longer records and subtree entries, the deployer created two
separate TXT records. This doesn't work as intended because the client
will receive the two records in arbitrary order. The fix is to encode
longer values as "string1""string2" instead of "string1", "string2".
This encoding creates a single record on AWS Route53.
Adds the 'geth dumpgenesis' command, which writes the configured
genesis in JSON format to stdout. This provides a way to generate the
data (structure and content) that can then be used with the 'geth init'
command.
* core/vm/runtime: add test for blockhash
* core/evm: less iteration in blockhash
* core/vm/runtime: nitpickfix
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
This change makes the client attempt to reconnect when a write fails.
We already had reconnect support, but the reconnect would previously
happen on the next call after an error. Being more eager leads to a
smoother experience overall.
This replaces the JavaScript interpreter used by the console with goja,
which is actively maintained and a lot faster than otto. Clef still uses otto
and eth/tracers still uses duktape, so we are currently dependent on three
different JS interpreters. We're looking to replace the remaining uses of otto
soon though.
* log: delete RotatingFileHandler
We added this for the dashboard, which is gone now. The
handler never really worked well and had data race and file
handling issues.
* internal/debug: remove unused RotatingFileHandler setup code
This change works around the 32k RDATA character limit per change
request and fixes several issues in the deployer which prevented it from
working for our production trees.
This is a temporary fix for a problem which started happening when the
dialer was changed to read nodes from an enode.Iterator. Before the
iterator change, discovery queries would always return within a couple
seconds even if there was no Internet access. Since the iterator won't
return unless a node is actually found, discoverTask can take much
longer. This means that the 'emergency connect' logic might not execute
in time, leading to a stuck node.
* build: remove env.sh
This removes the dirty symlink-to-self hack we've had for years. The
script was added to enable building without GOPATH and did that job
reliably for all this time. We can remove the workaround because modern
Go supports building without GOPATH natively.
* Makefile: add GO111MODULE=on to environment
* internal/testlog: print file+line number of log call in test log
This changes the unit test logger to print the actual file and line
number of the logging call instead of "testlog.go:44".
Output of 'go test -v -run TestServerListen ./p2p' before this change:
=== RUN TestServerListen
--- PASS: TestServerListen (0.00s)
testlog.go:44: DEBUG[01-08|15:16:31.651] UDP listener up addr=127.0.0.1:62678
testlog.go:44: DEBUG[01-08|15:16:31.651] TCP listener up addr=127.0.0.1:62678
testlog.go:44: TRACE[01-08|15:16:31.652] Accepted connection addr=127.0.0.1:62679
And after:
=== RUN TestServerListen
--- PASS: TestServerListen (0.00s)
server.go:868: DEBUG[01-08|15:25:35.679] TCP listener up addr=127.0.0.1:62712
server.go:557: DEBUG[01-08|15:25:35.679] UDP listener up addr=127.0.0.1:62712
server.go:912: TRACE[01-08|15:25:35.680] Accepted connection addr=127.0.0.1:62713
* internal/testlog: document use of t.Helper
* add node.go unit test file node_test.go
* add node_test.go file license and rollback trie_test.go
* fix unused variable v
* trie: fix license year
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* les: move the checkpoint oracle into its own package
It's first step of refactor LES package. LES package
basically can be divided into LES client and LES server.
However both sides will use checkpoint package for
status retrieval and verification. So this PR moves
checkpoint oracle into a separate package
* les: address comments
This commit intents to replicate the DialHTTPWithClient function which allows
creating a RPC Client using a custom dialer but for websockets.
We introduce a new DialWebsocketWithDialer function which allows the caller
to instantiate a new websocket client using a custom dialer.
* Add more functionality to the sim (#5)
* backends: implement more of ethclient in sim
* backends: add BlockByNumber to simulated backend
* backends: make simulated progress function agree with syncprogress interface for client
* backends: add more tests
* backends: add more comments
* backends: fix sim for index in tx and add tests
* backends: add lock back to estimategas
* backends: goimports
* backends: go ci lint
* Add more functionality to the sim (#5)
* backends: implement more of ethclient in sim
* backends: add BlockByNumber to simulated backend
* backends: make simulated progress function agree with syncprogress interface for client
* backends: add more tests
* backends: add more comments
* backends: fix sim for index in tx and add tests
* backends: add lock back to estimategas
* backends: goimports
* backends: go ci lint
* assert errs
* p2p/dnsdisc: add support for enode.Iterator
This changes the dnsdisc.Client API to support the enode.Iterator
interface.
* p2p/dnsdisc: rate-limit DNS requests
* p2p/dnsdisc: preserve linked trees across root updates
This improves the way links are handled when the link root changes.
Previously, sync would simply remove all links from the current tree and
garbage-collect all unreachable trees before syncing the new list of
links.
This behavior isn't great in certain cases: Consider a structure where
trees A, B, and C reference each other and D links to A. If D's link
root changed, the sync code would first remove trees A, B and C, only to
re-sync them later when the link to A was found again.
The fix for this problem is to track the current set of links in each
clientTree and removing old links only AFTER all links are synced.
* p2p/dnsdisc: deflake iterator test
* cmd/devp2p: adapt dnsClient to new p2p/dnsdisc API
* p2p/dnsdisc: tiny comment fix
This change:
- removes the PostChainEvents method on core.BlockChain.
- sorts 'removed log' events by block number.
- fire the NewChainHead event if we inject a canonical block into the chain
even if the entire insertion is not successful.
- guarantees correct event ordering in all cases.
The original idea behind this change was to remove a use of the
deprecated CancelRequest method. Simply removing it would've been an
option, but I couldn't resist and did a bit of a refactoring instead.
All remote sealing code was contained in a single giant function. Remote
sealing is now extracted into its own object, remoteSealer.
* go.mod, vendor: switch to Go modules
* travis: explicitly enable go modules in Go 1.11 and 1.12
* accounts/abi/bind: switch binding test to go modules
* travis, build: aggregate and upload go mod dependencies for PPA
* go.mod: tidy up the modules to avoid xgo writes to go.sum
* build, internal/build: drop own file/folder copier
* travis: fake build ppa only for go module dependencies
* mobile: fix CopyFile switch to package cp
* build, travis: use ephemeral debsrc GOPATH to get mod deps
* rpc: remove 'exported or builtin' restriction for parameters
There is no technial reason for this restriction because package reflect
can create values of any type. Requiring parameters and return values to
be exported causes a lot of noise in package exports.
* rpc: fix staticcheck warnings
This PR introduces clique_status which gives info about the health of
the clique network.
It's currently a bit PITA to find out how a clique network is
performing, and it can easily happen that sealers drop off -- and
everything is 'fine' until one more signer drops off, and the network
suddenly halts.
The new method provides the following stats:
- Which signers are currently active, and have signed blocks in the last
N (set to 64) blocks?
- How many blocks has each signer signed?
- What is the difficulty in the last N blocks, compared to the
theoretical maximum?
* core: s/isEIP155/isEIP2/ (fix)
This signature variable name reflects a spec'd change
in gas cost for creating contracts as documented in EIP2 (Homestead HF).
https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2.md#specification
* core: s/isEIP2/sIsHomestead/g
Use isHomestead since Homestead is what the caller
and rest of the code uses.
* build: use golangci-lint
This changes build/ci.go to download and run golangci-lint instead
of gometalinter.
* core/state: fix unnecessary conversion
* p2p/simulations: fix lock copying (found by go vet)
* signer/core: fix unnecessary conversions
* crypto/ecies: remove unused function cmpPublic
* core/rawdb: remove unused function print
* core/state: remove unused function xTestFuzzCutter
* core/vm: disable TestWriteExpectedValues in a different way
* core/forkid: remove unused function checksum
* les: remove unused type proofsData
* cmd/utils: remove unused functions prefixedNames, prefixFor
* crypto/bn256: run goimports
* p2p/nat: fix goimports lint issue
* cmd/clef: avoid using unkeyed struct fields
* les: cancel context in testRequest
* rlp: delete unreachable code
* core: gofmt
* internal/build: simplify DownloadFile for Go 1.11 compatibility
* build: remove go test --short flag
* .travis.yml: disable build cache
* whisper/whisperv6: fix ineffectual assignment in TestWhisperIdentityManagement
* .golangci.yml: enable goconst and ineffassign linters
* build: print message when there are no lint issues
* internal/build: refactor download a bit
* rpc: improve codec abstraction
rpc.ServerCodec is an opaque interface. There was only one way to get a
codec using existing APIs: rpc.NewJSONCodec. This change exports
newCodec (as NewFuncCodec) and NewJSONCodec (as NewCodec). It also makes
all codec methods non-public to avoid showing internals in godoc.
While here, remove codec options in tests because they are not
supported anymore.
* p2p/simulations: use github.com/gorilla/websocket
This package was the last remaining user of golang.org/x/net/websocket.
Migrating to the new library wasn't straightforward because it is no
longer possible to treat WebSocket connections as a net.Conn.
* vendor: delete golang.org/x/net/websocket
* rpc: fix godoc comments and run gofmt
This removes the dashboard project. The dashboard was an experimental
browser UI for geth which displayed metrics and chain information in
real time. We are removing it because it has marginal utility and nobody
on the team can maintain it.
Removing the dashboard removes a lot of dependency code and shaves
6 MB off the geth binary size.
This adds all dashboard changes from the last couple months.
We're about to remove the dashboard, but decided that we should
get all the recent work in first in case anyone wants to pick up this
project later on.
* cmd, dashboard, eth, p2p: send peer info to the dashboard
* dashboard: update npm packages, improve UI, rebase
* dashboard, p2p: remove println, change doc
* cmd, dashboard, eth, p2p: cleanup after review
* dashboard: send current block to the dashboard client
* travis: Enable ARM support
* Include fixes from 20039
* Add a trace to debug the invalid lookup issue
* Try increasing the timeout to see if the arm test passes
* Investigate the resolver issue
* Increase arm64 timeout for clique test
* increase timeout in tests for arm64
* Only test the failing tests
* Review feedback: don't export epsilon
* Remove investigation tricks+include fjl's feeback
* Revert the retry ahead of using the mock resolver
* Fix rebase errors
* build: bump PPAs to Go 1.13 (via longsleep), keep Trusty on 1.11
* travis, build, vendor: use own Go bundle for PPA builds
* travis, build, internal, vendor: smarter Go bundler, own untar
* build: updated ci-notes with new Go bundling, only make, don't test
* core/evm, contracts: avoid copying memory for input in calls + make ecrecover not modify input buffer
* core/vm: optimize mstore a bit
* core/vm: change Get -> GetCopy in vm memory access
* accounts/abi: fix various issues
The fixed issues include:
(1) If there is no return in a call function, unpack should
return nil error
(2) For some functions which have struct array as parameter,
it will also be detected and generate the struct definition
(3) For event, if it has non-indexed parameter, the parameter
name will also be assigned if empty. Also the internal struct
will be detected and generate struct defition if not exist.
(4) Fix annotation generation in event function
* accounts/abi: add new abi field internalType
* accounts: address comments and add tests
* accounts/abi: replace strings.ReplaceAll with strings.Replace
* core/asm: Fix encoding of pushed labels
EVM uses big-endian byte-order, so to pad a label value to 4 bytes,
zeros must be added to the front, not the end.
* core/asm: Fix PC calculations when a label is pushed
Incrementing PC by 5 is only correct if the label appears after a jump,
in which case there is an implicit push. When it appears after an explicit
push, PC should only be incremented by 4.
* core/asm: Allow JUMP with no argument
This way, a label can be pushed explicitly, or loaded from memory to
implement a jump table.
When we flush a batch of trie nodes into database during the state
sync, we should guarantee that all children should be flushed before
parent.
Actually the trie nodes commit order is strict by: children -> parent.
But when we flush all ready nodes into db, we don't need the order
anymore since
(1) they are all ready nodes (no more dependency)
(2) underlying database provides write atomicity
The gas price was not passed to the `EstimateGas` function. As a result,
conditional execution paths depending on `tx.gasprice` could be not
correctly processed and we could get invalid gas estimates for contract
function calls.
This PR adds a new unit test in miner package which will create some blocks from miner and then import into another chain. In this way, we can ensure all blocks generated by Geth miner obey consensus rules.
In Geth, we have two sources for configuration:
(1) Config file
(2) Command line flag
Basically geth will first resolve config file and then overwrite
configs with command line flags.
This issue is: geth should only overwrite configs if flags are truly
set. So before we apply any flag to configs, `GlobalIsSet` check
is necessary.
* update tests for istanbul
* tests: updated blockchaintests, see https://github.com/ethereum/tests/issues/637
* tests: update again, hopefully fixed this time
* tests: skip time consuming, run legacy tests
* tests: update again
* build: disable long-running tests on travis
* tests: fix formatting nits
* tests: I hate github's editor
This adds an implementation of node discovery via DNS TXT records to the
go-ethereum library. The implementation doesn't match EIP-1459 exactly,
the main difference being that this implementation uses separate merkle
trees for tree links and ENRs. The EIP will be updated to match p2p/dnsdisc.
To maintain DNS trees, cmd/devp2p provides a frontend for the p2p/dnsdisc
library. The new 'dns' subcommands can be used to create, sign and deploy DNS
discovery trees.
This change adds tests for the virtual clock and aligns the interface
with the time package by renaming Cancel to Stop. It also removes the
binary search from Stop because it complicates the code unnecessarily.
* rlp: improve nil pointer handling
In both encoder and decoder, the rules for encoding nil pointers were a
bit hard to understand, and didn't leave much choice. Since RLP allows
two empty values (empty list, empty string), any protocol built on RLP
must choose either of these values to represent the null value in a
certain context.
This change adds choice in the form of two new struct tags, "nilString"
and "nilList". These can be used to specify how a nil pointer value is
encoded. The "nil" tag still exists, but its implementation is now
explicit and defines exactly how nil pointers are handled in a single
place.
Another important change in this commit is how nil pointers and the
Encoder interface interact. The EncodeRLP method was previously called
even on nil values, which was supposed to give users a choice of how
their value would be handled when nil. It turns out this is a stupid
idea. If you create a network protocol containing an object defined in
another package, it's better to be able to say that the object should be
a list or string when nil in the definition of the protocol message
rather than defining the encoding of nil on the object itself.
As of this commit, the encoding rules for pointers now take precedence
over the Encoder interface rule. I think the "nil" tag will work fine
for most cases. For special kinds of objects which are a struct in Go
but strings in RLP, code using the object can specify the desired
encoding of nil using the "nilString" and "nilList" tags.
* rlp: propagate struct field type errors
If a struct contained fields of undecodable type, the encoder and
decoder would panic instead of returning an error. Fix this by
propagating type errors in makeStruct{Writer,Decoder} and add a test.
Most of these changes are related to the Go 1.13 changes to test binary
flag handling.
* cmd/geth: make attach tests more reliable
This makes the test wait for the endpoint to come up by polling
it instead of waiting for two seconds.
* tests: fix test binary flags for Go 1.13
Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.
* crypto/ecies: remove useless -dump flag in tests
* p2p/simulations: fix test binary flags for Go 1.13
Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.
* build: remove workaround for ./... vendor matching
This workaround was necessary for Go 1.8. The Go 1.9 release changed
the expansion rules to exclude vendored packages.
* Makefile: use relative path for GOBIN
This makes the "Run ./build/bin/..." line look nicer.
* les: fix test binary flags for Go 1.13
Calling flag.Parse during package initialization is prohibited
as of Go 1.13 and causes test failures. Call it in TestMain instead.
* Change chainId in genesis block to 8888 from 0
Change chainId in genesis block to 8888 from 0 per Moriteru in https://ethereum.stackexchange.com/a/28082/40230.
* Replace 8888 with “<arbitrary positive integer>”
Per PR review, replace Moriteru’s arbitrary positive integer 8888 with “<arbitrary positive integer>” in chainId field.
This change adds support for gzip encoding on HTTP responses.
Gzip encoding is used when the client sets the 'accept-encoding: gzip' header.
Original change by @brianosaurus, with fixes from @SjonHortensius.
This copies cli.printHelp but changes minwidth to 38. Custom flag
code is improved to print the default value using cli.FlagStringer like
all built-in flags do.
- Move the existing tests from memorydb into a generalized testsuite
that can be run by any ethdb backend implementation.
- Add several more test cases to clarify some non-obvious nuances when
implementing a custom ethdb backend, such as the behaviour of
NewIteratorWithPrefix vs NewIteratorWithStart.
- Add leveldb to the testsuite using in-memory storage for fast
execution.
The precompile at 0x09 wraps the BLAKE2b F compression function:
https://tools.ietf.org/html/rfc7693#section-3.2
The precompile requires 6 inputs tightly encoded, taking exactly 213
bytes, as explained below.
- `rounds` - the number of rounds - 32-bit unsigned big-endian word
- `h` - the state vector - 8 unsigned 64-bit little-endian words
- `m` - the message block vector - 16 unsigned 64-bit little-endian words
- `t_0, t_1` - offset counters - 2 unsigned 64-bit little-endian words
- `f` - the final block indicator flag - 8-bit word
[4 bytes for rounds][64 bytes for h][128 bytes for m][8 bytes for t_0]
[8 bytes for t_1][1 byte for f]
The boolean `f` parameter is considered as `true` if set to `1`.
The boolean `f` parameter is considered as `false` if set to `0`.
All other values yield an invalid encoding of `f` error.
The precompile should compute the F function as specified in the RFC
(https://tools.ietf.org/html/rfc7693#section-3.2) and return the updated
state vector `h` with unchanged encoding (little-endian).
See EIP-152 for details.
* Ref #19906 - replace passPHRASE with passWORD in any user interactions
this skips doccomments and variablenames to minimize impact. It does
however include a rename of the `ethkey` `changepassphrase` parameter
* console: fix JavaScript error capitalization
* graphql, internal/ethapi: extend eth_call
This PR offers the third option parameter for eth_call API.
Caller can specify a batch of contracts for overriding the
original account metadata(nonce, balance, code, state).
It has a few advantages:
* It's friendly for debugging
* It's can make on-chain contract lighter for getting rid of
state access functions
* core, internal: address comments
* accounts/mananger, internal/ethapi/api: Add new function AllAccounts on account manager to remove the duplication code on getting all wallets accounts
* Rename to Accounts
* Rename to AllAccounts
* params: add IsIstanbul to config + rules
IstanbulBlock, used to determine if the config IsIstanbul, is currently
left nil until an actual block is chosen.
* params, core/vm: implement EIP-1108
Old gas costs for elliptic curve operations are given the PreIstanbul
prefix, while current gas costs retain the unprefixed names. The actual
precompile implementations are the same, so they are factored out into
common functions that are called by the pre-Istanbul and current
precompile structs. Finally, an Istanbul precompile list is added that
references the new precompile structs, which in turn reference the new
gas costs.
* params: fix fork ordering, add missing chain compatibility check
* params, core/vm: deprecating gastable, part 1
* core/vm, params: deprecate gastable, use both constant and dynamic gas
* core/vm, params: remove gastable, remove copypaste
* core/vm: make use of the chainrules
* interpreter: make tracing count constant+dynamic gas
* core/vm: review concerns (param/method name changes)
* core/vm: make use of chainrules more
The abi package already supports function overload by adding a suffix to the overloaded function name, but it uses the function name with suffix to calculate signature(both for the event and method).
This PR fixes it by adding a new field named RawName, which can be used to calcuate all signatures but use Name to distinguish different overloaded function.
This change ensures 'blockHash', 'blockNumber' and 'transactionIndex'
are set to null for pending transactions. This behavior is required by
the Ethereum JSON-RPC spec.
* rpc: implement websockets with github.com/gorilla/websocket
This change makes package rpc use the github.com/gorilla/websocket
package for WebSockets instead of golang.org/x/net/websocket. The new
library is more robust and supports all WebSocket features including
continuation frames.
There are new tests for two issues with the previously-used library:
- TestWebsocketClientPing checks handling of Ping frames.
- TestWebsocketLargeCall checks whether the request size limit is
applied correctly.
* rpc: raise HTTP/WebSocket request size limit to 5MB
* rpc: remove default origin for client connections
The client used to put the local hostname into the Origin header because
the server wanted an origin to accept the connection, but that's silly:
Origin is for browsers/websites. The nobody would whitelist a particular
hostname.
Now that the server doesn't need Origin anymore, don't bother setting
one for clients. Users who need an origin can use DialWebsocket to
create a client with arbitrary origin if needed.
* vendor: put golang.org/x/net/websocket back
* rpc: don't set Origin header for empty (default) origin
* rpc: add HTTP status code to handshake error
This makes it easier to debug failing connections.
* ethstats: use github.com/gorilla/websocket
* rpc: fix lint
Gollvm has very aggressive dead code elimination that completely
removes one of these two benchmarks. To prevent this, use the
result of the benchmark (a boolean), and to be "fair", make the
transformation to both benchmarks.
To be reliably assured of not removing the code, "use" means
assigning to an exported global. Non-exported globals and
//go:noinline functions are possibly subject to this optimization.
* cmd/puppeth: integrate blockscout
* cmd/puppeth: expose debug namespace for blockscout
* cmd/puppeth: fix dbdir
* cmd/puppeth: run explorer in archive mode
* cmd/puppeth: ensure node is synced
* cmd/puppeth: fix explorer docker alignment + drop unneeded exec
* cmd/puppeth: polish up config saving and reloading
* cmd/puppeth: check both web and p2p port for explorer service
* eth: chain config (genesis + fork) ENR entry
* core/forkid, eth: protocol independent fork ID, update to CRC32 spec
* core/forkid, eth: make forkid a struct, next uint64, enr struct, RLP
* core/forkid: change forkhash rlp encoding from int to [4]byte
* eth: fixup eth entry a bit and update it every block
* eth: fix lint
* eth: fix crash in ethclient tests
This PR adds some hardening in the lower levels of the protocol stack, to bail early on invalid data. Primarily, attacks that this PR protects against are on the "annoyance"-level, which would otherwise write a couple of megabytes of data into the log output, which is a bit resource intensive.
* accounts, abigen: link dependent libs in deploy
* abigen: add java generation
* bind: Fix unit tests
* abigen: add unit test
* Fix CI
* Post-rebase fixes
* Fix rebase issue
* accounts/abi: Gary's review feedback
* accounts/abi: More Gary feedback
* accounts/abi: minor fixes
* accounts/abi, cmd/abigen: support tuple
accounts/abi/bind, cmd/abigen: add objc back
accounts/abi/bind: use byte[24] as function indicator
accounts/abi/bind: resolve struct slice or array
accounts/abi/bind: remove sort logic
accounts: fix issues in abi
* accounts/abi: address comment
This PR fixes an issue in chain indexer. Currently chain indexer will
validate whether the stored data is canonical by comparing section head
and canonical hash. But the header of the checkpoint may not exist in
the database. We should skip validation for sections below the
checkpoint.
* accounts/abi/bind: Accept function ptr parameter
They are translated as [24]byte
* Add Java template version
* accounts/abi/bind: fix merge issue
* Fix CI
* accounts/abi: Fix method overwritten by same name methods.
* accounts/abi: Fix method overwritten by same name methods.
* accounts/abi: avoid possible name conflict
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
* core/state, cmd/geth: streaming json output dump cmd + optional code+storage
* dump: add option to continue even if preimages are missing
* core, evm: lint nits
* cmd: use local flags for dump, omit empty code/storage
* core/state: fix state dump test
This function searches for an event+parameters in the ABI and returns it if found.
Co-authored-by: Victor Tran <vu.tran54@gmail.com>
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
This PR updates a comment about the maximum client subscription buffer
to reflect changes made previously, and fixes a test that wouldn't fail
when wantError == true but execution did not return an error.
* core: move TxPool reorg and events to background goroutine
This change moves internal queue re-shuffling work in TxPool to a
background goroutine, TxPool.runReorg. Requests to execute runReorg are
accumulated by the new scheduleReorgLoop. The new loop also accumulates
transaction events.
The motivation for this change is making sends to txFeed synchronous
instead of sending them in one-off goroutines launched by 'add' and
'promoteExecutables'. If a downstream consumer of txFeed is blocked for
a while, reorg requests and events will queue up.
* core: remove homestead check in TxPool
This change removes tracking of the homestead block number from TxPool.
The homestead field was used to enforce minimum gas of 53000 for
contract creations after the homestead fork, but not before it. Since
nobody would want configure a non-homestead chain nowadays and contract
creations usually take more than 53000 gas, the extra correctness is
redundant and can be removed.
* core: fixes for review comments
* core: remove BenchmarkPoolInsert
This is useless now because there is no separate code path for
individual transactions anymore.
* core: fix pending counter metric
* core: fix pool tests
* core: dedup txpool announced events, discard stales
* core: reorg tx promotion/demotion to avoid weird pending gaps
When cancelling the context for a call on a HTTP-based client while the
call is running, the select in requestOp.wait may hit the <-context.Done()
case instead of the <-op.resp case. This doesn't happen often -- our
cancel test hasn't caught this even though it ran thousands of times
on CI since the RPC client was added.
Fixes#19714
* les: reject client connection if it makes too much invalid req
* les: address comments
* les: use uint32
* les: fix variable name
* les: add invalid counter for duplicate invalid req
The dialer limits itself to one attempt every 30s. Apply the same limit
in Server and reject peers which try to connect too eagerly. The check
against the limit happens right after accepting the connection.
Further changes in this commit ensure we pass the Server logger
down to Peer instances, discovery and dialState. Unit test logging now
works in all Server tests.
* vendor: remove unused dependencies
These were used by swarm code, which has now migrated to its own repository.
* travis.yml: remove sudo requirement for test builders
These needed sudo to run FUSE tests for swarm.
* p2p/enr: add entries for for IPv4/IPv6 separation
This adds entry types for "ip6", "udp6", "tcp6" keys. The IP type stays
around because removing it would break a lot of code and force everyone
to care about the distinction.
* p2p/enode: track IPv4 and IPv6 address separately
LocalNode predicts the local node's UDP endpoint and updates the record.
This change makes it predict IPv4 and IPv6 endpoints separately since
they can now be in the record at the same time.
* p2p/enode: implement base64 text format
* all: switch to enode.Parse(...)
This allows passing base64-encoded node records to all the places that
previously accepted enode:// URLs. The URL format is still supported.
* cmd/bootnode, p2p: log node URL instead of ENR
...and return the base64 record in NodeInfo.
* p2p/discover: export Ping and RequestENR
These two are useful for checking the status of a node.
* cmd/devp2p: add devp2p debug tool
This is a new tool for debugging p2p issues. It supports a few
basic tasks for now, but many more things can and will be added
in the near future.
devp2p enrdump -- prints ENRs readably
devp2p discv4 ping -- checks if a node is up
devp2p discv4 requestenr -- gets a node's record
devp2p discv4 resolve -- finds a node through the DHT
Github has started supporting SECURITY.md to contain a project's security policy. Adding this information to the repository makes it easier to determine how to disclosure a vulnerability as SECURITY.md becomes a standard.
The pgp fingerprint and key is taken from bounty.ethereum.org.
* Add retesteth command
* Remove label and insert full version
* mineBlock - break the inner loop when the block is full
* Fixes for touched non-reward accounts, gas limit issues
* Not fail when SendTx has transaction with incorrect RLP
* Fix linter (unnecessary conversion)
* retesteth: add usage string to flag
* accounts/scwallet: Add a switch to enable smartcard support
* accounts: change the meaning of the switch
* disable card support in windows until tested
* only activate account if pcscd socket file is present
* the switch is now the path to the socket file
* accounts/scwallet: holiman's review feedback
* accounts/scwallet: send the path to go-pcsclite
* accounts/scwallet: add default, per platform path
* accounts/scwallet: fix error log warning
* accounts/scwallet: update pcsc lib to latest
* accounts/scwallet: use default path from pcsclite
* scwallet: forgot to change switch name
* cmd: minor style cleanups (error handling first, then happy path)
* Removed comment section referring to Cloudflare's bn curve parameters
* Added comment to clarify the nature of the parameters
* Changed value of xi to i+9
* core: reinit chain from freezer in batches
* core/rawdb: concurrent database reinit from freezer dump
* core/rawdb: reinit from freezer in sequential order
* core, eth: some fixes for freezer
* vendor, core/rawdb, cmd/geth: add db inspector
* core, cmd/utils: check ancient store path forceily
* cmd/geth, common, core/rawdb: a few fixes
* cmd/geth: support windows file rename and fix rename error
* core: support ancient plugin
* core, cmd: streaming file copy
* cmd, consensus, core, tests: keep genesis in leveldb
* core: write txlookup during ancient init
* core: bump database version
* all: freezer style syncing
core, eth, les, light: clean up freezer relative APIs
core, eth, les, trie, ethdb, light: clean a bit
core, eth, les, light: add unit tests
core, light: rewrite setHead function
core, eth: fix downloader unit tests
core: add receipt chain insertion test
core: use constant instead of hardcoding table name
core: fix rollback
core: fix setHead
core/rawdb: remove canonical block first and then iterate side chain
core/rawdb, ethdb: add hasAncient interface
eth/downloader: calculate ancient limit via cht first
core, eth, ethdb: lots of fixes
* eth/downloader: print ancient disable log only for fast sync
This change implements EIP-868. The UDPv4 transport announces support
for the extension in ping/pong and handles enrRequest messages.
There are two uses of the extension: If a remote node announces support
for EIP-868 in their pong, node revalidation pulls the node's record.
The Resolve method requests the record unconditionally.
These changes fix two corner cases related to internal handling of types
in package rlp: The "tail" struct tag can only be applied to the last field.
The check for this was wrong and didn't allow for private fields after the
field with the tag. Unsupported types (e.g. structs containing int) which
implement either the Encoder or Decoder interface but not both
couldn't be encoded/decoded.
Also fixes#19367
* core, eth, trie: bloom filter for trie node dedup during fast sync
* eth/downloader, trie: address review comments
* core, ethdb, trie: restart fast-sync bloom construction now and again
* eth/downloader: initialize fast sync bloom on startup
* eth: reenable eth/62 until we properly remove it
* core: fix import errors on clique crashes + empty blocks
* cosensus/clique, core: add test for the mirrored state issue
* core: address todo question wrt log count
* core: raise a louder warning for non-clique known blocks
swarm/api: integrate tags to count chunks being split and stored
swarm/api/http: integrate tags in middleware for HTTP `POST` calls and assert chunks being calculated and counted correctly
swarm: remove deprecated and unused code, add swarm hash to DoneSplit signature, remove calls to the api client from the http package
* swarm/shed: remove metrics fields from DB struct
* swarm/schunk: add String methods to modes
* swarm/storage/localstore: add metrics and traces
* swarm/chunk: unknown modes without spaces in String methods
* swarm/storage/localstore: remove bin number from pull subscription metrics
* swarm/storage/localstore: add resetting time metrics and code improvements
cmd/swarm/swarm-smoke: improve smoke tests (#1337)
swarm/network: remove dead code (#1339)
swarm/network: remove FetchStore and SyncChunkStore in favor of NetStore (#1342)
* add-date-to unstable
* fields-insteadof-split
* internal/build: support building with missing git
* docker: add git history back to support commit date in version
* internal/build: use PR commits hashes for PR builds
* core: import known blocks if they can be inserted as canonical blocks
* core: insert knowns blocks
* core: remove useless
* core: doesn't process head block in reorg function
* accounts: add note about backing up the keystore
* cmd, accounts: move the printout to accountCreate
* internal, signer: add info when new account is created via rpc
* cmd, internal, signer: split logs
* cmd/geth: make account new output a bit more verbose
This change makes getBalance, getCode, getStorageAt, getProof,
call, getTransactionCount return an error if the block number in
the request doesn't exist. getHeaderByNumber still returns null
for missing headers.
This change restructures the internals of p2p/discover to make room for
the discv5 code which will soon be added to this package.
- packet type names now have a "V4" suffix.
- ListenUDP returns *UDPv4 instead of *Table. This technically breaks
the API but the only caller in go-ethereum is package p2p, which uses
a compatible interface and doesn't need changes.
- The internal transport interface is changed to make Table reusable for v5.
- The 'lookup' code moves from table to transport. This required
updating the lookup unit test to use udpTest instead of a custom transport.
* swarm/network: fix data races in TestInitialPeersMsg test
* swarm/network: add Kademlia.Saturation method with lock
* swarm/network: add Hive.Peer method to safely retrieve a bzz peer
* swarm/network: remove duplicate comment
* p2p/testing: prevent goroutine leak in ProtocolTester
* swarm/network: fix data race in newBzzBaseTesterWithAddrs
* swarm/network: fix goroutone leaks in testInitialPeersMsg
* swarm/network: raise number of peer check attempts in testInitialPeersMsg
* swarm/network: use Hive.Peer in Hive.PeerInfo function
* swarm/network: reduce the scope of mutex lock in newBzzBaseTesterWithAddrs
* swarm/storage: disable TestCleanIndex with race detector
* core: lookup txs by block number instead of block hash
Transaction hashes now store a reference to their corresponding
block number as opposed to their hash. In benchmarks this was
shown to reduce storage by over 12 GB.
The main limitation of this approach is that transactions on
non-canonical blocks could never be looked up, however that is
currently not supported.
The database version has been upgraded to version 5 and the
transaction lookup process is backwards-compatible with the
prior two transaction lookup formats prexisting in the
database instance. Tests have been added to ensure this.
* core/rawdb: tiny review nit fixes
* cmd, eth, miner: disable advance sealing if user require
* cmd, console, miner, les, eth: wrap the miner config
* eth: remove todo
* cmd, miner: revert noadvance flag
The reason for this is: if the transaction execution is even longer
than block time, then this kind of transactions is DoS attack.
* swarm/storage/feed/lookup: Add context handling/forwarding
* swarm/storage/feed/lookup: Add test to catch bad hint
* swarm/storage/feed/lookup: Added context cancellation test
* swarm/api: fix file descriptor leak in NewTestSwarmServer
Swarm storage (localstore) was not closed. That resulted a
"too many open files" error if `TestClientUploadDownloadRawEncrypted`
was run with `-count 1000`.
* cmd/swarm: speed up StartNewNodes() by parallelization
Reduce cluster startup time from 13s to 7s.
* swarm/api: disable flaky TestClientUploadDownloadRawEncrypted with -race
* swarm/storage: disable flaky TestLDBStoreCollectGarbage (-race)
With race detection turned on the disabled cases often fail with:
"ldbstore_test.go:535: expected surplus chunk 150 to be missing, but got no error"
* cmd/swarm: fix process leak in TestACT and TestSwarmUp
Each test run we start 3 nodes, but we did not terminate them. So
those 3 nodes continued eating up 1.2GB (3.4GB with -race) after test
completion.
6b6c4d1c27 changed how we start clusters
to speed up tests. The changeset merged together test cases
and introduced a global cluster. But "forgot" about termination.
Let's get rid of "global cluster" so we have a clear owner of
termination (some time sacrifice), while leaving subtests to use the
same cluster.
* metrics/prometheus: added prometheus http server and metrics collector
* metrics/prometheus: minor cleanups
* metrics/prometheus: named keys instead name in tag
* metrics/prometheus: minor typo cleanups, sorted report
* accounts, core, internal, node: Add support for smartcard wallets
* accounts, internal: Changes in response to review
* vendor: pull in missing go-echd library
* accounts/scwallet, console: user friendly card opening
* accounts/scwallet: ordered wallets, tighter events, derivation logs
* accounts, console: frendly card errors, support pin unblock
* accounts/scwallet: fix crypto API change
* accounts/scwallet: rebase and update
* Fix some linter issues
* Remove the direct dependency on libpcsclite
Instead, use a go library that communicates with pcscd over a socket.
Also update the changes introduced by @gravityblast since this PR's
inception
* Temporary fix to the ADBU status call
* fix wallet status update
This is a temporary fix, better checks need to
be performed once the whole process has been
validated.
* Fix key derivation
* Add some documentation
* Update a comment to reflect the workings of the updated system
* Vendor keycard-go/derivationpath
* Formatting fixes
* Add instructions on how to install the card
* Achieve full transaction signature+sending
* PK derivation has to be supported by the card
* Fix linter issues
* Upgrade to keycard app v2.1.1
* Set gballet as codeowner of the smartcard wallet dir
* fix unnecessary condition linter warning
* refuse to overwrite the master key of a previously initialized card
* refresh the account list when initializing the card
* Update the card preparation instructions based on review feedback
* 'sanitize' JSON input
Co-Authored-By: gballet <gballet@gmail.com>
* Apply suggestions from code review
Co-Authored-By: gballet <gballet@gmail.com>
* fix a serialization error
* more review feedback
* More review feedback
* Can now specify the number of empty accounts to derive
* Fix rebase error: include norm package
* Update bip-39 ref and remove ebfe/scard from vendor
* Add missing dependency
This PR fixes this, moving domain.ChainId from the map's initializer down to a separate if statement which checks the existance of ChainId's value, similar to the rest of the fields, before adding it. I've also included a new test to demonstrate the issue
This resolves a minor issue where neighbors responses containing less
than 16 nodes would bump the failure counter, removing the node. One
situation where this can happen is a private deployment where the total
number of extant nodes is less than 16.
Issue found by @jsying.
This PR makes it easy to generate and execute testcases for VM arithmetic operations. By enabling and running the testcase TestWriteExpectedValues, a set of json files are created which contain input and output for each arith operation.
The test TestJsonTestcases executes all of those tests.
While meaningless as is, this PR makes it less risky to make changes (optimizations) to the vm operations, since there will be a larger body of testcases.
* swarm/network: fix hive bug not sending shallow peers
- hive bug: needed shallow peers were not sent to nodes beyond connection's proximity order
- add extensive protocol exchange tests for initial subPeersMsg-peersMsg exchange
- modify bzzProtocolTester to allow pregenerated overlay addresses
* swarm/network: attempt to fix hive persistance test
* swarm/network: fix TestHiveStatePersistance (#1320)
* swarm/network: remove trace lines from the hive persistance test
* address PR review comments
* swarm/network: address PR comments on TestInitialPeersMsg
* eliminate *testing.T argument from bzz/hive protocoltesters
* add sorting (only runs in test code) on peersMsg payload
* add random (0 to MaxPeersPerPO) peers for each po
* add extra peers closer to pivot than control
This PR is a more advanced form of the dirty-to-clean cacher (#18995),
where we reuse previous database write batches as datasets to uncache,
saving a dirty-trie-iteration and a dirty-trie-rlp-reencoding per block.
This PR fixes two issues in the PoW calculation of a Whisper envelope,
compared to the spec (see PoW Requirements):
- The pow is supposed to take the leading number of zeroes (i.e. most
significant zeroes) and what it did was to take the number of trailing
zeroes (i.e. least significant zeroes). It has been fixed to match what
the spec and Parity does.
- The spec expects to use the size of the RLP encoded envelope, and it took
something else, as described in #18070.
* travis: remove verbose from Swarm race tests
By removing -v our output will be cleaner, but the Travis job still
won't be terminated - due to 'no output for 10 minutes' - as keepalive
.sh produces a log line every 5 minutes.
* travis: extend Swarm race detection to p2p subpackages
As p2p/protocols, p2p/simulations and p2p/testing packages mostly
belong to the Swarm team.
* core/vm: remove function call for stack validation from evm runloop
* core/vm: separate gas calc into static + dynamic
* core/vm: optimize push1
* core/vm: reuse pooled bigints for ADDRESS, ORIGIN and CALLER
* core/vm: use generic error message for jump/jumpi, to avoid string interpolation
* testdata: fix tests for new error message
* core/vm: use 64-bit memory calculations
* core/vm: fix error in memory calculation
* core/vm: address review concerns
* core/vm: avoid unnecessary use of big.Int:BitLen()
dummyHook's fields were concurrently written by nodes and read by
the test. The simplest solution is to protect all fields with a mutex.
Enable: TestMultiplePeersDropSelf, TestMultiplePeersDropOther as they
seemingly accidentally stayed disabled during a refactor/rewrite
since 1836366ac1.
resolvesethersphere/go-ethereum#1286
* swarm/network: propagate span with ctx
* swarm/network: try to stop stream.send.request spans on time
* swarm/storage: add chunk ref as a log to netstore.fetcher span
Although current two implementations(ledgerDriver, trezorDriver) of interface driver.Close do not actually return any error. Instead, they only return nil.
But since the declaration of Close function returns error, it is better to check the returned error in case in future some new implementation of Close function returns error and we may forget to modify the function which invokes Close function at that time.
This PR will will break existing UIs, since it changes all calls like ApproveSignTransaction to be on the form ui_approveSignTransaction.
This is to make it possible for the UI to reuse the json-rpc library from go-ethereum, which uses this convention.
Also, this PR removes some unused structs, after import/export were removed from the external api (so no longer needs internal methods for approval)
One more breaking change is introduced, removing passwords from the ApproveSignTxResponse and the likes. This makes the manual interface more like the rulebased interface, and integrates nicely with the credential storage. Thus, the way it worked before, it would be tempting for the UI to implement 'remember password' functionality. The way it is now, it will be easy instead to tell clef to store passwords and use them.
If a pw is not found in the credential store, the user is prompted to provide the password.
These tests never run as the build tag excluded them from the CI
execution. As a results the (dead) code got out of sync with other
parts of Swarm and now they would not even compile. => Removed.
resolvesethersphere/go-ethereum#1238
* p2p/discover: remove unused function
* p2p/enode: use localItemKey for local sequence number
I added localItemKey for this purpose in #18963, but then
forgot to actually use it. This changes the database layout
yet again and requires bumping the version number.
* node: require LocalAppData variable
This avoids path inconsistencies on Windows XP.
Hat tip to @MicahZoltu for catching this so quickly.
* node: fix typo
* travis, build: switch to NDK 19b, fix gomobile builds
* travis, build: move NDK into its final bundle location
* travis: disable Android build on PRs once again
This change
- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
benchmarks which gives much more consistent results. Until now the
allowed number of light peers was just a guess which probably contributed
a lot to the fluctuating quality of available service. Everything related
to request cost is implemented in a single object, the 'cost tracker'. It
uses a fixed cost table with a global 'correction factor'. Benchmark code
is included and can be run at any time to adapt costs to low-level
implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
way, with added capabilities: There is now control over bandwidth, which
allows using the flow control parameters for client prioritization.
Target utilization over 100 percent is now supported to model concurrent
request processing. Total serving bandwidth is reduced during block
processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
assign priority bandwidth to certain clients and change prioritized
status even while the client is connected. The new API is meant for
cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
control functions through the new API.
* swarm/pss: fix data race on HandshakeController.symKeyIndex
The HandshakeController.symKeyIndex map was accessed concurrently.
Since insufficient test coverage the race is not detected every time.
However, running TestClientHandshake a 100 times seems to be enough to
reproduce the race.
Note: I've chosen HandshakeController.lock to protect
HandshakeController.symKeyIndex as that was already protected in a few
functions by that lock.
Additionally:
- removed unused testStore
- enabled tests in handshake_test.go as they pass
- removed code duplication by adding getSymKey()
* swarm/pss: fix a data race on HandshakeController.keyC
* swarm/pss: fix data races with on Pss.symKeyPool
* swarm/storage/mock: implement listings methods for mem and rpc stores
* swarm/storage/mock/rpc: add comments and newTestStore helper function
* swarm/storage/mock/mem: add missing comments
* swarm/storage/mock: add comments to new types and constants
* swarm/storage/mock/db: implement listings for mock/db global store
* swarm/storage/mock/test: add comments for MockStoreListings
* swarm/storage/mock/explorer: initial implementation
* cmd/swarm/global-store: add chunk explorer
* cmd/swarm/global-store: add chunk explorer tests
* swarm/storage/mock/explorer: add tests
* swarm/storage/mock/explorer: add swagger api definition
* swarm/storage/mock/explorer: not-zero test values for invalid addr and key
* swarm/storage/mock/explorer: test wildcard cors origin
* swarm/storage/mock/db: renames based on Fabio's suggestions
* swarm/storage/mock/explorer: add more comments to testHandler function
* cmd/swarm/global-store: terminate subprocess with Kill in tests
* swarm/storage/localstore: close localstore in two tests
* swarm/storage/localstore: fix a possible deadlock in tests
* swarm/storage/localstore: re-enable pull subs tests for travis race
* swarm/storage/localstore: stop sending to errChan on context done in tests
* swarm/storage/localstore: better want check in readPullSubscriptionBin
* swarm/storage/localstore: protect chunk put with addr lock in tests
* swamr/storage/localstore: wait for gc and writeGCSize workers on Close
* swarm/storage/localstore: more correct testDB_collectGarbageWorker
* swarm/storage/localstore: set DB Close timeout to 5s
The current loop continuation condition is always true as a uint8
is always being checked whether it is less than 255 (its maximum
value). Since the loop starts with the value 1, the loop termination
can be guarranteed to exit once the value overflows to 0.
* swarm/storage: increase mget timeout in common_test.go
TestDbStoreCorrect_1k sometimes timed out with -race on Travis.
--- FAIL: TestDbStoreCorrect_1k (24.63s)
common_test.go:194: testStore failed: timed out after 10s
* swarm: remove unused vars from TestSnapshotSyncWithServer
nodeCount and chunkCount is returned from setupSim and those values
we use.
* swarm: move race/norace helpers from stream to testutil
As we will need to use the flag in other packages, too.
* swarm: refactor TestSwarmNetwork case
Extract long running test cases for better visibility.
* swarm/network: skip TestSyncingViaGlobalSync with -race
As panics on Travis.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7e351b]
* swarm: run TestSwarmNetwork with fewer nodes with -race
As otherwise we always get test failure with `network_test.go:374:
context deadline exceeded` even with raised `Timeout`.
* swarm/network: run TestDeliveryFromNodes with fewer nodes with -race
Test on Travis times out with 8 or more nodes if -race flag is present.
* swarm/network: smaller node count for discovery tests with -race
TestDiscoveryPersistenceSimulationSimAdapters failed on Travis with
`-race` flag present. The failure was due to extensive memory usage,
coming from the CGO runtime. Using a smaller node count resolves the
issue.
=== RUN TestDiscoveryPersistenceSimulationSimAdapter
==7227==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL github.com/ethereum/go-ethereum/swarm/network/simulations/discovery 804.826s
* swarm/network: run TestFileRetrieval with fewer nodes with -race
Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes.
=== RUN TestFileRetrieval
==7366==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL github.com/ethereum/go-ethereum/swarm/network/stream 155.165s
* swarm/network: run TestRetrieval with fewer nodes with -race
Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes ("ThreadSanitizer failed to
allocate").
* swarm/network: skip flaky TestGetSubscriptionsRPC on Travis w/ -race
Test fails a lot with something like:
streamer_test.go:1332: Real subscriptions and expected amount don't match; real: 0, expected: 20
* swarm/storage: skip TestDB_SubscribePull* tests on Travis w/ -race
Travis just hangs...
ok github.com/ethereum/go-ethereum/swarm/storage/feed/lookup 1.307s
keepalive
keepalive
keepalive
or panics after a while.
Without these tests the race detector job is now stable. Let's
invetigate these tests in a separate issue:
https://github.com/ethersphere/go-ethereum/issues/1245
* swarm/newtork: WIP Span request span until delivery and put
* swarm/storage: Introduce new trace across single fetcher lifespan
* swarm/network: Put span ids for sendpriority in context value
* swarm: Add global span store in tracing
* swarm/tracing: Add context key constants
* swarm/tracing: Add comments
* swarm/storage: Remove redundant fix for filestore
* swarm/tracing: Elaborate constants comments
* swarm/network, swarm/storage, swarm:tracing: Minor cleanup
* swarm/network/stream: fix a goroutine leak in Registry
* swarm/network, swamr/network/stream: Kademlia close addr count and depth change chans
* swarm/network/stream: rename close channel to quit
* swarm/network/stream: fix sync between NewRegistry goroutine and Close method
Simplifies the transaction presense check to use a function to
determine if the transaction is present in the block provided
to trace, which originally had a redundant parenthesis and used
a `exist` flag to dictate control flow.
I added localItemKey for this purpose in #18963, but then
forgot to actually use it. This changes the database layout
yet again and requires bumping the version number.
Package crypto works with or without cgo, which is great. However, to make it
work without cgo required setting the build tag `nocgo`. It's common to disable
cgo by instead just setting the environment variable `CGO_ENABLED=0`. Setting
this environment variable does _not_ implicitly set the build tag `nocgo`. So
projects that try to build the crypto package with `CGO_ENABLED=0` will fail. I
have done this myself several times. Until today, I had just assumed that this
meant that this package requires cgo.
But a small build tag change will make this case work. Instead of using `nocgo`
and `!nocgo`, we can use `!cgo` and `cgo`, respectively. The `cgo` build tag is
automatically set if cgo is enabled, and unset if it is disabled.
This changes default location of the data directory to use the LOCALAPPDATA
environment variable, resolving issues with remote home directories an improving
compatibility with Cygwin.
Fixes#2239Fixes#2237Fixes#16437
* swarm/network: DRY out repeated giga comment
I not necessarily agree with the way we wait for event propagation.
But I truly disagree with having duplicated giga comments.
* p2p/simulations: encapsulate Node.Up field so we avoid data races
The Node.Up field was accessed concurrently without "proper" locking.
There was a lock on Network and that was used sometimes to access
the field. Other times the locking was missed and we had
a data race.
For example: https://github.com/ethereum/go-ethereum/pull/18464
The case above was solved, but there were still intermittent/hard to
reproduce races. So let's solve the issue permanently.
resolves: ethersphere/go-ethereum#1146
* p2p/simulations: fix unmarshal of simulations.Node
Making Node.Up field private in 13292ee897
broke TestHTTPNetwork and TestHTTPSnapshot. Because the default
UnmarshalJSON does not handle unexported fields.
Important: The fix is partial and not proper to my taste. But I cut
scope as I think the fix may require a change to the current
serialization format. New ticket:
https://github.com/ethersphere/go-ethereum/issues/1177
* p2p/simulations: Add a sanity test case for Node.Config UnmarshalJSON
* p2p/simulations: revert back to defer Unlock() pattern for Network
It's a good patten to call `defer Unlock()` right after `Lock()` so
(new) error cases won't miss to unlock. Let's get back to that pattern.
The patten was abandoned in 85a79b3ad3,
while fixing a data race. That data race does not exist anymore,
since the Node.Up field got hidden behind its own lock.
* p2p/simulations: consistent naming for test providers Node.UnmarshalJSON
* p2p/simulations: remove JSON annotation from private fields of Node
As unexported fields are not serialized.
* p2p/simulations: fix deadlock in Network.GetRandomDownNode()
Problem: GetRandomDownNode() locks -> getDownNodeIDs() ->
GetNodes() tries to lock -> deadlock
On Network type, unexported functions must assume that `net.lock`
is already acquired and should not call exported functions which
might try to lock again.
* p2p/simulations: ensure method conformity for Network
Connect* methods were moved to p2p/simulations.Network from
swarm/network/simulation. However these new methods did not follow
the pattern of Network methods, i.e., all exported method locks
the whole Network either for read or write.
* p2p/simulations: fix deadlock during network shutdown
`TestDiscoveryPersistenceSimulationSimAdapter` often got into deadlock.
The execution was stuck on two locks, i.e, `Kademlia.lock` and
`p2p/simulations.Network.lock`. Usually the test got stuck once in each
20 executions with high confidence.
`Kademlia` was stuck in `Kademlia.EachAddr()` and `Network` in
`Network.Stop()`.
Solution: in `Network.Stop()` `net.lock` must be released before
calling `node.Stop()` as stopping a node (somehow - I did not find
the exact code path) causes `Network.InitConn()` to be called from
`Kademlia.SuggestPeer()` and that blocks on `net.lock`.
Related ticket: https://github.com/ethersphere/go-ethereum/issues/1223
* swarm/state: simplify if statement in DBStore.Put()
* p2p/simulations: remove faulty godoc from private function
The comment started with the wrong method name.
The method is simple and self explanatory. Also, it's private.
=> Let's just remove the comment.
* swarm/network: new saturation for implementation
* swarm/network: re-added saturation func in Kademlia as it is used elsewhere
* swarm/network: saturation with higher MinBinSize
* swarm/network: PeersPerBin with depth check
* swarm/network: edited tests to pass new saturated check
* swarm/network: minor fix saturated check
* swarm/network/simulations/discovery: fixed renamed RPC call
* swarm/network: renamed to isSaturated and returns bool
* swarm/network: early depth check
* signer/clef: make use of json-rpc notification
* signer: tidy up output of OnApprovedTx
* accounts/external, signer: implement remote signing of text, make accounts_sign take hexdata
* clef: added basic testscript
* signer, external, api: add clique signing test to debug rpc, fix clique signing in clef
* signer: fix clique interoperability between geth and clef
* clef: rename networkid switch to chainid
* clef: enable chainid flag
* clef, signer: minor changes from review
* clef: more tests for signer
* common/fdlimit: cap on MacOS file limits, fixes#18994
* common/fdlimit: fix Maximum-check to respect OPEN_MAX
* common/fdlimit: return error if OPEN_MAX is exceeded in Raise()
* common/fdlimit: goimports
* common/fdlimit: check value after setting fdlimit
* common/fdlimit: make comment a bit more descriptive
* cmd/utils: make fdlimit happy path a bit cleaner
* node: close AccountsManager in new Close method
* p2p/simulations, p2p/simulations/adapters: handle node close on shutdown
* node: move node ephemeralKeystore cleanup to stop method
* node: call Stop in Node.Close method
* cmd/geth: close node.Node created with makeFullNode in cli commands
* node: close Node instances in tests
* cmd/geth, node: minor code style fixes
* cmd, console, miner, mobile: proper node Close() termination
* Named functions and defined a basic EIP191 content type list
* Written basic content type functions
* Added ecRecover method in the clef api
* Updated the extapi changelog and addded indications in the README
* Changed the version of the external API
* Added tests for 0x45
* Implementing UnmarshalJSON() for TypedData
* Working on TypedData
* Solved the auditlog issue
* Changed method to signTypedData
* Changed mimes and implemented the 'encodeType' function for EIP-712
* Polished docstrings, ran goimports and swapped fmt.Errorf with errors.New where possible
* Drafted recursive encodeData
* Ran goimports and gofmt
* Drafted first version of EIP-712, including tests
* Temporarily switched to using common.Address in tests
* Drafted text/validator and and rewritten []byte as hexutil.Bytes
* Solved stringified address encoding issue
* Changed the property type required by signData from bytes to interface{}
* Fixed bugs in 'data/typed' signs
* Brought legal warning back after temporarily disabling it for development
* Added example RPC calls for account_signData and account_signTypedData
* Named functions and defined a basic EIP191 content type list
* Written basic content type functions
* Added ecRecover method in the clef api
* Updated the extapi changelog and addded indications in the README
* Added tests for 0x45
* Implementing UnmarshalJSON() for TypedData
* Working on TypedData
* Solved the auditlog issue
* Changed method to signTypedData
* Changed mimes and implemented the 'encodeType' function for EIP-712
* Polished docstrings, ran goimports and swapped fmt.Errorf with errors.New where possible
* Drafted recursive encodeData
* Ran goimports and gofmt
* Drafted first version of EIP-712, including tests
* Temporarily switched to using common.Address in tests
* Drafted text/validator and and rewritten []byte as hexutil.Bytes
* Solved stringified address encoding issue
* Changed the property type required by signData from bytes to interface{}
* Fixed bugs in 'data/typed' signs
* Brought legal warning back after temporarily disabling it for development
* Added example RPC calls for account_signData and account_signTypedData
* Polished and fixed PR
* Polished and fixed PR
* Solved malformed data panics and also wrote tests
* Solved malformed data panics and also wrote tests
* Added alphabetical sorting to type dependencies
* Added alphabetical sorting to type dependencies
* Added pretty print to data/typed UI
* Added pretty print to data/typed UI
* signer: more tests for typed data
* signer: more tests for typed data
* Fixed TestMalformedData4 errors and renamed IsValid to Validate
* Fixed TestMalformedData4 errors and renamed IsValid to Validate
* Fixed more new failing tests and deanonymised some functions
* Fixed more new failing tests and deanonymised some functions
* Added types to EIP712 output in cliui
* Added types to EIP712 output in cliui
* Fixed regexp issues
* Fixed regexp issues
* Added pseudo-failing test
* Added pseudo-failing test
* Fixed false positive test
* Fixed false positive test
* Added PrettyPrint method
* Added PrettyPrint method
* signer: refactor formatting and UI
* signer: make ui use new message format for signing
* Fixed breaking changes
* Fixed rules_test failing test
* Added extra regexp for reference types
* signer: more hard types
* Fixed failing test, formatted files
* signer: use golang/x keccak
* Fixed goimports error
* clef, signer: address some review concerns
* Implemented latest recommendations
* Fixed comments and uintint256 issue
* accounts, signer: fix mimetypes, add interface to sign data with passphrase
* signer, accounts: remove duplicated code, pass hash preimages to signing
* signer: prevent panic in type assertions, make cliui print rawdata as quotable-safe
* signer: linter fixes, remove deprecated crypto dependency
* accounts: fix goimport
New APIs added:
client.RegisterName(namespace, service) // makes service available to server
client.Notify(ctx, method, args...) // sends a notification
ClientFromContext(ctx) // to get a client in handler method
This is essentially a rewrite of the server-side code. JSON-RPC
processing code is now the same on both server and client side. Many
minor issues were fixed in the process and there is a new test suite for
JSON-RPC spec compliance (and non-compliance in some cases).
List of behavior changes:
- Method handlers are now called with a per-request context instead of a
per-connection context. The context is canceled right after the method
returns.
- Subscription error channels are always closed when the connection
ends. There is no need to also wait on the Notifier's Closed channel
to detect whether the subscription has ended.
- Client now omits "params" instead of sending "params": null when there
are no arguments to a call. The previous behavior was not compliant
with the spec. The server still accepts "params": null.
- Floating point numbers are allowed as "id". The spec doesn't allow
them, but we handle request "id" as json.RawMessage and guarantee that
the same number will be sent back.
- Logging is improved significantly. There is now a message at DEBUG
level for each RPC call served.
This change clears up confusion around the two ways in which nodes
can be added to the table.
When a neighbors packet is received as a reply to findnode, the nodes
contained in the reply are added as 'seen' entries if sufficient space
is available.
When a ping is received and the endpoint verification has taken place,
the remote node is added as a 'verified' entry or moved to the front of
the bucket if present. This also updates the node's IP address and port
if they have changed.
* cmd, eth: Added in the flag to step geth once sync based on input
* cmd, eth: 16400 Add an option to stop geth once in sync.
* cmd: 16400 Add an option to stop geth once in sync. WIP
* cmd/geth/main, les/fletcher: added in light mode support
* cmd/geth/main, les/fletcher: Cleaned Comments and code for light mode
* cmd: 16400 Fixed formatting issue and cleaned code
* cmd, eth, les: 16400 Fixed formatting issues
* cmd, eth, les: Performed gofmt to update formatting
* cmd, eth, les: Fixed bugs resulting formatting
* cmd/geth, eth/, les: switched to downloader event
* eth: Fixed styling and gen_config
* eth/: Fix nil error in config file
* cmd/geth: Updated countdown log
* les/fetcher.go: Removed depcreated channel
* eth/downloader.go: Removed deprecated select
* cmd/geth, cmd/utils: Fixed minor issues
* eth: Reverted config files to proper format
* eth: Fixed typo in config file
* cmd/geth, eth/down: Updated code to use header time stamp
* eth/downloader: Changed the time threshold to 10 minutes
* cmd/geth, eth/downloader: Updated downloading event to pass latest header
* cmd/geth: Updated main to use right timer object
* cmd/geth: Removed unused failed event
* cmd/geth: added in correct time field with type assertion
* cmd/geth, cmd/utils: Updated flag to use boolean
* cmd/geth, cmd/utils, eth/downloader: Cleaned up code based on recommendations
* cmd/geth: Removed unneeded import
* cmd/geth, eth/downloader: fixed event field and suggested changes
* cmd/geth, cmd/utils: Updated flag and linting issue
This change resolves multiple issues around handling of endpoint proofs.
The proof is now done separately for each IP and completing the proof
requires a matching ping hash.
Also remove waitping because it's equivalent to sleep. waitping was
slightly more efficient, but that may cause issues with findnode if
packets are reordered and the remote end sees findnode before pong.
Logging of received packets was hitherto done after handling the packet,
which meant that sent replies were logged before the packet that
generated them. This change splits up packet handling into 'preverify'
and 'handle'. The error from 'preverify' is logged, but 'handle' happens
after the message is logged. This fixes the order. Packet logs now
contain the node ID.
dput --passive should make repo pushes from Travis work again.
dput --no-upload-log works around an issue I had while uploading locally.
debuild -d says that debuild shouldn't check for build dependencies when
creating the source package. This option is needed to make builds work
in environments where the installed Go version doesn't match the
declared dependency in the source package.
This PR is replacing the metrics.influxdb.host.tag cmd-line flag with metrics.influxdb.tags - a comma-separated key/value tags, that are passed to the InfluxDB reporter, so that we can index measurements with multiple tags, and not just one host tag.
This will be useful for Swarm, where we want to index measurements not just with the host tag, but also with bzzkey and git commit version (for long-running deployments).
When some of the same messages are redefined anywhere in a Go project,
the protobuf package panics (see
https://github.com/golang/protobuf/issues/178).
Since this package is internal, there is no way to work around it, as
one cannot use it directly, but also cannot define the same messages.
There is no downside in making the package accessible.
This change unbreaks the build and removes racy access to
disableCheckFreq. Even though the field is set while holding
the lock, it was read outside of the protected section.
When opening the wallet, ask for passphrase as well as for the PIN
and return the relevant error (PIN/passphrase required). Open must then
be called again with either PIN or passphrase to advance the process.
This also updates the console bridge to support passphrase authentication.
This PR adds a new fork which disables EIP-1283. Internally it's called Petersburg,
but the genesis/config field is ConstantinopleFix.
The block numbers are:
7280000 for Constantinople on Mainnet
7280000 for ConstantinopleFix on Mainnet
4939394 for ConstantinopleFix on Ropsten
9999999 for ConstantinopleFix on Rinkeby (real number decided later)
This PR also defaults to using the same ConstantinopleFix number as whatever
Constantinople is set to. That is, it will default to mainnet behaviour if ConstantinopleFix
is not set.This means that for private networks which have already transitioned
to Constantinople, this PR will break the network unless ConstantinopleFix is
explicitly set!
* Initial work on a graphql API
* Added receipts, and more transaction fields.
* Finish receipts, add logs
* Add transactionCount to block
* Add types and .
* Update Block type to be compatible with ethql
* Rename nonce to transactionCount in Account, to be compatible with ethql
* Update transaction, receipt and log to match ethql
* Add query operator, for a range of blocks
* Added ommerCount to Block
* Add transactionAt and ommerAt to Block
* Added sendRawTransaction mutation
* Add Call and EstimateGas to graphQL API
* Refactored to use hexutil.Bytes instead of HexBytes
* Replace BigNum with hexutil.Big
* Refactor call and estimateGas to use ethapi struct type
* Replace ethgraphql.Address with common.Address
* Replace ethgraphql.Hash with common.Hash
* Converted most quantities to Long instead of Int
* Add support for logs
* Fix bug in runFilter
* Restructured Transaction to work primarily with headers, so uncle data is reported properly
* Add gasPrice API
* Add protocolVersion API
* Add syncing API
* Moved schema into its own source file
* Move some single use args types into anonymous structs
* Add doc-comments
* Fixed backend fetching to use context
* Added (very) basic tests
* Add documentation to the graphql schema
* Fix reversion for formatting of big numbers
* Correct spelling error
* s/BigInt/Long/
* Update common/types.go
* Fixes in response to review
* Fix lint error
* Updated calls on private functions
* Fix typo in graphql.go
* Rollback ethapi breaking changes for graphql support
Co-Authored-By: Arachnid <arachnid@notdot.net>
receipts may be null for very short time in some condition. For this case, we should not add the null value into cache. Because you will not get the right result if you keep requesting that receipt.
* swarm/network: fix data race in stream.(*Peer).handleOfferedHashesMsg()
handleOfferedHashesMsg() contained a data race:
- read => in a goroutine, call to c.batchDone()
- write => in the main thread, write to c.sessionAt
c.batchDone() contained a call to c.AddInterval(). Client was a value
receiver for AddInterval. So on c.AddInterval() call the whole client
struct got copied (read) while one of its field was modified in
handleOfferedHashesMsg() (write).
fixesethersphere/go-ethereum#1086
* swarm/network: simplify some trivial statements
* accounts/abi: fix name styling when unpacking abi fields w/ underscores
ABI fields with underscores that are being unpacked
into structs expect structs with following form:
int_one -> Int_one
whereas in abigen the generated structs are camelcased
int_one -> IntOne
so updated the unpack method to expect camelcased structs as well.
* swarm/network: Revised depth calculation with tests
* swarm/network: WIP remove redundant "full" function
* swarm/network: WIP peerpot refactor
* swarm/network: Make test methods submethod of peerpot and embed kad
* swarm/network: Remove commented out code
* swarm/network: Rename health test functions
* swarm/network: Too many n's
* swarm/network: Change hive Healthy func to accept addresses
* swarm/network: Add Healthy proxy method for api in hive
* swarm/network: Skip failing test out of scope for PR
* swarm/network: Skip all tests dependent on SuggestPeers
* swarm/network: Remove commented code and useless kad Pof member
* swarm/network: Remove more unused code, add counter on depth test errors
* swarm/network: WIP Create Healthy assertion tests
* swarm/network: Roll back health related methods receiver change
* swarm/network: Hardwire network minproxbinsize in swarm sim
* swarm/network: Rework Health test to strict
Pending add test for saturation
And add test for as many as possible up to saturation
* swarm/network: Skip discovery tests (dependent on SuggestPeer)
* swarm/network: Remove useless minProxBinSize in stream
* swarm/network: Remove unnecessary testing.T param to assert health
* swarm/network: Implement t.Helper() in checkHealth
* swarm/network: Rename check back to assert now that we have helper magic
* swarm/network: Revert WaitTillHealthy change (deferred to nxt PR)
* swarm/network: Kademlia tests GotNN => ConnectNN
* swarm/network: Renames and comments
* swarm/network: Add comments
* p2p/simulation: WIP minimal snapshot test
* p2p/simulation: Add snapshot create, load and verify to snapshot test
* build: add test tag for tests
* p2p/simulations, build: Revert travis change, build test sym always
* p2p/simulations: Add comments, timeout check on additional events
* p2p/simulation: Add benchmark template for minimal peer protocol init
* p2p/simulations: Remove unused code
* p2p/simulation: Correct timer reset
* p2p/simulations: Put snapshot check events in buffer and call blocking
* p2p/simulations: TestSnapshot fail if Load function returns early
* p2p/simulations: TestSnapshot wait for all connections before returning
* p2p/simulation: Revert to before wait for snap load (5e75594)
* p2p/simulations: add "conns after load" subtest to TestSnapshot
and nudge
* swarm/shed: add metrics to each shed db
* swarm/shed: push metrics prefix up
* swarm/shed: rename prefix to metricsPrefix
* swarm/shed: unexport Meter, remove Mutex for quit channel
* geth/core/eth: implement constantinople override flag
* les: implemnent constantinople override flag for les clients
* cmd/geth, eth, les: fix typo, move flag to experimentals
* Rejects peers that respond with a different hash for any of the passed in block numbers.
* Meant for emergency situations when the network forks unexpectedly.
* swarm/network: Hive - do not notify peer if discovery is disabled
* p2p/simulations: validate all connections on loading a snapshot
* p2p/simulations: track all connections in on snapshot loading
* p2p/simulations: add snapshotLoadTimeout variable
* p2p/simulations: ignore control events in snapshot load
* p2p/simulations: simplify event loop synchronization
* p2p/simulations: return already connected error from Load function
* p2p/simulations: log warning on snapshot loading disconnection
Until this commit, when sending an RPC request that called `NewEVM`, a blank `vm.Config`
would be taken so as to set some options, based on the default configuration. If some extra
configuration switches were passed to the blockchain, those would be ignored.
This PR adds a function to get the config from the blockchain, and this is what is now used
for RPC calls.
Some subsequent changes need to be made, see https://github.com/ethereum/go-ethereum/pull/17955#pullrequestreview-182237244
for the details of the discussion.
* swarm/network/stream/: added stream protocol version match tests
* Increase BZZ version due to streamer version change; version tests
* swarm/network: increased hive and test protocol version
TryUpdate does not call t.trie.TryUpdate(key, value) and calls t.trie.TryDelete
instead. The update operation simply deletes the corresponding entry, though
it could retrieve later by odr. However, it adds further network overhead.
- Have `${DataDir}/bzzd.ipc` as IPC path default.
- Respect the `--datadir` flag.
- Keep only the global `--ipcpath` flag and drop the local `--ipcpath` flag
as flags might overwrite each other. (Note: before global `--ipcpath`
was ignored even if it was set)
fixes ethersphere#795
- Replace "crypto/rand" to "math/rand" for files content generation
- Remove swarm/network_test.go.Shuffle and swarm/btm/btm_test.go.Shuffle - because go1.9 support dropped (see https://github.com/ethereum/go-ethereum/pull/17807 and comments to swarm/network_test.go.Shuffle)
On file access LDBStore's tryAccessIdx() function created a faulty
GC Index Data entry, because not indexing the ikey correctly.
That caused the chunk addresses/hashes to start with '00' and the last
two digits were dropped. => Incorrect chunk address.
Besides the fix, the commit also contains a schema change which will
run the CleanGCIndex() function to clean the GC index from erroneous
entries.
Note: CleanGCIndex() rebuilds the index from scratch which can take
a really-really long time with a huge DB (possibly an hour).
Access count was not incremented when chunk was retrieved
from cache. So the garbage collector might have deleted the most
frequently accessed chunk from disk.
Co-authored-by: Ferenc Szabo <ferenc.szabo@ethereum.org>
ethereum/go-ethereum#16734 introduced BlockHash to the FilterQuery
struct. However, ethclient was not updated to include BlockHash in the actual
RPC request.
* RequestFromPeers does not use peers marked as lightnode
* fix warning about variable name
* write tests for RequestFromPeers
* lightnodes should be omitted from the addressbook
* resolve pr comments regarding logging, formatting and comments
* resolve pr comments regarding comments and added a missing newline
* add assertions to check peers in live connections
* core: speed up GenerateChain
Use a mock implementation of ChainReader instead of creating
and destroying a BlockChain object for each generated block.
* eth/downloader: speed up tests by generating chain only once
This change reworks the downloader tests so they share a common test
blockchain instead of generating a chain in every test. The tests are
roughly twice as fast now.
* swarm: clean up unused private types and functions
Those that were identified by code inspection tool.
* swarm/storage: move/add Proximity GoDoc from deleted private function
The mentioned proximity() private function was deleted in:
1ca8fc1e6f
* cmd/clef: replace password arg with prompt (#17829)
Entering passwords on the command line is not secure as it is easy to recover from bash_history or the process table.
1. The clef command addpw was renamed to setpw to better describe the functionality
2. The <password> argument was removed and replaced with an interactive prompt
* cmd/clef: remove undeclared variable
This adds the global accumulated refund counter to the standard
json output as a numeric json value. Previously this was not very
interesting since it was not used much, but with the new sstore
gas changes the value is a lot more interesting from a consensus
investigation perspective.
* swarm/network/stream: disambiguate chunk delivery messages (retrieval vs syncing)
* swarm/network/stream: addressed PR comments
* swarm/network/stream: stream protocol version change due to new message types in this PR
* first impl of eth_getProof
* fixed docu
* added comments and refactored based on comments from holiman
* created structs
* handle errors correctly
* change Value to *hexutil.Big in order to have the same output as parity
* use ProofList as return type
This change extends the peer metrics collection:
- traces the life-cycle of the peers
- meters the peer traffic separately for every peer
- creates event feed for the peer events
- emits the peer events
* swarm/network/stream: generalize SetNextBatch and add Server SessionIndex
* swarm/network/stream: fix a typo in comment
* swarm/network/stream: remove live argument from NewSwarmSyncerServer
This PR adds enode.LocalNode and integrates it into the p2p
subsystem. This new object is the keeper of the local node
record. For now, a new version of the record is produced every
time the client restarts. We'll make it smarter to avoid that in
the future.
There are a couple of other changes in this commit: discovery now
waits for all of its goroutines at shutdown and the p2p server
now closes the node database after discovery has shut down. This
fixes a leveldb crash in tests. p2p server startup is faster
because it doesn't need to wait for the external IP query
anymore.
This fixes a rare deadlock with the inproc adapter:
- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
which waits for Network.lock.
The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.
Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.
In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.
A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
Notifier tracks whether subscription are 'active'. A subscription
becomes active when the subscription ID has been sent to the client. If
the client sends notifications in the request handler before the
subscription becomes active they are dropped. The tests tried to work
around this problem by always waiting 5s before sending the first
notification.
Fix it by buffering notifications until the subscription becomes active.
This speeds up all subscription tests.
Also fix TestSubscriptionMultipleNamespaces to wait for three messages
per subscription instead of six. The test now finishes just after all
notifications have been received and doesn't hit the 30s timeout anymore.
* travis: exclude non-test jobs for PRs
We don't usually look at these builders and not starting them
removes ~15min of build time.
* build: don't run vet before tests
Recent versions of Go run vet during 'go test' and we have
a dedicated lint job.
* build: use -timeout 5m for tests
Tests sometimes hang on Travis. CI runs are aborted after 10min with no
output. Adding the timeout means we get to see the stack trace for
timeouts.
- Mime types generator (Standard "mime" package rely on system-settings, see mime.osInitMime)
- Changed swarm/api.Upload:
- simplify I/O throttling by semaphore primitive and use file name where possible
- f.Close() must be called in Defer - otherwise panic or future added early return will cause leak of file descriptors
- one error was suppressed
When CLI tests were spanning new nodes, the log level verbosity was
hard coded as 6. So the Swarm process was always polluting the test
output with TRACE level logs.
Now `go test -v ./cmd/swarm -loglevel 0` works as expected.
* swarm/storage/mru: Adaptive Frequency
swarm/storage/mru/lookup: fixed getBaseTime
Added NewEpoch constructor
swarm/api/client: better error handling in GetResource()
swarm/storage/mru: Renamed structures.
Renamed ResourceMetadata to ResourceID.
Renamed ResourceID.Name to ResourceID.Topic
swarm/storage/mru: Added binarySerializer interface and test tools
swarm/storage/mru/lookup: Changed base time to time and + marshallers
swarm/storage/mru: Added ResourceID (former resourceMetadata)
swarm/storage/mru: Added ResourceViewId and serialization tests
swarm/storage/mru/lookup: fixed epoch unmarshaller. Added Epoch Equals
swarm/storage/mru: Fixes as per review comments
cmd/swarm: reworded resource create/update help text regarding topic
swarm/storage/mru: Added UpdateLookup and serializer tests
swarm/storage/mru: Added UpdateHeader, serializers and tests
swarm/storage/mru: changed UpdateAddr / epoch to Base()
swarm/storage/mru: Added resourceUpdate serializer and tests
swarm/storage/mru: Added SignedResourceUpdate tests and serializers
swarm/storage/mru/lookup: fixed GetFirstEpoch bug
swarm/storage/mru: refactor, comments, cleanup
Also added tests for Topic
swarm/storage/mru: handler tests pass
swarm/storage/mru: all resource package tests pass
swarm/storage/mru: resource test pass after adding
timestamp checking support
swarm/storage/mru: Added JSON serializers to ResourceIDView structures
swarm/storage/mru: Sever, client, API test pass
swarm/storage/mru: server test pass
swarm/storage/mru: Added topic length check
swarm/storage/mru: removed some literals,
improved "previous lookup" test case
swarm/storage/mru: some fixes and comments as per review
swarm/storage/mru: first working version without metadata chunk
swarm/storage/mru: Various fixes as per review
swarm/storage/mru: client test pass
swarm/storage/mru: resource query strings and manifest-less queries
swarm/storage/mru: simplify naming
swarm/storage/mru: first autofreq working version
swarm/storage/mru: renamed ToValues to AppendValues
swarm/resource/mru: Added ToValues / FromValues for URL query strings
swarm/storage/mru: Changed POST resource to work with query strings.
No more JSON.
swarm/storage/mru: removed resourceid
swarm/storage/mru: Opened up structures
swarm/storage/mru: Merged Request and SignedResourceUpdate
swarm/storage/mru: removed initial data from CLI resource create
swarm/storage/mru: Refactor Topic as a direct fixed-length array
swarm/storage/mru/lookup: Comprehensive GetNextLevel tests
swarm/storage/mru: Added comments
Added length checks in Topic
swarm/storage/mru: fixes in tests and some code comments
swarm/storage/mru/lookup: new optimized lookup algorithm
swarm/api: moved getResourceView to api out of server
swarm/storage/mru: Lookup algorithm working
swarm/storage/mru: comments and renamed NewLookupParams
Deleted commented code
swarm/storage/mru/lookup: renamed Epoch.LaterThan to After
swarm/storage/mru/lookup: Comments and tidying naming
swarm/storage/mru: fix lookup algorithm
swarm/storage/mru: exposed lookup hint
removed updateheader
swarm/storage/mru/lookup: changed GetNextEpoch for initial values
swarm/storage/mru: resource tests pass
swarm/storage/mru: valueSerializer interface and tests
swarm/storage/mru/lookup: Comments, improvements, fixes, more tests
swarm/storage/mru: renamed UpdateLookup to ID, LookupParams to Query
swarm/storage/mru: renamed query receiver var
swarm/cmd: MRU CLI tests
* cmd/swarm: remove rogue fmt
* swarm/storage/mru: Add version / header for future use
* swarm/storage/mru: Fixes/comments as per review
cmd/swarm: remove rogue fmt
swarm/storage/mru: Add version / header for future use-
* swarm/storage/mru: fix linter errors
* cmd/swarm: Speeded up TestCLIResourceUpdate
* signer: remove local path disclosure from extapi
* signer: show more data in cli ui
* rpc: make http server forward UA and Origin via Context
* signer, clef/core: ui changes + display UA and Origin
* signer: cliui - indicate less trust in remote headers, see https://github.com/ethereum/go-ethereum/issues/17637
* signer: prevent possibility swap KV-entries in aes_gcm storage, fixes#17635
* signer: remove ecrecover from external API
* signer,clef: default reject instead of warn + valideate new passwords. fixes#17632 and #17631
* signer: check calldata length even if no ABI signature is present
* signer: fix failing testcase
* clef: remove account import from external api
* signer: allow space in passwords, improve error messsage
* signer/storage: fix typos
The contributing instructions in the README are not in the GitHub contributing
guide, which means that people coming from the GitHub issues are less likely to
see them.
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.
Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.
The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.
* p2p/discover: port to p2p/enode
This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:
- Table.Lookup is not available anymore. It used to take a public key
as argument because v4 protocol requires one. Its replacement is
LookupRandom.
- Table.Resolve takes *enode.Node instead of NodeID. This is also for
v4 protocol compatibility because nodes cannot be looked up by ID
alone.
- Types Node and NodeID are gone. Further commits in the series will be
fixes all over the the codebase to deal with those removals.
* p2p: port to p2p/enode and discovery changes
This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.
New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.
* p2p/simulations, p2p/testing: port to p2p/enode
No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:
- testing.ProtocolSession tracks complete nodes, not just their IDs.
- adapters.NodeConfig has a new method to create a complete node.
These changes were needed to make swarm tests work.
Note that the NodeID change makes the code incompatible with old
simulation snapshots.
* whisper/whisperv5, whisper/whisperv6: port to p2p/enode
This port was easy because whisper uses []byte for node IDs and
URL strings in the API.
* eth: port to p2p/enode
Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.
* les: port to p2p/enode
Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.
* node: port to p2p/enode
This change simply replaces discover.Node and discover.NodeID with their
new equivalents.
* swarm/network: port to p2p/enode
Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).
There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.
Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
Interpreter initialization is left to the PRs implementing them.
Options for external interpreters are passed after a colon in the
`--vm.ewasm` and `--vm.evm` switches.
* ethdb: unified code comment style.
* puppeth: it is unnecessary to alloc pre-funded to 256 addresses
* Revert "puppeth: it is unnecessary to alloc pre-funded to 256 addresses"
This reverts commit 5e04fbccf0.
* puppeth: fix comment typo
* Revert "ethdb: unified code comment style."
This reverts commit a581efb3f0.
* swarm/network: simplify kademlia/hive; rid interfaces
* swarm, swarm/network/stream, swarm/netork/simulations,, swarm/pss: adapt to new Kad API
* swarm/network: minor changes re review; add missing lock to NeighbourhoodDepthC
* swarm/storage/encryption: async segmentwise encryption/decryption
* swarm/storage: adapt hasherstore to encryption API change
* swarm/api: adapt RefEncryption for AC to new Encryption API
* swarm/storage/encryption: address review comments
Makes Interface interface a bit more stateless and abstract.
Obviously this change is dictated by EVMC design. The EVMC tries to keep the responsibility for EVM features totally inside the VMs, if feasible. This makes VM "stateless" because VM does not need to pass any information between executions, all information is included in parameters of the execute function.
This commit does a few things at once:
- Updates the tests to contain the latest data from ethereum/tests repo.
- Enables Constantinople state tests. This is needed to be able to
fuzz-test the evm with constantinople rules.
- Fixes the error in opSAR that we've known about for some time. I was
kind of saving it to see if we hit upon it with the random test
generator, but it's difficult to both enable the tests and have the
bug there -- we don't want to forget about it, so maybe it's better
to just fix it.
* miner: commit state which is relative with sealing result
* consensus, core, miner, mobile: introduce sealHash interface
* miner: evict pending task with threshold
* miner: go fmt
* les: fix crasher in NodeInfo when running as server
The ProtocolManager computes CHT and Bloom trie roots by asking the
indexers for their current head. It tried to get the indexers from
LesOdr, but no LesOdr instance is created in server mode.
Attempt to fix this by moving the indexers, protocol creation and
NodeInfo to a new lesCommons struct which is embedded into both server
and client.
All this setup code should really be cleaned up, but this is just a
hotfix so we have to do that some other time.
* les: fix commons protocol maker
This PR enables the indexers to work in light client mode by
downloading a part of these tries (the Merkle proofs of the last
values of the last known section) in order to be able to add new
values and recalculate subsequent hashes. It also adds CHT data to
NodeInfo.
* backends: increase gaslimit in order to allow tests of large contracts
* backends: increase gaslimit in order to allow tests of large contracts
* backends: increase gaslimit in order to allow tests of large contracts
This PR implements les.freeClientPool. It also adds a simulated clock
in common/mclock, which enables time-sensitive tests to run quickly
and still produce accurate results, and package common/prque which is
a generalised variant of prque that enables removing elements other
than the top one from the queue.
les.freeClientPool implements a client database that limits the
connection time of each client and manages accepting/rejecting
incoming connections and even kicking out some connected clients. The
pool calculates recent usage time for each known client (a value that
increases linearly when the client is connected and decreases
exponentially when not connected). Clients with lower recent usage are
preferred, unknown nodes have the highest priority. Already connected
nodes receive a small bias in their favor in order to avoid accepting
and instantly kicking out clients.
Note: the pool can use any string for client identification. Using
signature keys for that purpose would not make sense when being known
has a negative value for the client. Currently the LES protocol
manager uses IP addresses (without port address) to identify clients.
* swarm: Added lightnode flag
Added --lightnode command line parameter
Added LightNode to Handshake message
* swarm/config: Fixed variable naming
* cmd/swarm: Changed BoolTFlag to BoolFlag for SwarmLightNodeEnabled
* swarm/network: Changed logging
* swarm/network: Changed protocol version testing
* swarm/network: Renamed DefaultNetworkID variable to TestProtocolNetworkID
* swarm/network: Bumped protocol version
* swarm/network: Changed LightNode handhsake test to table driven
* swarm/network: Changed back TestProtocolVersion to 5 for now
* swarm/network: Moved the test configuration inside the test function scope
* swarm/network/simulation: increase the sleep duration for TestRun
* cmd/swarm, swarm: fix failing tests on mac
* cmd/swarm: update TestCLISwarmFs skip comment
* swarm/network/simulation: adjust disconnections on simulation close
* swarm/network/simulation: call cleanups after net shutdown
* consensus/ethash: start remote ggoroutine to handle remote mining
* consensus/ethash: expose remote miner api
* consensus/ethash: expose submitHashrate api
* miner, ethash: push empty block to sealer without waiting execution
* consensus, internal: add getHashrate API for ethash
* consensus: add three method for consensus interface
* miner: expose consensus engine running status to miner
* eth, miner: specify etherbase when miner created
* miner: commit new work when consensus engine is started
* consensus, miner: fix some logics
* all: delete useless interfaces
* consensus: polish a bit
- Update benchmarks to use a pool of int pools.
Unless benchmarks are aborted with segmentation fault.
Signed-off-by: Hyung-Kyu Choi <hqueue@users.noreply.github.com>
* rpc: Make HTTP server timeout values configurable
* rpc: Remove flags for setting HTTP Timeouts, configuring via .toml is sufficient.
* rpc: Replace separate constants with a single default struct.
* rpc: Update HTTP Server Read and Write Timeouts to 30s.
* rpc: Remove redundant NewDefaultHTTPTimeouts function.
* rpc: document HTTPTimeouts.
* rpc: sanitize timeout values for library use
* build: add support for different package and binary names
* build: bump up copyright date
* build: change default PackageName to empty string
* build, internal, swarm: enhance build/release process
* build: hack ethereum-swarm as a "depends" in deb package
* build/ci: remove redundant variables
* build, cmd, mobile, params, swarm: remove VERSION file; rename Version to VersionMeta;
* internal: remove VERSION() method which reads VERSION file
* build: fix VersionFilePath to Version
* Makefile: remove clean_go_build_cache.sh until it works
* Makefile: revert removal of clean_go_build_cache.sh
- Define an Interpreter interface
- One contract can call contracts from other interpreter types.
- Pass the interpreter to the operands instead of the evm.
This is meant to prevent type assertions in operands.
Our original wrapper code had two parts. One taken from a third
party repository (who took it from upstream Go) licensed under
BSD-3. The second written by Jeff, Felix and Gustav, licensed
under LGPL. This made this package problematic to use from the
outside.
With the agreement of the original copyright holders, this commit
changes the license of the LGPL portions of the code to BSD-3:
---
I agree changing from LGPL to a BSD style license.
Jeff
---
Hey guys,
My preference would be to relicense to GNUBL, but I'm also OK with BSD.
Cheers,
Gustav
---
Felix Lange (fjl):
I would approve anything that makes our licensing less complicated
---
* swarm/storage/mru: Add embedded publickey and remove ENS dep
This commit breaks swarm, swarm/api...
but tests in swarm/storage/mru pass
* swarm: Refactor swarm, swarm/api to mru changes, make tests pass
* swarm/storage/mru: Remove self from recv, remove test ens vldtr
* swarm/storage/mru: Remove redundant test, expose ResourceHash mthd
* swarm/storage/mru: Make HeaderGetter mandatory + godoc fixes
* swarm/storage: Remove validator prefix for metadata chunk
* swarm/storage/mru: Use Address instead of PublicKey
* swarm/storage/mru: Change index from name to metadata chunk addr
* swarm/storage/mru: Refactor swarm/api/... to MRU index changes
* swarm/storage/mru: Refactor cleanup
* swarm/storage/mru: Rebase cleanup
* swarm: Use constructor for GenericSigner MRU in swarm.go
* swarm/storage: Change to BMTHash for MRU hashing
* swarm/storage: Reduce loglevel on chunk validator logs
* swarm/storage/mru: Delint
* swarm: MRU Rebase cleanup
* swarm/storage/mru: client-side mru signatures
Rebase to PR #668 and fix all conflicts
* swarm/storage/mru: refactor and documentation
* swarm/resource/mru: error-checking tests for parseUpdate/newUpdateChunk
* swarm/storage/mru: Added resourcemetadata tests
* swarm/storage/mru: Added tests for UpdateRequest
* swarm/storage/mru: more test coverage for UpdateRequest and comments
* swarm/storage/mru: Avoid fake chunks in parseUpdate()
* swarm/storage/mru: Documented resource.go extensively
moved some functions where they make most sense
* swarm/storage/mru: increase test coverage for UpdateRequest and
variable name changes throughout to increase consistency
* swarm/storage/mru: moved default timestamp to NewCreateRequest-
* swarm/storage/mru: lookup refactor
* swarm/storage/mru: added comments and renamed raw flag to rawmru
* swarm/storage/mru: fix receiver typo
* swarm/storage/mru: refactored update chunk new/create
* swarm/storage/mru: refactored signature digest to avoid malleability
* swarm/storage/mru: optimize update data serialization
* swarm/storage/mru: refactor and cleanup
* swarm/storage/mru: add timestamp struct and serialization
* swarm/storage/mru: fix lint error and mark some old code for deletion
* swarm/storage/mru: remove unnecessary variable
* swarm/storage/mru: Added more comments throughout
* swarm/storage/mru: Refactored metadata chunk layout + extensive error...
* swarm/storage/mru: refactor cli parser
Changed resource info output to JSON
* swarm/storage/mru: refactor serialization for extensibility
refactored error messages to NewErrorf
* swarm/storage/mru: Moved Signature to resource_sign.
Check Sign errors in server tests
* swarm/storage/mru: Remove isSafeName() checks
* swarm/storage/mru: scrubbed off all references to "block" for time
* swarm/storage/mru: removed superfluous isSynced() call.
* swarm/storage/mru: remove isMultihash() and ToSafeName functions
* swarm/storage/mru: various fixes and comments
* swarm/storage/mru: decoupled cli for independent create/update
* Made resource name optional
* Removed unused LookupPrevious
* swarm/storage/mru: Decoupled resource create / update & refactor
* swarm/storage/mru: Fixed some comments as per issues raised in PR #743
* swarm/storage/mru: Cosmetic changes as per #743 comments
* swarm/storage/mru: refct request encoder/decoder > marshal/unmarshal
* swarm/storage/mru: Cosmetic changes as per review in #748
* swarm/storage/mru: removed timestamp proof placeholder
* swarm/storage/mru: cosmetic/doc/fixes changes as per comments in #704
* swarm/storage/mru: removed unnecessary check in Handler.update
* swarm/storage/mru: Implemented Marshaler/Unmarshaler iface in Request
* swarm/storage/mru: Fixed linter error
* swarm/storage/mru: removed redundant address in signature digest
* swarm/storage/mru: fixed bug: LookupLatestVersionInPeriod not working
* swarm/storage/mru: Unfold Request creation API for create or update+create
set common time source for mru package
* swarm/api/http: fix HandleGetResource error variable shadowed
when requesting a resource that does not exist
* swarm/storage/mru: Add simple check to detect duplicate updates
* swarm/storage/mru: moved Multihash() to the right place.
* cmd/swarm: remove unneeded clientaccountmanager.go
* swarm/storage/mru: Changed some comments as per reviews in #784
* swarm/storage/mru: Made SignedResourceUpdate.GetDigest() public
* swarm/storage/mru: cosmetic changes as per comments in #784
* cmd/swarm: Inverted --multihash flag default
* swarm/storage/mru: removed Verify from SignedResourceUpdate.fromChunk
* swarm/storage/mru: Moved validation code out of serializer
Cosmetic / comment changes
* swarm/storage/mru: Added unit tests for UpdateLookup
* swarm/storage/mru: Increased coverage of metadata serialization
* swarm/storage/mru: Increased test coverage of updateHeader serializers
* swarm/storage/mru: Add resourceUpdate serializer test
* cmd/swarm: minor cli flag text adjustments
* cmd/swarm, swarm/storage, swarm: fix mingw on windows test issues
* cmd/swarm: support for smoke tests on the production swarm cluster
* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion
* changed colour of landing page
* landing page reacts to enter keypress
* swarm/api/http: sticky footer for swarm landing page using flex
* swarm/api/http: sticky footer for error pages and fix for multiple choices
* swarm: propagate ctx to internal apis (#754)
* swarm/simnet: add basic node/service functions
* swarm/netsim: add buckets for global state and kademlia health check
* swarm/netsim: Use sync.Map as bucket and provide cleanup function for...
* swarm, swarm/netsim: adjust SwarmNetworkTest
* swarm/netsim: fix tests
* swarm: added visualization option to sim net redesign
* swarm/netsim: support multiple services per node
* swarm/netsim: remove redundant return statement
* swarm/netsim: add comments
* swarm: shutdown HTTP in Simulation.Close
* swarm: sim HTTP server timeout
* swarm/netsim: add more simulation methods and peer events examples
* swarm/netsim: add WaitKademlia example
* swarm/netsim: fix comments
* swarm/netsim: terminate peer events goroutines on simulation done
* swarm, swarm/netsim: naming updates
* swarm/netsim: return not healthy kademlias on WaitTillHealthy
* swarm: fix WaitTillHealthy call in testSwarmNetwork
* swarm/netsim: allow bucket to have any type for a key
* swarm: Added snapshots to new netsim
* swarm/netsim: add more tests for bucket
* swarm/netsim: move http related things into separate files
* swarm/netsim: add AddNodeWithService option
* swarm/netsim: add more tests and Start* methods
* swarm/netsim: add peer events and kademlia tests
* swarm/netsim: fix some tests flakiness
* swarm/netsim: improve random nodes selection, fix TestStartStop* tests
* swarm/netsim: remove time measurement from TestClose to avoid flakiness
* swarm/netsim: builder pattern for netsim HTTP server (#773)
* swarm/netsim: add connect related tests
* swarm/netsim: add comment for TestPeerEvents
* swarm: rename netsim package to network/simulation
- AsyncHasher implements AsyncWriter interface
- add extra level for zerohashes in pool to lookup empty data hash
- remove unused segment, hash and depth fields from Tree
- Hash pkg function -> syncHash moved to test
- add asyncHash helper func to tests using shuffle
- add TestAsyncCorrectness to tests
- add BenchmarkBMTAsync to tests
- refactor benchmarks using subbenchmarks
- improved comments
- preinitialise base hashers on the nodes
* cmd/swarm: minor cli flag text adjustments
* swarm/api/http: sticky footer for swarm landing page using flex
* swarm/api/http: sticky footer for error pages and fix for multiple choices
* cmd/swarm, swarm/storage, swarm: fix mingw on windows test issues
* cmd/swarm: update description of swarm cmd
* swarm: added network ID test
* cmd/swarm: support for smoke tests on the production swarm cluster
* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion
* swarm: propagate ctx to internal apis (#754)
* swarm/metrics: collect disk measurements
* swarm/bmt: fix io.Writer interface
* Write now tolerates arbitrary variable buffers
* added variable buffer tests
* Write loop and finalise optimisation
* refactor / rename
* add tests for empty input
* swarm/pss: (UPDATE) Generic notifications package (#744)
swarm/pss: Generic package for creating pss notification svcs
* swarm: Adding context to more functions
* swarm/api: change colour of landing page in templates
* swarm/api: change landing page to react to enter keypress
* core: fix func TxDifference
fix a typo in func comment;
change named return to unnamed as there's explicit return in the body
* fix another typo in TxDifference
* p2p/discover: move bond logic from table to transport
This commit moves node endpoint verification (bonding) from the table to
the UDP transport implementation. Previously, adding a node to the table
entailed pinging the node if needed. With this change, the ping-back
logic is embedded in the packet handler at a lower level.
It is easy to verify that the basic protocol is unchanged: we still
require a valid pong reply from the node before findnode is accepted.
The node database tracked the time of last ping sent to the node and
time of last valid pong received from the node. Node endpoints are
considered verified when a valid pong is received and the time of last
pong was called 'bond time'. The time of last ping sent was unused. In
this commit, the last ping database entry is repurposed to mean last
ping _received_. This entry is now used to track whether the node needs
to be pinged back.
The other big change is how nodes are added to the table. We used to add
nodes in Table.bond, which ran when a remote node pinged us or when we
encountered the node in a neighbors reply. The transport now adds to the
table directly after the endpoint is verified through ping. To ensure
that the Table can't be filled just by pinging the node repeatedly, we
retain the isInitDone check. During init, only nodes from neighbors
replies are added.
* p2p/discover: reduce findnode failure counter on success
* p2p/discover: remove unused parameter of loadSeedNodes
* p2p/discover: improve ping-back check and comments
* p2p/discover: add neighbors reply nodes always, not just during init
These RPC calls are analogous to Parity's parity_addReservedPeer and
parity_removeReservedPeer.
They are useful for adjusting the trusted peer set during runtime,
without requiring restarting the server.
The current trie memory database/cache that we do pruning on stores
trie nodes as binary rlp encoded blobs, and also stores the node
relationships/references for GC purposes. However, most of the trie
nodes (everything apart from a value node) is in essence just a
collection of references.
This PR switches out the RLP encoded trie blobs with the
collapsed-but-not-serialized trie nodes. This permits most of the
references to be recovered from within the node data structure,
avoiding the need to track them a second time (expensive memory wise).
* debug: Use pprof goroutine writer in debug.Stacks() to ensure all goroutines are captured.
* Up to 64MB limit, previous code only captured first 1MB of goroutines.
* internal/debug: simplify stacks handler
* fix typo
* fix pointer receiver
This package was meant to hold an improved 256 bit integer library, but
the effort was abandoned in 2015. AFAIK nothing ever used this package.
Time to say goodbye.
Prior to this change, when geth was started with `geth -dev -rpc`,
it would report a network id of `1` in response to the `net_version` RPC
request. But the actual network id it used to verify transactions
was `1337`.
This change causes geth instead respond with `1337` to the `net_version`
RPC when geth is started with `geth -dev -rpc`.
* vm/test: add tests+benchmarks for mstore
* core/vm: less alloc and copying for mstore
* core/vm: less allocs in sload
* vm: check for errors more correctly
This commit adds all changes needed for the merge of swarm-network-rewrite.
The changes:
- build: increase linter timeout
- contracts/ens: export ensNode
- log: add Output method and enable fractional seconds in format
- metrics: relax test timeout
- p2p: reduced some log levels, updates to simulation packages
- rpc: increased maxClientSubscriptionBuffer to 20000
Improves test portability by resolving 127.0.0.1:0
to get a random free port instead of the hard coded one. Now
the test works if you have a running node on the same
interface already.
Fixes#15685
This PR fixes a retriever logic bug. When a peer had a soft timeout
and then a response arrived, it always assumed it was the same peer
even though it could have been a later requested one that did not time
out at all yet. In this case the logic went to an illegal state and
deadlocked, causing a goroutine leak.
Fixes#16243 and replaces #16359.
Thanks to @riceke for finding the bug in the logic.
ToECDSAPub was unsafe because it returned a non-nil key with nil X, Y in
case of invalid input. This change replaces ToECDSAPub with
UnmarshalPubkey across the codebase.
The error produced when using a Parity RPC was the following:
ERROR: transaction did not get mined: failed to get tx for txid 0xbdeb094b3278019383c8da148ff1cb5b5dbd61bf8731bc2310ac1b8ed0235226: json: cannot unmarshal non-string into Go struct field txExtraInfo.blockHash of type common.Hash
Allow the --abi flag to be given - to indicate that it should read the
ABI information from standard input. It expects to read the solc output
with the --combined-json flag providing bin, abi, userdoc, devdoc, and
metadata, and works very similarly to the internal invocation of solc,
except it allows external invocation of solc.
This facilitates integration with more complex solc invocations, such
as invocations that require path remapping or --allow-paths tweaks.
Simple usage example:
solc --combined-json bin,abi,userdoc,devdoc,metadata *.sol | abigen --abi -
* metrics: expvar support for ResettingTimer
* metrics: use integers for percentiles; remove Overall
* metrics: fix edge-case panic for index-out-of-range
This removes a golint warning: type name will be used as trie.TrieSync by
other packages, and that stutters; consider calling this Sync.
In hexToKeybytes len(hex) is even and (even+1)/2 == even/2, remove the +1.
* core: use a wrapped `map` and `sync.RWMutex` for `TxPool.all` to remove contention in `TxPool.Get`.
* core: Remove redundant `txLookup.Find` and improve comments on txLookup methods.
This applies spec changes from ethereum/EIPs#1049 and adds support for
pluggable identity schemes.
Some care has been taken to make the "v4" scheme standalone. It uses
public APIs only and could be moved out of package enr at any time.
A couple of minor changes were needed to make identity schemes work:
- The sequence number is now updated in Set instead of when signing.
- Record is now copy-safe, i.e. calling Set on a shallow copy doesn't
modify the record it was copied from.
Feed keeps active subscription channels in a slice called 'f.sendCases'.
The Send method tracks the active cases in a local variable 'cases'
whose value is f.sendCases initially. 'cases' shrinks to a shorter
prefix of f.sendCases every time a send succeeds, moving the successful
case out of range of the active case list.
This can be confusing because the two slices share a backing array. Add
more comments to document what is going on. Also add a test for removing
a case that is in 'f.sentCases' but not 'cases'.
* whisper/mailserver: pass init error to the caller
* whisper/mailserver: add returns to fmt.Errorf
* whisper/mailserver: check err in mailserver init test
This commit is meant to allow ecosystem projects such as ethersphere
to minimize CI build times by specifying an environment variable with
the packages to run tests on.
If the environment variable isn't defined the build script will test
all packages so this shouldn't affect the main go-ethereum repository.
The 'from' and 'to' methods on StateTransitions are reader methods and
shouldn't have inadvertent side effects on state.
It is safe to remove the check in 'from' because account existence is
implicitly checked by the nonce and balance checks. If the account has
non-zero balance or nonce, it must exist. Even if the sender account has
nonce zero at the start of the state transition or no balance, the nonce
is incremented before execution and the account will be created at that
time.
It is safe to remove the check in 'to' because the EVM creates the
account if necessary.
Fixes#15119
* common: delete StringToAddress, StringToHash
These functions are confusing because they don't parse hex, but use the
bytes of the string. This change removes them, replacing all uses of
StringToAddress(s) by BytesToAddress([]byte(s)).
* eth/filters: remove incorrect use of common.BytesToAddress
Most of these methods did not contain all the relevant information
inside the object and were not using a similar formatting type.
Moreover, the existence of a suboptimal String method breaks usage
with more advanced data dumping tools like go-spew.
- Fixes#16271. What was appeneded was a pointer to
an object that changes during the iteration.
- The topic is allocated as a 4-byte array, fill partial topics
with 0s. Partial topics are currently disabled, but would
crash as they rely on the presence of byte number 3.
* crypto/bn256: full switchover to cloudflare's code
* crypto/bn256: only use cloudflare for optimized architectures
* crypto/bn256: upstream fallback for non-optimized code
* .travis, build: drop support for Go 1.8 (need type aliases)
* crypto/bn256/cloudflare: enable curve mul lattice optimization
* core/vm, crypto/bn256: switch over to cloudflare library
* crypto/bn256: unmarshal constraint + start pure go impl
* crypto/bn256: combo cloudflare and google lib
* travis: drop 386 test job
whisper: refactoring go-routines workflow
Move the call mailServer.Init() down (to the bottom of the function) because if the function initialize() completes successfully, then it will be followed by mailServer.Close() in shutdown(). The workflow of the corresponding goroutines is clearer now.
* accounts/abi/bind: support for multi-dim arrays
Also:
- reduce usage of regexes a bit.
- fix minor Java syntax problems
Fixes#15648
* accounts/abi/bind: Add some more documentation
* accounts/abi/bind: Improve code readability
* accounts/abi: bugfix for unpacking nested arrays
The code previously assumed the arrays/slices were always 1 level
deep. While the packing supports nested arrays (!!!).
The current code for unpacking doesn't return the "consumed" length, so
this fix had to work around that by calculating it (i.e. packing and
getting resulting length) after the unpacking of the array element.
It's far from ideal, but unpacking behaviour is fixed now.
* accounts/abi: Fix unpacking of nested arrays
Removed the temporary workaround of packing to calculate size, which was
incorrect for slice-like types anyway.
Full size of nested arrays is used now.
* accounts/abi: deeply nested array unpack test
Test unpacking of an array nested more than one level.
* accounts/abi: Add deeply nested array pack test
Same as the deep nested array unpack test, but the other way around.
* accounts/abi/bind: deeply nested arrays bind test
Test the usage of bindings that were generated
for methods with multi-dimensional (and not
just a single extra dimension, like foo[2][3])
array arguments and returns.
edit: trigger rebuild, CI failed to fetch linter module.
* accounts/abi/bind: improve array binding
wrapArray uses a regex now, and arrayBindingJava is improved.
* accounts/abi: Improve naming of element size func
The full step size for unpacking an array
is now retrieved with "getFullElemSize".
* accounts/abi: support nested nested array args
Previously, the code only considered the outer-size of the array,
ignoring the size of the contents. This was fine for most types,
but nested arrays are packed directly into it, and count towards
the total size. This resulted in arguments following a nested
array to replicate some of the binary contents of the array.
The fix: for arrays, calculate their complete contents size:
count the arg.Type.Elem.Size when Elem is an Array, and
repeat when their child is an array too, etc.
The count is the number of 32 byte elements, similar to how it
previously counted, but nested.
* accounts/abi: Test deep nested arr multi-arguments
Arguments with a deeply nested array should not cause the next arguments
to be read from the wrong position.
The diagnostic tool was saving the unencrypted version of the messages, which is an obvious
security flaw. As of this commit:
* encrypted messages saved instead of plain text.
* all messages are stored, even that created by the user of wnode.
whisper: mailserver no longer supports the signature validation
Mailserver is provided as an example, but client validation belongs to the upper layer protocol and
needs not be covered in this example. The check that was previously available hinders the switch
to libp2p so we agreed not to include that check in that example code anymore.
- added a case error struct that contains information about certain error cases
in which we would like to output more information to the client
- added a validation method that iterates and adds the information that is
stored in the error cases
* swarm: began work on GetHandleFile method re: issue #155
* swarm: now able to serve landing page template
* swarm: added landing page template
* swarm: landing page has working input
* swarm: fixed CSS issue in template
* swarm: deleted extra lines
* swarm: deleted time header and made redirect a relative path
* swarm: removed code mistakenly left
This change removes a peer information from dialing history
when peer is removed from static list. It allows to force a
server to re-dial concrete peer if it is needed.
In our case we are running geth node on mobile devices, and
it is common for a network connection to flap on mobile.
Almost every time it flaps or network connection is changed
from cellular to wifi peers are disconnected with read
timeout. And usually it takes 30 seconds (default expiration
timeout) to recover connection with static peers after
connectivity is restored.
This change allows us to reconnect with peers almost
immediately and it seems harmless enough.
I forgot to change the check in udp.go when I changed Table.bond to be
based on lastPong instead of node presence in db. Rename lastPong to
bondTime and add hasBond so it's clearer what this DB key is used for
now.
* swarm: added script to HTML header to create favicon addresses #153
* swarm: moved data blob direclty into link tag, removed script
* swarm: added favicon info to other html templates
* swarm: fixing test errors
* swarm: fixing favicon test
* swarm: fixing travis tests
* swarm: added script to HTML header to create favicon addresses #153
* swarm: moved data blob direclty into link tag, removed script
* swarm: added favicon info to other html templates
* swarm: fixing test errors
* swarm: fixing favicon test
* swarm: fixing travis tests
This is in preparation for the switch to libp2p: the ID generated
will be from a private key created with the help of libp2p's crypto
library, while Whisper will still use Go's default crypto libraries
for encrypting its messages. This change removes a conflict.
It shouldn't have any impact as the person receiving emails is
the user, not the node.
- according to implementation of `IntrinsicGas`
we can continue execution since problem will be detected
later. However, early return is future-proof for changes.
Talk about "state" instead of "trie timing", "trie memory" and remove
the overzealous warning when the limit is just reached. Since the time
limit is always reached on slow machines, move the message to info level
so users don't freak out about internal details.
* cmd,node,rpc: add allowedHosts to prevent dns rebinding attacks
* p2p,node: Fix bug with dumpconfig introduced in r54aeb8e4c0bb9f0e7a6c67258af67df3b266af3d
* rpc: add wildcard support for rpcallowedhosts + go fmt
* cmd/geth, cmd/utils, node, rpc: ignore direct ip(v4/6) addresses in rpc virtual hostnames check
* http, rpc, utils: make vhosts into map, address review concerns
* node: change log messages to use geth standard (not sprintf)
* rpc: fix spelling
* p2p: add DialRatio for configuration of inbound vs. dialed connections
* p2p: add connection flags to PeerInfo
* p2p/netutil: add SameNet, DistinctNetSet
* p2p/discover: improve revalidation and seeding
This changes node revalidation to be periodic instead of on-demand. This
should prevent issues where dead nodes get stuck in closer buckets
because no other node will ever come along to replace them.
Every 5 seconds (on average), the last node in a random bucket is
checked and moved to the front of the bucket if it is still responding.
If revalidation fails, the last node is replaced by an entry of the
'replacement list' containing recently-seen nodes.
Most close buckets are removed because it's very unlikely we'll ever
encounter a node that would fall into any of those buckets.
Table seeding is also improved: we now require a few minutes of table
membership before considering a node as a potential seed node. This
should make it less likely to store short-lived nodes as potential
seeds.
* p2p/discover: fix nits in UDP transport
We would skip sending neighbors replies if there were fewer than
maxNeighbors results and CheckRelayIP returned an error for the last
one. While here, also resolve a TODO about pong reply tokens.
* les, light: fix CHT trie retrievals
* les, light: minor polishes, test remote CHT retrievals
* les, light: deterministic nodeset rlp, bloombits test skeleton
* les: add an event emission to the les bloombits test
* les: drop dead tester code
The bulk of the issue was to adapt to the new requirement
that a v6 filter has to either contain a symmertric key or
an asymmetric one.
This commits revert one of the fixes that I made to remove
a linter warning: unexporting NewSentMessage. This is not
really a problem as I have a cleanup in the pipe that will
solve this issue.
* Remove --fast flag and clarify default
`--fast` is no longer a flag it's `--syncmode "fast"` and that is the default
* Remove --cache flag
--cache=512 is no longer required as of 1.8 as the default has been increased
* README: Minor cache amount fix, mention Rinkeby
* whisper: fixes warnings from the code linter
* whisper: more non-API-breaking changes
The remaining lint errors are because of auto-generated
files and one is because an exported function has a non-
exported return type. Changing this would break the API,
and will be part of another commit for easier reversal.
* whisper: un-export NewSentMessage to please the linter
This is an API change, which is why it's in its own commit.
This change was initiated after the linter complained that
the returned type wasn't exported. I chose to un-export
the function instead of exporting the type, because that
type is an implementation detail that I would like to
change in the near future to make the code more
readable and with an increased coverage.
* whisper: update gencodec output after upgrading it to new lint standards
* rpc: Support specifying HTTP client in RPC dialing
Adds a minimal interface that captures http.Client and adds a new method
rpc.DialHTTPClient that takes a client using that interface. The existing
rpc.DialHTTP method is then alternatively implemented by using the new
rpc.DialHTTPClient method provided with a standard *http.Client.
* rpc: fix minor doc typos
* dashboard: footer, deep state update
* dashboard: resolve asset path
* dashboard: prevent state update on every reconnection
* dashboard: fix linter issue
* dashboard, cmd: minor UI fix, include commit hash
* dashboard: gitCommit renamed to commit
* dashboard: move the geth version to the right, make commit optional
* dashboard: memory, traffic and CPU on footer
* dashboard: fix merge
* dashboard: CPU, diskIO on footer
* dashboard: rename variables, use group declaration
* dashboard: docs
* consensus/ethash: add maxEpoch constant
* consensus/ethash: improve cache/dataset handling
There are two fixes in this commit:
Unmap the memory through a finalizer like the libethash wrapper did. The
release logic was incorrect and freed the memory while it was being
used, leading to crashes like in #14495 or #14943.
Track caches and datasets using simplelru instead of reinventing LRU
logic. This should make it easier to see whether it's correct.
* consensus/ethash: restore 'future item' logic in lru
* consensus/ethash: use mmap even in test mode
This makes it possible to shorten the time taken for TestCacheFileEvict.
* consensus/ethash: shuffle func calc*Size comments around
* consensus/ethash: ensure future cache/dataset is in the lru cache
* consensus/ethash: add issue link to the new test
* consensus/ethash: fix vet
* consensus/ethash: fix test
* consensus: tiny issue + nitpick fixes
This commit affects p2p/discv5 "topic discovery" by running it on
the same UDP port where the old discovery works. This is realized
by giving an "unhandled" packet channel to the old v4 discovery
packet handler where all invalid packets are sent. These packets
are then processed by v5. v5 packets are always invalid when
interpreted by v4 and vice versa. This is ensured by adding one
to the first byte of the packet hash in v5 packets.
DiscoveryV5Bootnodes is also changed to point to new bootnodes
that are implementing the changed packet format with modified
hash. Existing and new v5 bootnodes are both running on different
ports ATM.
This commit:
- Adds a --msgfile option to read the message to sign from a file
instead of command line argument.
- Adds a unit test for signing subcommands.
- Removes some weird whitespace in the code.
* dashboard: footer, deep state update
* dashboard: resolve asset path
* dashboard: remove bundle.js
* dashboard: prevent state update on every reconnection
* dashboard: fix linter issue
* dashboard, cmd: minor UI fix, include commit hash
* remove geth binary
* dashboard: gitCommit renamed to commit
* dashboard: move the geth version to the right, make commit optional
* dashboard: commit limited to 7 characters
* dashboard: limit commit length on client side
* dashboard: run go generate
* common/fdlimit: Move fdlimit files to separate package
When go-ethereum is used as a library the calling program need to set
the FD limit.
This commit extract fdlimit files to a separate package so it can be
used outside of go-ethereum.
* common/fdlimit: Remove FdLimit from functions signature
* common/fdlimit: Rename fdlimit functions
The first should address a long term issue where we recommend a gas
price that is greater than that required for 50% of transactions in
recent blocks, which can lead to gas price inflation as people take
this figure and add a margin to it, resulting in a positive feedback
loop.
* core/types, core/vm, eth, tests: regenerate gencodec files
* Makefile: update devtools target
Install protoc-gen-go and print reminders about npm, solc and protoc.
Also switch to github.com/kevinburke/go-bindata because it's more
maintained.
* contracts/ens: update contracts and regenerate with solidity v0.4.19
The newer upstream version of the FIFSRegistrar contract doesn't set the
resolver anymore. The resolver is now deployed separately.
* contracts/release: regenerate with solidity v0.4.19
* contracts/chequebook: fix fallback and regenerate with solidity v0.4.19
The contract didn't have a fallback function, payments would be rejected
when compiled with newer solidity. References to 'mortal' and 'owned'
use the local file system so we can compile without network access.
* p2p/discv5: regenerate with recent stringer
* cmd/faucet: regenerate
* dashboard: regenerate
* eth/tracers: regenerate
* internal/jsre/deps: regenerate
* dashboard: avoid sed -i because it's not portable
* accounts/usbwallet/internal/trezor: fix go generate warnings
* go-ethereum/dashboard: update assets.go
Use current rsc/go-bindata instead of jteeuwen/go-bindata, to get
balanced tree in very long string concatenations.
This works around problems in current Go distributions.
For golang/go#23222.
* dashboard: run last two go:generate steps for linter
+ The event slice unpacker doesn't correctly extract element from the
slice. The indexed arguments are not ignored as they should be
(the data offset should not include the indexed arguments).
+ The `Elem()` call in the slice unpack doesn't work.
The Slice related tests fails because of that.
+ the check in the loop are suboptimal and have been extracted
out of the loop.
+ extracted common code from event and method tupleUnpack
* cmd/utils: Add check on hard limit, skip test if below target
* cmd/utils: Cross platform compatible fd limit test
* cmd/utils: Remove syscall.Rlimit in test
* cmd/utils: comment fd utility method
* swarm/api: url scheme bzz-hash to get hashes of swarm content (#15238)
Update URI to support bzz-hash scheme and handle such HTTP requests by
responding with hash of the content as a text/plain response.
* swarm/api: return hash of the content for bzz-hash:// requests
* swarm/api: revert "return hash of the content for bzz-hash:// requests"
Return hashes of the content that would be returned by bzz-raw
request.
* swarm/api/http: handle error in TestBzzGetPath
* swarm/api: remove extra blank line in comment
ethkey is a new tool that serves as a command line interface to
the basic key management functionalities of geth. It currently
supports:
- generating keyfiles
- inspecting keyfiles (print public and private key)
- signing messages
- verifying signed messages
This change inlines the logic of bytesAreProper at its sole
callsite, ABI.Unpack, and applies the multiple-of-32 test only in
the case of unpacking methods. Event data is not required to be a
multiple of 32 bytes long.
Changed the communication protocol for ordinary message,
according to EIP 627. Messages will be send in bundles, i.e.
array of messages will be sent instead of single message.
* crypto: ensure that VerifySignature rejects malleable signatures
It already rejected them when using libsecp256k1, make sure the nocgo
version does the same thing.
* crypto: simplify check
* crypto: fix build
* swarm/api: url scheme bzz-list for getting list of files from manifest
Replace query parameter list=true for listing all files contained
in a swarm manifest with a new URL scheme bzz-list.
* swarm: replaace bzzr and bzzi schemes with bzz-raw and bzz-immutable
New URI Shemes are added and old ones are deprecated, but not removed.
Old Schemes bzzr and bzzi are functional for backward compatibility.
* swarm/api: completely remove bzzr and bzzi schemes
Remove old schemes in favour of bzz-raw and
bzz-immutable.
* swarm/api: revert "completely remove bzzr and bzzi schemes"
Keep bzzr and bzzi schemes for backward compatibility. At least
until 0.3 swarm release.
Specifying ENS API CLI flag, env variable or configuration
field is required for ENS resolving. Backward compatibility is
preserved with --ens-api="" CLI flag value.
Merge with changes that implement config file PR #15548.
Field *EnsApi string* in swarm/api.Config is replaced with
*EnsAPIs []string*.
A new field *EnsDisabled bool* is added to swarm/api.Config for
easy way to disable ENS resolving with config file.
Signature of function swarm.NewSwarm is changed and simplified.
This commit adds a TOML configuration option to swarm. It reuses
the TOML configuration structure used in geth with swarm
customized items.
The commit:
* Adds a "dumpconfig" command to the swarm executable which
allows printing the (default) configuration to stdout, which
then can be redirected to a file in order to customize it.
* Adds a "--config <file>" option to the swarm executable which will
allow to load a configuration file in TOML format from the
specified location in order to initialize the Swarm node The
override priorities are like follows: environment variables
override command line arguments override config file override
default config.
With this change,
key, err := crypto.HexToECDSA("000000...")
returns nil key and an error instead of a non-nil key with nil X
and Y inside. Issue found by @guidovranken.
Now that the AES salt has been moved to the payload, padding must
be adjusted to hide it, lest an attacker guesses that the packet
uses symmetric encryption.
We need those operations for p2p/enr.
Also upgrade github.com/btcsuite/btcd/btcec to the latest version
and improve BenchmarkSha3. The benchmark printed extra output
that confused tools like benchstat and ignored N.
Allow multiple --ens-api flags to be specified with value format
[tld:][contract-addr@]url.
Backward compatibility with only one --ens-api flag and --ens-addr
flag is preserved and conflict cases are handled:
- multiple --ens-api with --ens-addr returns an error
- single --ens-api with contract address and --ens-addr with
different contract address returns an error
Previously implemented --ens-endpoint is removed. Its functionality
is replaced with multiple --ens-api flags.
Generator in the current lib uses -2 as the y point when doing
ScalarBaseMult, this makes it so that points/signatures generated
from libs like py_ecc don't match/validate as pretty much all
other libs (including libsnark) have (1, 2) as the standard
generator.
This does not affect consensus as the generator is never used in
the VM, points are always explicitly defined and there is not
ScalarBaseMult op - it only makes it so that doing "import
github.com/ethereum/go-ethereum/crypto/bn256" doesn't generate
bad points in userland tools.
Updated use of Parallel and added some subtests to help isolate
them. Increased timeout in RequestHeadersByNumber so it
doesn't time out and causes other tests to break.
p2p/simulations: introduce dialBan
- Refactor simulations/network connection getters to support
avoiding simultaneous dials between two peers If two peers dial
simultaneously, the connection will be dropped to help avoid
that, we essentially lock the connection object with a
timestamp which serves as a ban on dialing for a period of time
(dialBanTimeout).
- The connection getter InitConn can be wrapped and passed to the
nodes via adapters.NodeConfig#Reachable field and then used by
the respective services when they initiate connections. This
massively stablise the emerging connectivity when running with
hundreds of nodes bootstrapping a network.
p2p: add Inbound public method to p2p.Peer
p2p/simulations: Add server id to logs to support debugging
in-memory network simulations when multiple peers are logging.
p2p: SetupConn now returns error. The dialer checks the error and
only calls resolve if the actual TCP dial fails.
* core/vm: track 63/64 call gas off stack
Gas calculations in gasCall* relayed the available gas for calls by
replacing it on the stack. This lead to inconsistent traces, which we
papered over by copying the pre-execution stack in trace mode.
This change relays available gas using a temporary variable, off the
stack, and allows removing the weird copy.
* core/vm: remove stackCopy
* core/vm: pop call gas into pool
* core/vm: to -> addr
* trie: make fullnode children hash calculation concurrently
* trie: thread out only on topmost fullnode
* trie: clean up full node children hash calculation
* trie: minor code fixups
* cmd, consensus, eth: split ethash related config to it own
* eth, consensus: minor polish
* eth, consenus, console: compress pow testing config field to single one
* consensus, eth: document pow mode
* eth, internal: Implement using trie diffs
* eth, internal: Changes in response to review
* eth: More fixes to getModifiedAccountsBy*
* eth: minor polishes on error capitalization
* .dockerignore, internal/build: Read git information directly from file
This commit changes the way of retrieving git commit and branch for build
environment from running git command to reading git files directly.
This commit also adds required git files into Docker build context.
fixes: #15346
* .dockerignore: workaround for including some files in .git
github.com/stretchr/testify is a useful library for doing
assertion in tests. It makes assertions in test more less verbose and
more comfortable to read and use.
This changes behaviour: before if the owner/amount didn't match, it
resulted in a successful execution without doing anything; now it
results in a failed execution using the revert opcode (remainder gas
is not consumed).
* core: allow price bump at threshold
* core: test changes to allow price bump at threshold
* core: reinstate tx replacement test underneath threshold
* core: minor test failure message cleanups
This PR implements the new LES protocol version extensions:
* new and more efficient Merkle proofs reply format (when replying to
a multiple Merkle proofs request, we just send a single set of trie
nodes containing all necessary nodes)
* BBT (BloomBitsTrie) works similarly to the existing CHT and contains
the bloombits search data to speed up log searches
* GetTxStatusMsg returns the inclusion position or the
pending/queued/unknown state of a transaction referenced by hash
* an optional signature of new block data (number/hash/td) can be
included in AnnounceMsg to provide an option for "very light
clients" (mobile/embedded devices) to skip expensive Ethash check
and accept multiple signatures of somewhat trusted servers (still a
lot better than trusting a single server completely and retrieving
everything through RPC). The new client mode is not implemented in
this PR, just the protocol extension.
* cmd, consensus, core, miner: instatx clique for --dev
* cmd, consensus, clique: support configurable --dev block times
* cmd, core: allow --dev to use persistent storage too
If the file already existed, the WriteResponse.Size was being set
as the length of the entire file, not just the amount that was
written to the existing file.
Fixes#15216
The accountCache contains a file cache, and remembers from
scan to scan what files were present earlier. Thus, whenever
there's a change, the scan phase only bothers processing new
and removed files.
* core/types: make Signer derive address instead of public key
There are two reasons to do this now: The upcoming ethclient signer
doesn't know the public key, just the address. EIP 208 will introduce a
new signer which derives the 'entry point' address for transactions with
zero signature. The entry point has no public key.
Other changes to the interface ease the path make to moving signature
crypto out of core/types later.
* ethclient, mobile: add TransactionSender
The new method can get the right signer without any crypto, and without
knowledge of the signature scheme that was used when the transaction was
included.
When implementing the new bloombits based filter, I've accidentally broke null
topics by removing the special casing of common.Hash{} filter rules, which
acted as the wildcard topic until now.
This PR fixes the regression, but instead of using the magic hash
common.Hash{} as the null wildcard, the PR reworks the code to handle nil
topics during parsing, converting a JSON null into nil []common.Hash topic.
This patch fixes the following issues:
* The contract executes send() when it does not have enough balance.
* The contract always sends a total amount of zero.
This commit introduces a network simulation framework which
can be used to run simulated networks of devp2p nodes. The
intention is to use this for testing protocols, performing
benchmarks and visualising emergent network behaviour.
This should only be used in very rare cases e.g. if you are not 100% sure if something is a bug or asking a question that leads to improving the documentation. For general questions please use [discord](https://discord.gg/nthXNEv) or the Ethereum stack exchange at https://ethereum.stackexchange.com.
For prerequisites and detailed build instructions please read the [Installation Instructions](https://geth.ethereum.org/docs/install-and-build/installing-geth).
Building geth requires both a Go (version 1.7 or later) and a C compiler.
You can install them using your favourite package manager.
Once the dependencies are installed, run
Building `geth` requires both a Go (version 1.13 or later) and a C compiler. You can install
them using your favourite package manager. Once the dependencies are installed, run
make geth
```shell
make geth
```
or, to build the full suite of utilities:
make all
```shell
make all
```
## Executables
The go-ethereum project comes with several wrappers/executables found in the `cmd` directory.
The go-ethereum project comes with several wrappers/executables found in the `cmd`
directory.
| Command | Description |
|:----------:|-------------|
| **`geth`** | Our main Ethereum CLI client. It is the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default) archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. `geth --help` and the [CLI Wiki page](https://github.com/ethereum/go-ethereum/wiki/Command-Line-Options) for command line options. |
| `abigen` | Source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages. It operates on plain [Ethereum contract ABIs](https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI) with expanded functionality if the contract bytecode is also available. However it also accepts Solidity source files, making development much more streamlined. Please see our [Native DApps](https://github.com/ethereum/go-ethereum/wiki/Native-DApps:-Go-bindings-to-Ethereum-contracts) wiki page for details. |
|**`geth`** | Our main Ethereum CLI client. It is the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default), archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. `geth --help` and the [CLI page](https://geth.ethereum.org/docs/interface/command-line-options) for command line options. |
|`abigen` | Source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages. It operates on plain [Ethereum contract ABIs](https://docs.soliditylang.org/en/develop/abi-spec.html) with expanded functionality if the contract bytecode is also available. However, it also accepts Solidity source files, making development much more streamlined. Please see our [Native DApps](https://geth.ethereum.org/docs/dapp/native-bindings) page for details. |
| `bootnode` | Stripped down version of our Ethereum client implementation that only takes part in the network node discovery protocol, but does not run any of the higher level application protocols. It can be used as a lightweight bootstrap node to aid in finding peers in private networks. |
| `evm` | Developer utility version of the EVM (Ethereum Virtual Machine) that is capable of running bytecode snippets within a configurable environment and execution mode. Its purpose is to allow insolated, fine-grained debugging of EVM opcodes (e.g. `evm --code 60ff60ff --debug`). |
| `gethrpctest` | Developer utility tool to support our [ethereum/rpc-test](https://github.com/ethereum/rpc-tests) test suite which validates baseline conformity to the [Ethereum JSON RPC](https://github.com/ethereum/wiki/wiki/JSON-RPC) specs. Please see the [test suite's readme](https://github.com/ethereum/rpc-tests/blob/master/README.md) for details. |
| `rlpdump` | Developer utility tool to convert binary RLP ([Recursive Length Prefix](https://github.com/ethereum/wiki/wiki/RLP)) dumps (data encoding used by the Ethereum protocol both network as well as consensus wise) to userfriendlier hierarchical representation (e.g. `rlpdump --hex CE0183FFFFFFC4C304050583616263`). |
| `swarm` | swarm daemon and tools. This is the entrypoint for the swarm network. `swarm --help` for command line options and subcommands. See https://swarm-guide.readthedocs.io for swarm documentation. |
|`evm` | Developer utility version of the EVM (Ethereum Virtual Machine) that is capable of running bytecode snippets within a configurable environment and execution mode. Its purpose is to allow isolated, fine-grained debugging of EVM opcodes (e.g. `evm --code 60ff60ff --debug run`). |
| `gethrpctest` | Developer utility tool to support our [ethereum/rpc-test](https://github.com/ethereum/rpc-tests) test suite which validates baseline conformity to the [Ethereum JSON RPC](https://eth.wiki/json-rpc/API) specs. Please see the [test suite's readme](https://github.com/ethereum/rpc-tests/blob/master/README.md) for details. |
|`rlpdump` | Developer utility tool to convert binary RLP ([Recursive Length Prefix](https://eth.wiki/en/fundamentals/rlp)) dumps (data encoding used by the Ethereum protocol both network as well as consensus wise) to user-friendlier hierarchical representation (e.g. `rlpdump --hex CE0183FFFFFFC4C304050583616263`). |
| `puppeth` | a CLI wizard that aids in creating a new Ethereum network. |
## Running geth
## Running `geth`
Going through all the possible command line flags is out of scope here (please consult our
[CLI Wiki page](https://github.com/ethereum/go-ethereum/wiki/Command-Line-Options)), but we've
enumerated a few common parameter combos to get you up to speed quickly on how you can run your
own Geth instance.
[CLI Wiki page](https://geth.ethereum.org/docs/interface/command-line-options)),
but we've enumerated a few common parameter combos to get you up to speed quickly
on how you can run your own `geth` instance.
### Full node on the main Ethereum network
By far the most common scenario is people wanting to simply interact with the Ethereum network:
create accounts; transfer funds; deploy and interact with contracts. For this particular use-case
the user doesn't care about years-old historical data, so we can fast-sync quickly to the current
state of the network. To do so:
By far the most common scenario is people wanting to simply interact with the Ethereum
network: create accounts; transfer funds; deploy and interact with contracts. For this
particular use-case the user doesn't care about years-old historical data, so we can
fast-sync quickly to the current state of the network. To do so:
```
$ geth --fast --cache=512 console
```shell
$ geth console
```
This command will:
* Start `geth` in fast sync mode (default, can be changed with the `--syncmode` flag),
causing it to download more data in exchange for avoiding processing the entire history
of the Ethereum network, which is very CPU intensive.
* Start up `geth`'s built-in interactive [JavaScript console](https://geth.ethereum.org/docs/interface/javascript-console),
(via the trailing `console` subcommand) through which you can invoke all official [`web3` methods](https://web3js.readthedocs.io/en/)
as well as `geth`'s own [management APIs](https://geth.ethereum.org/docs/rpc/server).
This tool is optional and if you leave it out you can always attach to an already running
`geth` instance with `geth attach`.
* Start geth in fast sync mode (`--fast`), causing it to download more data in exchange for avoiding
processing the entire history of the Ethereum network, which is very CPU intensive.
* Bump the memory allowance of the database to 512MB (`--cache=512`), which can help significantly in
sync times especially for HDD users. This flag is optional and you can set it as high or as low as
you'd like, though we'd recommend the 512MB - 2GB range.
* Start up Geth's built-in interactive [JavaScript console](https://github.com/ethereum/go-ethereum/wiki/JavaScript-Console),
(via the trailing `console` subcommand) through which you can invoke all official [`web3` methods](https://github.com/ethereum/wiki/wiki/JavaScript-API)
as well as Geth's own [management APIs](https://github.com/ethereum/go-ethereum/wiki/Management-APIs).
This too is optional and if you leave it out you can always attach to an already running Geth instance
with `geth attach`.
### A Full node on the Görli test network
### Full node on the Ethereum test network
Transitioning towards developers, if you'd like to play around with creating Ethereum
contracts, you almost certainly would like to do that without any real money involved until
you get the hang of the entire system. In other words, instead of attaching to the main
network, you want to join the **test** network with your node, which is fully equivalent to
the main network, but with play-Ether only.
Transitioning towards developers, if you'd like to play around with creating Ethereum contracts, you
almost certainly would like to do that without any real money involved until you get the hang of the
entire system. In other words, instead of attaching to the main network, you want to join the **test**
network with your node, which is fully equivalent to the main network, but with play-Ether only.
```
$ geth --testnet --fast --cache=512 console
```shell
$ geth --goerli console
```
The `--fast`, `--cache` flags and `console` subcommand have the exact same meaning as above and they
are equally useful on the testnet too. Please see above for their explanations if you've skipped to
here.
The `console` subcommand has the exact same meaning as above and they are equally
useful on the testnet too. Please, see above for their explanations if you've skipped here.
Specifying the `--testnet` flag however will reconfigure your Geth instance a bit:
Specifying the `--goerli` flag, however, will reconfigure your `geth` instance a bit:
* Instead of using the default data directory (`~/.ethereum` on Linux for example), Geth will nest
itself one level deeper into a `testnet` subfolder (`~/.ethereum/testnet` on Linux). Note, on OSX
and Linux this also means that attaching to a running testnet node requires the use of a custom
endpoint since `geth attach` will try to attach to a production node endpoint by default. E.g.
`geth attach <datadir>/testnet/geth.ipc`. Windows users are not affected by this.
* Instead of connecting the main Ethereum network, the client will connect to the testnetwork,
which uses different P2P bootnodes, different network IDs and genesis states.
* Instead of connecting the main Ethereum network, the client will connect to the Görli
test network, which uses different P2P bootnodes, different network IDs and genesis
states.
* Instead of using the default data directory (`~/.ethereum` on Linux for example), `geth`
will nest itself one level deeper into a `goerli` subfolder (`~/.ethereum/goerli` on
Linux). Note, on OSX and Linux this also means that attaching to a running testnet node
requires the use of a custom endpoint since `geth attach` will try to attach to a
production node endpoint by default, e.g.,
`geth attach <datadir>/goerli/geth.ipc`. Windows users are not affected by
this.
*Note: Although there are some internal protective measures to prevent transactions from crossing
over between the main network and test network, you should make sure to always use separate accounts
for play-money and real-money. Unless you manually move accounts, Geth will by default correctly
separate the two networks and will not make any accounts available between them.*
*Note: Although there are some internal protective measures to prevent transactions from
crossing over between the main network and test network, you should make sure to always
use separate accounts for play-money and real-money. Unless you manually move
accounts, `geth` will by default correctly separate the two networks and will not make any
accounts available between them.*
### Full node on the Rinkeby test network
Go Ethereum also supports connecting to the older proof-of-authority based test network
called [*Rinkeby*](https://www.rinkeby.io) which is operated by members of the community.
```shell
$ geth --rinkeby console
```
### Full node on the Ropsten test network
In addition to Görli and Rinkeby, Geth also supports the ancient Ropsten testnet. The
Ropsten test network is based on the Ethash proof-of-work consensus algorithm. As such,
it has certain extra overhead and is more susceptible to reorganization attacks due to the
network's low difficulty/security.
```shell
$ geth --ropsten console
```
*Note: Older Geth configurations store the Ropsten database in the `testnet` subdirectory.*
### Configuration
As an alternative to passing the numerous flags to the `geth` binary, you can also pass a configuration file via:
As an alternative to passing the numerous flags to the `geth` binary, you can also pass a
configuration file via:
```
```shell
$ geth --config /path/to/your_config.toml
```
To get an idea how the file should look like you can use the `dumpconfig` subcommand to export your existing configuration:
To get an idea how the file should look like you can use the `dumpconfig` subcommand to
export your existing configuration:
```
```shell
$ geth --your-favourite-flags dumpconfig
```
*Note: This works only with geth v1.6.0 and above.*
*Note: This works only with `geth` v1.6.0 and above.*
#### Docker quick start
One of the quickest ways to get Ethereum up and running on your machine is by using Docker:
One of the quickest ways to get Ethereum up and running on your machine is by using
Docker:
```
```shell
docker run -d --name ethereum-node -v /Users/alice/ethereum:/root \
-p 8545:8545 -p 30303:30303 \
ethereum/client-go --fast --cache=512
ethereum/client-go
```
This will start geth in fastsync mode with a DB memory allowance of 512MB just as the above command does. It will also create a persistent volume in your home directory for saving your blockchain as well as map the default ports. There is also an `alpine` tag available for a slim version of the image.
This will start `geth` in fast-sync mode with a DB memory allowance of 1GB just as the
above command does. It will also create a persistent volume in your home directory for
saving your blockchain as well as map the default ports. There is also an `alpine` tag
available for a slim version of the image.
### Programatically interfacing Geth nodes
Do not forget `--http.addr 0.0.0.0`, if you want to access RPC from other containers
and/or hosts. By default, `geth` binds to the local interface and RPC endpoints is not
accessible from the outside.
As a developer, sooner rather than later you'll want to start interacting with Geth and the Ethereum
network via your own programs and not manually through the console. To aid this, Geth has built in
support for a JSON-RPC based APIs ([standard APIs](https://github.com/ethereum/wiki/wiki/JSON-RPC) and
[Geth specific APIs](https://github.com/ethereum/go-ethereum/wiki/Management-APIs)). These can be
exposed via HTTP, WebSockets and IPC (unix sockets on unix based platforms, and named pipes on Windows).
### Programmatically interfacing `geth` nodes
The IPC interface is enabled by default and exposes all the APIs supported by Geth, whereas the HTTP
and WS interfaces need to manually be enabled and only expose a subset of APIs due to security reasons.
These can be turned on/off and configured as you'd expect.
As a developer, sooner rather than later you'll want to start interacting with `geth` and the
Ethereum network via your own programs and not manually through the console. To aid
this, `geth` has built-in support for a JSON-RPC based APIs ([standard APIs](https://eth.wiki/json-rpc/API)
and [`geth` specific APIs](https://geth.ethereum.org/docs/rpc/server)).
These can be exposed via HTTP, WebSockets and IPC (UNIX sockets on UNIX based
platforms, and named pipes on Windows).
The IPC interface is enabled by default and exposes all the APIs supported by `geth`,
whereas the HTTP and WS interfaces need to manually be enabled and only expose a
subset of APIs due to security reasons. These can be turned on/off and configured as
you'd expect.
HTTP based JSON-RPC API options:
*`--rpc` Enable the HTTP-RPC server
*`--rpcaddr` HTTP-RPC server listening interface (default: "localhost")
*`--rpcport` HTTP-RPC server listening port (default: 8545)
*`--rpcapi` API's offered over the HTTP-RPC interface (default: "eth,net,web3")
*`--rpccorsdomain` Comma separated list of domains from which to accept cross origin requests (browser enforced)
*`--http` Enable the HTTP-RPC server
*`--http.addr` HTTP-RPC server listening interface (default: `localhost`)
*`--http.port` HTTP-RPC server listening port (default: `8545`)
*`--http.api` API's offered over the HTTP-RPC interface (default: `eth,net,web3`)
*`--http.corsdomain` Comma separated list of domains from which to accept cross origin requests (browser enforced)
*`--ws` Enable the WS-RPC server
*`--wsaddr` WS-RPC server listening interface (default: "localhost")
*`--wsport` WS-RPC server listening port (default: 8546)
*`--wsapi` API's offered over the WS-RPC interface (default: "eth,net,web3")
*`--wsorigins` Origins from which to accept websockets requests
*`--ws.addr` WS-RPC server listening interface (default: `localhost`)
*`--ws.port` WS-RPC server listening port (default: `8546`)
*`--ws.api` API's offered over the WS-RPC interface (default: `eth,net,web3`)
*`--ws.origins` Origins from which to accept websockets requests
*`--ipcdisable` Disable the IPC-RPC server
*`--ipcapi` API's offered over the IPC-RPC interface (default: "admin,debug,eth,miner,net,personal,shh,txpool,web3")
*`--ipcapi` API's offered over the IPC-RPC interface (default: `admin,debug,eth,miner,net,personal,shh,txpool,web3`)
*`--ipcpath` Filename for IPC socket/pipe within the datadir (explicit paths escape it)
You'll need to use your own programming environments' capabilities (libraries, tools, etc) to connect
via HTTP, WS or IPC to a Geth node configured with the above flags and you'll need to speak [JSON-RPC](http://www.jsonrpc.org/specification)
on all transports. You can reuse the same connection for multiple requests!
You'll need to use your own programming environments' capabilities (libraries, tools, etc) to
connect via HTTP, WS or IPC to a `geth` node configured with the above flags and you'll
need to speak [JSON-RPC](https://www.jsonrpc.org/specification) on all transports. You
can reuse the same connection for multiple requests!
**Note: Please understand the security implications of opening up an HTTP/WS based transport before
doing so! Hackers on the internet are actively trying to subvert Ethereum nodes with exposed APIs!
Further, all browser tabs can access locally running webservers, so malicious webpages could try to
subvert locally available APIs!**
**Note: Please understand the security implications of opening up an HTTP/WS based
transport before doing so! Hackers on the internet are actively trying to subvert
Ethereum nodes with exposed APIs! Further, all browser tabs can access locally
running web servers, so malicious web pages could try to subvert locally available
APIs!**
### Operating a private network
Maintaining your own private network is more involved as a lot of configurations taken for granted in
the official networks need to be manually set up.
Maintaining your own private network is more involved as a lot of configurations taken for
granted in the official networks need to be manually set up.
#### Defining the private genesis state
First, you'll need to create the genesis state of your networks, which all nodes need to be aware of
and agree upon. This consists of a small JSON file (e.g. call it `genesis.json`):
First, you'll need to create the genesis state of your networks, which all nodes need to be
aware of and agree upon. This consists of a small JSON file (e.g. call it `genesis.json`):
Which will start mining bocks and transactions on a single CPU thread, crediting all proceedings to
the account specified by `--etherbase`. You can further tune the mining by changing the default gas
limit blocks converge to (`--targetgaslimit`) and the price transactions are accepted at (`--gasprice`).
Which will start mining blocks and transactions on a single CPU thread, crediting all
proceedings to the account specified by `--miner.etherbase`. You can further tune the mining
by changing the default gas limit blocks converge to (`--miner.targetgaslimit`) and the price
transactions are accepted at (`--miner.gasprice`).
## Contribution
Thank you for considering to help out with the source code! We welcome contributions from
anyone on the internet, and are grateful for even the smallest of fixes!
Thank you for considering to help out with the source code! We welcome contributions
from anyone on the internet, and are grateful for even the smallest of fixes!
If you'd like to contribute to go-ethereum, please fork, fix, commit and send a pull request
for the maintainers to review and merge into the main code base. If you wish to submit more
complex changes though, please check up with the core devs first on [our gitter channel](https://gitter.im/ethereum/go-ethereum)
to ensure those changes are in line with the general philosophy of the project and/or get some
early feedback which can make both your efforts much lighter as well as our review and merge
procedures quick and simple.
for the maintainers to review and merge into the main code base. If you wish to submit
more complex changes though, please check up with the core devs first on [our gitter channel](https://gitter.im/ethereum/go-ethereum)
to ensure those changes are in line with the general philosophy of the project and/or get
some early feedback which can make both your efforts much lighter as well as our review
and merge procedures quick and simple.
Please make sure your contributions adhere to our coding guidelines:
* Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)).
* Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines.
* Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting)
Please see [Releases](https://github.com/ethereum/go-ethereum/releases). We recommend using the [most recently released version](https://github.com/ethereum/go-ethereum/releases/latest).
## Audit reports
Audit reports are published in the `docs` folder: https://github.com/ethereum/go-ethereum/tree/master/docs/audits
**Please do not file a public ticket** mentioning the vulnerability.
To find out how to disclose a vulnerability in Ethereum visit [https://bounty.ethereum.org](https://bounty.ethereum.org) or email bounty@ethereum.org. Please read the [disclosure page](https://github.com/ethereum/go-ethereum/security/advisories?state=published) for more information about publically disclosed security vulnerabilities.
Use the built-in `geth version-check` feature to check whether the software is affected by any known vulnerability. This command will fetch the latest [`vulnerabilities.json`](https://geth.ethereum.org/docs/vulnerabilities/vulnerabilities.json) file which contains known security vulnerabilities concerning `geth`, and cross-check the data against its own version number.
The following key may be used to communicate sensitive information to developers.
// {{$contract.Type}}{{.Normalized.Name}}Iterator is returned from Filter{{.Normalized.Name}} and is used to iterate over the raw logs and unpacked data for {{.Normalized.Name}} events raised by the {{$contract.Type}} contract.
type {{$contract.Type}}{{.Normalized.Name}}Iterator struct {
Event *{{$contract.Type}}{{.Normalized.Name}} // Event containing the contract specifics and raw log
contract *bind.BoundContract // Generic contract to use for unpacking event data
event string // Event name to use for unpacking event data
logs chan types.Log // Log channel receiving the found contract events
sub ethereum.Subscription // Subscription for errors, completion and termination
done bool // Whether the subscription completed delivering logs
fail error // Occurred error to stop iteration
}
// Next advances the iterator to the subsequent event, returning whether there
// are any more events found. In case of a retrieval or parsing error, false is
// returned and Error() can be queried for the exact failure.
* PCSCD version 4.3 running on your system **Only version 4.3 is currently supported**
## Preparing the smartcard
**WARNING: FOILLOWING THESE INSTRUCTIONS WILL DESTROY THE MASTER KEY ON YOUR CARD. ONLY PROCEED IF NO FUNDS ARE ASSOCIATED WITH THESE ACCOUNTS**
You can use status' [keycard-cli](https://github.com/status-im/keycard-cli) and you should get _at least_ version 2.1.1 of their [smartcard application](https://github.com/status-im/status-keycard/releases/download/2.2.1/keycard_v2.2.1.cap)
You also need to make sure that the PCSC daemon is running on your system.
Then, you can install the application to the card by typing:
```
keycard install -a keycard_v2.2.1.cap && keycard init
```
At the end of this process, you will be provided with a PIN, a PUK and a pairing password. Write them down, you'll need them shortly.
Start `geth` with the `console` command. You will notice the following warning:
```
WARN [04-09|16:58:38.898] Failed to open wallet url=keycard://044def09 err="smartcard: pairing password needed"
```
Write down the URL (`keycard://044def09` in this example). Then ask `geth` to open the wallet:
//lint:ignore ST1005 brand name displayed on the console
returncommon.Address{},nil,fmt.Errorf("Ledger v%d.%d.%d doesn't support signing this transaction, please update to v1.0.3 at least",w.version[0],w.version[1],w.version[2])
}
// All infos gathered and metadata checks out, request signing
@ -259,7 +259,9 @@ func (w *ledgerDriver) ledgerDerive(derivationPath []uint32) (common.Address, er
// Decode the hex sting into an Ethereum address and return
optionalbytesvalue=6;// <=256 bit unsigned big endian (in wei)
optionalbytesdata_initial_chunk=7;// The initial data chunk (<= 1024 bytes)
optionaluint32data_length=8;// Length of transaction payload
optionaluint32chain_id=9;// Chain Id for EIP 155
optionaluint32tx_type=10;// (only for Wanchain)
}
/**
* Response: Device asks for more data from transaction payload, or returns the signature.
* If data_length is set, device awaits that many more bytes of payload.
* Otherwise, the signature_* fields contain the computed transaction signature. All three fields will be present.
* @end
* @next EthereumTxAck
*/
messageEthereumTxRequest{
optionaluint32data_length=1;// Number of bytes being requested (<= 1024)
optionaluint32signature_v=2;// Computed signature (recovery parameter, limited to 27 or 28)
optionalbytessignature_r=3;// Computed signature R component (256 bit)
optionalbytessignature_s=4;// Computed signature S component (256 bit)
}
/**
* Request: Transaction payload data.
* @next EthereumTxRequest
*/
messageEthereumTxAck{
optionalbytesdata_chunk=1;// Bytes from transaction payload (<= 1024 bytes)
}
/**
* Request: Ask device to sign message
* @start
* @next EthereumMessageSignature
* @next Failure
*/
messageEthereumSignMessage{
repeateduint32address_n=1;// BIP-32 path to derive the key from master node
optionalbytesmessage=2;// message to be signed
}
/**
* Response: Signed message
* @end
*/
messageEthereumMessageSignature{
optionalbytesaddressBin=1;// address used to sign the message (20 bytes, legacy firmware)
optionalbytessignature=2;// signature of the message
optionalstringaddressHex=3;// address used to sign the message (hex string, newer firmware)
}
/**
* Request: Ask device to verify message
* @start
* @next Success
* @next Failure
*/
messageEthereumVerifyMessage{
optionalbytesaddressBin=1;// address to verify (20 bytes, legacy firmware)
optionalbytessignature=2;// signature to verify
optionalbytesmessage=3;// message to verify
optionalstringaddressHex=4;// address to verify (hex string, newer firmware)
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
