gaspersic/go-ethereum
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

docs: major overhaul (#19395)

Browse Source
This commit is contained in:
Martin Holst Swende
2019-04-05 00:21:15 +02:00
committed by GitHub
parent 6f029479b2
commit 1122480aa3
39 changed files with 103 additions and 218 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
docs/_developers/Code-Review-Guidelines.md Normal file
View File

@ -0,0 +1,101 @@
---
title: Code Review Guidelines
---
The only way to get code into go-ethereum is to send a pull request. Those pull requests
need to be reviewed by someone. This document is a guide that explains our expectations
around PRs for both authors and reviewers.
## Terminology
* The **author** of a pull request is the entity who wrote the diff and submitted it to
GitHub.
* The **team** consists of people with commit rights on the go-ethereum repository.
* The **reviewer** is the person assigned to review the diff. The reviewer must be a team
member.
* The **code owner** is the person responsible for the subsystem being modified by the PR.
## The Process
The first decision to make for any PR is whether it's worth including at all. This
decision lies primarily with the code owner, but may be negotiated with team members.
To make the decision we must understand what the PR is about. If there isn't enough
description content or the diff is too large, request an explanation. Anyone can do this
part.
We expect that reviewers check the style and functionality of the PR, providing comments
to the author using the GitHub review system. Reviewers should follow up with the PR until
it is in good shape, then **approve** the PR. Approved PRs can be merged by any code owner.
When communicating with authors, be polite and respectful.
### Code Style
We expect `gofmt`ed code. For contributions of significant size, we expect authors to
understand and use the guidelines in [Effective Go][effgo]. Authors should avoid common
mistakes explained in the [Go Code Review Comments][revcomment] page.
### Functional Checks
For PRs that fix an issue, reviewers should try reproduce the issue and verify that the
pull request actually fixes it. Authors can help with this by including a unit test that
fails without (and passes with) the change.
For PRs adding new features, reviewers should attempt to use the feature and comment on
how it feels to use it. Example: if a PR adds a new command line flag, use the program
with the flag and comment on whether the flag feels useful.
We expect appropriate unit test coverage. Reviewers should verify that new code is covered
by unit tests.
### CI
Code submitted must pass all unit tests and static analysis ("lint") checks. We use Travis
CI to test code on Linux, macOS and AppVeyor to test code on Microsoft Windows.
For failing CI builds, the issue may not be related to the PR itself. Such failures are
usually related to flakey tests. These failures can be ignored (authors don't need to fix
unrelated issues), but please file a GH issue so the test gets fixed eventually.
### Commit Messages
Commit messages on the master branch should follow the rule below. PR authors are not
required to use any particular style because the message can be modified at merge time.
Enforcing commit message style is the responsibility of the person merging the PR.
The commit message style we use is similar to the style used by the Go project:
The first line of the change description is conventionally a one-line summary of the
change, prefixed by the primary affected Go package. It should complete the sentence "This
change modifies go-ethereum to _____." The rest of the description elaborates and should
provide context for the change and explain what it does.
Template:
```text
package/path: change XYZ
Longer explanation of the change in the commit. You can use
multiple sentences here. It's usually best to include content
from the PR description in the final commit message.
issue notices, e.g. "Fixes #42353".
```
### Special Situations And How To Deal With Them
As a reviewer, you may find yourself in one of the sitations below. Here's how to deal
with those:
* The author doesn't follow up: ping them after a while (i.e. after a few days). If there
is no further response, close the PR or complete the work yourself.
* Author insists on including refactoring changes alongside bug fix: We can tolerate small
refactorings alongside any change. If you feel lost in the diff, ask the author to
submit the refactoring as an independent PR, or at least as an independent commit in the
same PR.
* Author keeps rejecting your feedback: reviewers have authority to reject any change for technical reasons. If you're unsure, ask the team for a second opinion. You may close the PR if no consensus can be reached.
[effgo]: https://golang.org/doc/effective_go.html
[revcomment]: https://github.com/golang/go/wiki/CodeReviewComments

167
docs/_developers/Cross-compiling-Ethereum.md Normal file
View File

@ -0,0 +1,167 @@
---
title: Cross-compiling Ethereum
---
**Note: All of these and much more have been merged into the project Makefile.
You can cross build via `make geth-<os>-<platform>` without needing to know any
of these details from below.**
Developers usually have a preferred platform that they feel most comfortable
working in, with all the necessary tools, libraries and environments set up for
an optimal workflow. However, there's often need to build for either a different
CPU architecture, or an entirely different operating system; but maintaining a
development environment for each and switching between the them quickly becomes
unwieldy.
Here we present a very simple way to cross compile Ethereum to various operating
systems and architectures using a minimal set of prerequisites and a completely
containerized approach, guaranteeing that your development environment remains
clean even after the complex requirements and mechanisms of a cross compilation.
The currently supported target platforms are:
- ARMv7 Android and iOS
- 32 bit, 64 bit and ARMv5 Linux
- 32 bit and 64 bit Mac OSX
- 32 bit and 64 bit Windows
Please note, that cross compilation does not replace a release build. Although
resulting binaries can usually run perfectly on the desired platform, compiling
on a native system with the specialized tools provided by the official vendor
can often result in more a finely optimized code.
## Cross compilation environment
Although the `go-ethereum` project is written in Go, it does include a bit of C
code shared between all implementations to ensure that all perform equally well,
including a dependency to the GNU Multiple Precision Arithmetic Library. Because
of these, Go cannot by itself compile to a different platform than the host. To
overcome this limitation, we will use [`xgo`](https://github.com/karalabe/xgo),
a Go cross compiler package based on Docker containers that has been architected
specifically to allow both embedded C snippets as well as simpler external C
dependencies during compilation.
The `xgo` project has two simple dependencies: Docker (to ensure that the build
environment is completely contained) and Go. On most platforms these should be
available from the official package repositories. For manually installing them,
please consult their install guides at [Docker](https://docs.docker.com/installation/)
and [Go](https://golang.org/doc/install) respectively. This guide assumes that these
two dependencies are met.
To install and/or update xgo, simply type:
$ go get -u github.com/karalabe/xgo
You can test whether `xgo` is functioning correctly by requesting it to cross
compile itself and verifying that all cross compilations succeeded or not.
$ xgo github.com/karalabe/xgo
...
$ ls -al
-rwxr-xr-x 1 root root 2792436 Sep 14 16:45 xgo-android-21-arm
-rwxr-xr-x 1 root root 2353212 Sep 14 16:45 xgo-darwin-386
-rwxr-xr-x 1 root root 2906128 Sep 14 16:45 xgo-darwin-amd64
-rwxr-xr-x 1 root root 2388288 Sep 14 16:45 xgo-linux-386
-rwxr-xr-x 1 root root 2960560 Sep 14 16:45 xgo-linux-amd64
-rwxr-xr-x 1 root root 2437864 Sep 14 16:45 xgo-linux-arm
-rwxr-xr-x 1 root root 2551808 Sep 14 16:45 xgo-windows-386.exe
-rwxr-xr-x 1 root root 3130368 Sep 14 16:45 xgo-windows-amd64.exe
## Building Ethereum
Cross compiling Ethereum is analogous to the above example, but an additional
flags is required to satisfy the dependencies:
- `--deps` is used to inject arbitrary C dependency packages and pre-build them
Injecting the GNU Arithmetic Library dependency and selecting `geth` would be:
$ xgo --deps=https://gmplib.org/download/gmp/gmp-6.0.0a.tar.bz2 \
github.com/ethereum/go-ethereum/cmd/geth
...
$ ls -al
-rwxr-xr-x 1 root root 23213372 Sep 14 17:59 geth-android-21-arm
-rwxr-xr-x 1 root root 14373980 Sep 14 17:59 geth-darwin-386
-rwxr-xr-x 1 root root 17373676 Sep 14 17:59 geth-darwin-amd64
-rwxr-xr-x 1 root root 21098910 Sep 14 17:59 geth-linux-386
-rwxr-xr-x 1 root root 25049693 Sep 14 17:59 geth-linux-amd64
-rwxr-xr-x 1 root root 20578535 Sep 14 17:59 geth-linux-arm
-rwxr-xr-x 1 root root 16351260 Sep 14 17:59 geth-windows-386.exe
-rwxr-xr-x 1 root root 19418071 Sep 14 17:59 geth-windows-amd64.exe
As the cross compiler needs to build all the dependencies as well as the main
project itself for each platform, it may take a while for the build to complete
(approximately 3-4 minutes on a Core i7 3770K machine).
### Fine tuning the build
By default Go, and inherently `xgo`, checks out and tries to build the master
branch of a source repository. However, more often than not, you'll probably
want to build a different branch from possibly an entirely different remote
repository. These can be controlled via the `--remote` and `--branch` flags.
To build the `develop` branch of the official `go-ethereum` repository instead
of the default `master` branch, you just need to specify it as an additional
command line flag (`--branch`):
$ xgo --deps=https://gmplib.org/download/gmp/gmp-6.0.0a.tar.bz2 \
--branch=develop \
github.com/ethereum/go-ethereum/cmd/geth
Additionally, during development you will most probably want to not only build
a custom branch, but also one originating from your own fork of the repository
instead of the upstream one. This can be done via the `--remote` flag:
$ xgo --deps=https://gmplib.org/download/gmp/gmp-6.0.0a.tar.bz2 \
--remote=https://github.com/karalabe/go-ethereum \
--branch=rpi-staging \
github.com/ethereum/go-ethereum/cmd/geth
By default `xgo` builds binaries for all supported platforms and architectures,
with Android binaries defaulting to the highest released Android NDK platform.
To limit the build targets or compile to a different Android platform, use the
`--targets` CLI parameter.
$ xgo --deps=https://gmplib.org/download/gmp/gmp-6.0.0a.tar.bz2 \
--targets=android-16/arm,windows/* \
github.com/ethereum/go-ethereum/cmd/geth
### Building locally
If you would like to cross compile your local development version, simply specify
a local path (starting with `.` or `/`), and `xgo` will use all local code from
`GOPATH`, only downloading missing dependencies. In such a case of course, the
`--branch`, `--remote` and `--pkg` arguments are no-op:
$ xgo --deps=https://gmplib.org/download/gmp/gmp-6.0.0a.tar.bz2 \
./cmd/geth
## Using the Makefile
Having understood the gist of `xgo` based cross compilation, you do not need to
actually memorize and maintain these commands, as they have been incorporated into
the official [Makefile](https://github.com/ethereum/go-ethereum/blob/master/Makefile)
and can be invoked with a trivial `make` request:
* `make geth-cross`: Cross compiles to every supported OS and architecture
* `make geth-<os>`: Cross compiles supported architectures of a particular OS (e.g. `linux`)
* `make geth-<os>-<arch>`: Cross compiles to a specific OS/architecture (e.g. `linux`, `arm`)
We advise using the `make` based commands opposed to manually invoking `xgo` as we do
maintain the Makefile actively whereas we cannot guarantee that this document will be
always readily updated to latest advancements.
### Tuning the cross builds
A few of the `xgo` build options have also been surfaced directly into the Makefile to
allow fine tuning builds to work around either upstream Go issues, or to enable some
fancier mechanics.
- `make ... GO=<go>`: Use a specific Go runtime (e.g. `1.5.1`, `1.5-develop`, `develop`)
- `make ... MODE=<mode>`: Build a specific target type (e.g. `exe`, `c-archive`).
Please note that these are not yet fully finalized, so they may or may not change in
the future as our code and the Go runtime features change.

15
docs/_developers/Developer-Guide.md Normal file
View File

@ -0,0 +1,15 @@
---
title: Developer guide
---
### Native DApps
[Introduction and packages](Native:-Introduction)
[Account management](Native:-Account-management)
### Mobile platforms
[Introduction and packages](Mobile:-Introduction)
[Account management](Mobile:-Account-management)

161
docs/_developers/Peer-to-Peer.md Normal file
View File

@ -0,0 +1,161 @@
---
title: Peer-to-peer
---
The peer to peer package ([go-ethereum/p2p](https://github.com/ethereum/go-ethereum/tree/develop/p2p)) allows you to rapidly and easily add peer to peer networking to any type of application. The p2p package is set up in a modular structure and extending the p2p with your own additional sub protocols is easy and straight forward.
Starting the p2p service only requires you setup a `p2p.Server{}` with a few settings:
```go
import "github.com/ethereum/go-ethereum/crypto"
import "github.com/ethereum/go-ethereum/p2p"
nodekey, _ := crypto.GenerateKey()
srv := p2p.Server{
MaxPeers: 10,
PrivateKey: nodekey,
Name: "my node name",
ListenAddr: ":30300",
Protocols: []p2p.Protocol{},
}
srv.Start()
```
If we wanted to extend the capabilities of our p2p server we'd need to pass it an additional sub protocol in the `Protocol: []p2p.Protocol{}` array.
An additional sub protocol that has the ability to respond to the message "foo" with "bar" requires you to setup an `p2p.Protocol{}`:
```go
func MyProtocol() p2p.Protocol {
return p2p.Protocol{ // 1.
Name: "MyProtocol", // 2.
Version: 1, // 3.
Length: 1, // 4.
Run: func(peer *p2p.Peer, ws p2p.MsgReadWriter) error { return nil }, // 5.
}
}
```
1. A sub-protocol object in the p2p package is called `Protocol{}`. Each time a peer connects with the capability of handling this type of protocol will use this;
2. The name of your protocol to identify the protocol on the network;
3. The version of the protocol.
4. The amount of messages this protocol relies on. Because the p2p is extendible and thus has the ability to send an arbitrary amount of messages (with a type, which we'll see later) the p2p handler needs to know how much space it needs to reserve for your protocol, this to ensure consensus can be reached between the peers doing a negotiation over the message IDs. Our protocol supports only one; `message` (as you'll see later).
5. The main handler of your protocol. We've left this intentionally blank for now. The `peer` variable is the peer connected to you and provides you with some basic information regarding the peer. The `ws` variable which is a reader and a writer allows you to communicate with the peer. If a message is being send to us by that peer the `MsgReadWriter` will handle it and vice versa.
Lets fill in the blanks and create a somewhat useful peer by allowing it to communicate with another peer:
```go
const messageId = 0 // 1.
type Message string // 2.
func msgHandler(peer *p2p.Peer, ws p2p.MsgReadWriter) error {
for {
msg, err := ws.ReadMsg() // 3.
if err != nil { // 4.
return err // if reading fails return err which will disconnect the peer.
}
var myMessage [1]Message
err = msg.Decode(&myMessage) // 5.
if err != nil {
// handle decode error
continue
}
switch myMessage[0] {
case "foo":
err := p2p.SendItems(ws, messageId, "bar") // 6.
if err != nil {
return err // return (and disconnect) error if writing fails.
}
default:
fmt.Println("recv:", myMessage)
}
}
return nil
}
```
1. The one and only message we know about;
2. A typed string we decode in to;
3. `ReadMsg` waits on the line until it receives a message, an error or EOF.
4. In case of an error during reading it's best to return that error and let the p2p server handle it. This usually results in a disconnect from the peer.
5. `msg` contains two fields and a decoding method:
* `Code` contains the message id, `Code == messageId` (i.e., 0)
* `Payload` the contents of the message.
* `Decode(<ptr>)` is a helper method for: take `msg.Payload` and decodes the rest of the message in to the given interface. If it fails it will return an error.
6. If the message we decoded was `foo` respond with a `NewMessage` using the `messageId` message identifier and respond with the message `bar`. The `bar` message would be handled in the `default` case in the same switch.
Now if we'd tie this all up we'd have a working p2p server with a message passing sub protocol.
```go
package main
import (
"fmt"
"os"
"github.com/ethereum/go-ethereum/crypto"
"github.com/ethereum/go-ethereum/p2p"
)
const messageId = 0
type Message string
func MyProtocol() p2p.Protocol {
return p2p.Protocol{
Name: "MyProtocol",
Version: 1,
Length: 1,
Run: msgHandler,
}
}
func main() {
nodekey, _ := crypto.GenerateKey()
srv := p2p.Server{
MaxPeers: 10,
PrivateKey: nodekey,
Name: "my node name",
ListenAddr: ":30300",
Protocols: []p2p.Protocol{MyProtocol()},
}
if err := srv.Start(); err != nil {
fmt.Println(err)
os.Exit(1)
}
select {}
}
func msgHandler(peer *p2p.Peer, ws p2p.MsgReadWriter) error {
for {
msg, err := ws.ReadMsg()
if err != nil {
return err
}
var myMessage Message
err = msg.Decode(&myMessage)
if err != nil {
// handle decode error
continue
}
switch myMessage {
case "foo":
err := p2p.SendItems(ws, messageId, "bar"))
if err != nil {
return err
}
default:
fmt.Println("recv:", myMessage)
}
}
return nil
}
```

145
docs/_developers/RPC-PUB-SUB.md Normal file
View File

@ -0,0 +1,145 @@
---
title: RPS pub-sub
---
# Introduction
From version 1.4 geth has **_experimental_** support for pub/sub using subscriptions as defined in the JSON-RPC 2.0 specification. This allows clients to wait for events instead of polling for them.
It works by subscribing to particular events. The node will return a subscription id. For each event that matches the subscription a notification with relevant data is send together with the subscription id.
Example:
// create subscription
>> {"id": 1, "method": "eth_subscribe", "params": ["newHeads", {}]}
<< {"jsonrpc":"2.0","id":1,"result":"0xcd0c3e8af590364c09d0fa6a1210faf5"}
// incoming notifications
<< {"jsonrpc":"2.0","method":"eth_subscription","params":{"subscription":"0xcd0c3e8af590364c09d0fa6a1210faf5","result":{"difficulty":"0xd9263f42a87",<...>, "uncles":[]}}}
<< {"jsonrpc":"2.0","method":"eth_subscription","params":{"subscription":"0xcd0c3e8af590364c09d0fa6a1210faf5","result":{"difficulty":"0xd90b1a7ad02", <...>, "uncles":["0x80aacd1ea4c9da32efd8c2cc9ab38f8f70578fcd46a1a4ed73f82f3e0957f936"]}}}
// cancel subscription
>> {"id": 1, "method": "eth_unsubscribe", "params": ["0xcd0c3e8af590364c09d0fa6a1210faf5"]}
<< {"jsonrpc":"2.0","id":1,"result":true}
# Considerations
1. notifications are send for current events and not for past events. If your use case requires you not to miss any notifications than subscriptions are probably not the best option.
2. subscriptions require a full duplex connection. Geth offers such connections in the form of websockets (enable with --ws) and ipc (enabled by default).
3. subscriptions are coupled to a connection. If the connection is closed all subscriptions that are created over this connection are removed.
4. notifications are stored in an internal buffer and sent from this buffer to the client. If the client is unable to keep up and the number of buffered notifications reaches a limit (currently 10k) the connection is closed. Keep in mind that subscribing to some events can cause a flood of notifications, e.g. listening for all logs/blocks when the node starts to synchronize.
## Create subscription
Subscriptions are creates with a regular RPC call with `eth_subscribe` as method and the subscription name as first parameter. If successful it returns the subscription id.
### Parameters
1. subscription name
2. optional arguments
### Example
>> {"id": 1, "method": "eth_subscribe", "params": ["newHeads", {"includeTransactions": true}]}
<< {"id": 1, "jsonrpc": "2.0", "result": "0x9cef478923ff08bf67fde6c64013158d"}
## Cancel subscription
Subscriptions are cancelled with a regular RPC call with `eth_unsubscribe` as method and the subscription id as first parameter. It returns a bool indicating if the subscription was cancelled successful.
### Parameters
1. subscription id
### Example
>> {"id": 1, "method": "eth_unsubscribe", "params": ["0x9cef478923ff08bf67fde6c64013158d"]}
<< {"jsonrpc":"2.0","id":1,"result":true}
# Supported subscriptions
## newHeads
Fires a notification each time a new header is appended to the chain, including chain reorganizations. Users can use the bloom filter to determine if the block contains logs that are interested to them.
In case of a chain reorganization the subscription will emit all new headers for the new chain. Therefore the subscription can emit multiple headers on the same height.
### Example
```
>> {"id": 1, "method": "eth_subscribe", "params": ["newHeads"]}
<< {"jsonrpc":"2.0","id":2,"result":"0x9ce59a13059e417087c02d3236a0b1cc"}
<< {
"jsonrpc": "2.0",
"method": "eth_subscription",
"params": {
"result": {
"difficulty": "0x15d9223a23aa",
"extraData": "0xd983010305844765746887676f312e342e328777696e646f7773",
"gasLimit": "0x47e7c4",
"gasUsed": "0x38658",
"logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"miner": "0xf8b483dba2c3b7176a3da549ad41a48bb3121069",
"nonce": "0x084149998194cc5f",
"number": "0x1348c9",
"parentHash": "0x7736fab79e05dc611604d22470dadad26f56fe494421b5b333de816ce1f25701",
"receiptRoot": "0x2fab35823ad00c7bb388595cb46652fe7886e00660a01e867824d3dceb1c8d36",
"sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
"stateRoot": "0xb3346685172db67de536d8765c43c31009d0eb3bd9c501c9be3229203f15f378",
"timestamp": "0x56ffeff8",
"transactionsRoot": "0x0167ffa60e3ebc0b080cdb95f7c0087dd6c0e61413140e39d94d3468d7c9689f"
},
"subscription": "0x9ce59a13059e417087c02d3236a0b1cc"
}
}
```
## logs
Returns logs that are included in new imported blocks and match the given filter criteria.
In case of a chain reorganization previous sent logs that are on the old chain will be resend with the `removed` property set to true. Logs from transactions that ended up in the new chain are emitted. Therefore a subscription can emit logs for the same transaction multiple times.
### Parameters
1. `object` with the following (optional) fields
- **address**, either an address or an array of addresses. Only logs that are created from these addresses are returned (optional)
- **topics**, only logs which match the specified topics (optional)
### Example
>> {"id": 1, "method": "eth_subscribe", "params": ["logs", {"address": "0x8320fe7702b96808f7bbc0d4a888ed1468216cfd", "topics": ["0xd78a0cb8bb633d06981248b816e7bd33c2a35a6089241d099fa519e361cab902"]}]}
<< {"jsonrpc":"2.0","id":2,"result":"0x4a8a4c0517381924f9838102c5a4dcb7"}
<< {"jsonrpc":"2.0","method":"eth_subscription","params": {"subscription":"0x4a8a4c0517381924f9838102c5a4dcb7","result":{"address":"0x8320fe7702b96808f7bbc0d4a888ed1468216cfd","blockHash":"0x61cdb2a09ab99abf791d474f20c2ea89bf8de2923a2d42bb49944c8c993cbf04","blockNumber":"0x29e87","data":"0x00000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000003","logIndex":"0x0","topics":["0xd78a0cb8bb633d06981248b816e7bd33c2a35a6089241d099fa519e361cab902"],"transactionHash":"0xe044554a0a55067caafd07f8020ab9f2af60bdfe337e395ecd84b4877a3d1ab4","transactionIndex":"0x0"}}}
## newPendingTransactions
Returns the hash for all transactions that are added to the pending state and are signed with a key that is available in the node.
When a transaction that was previously part of the canonical chain isn't part of the new canonical chain after a reogranization its again emitted.
### Parameters
none
### Example
>> {"id": 1, "method": "eth_subscribe", "params": ["newPendingTransactions"]}
<< {"jsonrpc":"2.0","id":2,"result":"0xc3b33aa549fb9a60e95d21862596617c"}
<< {
"jsonrpc":"2.0",
"method":"eth_subscription",
"params":{
"subscription":"0xc3b33aa549fb9a60e95d21862596617c",
"result":"0xd6fdc5cc41a9959e922f30cb772a9aef46f4daea279307bc5f7024edc4ccd7fa"
}
}
## syncing
Indicates when the node starts or stops synchronizing. The result can either be a boolean indicating that the synchronization has started (true), finished (false) or an object with various progress indicators.
### Parameters
none
### Example
>> {"id": 1, "method": "eth_subscribe", "params": ["syncing"]}
<< {"jsonrpc":"2.0","id":2,"result":"0xe2ffeb2703bcf602d42922385829ce96"}
<< {"subscription":"0xe2ffeb2703bcf602d42922385829ce96","result":{"syncing":true,"status":{"startingBlock":674427,"currentBlock":67400,"highestBlock":674432,"pulledStates":0,"knownStates":0}}}}
## Possible future subscription:
- balance changes
- account changes
- nonce changes
- storage changes in contracts

141
docs/_developers/Tracing_Introduction.md Normal file
View File

@ -0,0 +1,141 @@
---
title: Tracing / Introduction
---
There are two different types of transactions in Ethereum: plain value transfers and contract executions. A plain value transfer just moves Ether from one account to another and as such is uninteresting from this guide's perspective. If however the recipient of a transaction is a contract account with associated EVM (Ethereum Virtual Machine) bytecode - beside transferring any Ether - the code will also be executed as part of the transaction.
Having code associated with Ethereum accounts permits transactions to do arbitrarilly complex data storage and enables them to act on the previously stored data by further transacting internally with outside accounts and contracts. This creates an intertwined ecosystem of contracts, where a single transaction can interact with tens or hunderds of accounts.
The downside of contract execution is that it is very hard to say what a transaction actually did. A transaction receipt does contain a status code to check whether execution succeeded or not, but there's no way to see what data was modified, nor what external contracts where invoked. In order to introspect a transaction, we need to trace its execution.
## Tracing prerequisites
In its simplest form, tracing a transaction entails requesting the Ethereum node to reexecute the desired transaction with varying degrees of data collection and have it return the aggregated summary for post processing. Reexecuting a transaction however has a few prerequisites to be met.
In order for an Ethereum node to reexecute a transaction, it needs to have available all historical state accessed by the transaction:
* Balance, nonce, bytecode and storage of both the recipient as well as all internally invoked contracts.
* Block metadata referenced during execution of both the outer as well as all internally created transactions.
* Intermediate state generated by all preceding transactions contained in the same block as the one being traced.
Depending on your node's mode of synchronization and pruning, different configurations result in different capabilities:
* An **archive** node retaining **all historical data** can trace arbitrary transactions at any point in time. Tracing a single transaction also entails reexecuting all preceding transactions in the same block.
* A **fast synced** node retaining **all historical data** after initial sync can only trace transactions from blocks following the initial sync point. Tracing a single transaction also entails reexecuting all preceding transactions in the same block.
* A **fast synced** node retaining only **periodic state data** after initial sync can only trace transactions from blocks following the initial sync point. Tracing a single transaction entails reexecuting all preceding transactions **both** in the same block, as well as all preceding blocks until the previous stored snapshot.
* A **light synced** node retrieving data **on demand** can in theory trace transactions for which all required historical state is readily available in the network. In practice, data availability is **not** a feasible assumption.
*There are exceptions to the above rules when running batch traces of entire blocks or chain segments. Those will be detailed later.*
## Basic traces
The simplest type of transaction trace that `go-ethereum` can generate are raw EVM opcode traces. For every VM instruction the transaction executes, a structured log entry is emitted, containing all contextual metadata deemed useful. This includes the *program counter*, *opcode name*, *opcode cost*, *remaining gas*, *execution depth* and any *occurred error*. The structured logs can optionally also contain the content of the *execution stack*, *execution memory* and *contract storage*.
An example log entry for a single opcode looks like:
```json
{
"pc": 48,
"op": "DIV",
"gasCost": 5,
"gas": 64532,
"depth": 1,
"error": null,
"stack": [
"00000000000000000000000000000000000000000000000000000000ffffffff",
"0000000100000000000000000000000000000000000000000000000000000000",
"2df07fbaabbe40e3244445af30759352e348ec8bebd4dd75467a9f29ec55d98d"
],
"memory": [
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000000",
"0000000000000000000000000000000000000000000000000000000000000060"
],
"storage": {
}
}
```
The entire output of an raw EVM opcode trace is a JSON object having a few metadata fields: *consumed gas*, *failure status*, *return value*; and a list of *opcode entries* that take the above form:
```json
{
"gas": 25523,
"failed": false,
"returnValue": "",
"structLogs": []
}
```
### Generating basic traces
To generate a raw EVM opcode trace, `go-ethereum` provides a few [RPC API endpoints](https://github.com/ethereum/go-ethereum/wiki/Management-APIs), out of which the most commonly used is [`debug_traceTransaction`](https://github.com/ethereum/go-ethereum/wiki/Management-APIs#debug_tracetransaction).
In its simplest form, `traceTransaction` accepts a transaction hash as its sole argument, traces the transaction, aggregates all the generated data and returns it as a **large** JSON object. A sample invocation from the Geth console would be:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f")
```
The same call can of course be invoked from outside the node too via HTTP RPC. In this case, please make sure the HTTP endpoint is enabled via `--rpc` and the `debug` API namespace exposed via `--rpcapi=debug`.
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f"]}' localhost:8545
```
Running the above operation on the Rinkeby network (with a node retaining enough history) will result in this [trace dump](https://gist.github.com/karalabe/c91f95ac57f5e57f8b950ec65ecc697f).
### Tuning basic traces
By default the raw opcode tracer emits all relevant events that occur within the EVM while processing a transaction, such as *EVM stack*, *EVM memory* and *updated storage slots*. Certain use cases however may not need some of these data fields reported. To cater for those use cases, these massive fields may be omitted using a second *options* parameter for the tracer:
```json
{
"disableStack": true,
"disableMemory": true,
"disableStorage": true
}
```
Running the previous tracer invocation from the Geth console with the data fields disabled:
```js
debug.traceTransaction("0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {disableStack: true, disableMemory: true, disableStorage: true})
```
Analogously running the filtered tracer from outside the node too via HTTP RPC:
```
$ curl -H "Content-Type: application/json" -d '{"id": 1, "method": "debug_traceTransaction", "params": ["0xfc9359e49278b7ba99f59edac0e3de49956e46e530a53c15aa71226b7aa92c6f", {"disableStack": true, "disableMemory": true, "disableStorage": true}]}' localhost:8545
```
Running the above operation on the Rinkeby network will result in this significantly shorter [trace dump](https://gist.github.com/karalabe/d74a7cb33a70f2af75e7824fc772c5b4).
### Limits of basic traces
Although the raw opcode traces we've generated above have their use, this basic way of tracing is problematic in the real world. Having an individual log entry for every single opcode is too low level for most use cases, and will require developers to create additional tools to post-process the traces. Additionally, a full opcode trace can easily go into the hundreds of megabytes, making them very resource intensive to get out of the node and process externally.
To avoid all of the previously mentioned issues, `go-ethereum` supports running custom JavaScript tracers *within* the Ethereum node, which have full access to the EVM stack, memory and contract storage. This permits developers to only gather the data they need, and do any processing **at** the data. Please see the next section for our *custom in-node tracers*.
### Pruning
Geth by default does in-memory pruning of state, discarding state entries that it deems is no longer necessary to maintain. This is configured via the `--gcmode` option. Often, people run into the error that state is not available.
Say you want to do a trace on block `B`. Now there are a couple of cases:
1. You have done a fast-sync, pivot block `P` where `P <= B`.
2. You have done a fast-sync, pivot block `P` where `P > B`.
3. You have done a full-sync, with pruning
4. You have done a full-sync, without pruning (`--gcmode=archive`)
Here's what happens in each respective case:
1. Geth will regenerate the desired state by replaying blocks from the closest point in time before `B` where it has full state. This defaults to [`128`](https://github.com/ethereum/go-ethereum/blob/master/eth/api_tracer.go#L52) blocks max, but you can specify more in the actual call `... "reexec":1000 .. }` to the tracer.
2. Sorry, can't be done without replaying from genesis.
3. Same as 1)
4. Does not need to replay anything, can immediately load up the state and serve the request.
There is one other option available to you, which may or may not suit your needs. That is to use [Evmlab]( https://github.com/holiman/evmlab).
```
docker pull holiman/evmlab && docker run -it holiman/evmlab
```
There you can use the reproducer. The reproducer will incrementally fetch data from infura until it has all the information required to create the trace locally on an evm which is bundled with the image. It will create a custom genesis containing the state that the transaction touches (balances, code, nonce etc). It should be mentioned that the evmlab reproducer is strictly guaranteed to be totally exact with regards to gascosts incurred by the outer transaction, as evmlab does not fully calculate the gascosts for nonzero data etc, but is usually sufficient to analyze contracts and events.