Move from gitbook to docusaurus, build docs in Travis CI (#10970)
* fix: ignore unknown fields in more RPC responses * Remove mdbook infrastructure * Delete gitattributes and other theme related items Move all docs to /docs folder to support Docusaurus * all docs need to be moved to /docs * can be changed in the future Add Docusaurus infrastructure * initialize docusaurus repo Remove trailing whitespace, add support for eslint Change Docusaurus configuration to support `src` * No need to rename the folder! Change a setting and we're all good to go. * Fixing rebase items * Remove unneccessary markdown file, fix type * Some fonts are hard to read. Others, not so much. Rubik, you've been sidelined. Roboto, into the limelight! * As much as we all love tutorials, I think we all can navigate around a markdown file. Say goodbye, `mdx.md`. * Setup deployment infrastructure * Move docs job from buildkite to travic * Fix travis config * Add vercel token to travis config * Only deploy docs after merge * Docker rust env * Revert "Docker rust env" This reverts commit f84bc208e807aab1c0d97c7588bbfada1fedfa7c. * Build CLI usage from docker * Pacify shellcheck * Run job on PR and new commits for publication * Update README * Fix svg image building * shellcheck Co-authored-by: Michael Vines <mvines@gmail.com> Co-authored-by: Ryan Shea <rmshea@users.noreply.github.com> Co-authored-by: publish-docs.sh <maintainers@solana.com>
This commit is contained in:
Dan Albert
GitHub
@@ -1,4 +1,6 @@
|
||||
# A Solana Cluster
|
||||
---
|
||||
title: A Solana Cluster
|
||||
---
|
||||
|
||||
A Solana cluster is a set of validators working together to serve client transactions and maintain the integrity of the ledger. Many clusters may coexist. When two clusters share a common genesis block, they attempt to converge. Otherwise, they simply ignore the existence of the other. Transactions sent to the wrong one are quietly rejected. In this section, we'll discuss how a cluster is created, how nodes join the cluster, how they share the ledger, how they ensure the ledger is replicated, and how they cope with buggy and malicious nodes.
|
||||
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Benchmark a Cluster
|
||||
---
|
||||
title: Benchmark a Cluster
|
||||
---
|
||||
|
||||
The Solana git repository contains all the scripts you might need to spin up your own local testnet. Depending on what you're looking to achieve, you may want to run a different variation, as the full-fledged, performance-enhanced multinode testnet is considerably more complex to set up than a Rust-only, singlenode testnode. If you are looking to develop high-level features, such as experimenting with smart contracts, save yourself some setup headaches and stick to the Rust-only singlenode demo. If you're doing performance optimization of the transaction pipeline, consider the enhanced singlenode demo. If you're doing consensus work, you'll need at least a Rust-only multinode demo. If you want to reproduce our TPS metrics, run the enhanced multinode demo.
|
||||
|
||||
@@ -92,17 +94,17 @@ What just happened? The client demo spins up several threads to send 500,000 tra
|
||||
|
||||
### Testnet Debugging
|
||||
|
||||
There are some useful debug messages in the code, you can enable them on a per-module and per-level basis. Before running a leader or validator set the normal RUST\_LOG environment variable.
|
||||
There are some useful debug messages in the code, you can enable them on a per-module and per-level basis. Before running a leader or validator set the normal RUST_LOG environment variable.
|
||||
|
||||
For example
|
||||
|
||||
* To enable `info` everywhere and `debug` only in the solana::banking\_stage module:
|
||||
- To enable `info` everywhere and `debug` only in the solana::banking_stage module:
|
||||
|
||||
```bash
|
||||
$ export RUST_LOG=solana=info,solana::banking_stage=debug
|
||||
```
|
||||
|
||||
* To enable BPF program logging:
|
||||
- To enable BPF program logging:
|
||||
|
||||
```bash
|
||||
$ export RUST_LOG=solana_bpf_loader=trace
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Fork Generation
|
||||
---
|
||||
title: Fork Generation
|
||||
---
|
||||
|
||||
This section describes how forks naturally occur as a consequence of [leader rotation](leader-rotation.md).
|
||||
|
||||
@@ -58,7 +60,7 @@ Validators vote based on a greedy choice to maximize their reward described in [
|
||||
|
||||
The diagram below represents a validator's view of the PoH stream with possible forks over time. L1, L2, etc. are leader slots, and `E`s represent entries from that leader during that leader's slot. The `x`s represent ticks only, and time flows downwards in the diagram.
|
||||
|
||||
|
||||
|
||||
|
||||
Note that an `E` appearing on 2 forks at the same slot is a slashable condition, so a validator observing `E3` and `E3'` can slash L3 and safely choose `x` for that slot. Once a validator commits to a forks, other forks can be discarded below that tick count. For any slot, validators need only consider a single "has entries" chain or a "ticks only" chain to be proposed by a leader. But multiple virtual entries may overlap as they link back to the a previous slot.
|
||||
|
||||
@@ -66,10 +68,10 @@ Note that an `E` appearing on 2 forks at the same slot is a slashable condition,
|
||||
|
||||
It's useful to consider leader rotation over PoH tick count as time division of the job of encoding state for the cluster. The following table presents the above tree of forks as a time-divided ledger.
|
||||
|
||||
| leader slot | L1 | L2 | L3 | L4 | L5 |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| data | E1 | E2 | E3 | E4 | E5 |
|
||||
| ticks since prev | | | | x | xx |
|
||||
| leader slot | L1 | L2 | L3 | L4 | L5 |
|
||||
| :--------------- | :-- | :-- | :-- | :-- | :-- |
|
||||
| data | E1 | E2 | E3 | E4 | E5 |
|
||||
| ticks since prev | | | | x | xx |
|
||||
|
||||
Note that only data from leader L3 will be accepted during leader slot L3. Data from L3 may include "catchup" ticks back to a slot other than L2 if L3 did not observe L2's data. L4 and L5's transmissions include the "ticks to prev" PoH entries.
|
||||
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Leader Rotation
|
||||
---
|
||||
title: Leader Rotation
|
||||
---
|
||||
|
||||
At any given moment, a cluster expects only one validator to produce ledger entries. By having only one leader at a time, all validators are able to replay identical copies of the ledger. The drawback of only one leader at a time, however, is that a malicious leader is capable of censoring votes and transactions. Since censoring cannot be distinguished from the network dropping packets, the cluster cannot simply elect a single node to hold the leader role indefinitely. Instead, the cluster minimizes the influence of a malicious leader by rotating which node takes the lead.
|
||||
|
||||
@@ -31,8 +33,8 @@ Two partitions that are generating half of the blocks each. Neither is coming to
|
||||
|
||||
In this unstable scenario, multiple valid leader schedules exist.
|
||||
|
||||
* A leader schedule is generated for every fork whose direct parent is in the previous epoch.
|
||||
* The leader schedule is valid after the start of the next epoch for descendant forks until it is updated.
|
||||
- A leader schedule is generated for every fork whose direct parent is in the previous epoch.
|
||||
- The leader schedule is valid after the start of the next epoch for descendant forks until it is updated.
|
||||
|
||||
Each partition's schedule will diverge after the partition lasts more than an epoch. For this reason, the epoch duration should be selected to be much much larger then slot time and the expected length for a fork to be committed to root.
|
||||
|
||||
@@ -73,8 +75,8 @@ The seed that is selected is predictable but unbiasable. There is no grinding at
|
||||
|
||||
A leader can bias the active set by censoring validator votes. Two possible ways exist for leaders to censor the active set:
|
||||
|
||||
* Ignore votes from validators
|
||||
* Refuse to vote for blocks with votes from validators
|
||||
- Ignore votes from validators
|
||||
- Refuse to vote for blocks with votes from validators
|
||||
|
||||
To reduce the likelihood of censorship, the active set is calculated at the leader schedule offset boundary over an _active set sampling duration_. The active set sampling duration is long enough such that votes will have been collected by multiple leaders.
|
||||
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Managing Forks
|
||||
---
|
||||
title: Managing Forks
|
||||
---
|
||||
|
||||
The ledger is permitted to fork at slot boundaries. The resulting data structure forms a tree called a _blockstore_. When the validator interprets the blockstore, it must maintain state for each fork in the chain. We call each instance an _active fork_. It is the responsibility of a validator to weigh those forks, such that it may eventually select a fork.
|
||||
|
||||
@@ -8,14 +10,14 @@ A validator selects a fork by submiting a vote to a slot leader on that fork. Th
|
||||
|
||||
An active fork is as a sequence of checkpoints that has a length at least one longer than the rollback depth. The shortest fork will have a length exactly one longer than the rollback depth. For example:
|
||||
|
||||
|
||||
|
||||
|
||||
The following sequences are _active forks_:
|
||||
|
||||
* {4, 2, 1}
|
||||
* {5, 2, 1}
|
||||
* {6, 3, 1}
|
||||
* {7, 3, 1}
|
||||
- {4, 2, 1}
|
||||
- {5, 2, 1}
|
||||
- {6, 3, 1}
|
||||
- {7, 3, 1}
|
||||
|
||||
## Pruning and Squashing
|
||||
|
||||
@@ -23,12 +25,12 @@ A validator may vote on any checkpoint in the tree. In the diagram above, that's
|
||||
|
||||
Starting from the example above, wth a rollback depth of 2, consider a vote on 5 versus a vote on 6. First, a vote on 5:
|
||||
|
||||
|
||||
|
||||
|
||||
The new root is 2, and any active forks that are not descendants from 2 are pruned.
|
||||
|
||||
Alternatively, a vote on 6:
|
||||
|
||||
|
||||
|
||||
|
||||
The tree remains with a root of 1, since the active fork starting at 6 is only 2 checkpoints from the root.
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Performance Metrics
|
||||
---
|
||||
title: Performance Metrics
|
||||
---
|
||||
|
||||
Solana cluster performance is measured as average number of transactions per second that the network can sustain \(TPS\). And, how long it takes for a transaction to be confirmed by super majority of the cluster \(Confirmation Time\).
|
||||
|
||||
@@ -21,4 +23,3 @@ The validator software is deployed to GCP n1-standard-16 instances with 1TB pd-s
|
||||
solana-bench-tps is started after the network converges from a client machine with n1-standard-16 CPU-only instance with the following arguments: `--tx\_count=50000 --thread-batch-sleep 1000`
|
||||
|
||||
TPS and confirmation metrics are captured from the dashboard numbers over a 5 minute average of when the bench-tps transfer stage begins.
|
||||
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Stake Delegation and Rewards
|
||||
---
|
||||
title: Stake Delegation and Rewards
|
||||
---
|
||||
|
||||
Stakers are rewarded for helping to validate the ledger. They do this by delegating their stake to validator nodes. Those validators do the legwork of replaying the ledger and send votes to a per-node vote account to which stakers can delegate their stakes. The rest of the cluster uses those stake-weighted votes to select a block when forks arise. Both the validator and staker need some economic incentive to play their part. The validator needs to be compensated for its hardware and the staker needs to be compensated for the risk of getting its stake slashed. The economics are covered in [staking rewards](../implemented-proposals/staking-rewards.md). This section, on the other hand, describes the underlying mechanics of its implementation.
|
||||
|
||||
@@ -22,18 +24,18 @@ The rewards process is split into two on-chain programs. The Vote program solves
|
||||
|
||||
VoteState is the current state of all the votes the validator has submitted to the network. VoteState contains the following state information:
|
||||
|
||||
* `votes` - The submitted votes data structure.
|
||||
* `credits` - The total number of rewards this vote program has generated over its lifetime.
|
||||
* `root_slot` - The last slot to reach the full lockout commitment necessary for rewards.
|
||||
* `commission` - The commission taken by this VoteState for any rewards claimed by staker's Stake accounts. This is the percentage ceiling of the reward.
|
||||
* Account::lamports - The accumulated lamports from the commission. These do not count as stakes.
|
||||
* `authorized_voter` - Only this identity is authorized to submit votes. This field can only modified by this identity.
|
||||
* `node_pubkey` - The Solana node that votes in this account.
|
||||
* `authorized_withdrawer` - the identity of the entity in charge of the lamports of this account, separate from the account's address and the authorized vote signer
|
||||
- `votes` - The submitted votes data structure.
|
||||
- `credits` - The total number of rewards this vote program has generated over its lifetime.
|
||||
- `root_slot` - The last slot to reach the full lockout commitment necessary for rewards.
|
||||
- `commission` - The commission taken by this VoteState for any rewards claimed by staker's Stake accounts. This is the percentage ceiling of the reward.
|
||||
- Account::lamports - The accumulated lamports from the commission. These do not count as stakes.
|
||||
- `authorized_voter` - Only this identity is authorized to submit votes. This field can only modified by this identity.
|
||||
- `node_pubkey` - The Solana node that votes in this account.
|
||||
- `authorized_withdrawer` - the identity of the entity in charge of the lamports of this account, separate from the account's address and the authorized vote signer
|
||||
|
||||
### VoteInstruction::Initialize\(VoteInit\)
|
||||
|
||||
* `account[0]` - RW - The VoteState
|
||||
- `account[0]` - RW - The VoteState
|
||||
|
||||
`VoteInit` carries the new vote account's `node_pubkey`, `authorized_voter`, `authorized_withdrawer`, and `commission`
|
||||
|
||||
@@ -43,16 +45,16 @@ VoteState is the current state of all the votes the validator has submitted to t
|
||||
|
||||
Updates the account with a new authorized voter or withdrawer, according to the VoteAuthorize parameter \(`Voter` or `Withdrawer`\). The transaction must be by signed by the Vote account's current `authorized_voter` or `authorized_withdrawer`.
|
||||
|
||||
* `account[0]` - RW - The VoteState
|
||||
- `account[0]` - RW - The VoteState
|
||||
`VoteState::authorized_voter` or `authorized_withdrawer` is set to to `Pubkey`.
|
||||
|
||||
### VoteInstruction::Vote\(Vote\)
|
||||
|
||||
* `account[0]` - RW - The VoteState
|
||||
- `account[0]` - RW - The VoteState
|
||||
`VoteState::lockouts` and `VoteState::credits` are updated according to voting lockout rules see [Tower BFT](../implemented-proposals/tower-bft.md)
|
||||
|
||||
* `account[1]` - RO - `sysvar::slot_hashes` A list of some N most recent slots and their hashes for the vote to be verified against.
|
||||
* `account[2]` - RO - `sysvar::clock` The current network time, expressed in slots, epochs.
|
||||
- `account[1]` - RO - `sysvar::slot_hashes` A list of some N most recent slots and their hashes for the vote to be verified against.
|
||||
- `account[2]` - RO - `sysvar::clock` The current network time, expressed in slots, epochs.
|
||||
|
||||
### StakeState
|
||||
|
||||
@@ -62,15 +64,15 @@ A StakeState takes one of four forms, StakeState::Uninitialized, StakeState::Ini
|
||||
|
||||
StakeState::Stake is the current delegation preference of the **staker** and contains the following state information:
|
||||
|
||||
* Account::lamports - The lamports available for staking.
|
||||
* `stake` - the staked amount \(subject to warm up and cool down\) for generating rewards, always less than or equal to Account::lamports
|
||||
* `voter_pubkey` - The pubkey of the VoteState instance the lamports are delegated to.
|
||||
* `credits_observed` - The total credits claimed over the lifetime of the program.
|
||||
* `activated` - the epoch at which this stake was activated/delegated. The full stake will be counted after warm up.
|
||||
* `deactivated` - the epoch at which this stake was de-activated, some cool down epochs are required before the account is fully deactivated, and the stake available for withdrawal
|
||||
- Account::lamports - The lamports available for staking.
|
||||
- `stake` - the staked amount \(subject to warm up and cool down\) for generating rewards, always less than or equal to Account::lamports
|
||||
- `voter_pubkey` - The pubkey of the VoteState instance the lamports are delegated to.
|
||||
- `credits_observed` - The total credits claimed over the lifetime of the program.
|
||||
- `activated` - the epoch at which this stake was activated/delegated. The full stake will be counted after warm up.
|
||||
- `deactivated` - the epoch at which this stake was de-activated, some cool down epochs are required before the account is fully deactivated, and the stake available for withdrawal
|
||||
|
||||
* `authorized_staker` - the pubkey of the entity that must sign delegation, activation, and deactivation transactions
|
||||
* `authorized_withdrawer` - the identity of the entity in charge of the lamports of this account, separate from the account's address, and the authorized staker
|
||||
- `authorized_staker` - the pubkey of the entity that must sign delegation, activation, and deactivation transactions
|
||||
- `authorized_withdrawer` - the identity of the entity in charge of the lamports of this account, separate from the account's address, and the authorized staker
|
||||
|
||||
### StakeState::RewardsPool
|
||||
|
||||
@@ -82,17 +84,17 @@ The Stakes and the RewardsPool are accounts that are owned by the same `Stake` p
|
||||
|
||||
The Stake account is moved from Initialized to StakeState::Stake form, or from a deactivated (i.e. fully cooled-down) StakeState::Stake to activated StakeState::Stake. This is how stakers choose the vote account and validator node to which their stake account lamports are delegated. The transaction must be signed by the stake's `authorized_staker`.
|
||||
|
||||
* `account[0]` - RW - The StakeState::Stake instance. `StakeState::Stake::credits_observed` is initialized to `VoteState::credits`, `StakeState::Stake::voter_pubkey` is initialized to `account[1]`. If this is the initial delegation of stake, `StakeState::Stake::stake` is initialized to the account's balance in lamports, `StakeState::Stake::activated` is initialized to the current Bank epoch, and `StakeState::Stake::deactivated` is initialized to std::u64::MAX
|
||||
* `account[1]` - R - The VoteState instance.
|
||||
* `account[2]` - R - sysvar::clock account, carries information about current Bank epoch
|
||||
* `account[3]` - R - sysvar::stakehistory account, carries information about stake history
|
||||
* `account[4]` - R - stake::Config accoount, carries warmup, cooldown, and slashing configuration
|
||||
- `account[0]` - RW - The StakeState::Stake instance. `StakeState::Stake::credits_observed` is initialized to `VoteState::credits`, `StakeState::Stake::voter_pubkey` is initialized to `account[1]`. If this is the initial delegation of stake, `StakeState::Stake::stake` is initialized to the account's balance in lamports, `StakeState::Stake::activated` is initialized to the current Bank epoch, and `StakeState::Stake::deactivated` is initialized to std::u64::MAX
|
||||
- `account[1]` - R - The VoteState instance.
|
||||
- `account[2]` - R - sysvar::clock account, carries information about current Bank epoch
|
||||
- `account[3]` - R - sysvar::stakehistory account, carries information about stake history
|
||||
- `account[4]` - R - stake::Config accoount, carries warmup, cooldown, and slashing configuration
|
||||
|
||||
### StakeInstruction::Authorize\(Pubkey, StakeAuthorize\)
|
||||
|
||||
Updates the account with a new authorized staker or withdrawer, according to the StakeAuthorize parameter \(`Staker` or `Withdrawer`\). The transaction must be by signed by the Stakee account's current `authorized_staker` or `authorized_withdrawer`. Any stake lock-up must have expired, or the lock-up custodian must also sign the transaction.
|
||||
Updates the account with a new authorized staker or withdrawer, according to the StakeAuthorize parameter \(`Staker` or `Withdrawer`\). The transaction must be by signed by the Stakee account's current `authorized_staker` or `authorized_withdrawer`. Any stake lock-up must have expired, or the lock-up custodian must also sign the transaction.
|
||||
|
||||
* `account[0]` - RW - The StakeState
|
||||
- `account[0]` - RW - The StakeState
|
||||
|
||||
`StakeState::authorized_staker` or `authorized_withdrawer` is set to to `Pubkey`.
|
||||
|
||||
@@ -101,8 +103,8 @@ Updates the account with a new authorized staker or withdrawer, according to the
|
||||
A staker may wish to withdraw from the network. To do so he must first deactivate his stake, and wait for cool down.
|
||||
The transaction must be signed by the stake's `authorized_staker`.
|
||||
|
||||
* `account[0]` - RW - The StakeState::Stake instance that is deactivating.
|
||||
* `account[1]` - R - sysvar::clock account from the Bank that carries current epoch
|
||||
- `account[0]` - RW - The StakeState::Stake instance that is deactivating.
|
||||
- `account[1]` - R - sysvar::clock account from the Bank that carries current epoch
|
||||
|
||||
StakeState::Stake::deactivated is set to the current epoch + cool down. The account's stake will ramp down to zero by that epoch, and Account::lamports will be available for withdrawal.
|
||||
|
||||
@@ -110,21 +112,21 @@ StakeState::Stake::deactivated is set to the current epoch + cool down. The acco
|
||||
|
||||
Lamports build up over time in a Stake account and any excess over activated stake can be withdrawn. The transaction must be signed by the stake's `authorized_withdrawer`.
|
||||
|
||||
* `account[0]` - RW - The StakeState::Stake from which to withdraw.
|
||||
* `account[1]` - RW - Account that should be credited with the withdrawn lamports.
|
||||
* `account[2]` - R - sysvar::clock account from the Bank that carries current epoch, to calculate stake.
|
||||
* `account[3]` - R - sysvar::stake\_history account from the Bank that carries stake warmup/cooldown history
|
||||
- `account[0]` - RW - The StakeState::Stake from which to withdraw.
|
||||
- `account[1]` - RW - Account that should be credited with the withdrawn lamports.
|
||||
- `account[2]` - R - sysvar::clock account from the Bank that carries current epoch, to calculate stake.
|
||||
- `account[3]` - R - sysvar::stake_history account from the Bank that carries stake warmup/cooldown history
|
||||
|
||||
## Benefits of the design
|
||||
|
||||
* Single vote for all the stakers.
|
||||
* Clearing of the credit variable is not necessary for claiming rewards.
|
||||
* Each delegated stake can claim its rewards independently.
|
||||
* Commission for the work is deposited when a reward is claimed by the delegated stake.
|
||||
- Single vote for all the stakers.
|
||||
- Clearing of the credit variable is not necessary for claiming rewards.
|
||||
- Each delegated stake can claim its rewards independently.
|
||||
- Commission for the work is deposited when a reward is claimed by the delegated stake.
|
||||
|
||||
## Example Callflow
|
||||
|
||||
|
||||
|
||||
|
||||
## Staking Rewards
|
||||
|
||||
@@ -171,22 +173,22 @@ Consider the situation of a single stake of 1,000 activated at epoch N, with net
|
||||
At epoch N+1, the amount available to be activated for the network is 400 \(20% of 200\), and at epoch N, this example stake is the only stake activating, and so is entitled to all of the warmup room available.
|
||||
|
||||
| epoch | effective | activating | total effective | total activating |
|
||||
| :--- | ---: | ---: | ---: | ---: |
|
||||
| N-1 | | | 2,000 | 0 |
|
||||
| N | 0 | 1,000 | 2,000 | 1,000 |
|
||||
| N+1 | 400 | 600 | 2,400 | 600 |
|
||||
| N+2 | 880 | 120 | 2,880 | 120 |
|
||||
| N+3 | 1000 | 0 | 3,000 | 0 |
|
||||
| :---- | --------: | ---------: | --------------: | ---------------: |
|
||||
| N-1 | | | 2,000 | 0 |
|
||||
| N | 0 | 1,000 | 2,000 | 1,000 |
|
||||
| N+1 | 400 | 600 | 2,400 | 600 |
|
||||
| N+2 | 880 | 120 | 2,880 | 120 |
|
||||
| N+3 | 1000 | 0 | 3,000 | 0 |
|
||||
|
||||
Were 2 stakes \(X and Y\) to activate at epoch N, they would be awarded a portion of the 20% in proportion to their stakes. At each epoch effective and activating for each stake is a function of the previous epoch's state.
|
||||
|
||||
| epoch | X eff | X act | Y eff | Y act | total effective | total activating |
|
||||
| :--- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| N-1 | | | | | 2,000 | 0 |
|
||||
| N | 0 | 1,000 | 0 | 200 | 2,000 | 1,200 |
|
||||
| N+1 | 333 | 667 | 67 | 133 | 2,400 | 800 |
|
||||
| N+2 | 733 | 267 | 146 | 54 | 2,880 | 321 |
|
||||
| N+3 | 1000 | 0 | 200 | 0 | 3,200 | 0 |
|
||||
| :---- | ----: | ----: | ----: | ----: | --------------: | ---------------: |
|
||||
| N-1 | | | | | 2,000 | 0 |
|
||||
| N | 0 | 1,000 | 0 | 200 | 2,000 | 1,200 |
|
||||
| N+1 | 333 | 667 | 67 | 133 | 2,400 | 800 |
|
||||
| N+2 | 733 | 267 | 146 | 54 | 2,880 | 321 |
|
||||
| N+3 | 1000 | 0 | 200 | 0 | 3,200 | 0 |
|
||||
|
||||
### Withdrawal
|
||||
|
||||
@@ -194,4 +196,4 @@ Only lamports in excess of effective+activating stake may be withdrawn at any ti
|
||||
|
||||
### Lock-up
|
||||
|
||||
Stake accounts support the notion of lock-up, wherein the stake account balance is unavailable for withdrawal until a specified time. Lock-up is specified as an epoch height, i.e. the minimum epoch height that must be reached by the network before the stake account balance is available for withdrawal, unless the transaction is also signed by a specified custodian. This information is gathered when the stake account is created, and stored in the Lockup field of the stake account's state. Changing the authorized staker or withdrawer is also subject to lock-up, as such an operation is effectively a transfer.
|
||||
Stake accounts support the notion of lock-up, wherein the stake account balance is unavailable for withdrawal until a specified time. Lock-up is specified as an epoch height, i.e. the minimum epoch height that must be reached by the network before the stake account balance is available for withdrawal, unless the transaction is also signed by a specified custodian. This information is gathered when the stake account is created, and stored in the Lockup field of the stake account's state. Changing the authorized staker or withdrawer is also subject to lock-up, as such an operation is effectively a transfer.
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Synchronization
|
||||
---
|
||||
title: Synchronization
|
||||
---
|
||||
|
||||
Fast, reliable synchronization is the biggest reason Solana is able to achieve such high throughput. Traditional blockchains synchronize on large chunks of transactions called blocks. By synchronizing on blocks, a transaction cannot be processed until a duration called "block time" has passed. In Proof of Work consensus, these block times need to be very large \(~10 minutes\) to minimize the odds of multiple validators producing a new valid block at the same time. There's no such constraint in Proof of Stake consensus, but without reliable timestamps, a validator cannot determine the order of incoming blocks. The popular workaround is to tag each block with a [wallclock timestamp](https://en.bitcoin.it/wiki/Block_timestamp). Because of clock drift and variance in network latencies, the timestamp is only accurate within an hour or two. To workaround the workaround, these systems lengthen block times to provide reasonable certainty that the median timestamp on each block is always increasing.
|
||||
|
||||
@@ -22,6 +24,5 @@ Proof of History is not a consensus mechanism, but it is used to improve the per
|
||||
|
||||
## More on Proof of History
|
||||
|
||||
* [water clock analogy](https://medium.com/solana-labs/proof-of-history-explained-by-a-water-clock-e682183417b8)
|
||||
* [Proof of History overview](https://medium.com/solana-labs/proof-of-history-a-clock-for-blockchain-cf47a61a9274)
|
||||
|
||||
- [water clock analogy](https://medium.com/solana-labs/proof-of-history-explained-by-a-water-clock-e682183417b8)
|
||||
- [Proof of History overview](https://medium.com/solana-labs/proof-of-history-a-clock-for-blockchain-cf47a61a9274)
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Turbine Block Propagation
|
||||
---
|
||||
title: Turbine Block Propagation
|
||||
---
|
||||
|
||||
A Solana cluster uses a multi-layer block propagation mechanism called _Turbine_ to broadcast transaction shreds to all nodes with minimal amount of duplicate messages. The cluster divides itself into small collections of nodes, called _neighborhoods_. Each node is responsible for sharing any data it receives with the other nodes in its neighborhood, as well as propagating the data on to a small set of nodes in other neighborhoods. This way each node only has to communicate with a small number of nodes.
|
||||
|
||||
@@ -20,15 +22,15 @@ This way each node only has to communicate with a maximum of `2 * DATA_PLANE_FAN
|
||||
|
||||
The following diagram shows how the Leader sends shreds with a Fanout of 2 to Neighborhood 0 in Layer 0 and how the nodes in Neighborhood 0 share their data with each other.
|
||||
|
||||
|
||||
|
||||
|
||||
The following diagram shows how Neighborhood 0 fans out to Neighborhoods 1 and 2.
|
||||
|
||||
|
||||
|
||||
|
||||
Finally, the following diagram shows a two layer cluster with a Fanout of 2.
|
||||
|
||||
|
||||
|
||||
|
||||
### Configuration Values
|
||||
|
||||
@@ -38,59 +40,62 @@ Currently, configuration is set when the cluster is launched. In the future, the
|
||||
|
||||
## Calcuating the required FEC rate
|
||||
|
||||
Turbine relies on retransmission of packets between validators. Due to
|
||||
Turbine relies on retransmission of packets between validators. Due to
|
||||
retransmission, any network wide packet loss is compounded, and the
|
||||
probability of the packet failing to reach is destination increases
|
||||
on each hop. The FEC rate needs to take into account the network wide
|
||||
on each hop. The FEC rate needs to take into account the network wide
|
||||
packet loss, and the propagation depth.
|
||||
|
||||
A shred group is the set of data and coding packets that can be used
|
||||
to reconstruct each other. Each shred group has a chance of failure,
|
||||
to reconstruct each other. Each shred group has a chance of failure,
|
||||
based on the likelyhood of the number of packets failing that exceeds
|
||||
the FEC rate. If a validator fails to reconstruct the shred group,
|
||||
then the block cannot be reconstructed, and the validator has to rely
|
||||
on repair to fixup the blocks.
|
||||
|
||||
The probability of the shred group failing can be computed using the
|
||||
binomial distribution. If the FEC rate is `16:4`, then the group size
|
||||
binomial distribution. If the FEC rate is `16:4`, then the group size
|
||||
is 20, and at least 4 of the shreds must fail for the group to fail.
|
||||
Which is equal to the sum of the probability of 4 or more trails failing
|
||||
out of 20.
|
||||
|
||||
Probability of a block succeeding in turbine:
|
||||
|
||||
* Probability of packet failure: `P = 1 - (1 - network_packet_loss_rate)^2`
|
||||
* FEC rate: `K:M`
|
||||
* Number of trials: `N = K + M`
|
||||
* Shred group failure rate: `S = SUM of i=0 -> M for binomial(prob_failure = P, trials = N, failures = i)`
|
||||
* Shreds per block: `G`
|
||||
* Block success rate: `B = (1 - S) ^ (G / N) `
|
||||
* Binomial distribution for exactly `i` results with probability of P in N trials is defined as `(N choose i) * P^i * (1 - P)^(N-i)`
|
||||
- Probability of packet failure: `P = 1 - (1 - network_packet_loss_rate)^2`
|
||||
- FEC rate: `K:M`
|
||||
- Number of trials: `N = K + M`
|
||||
- Shred group failure rate: `S = SUM of i=0 -> M for binomial(prob_failure = P, trials = N, failures = i)`
|
||||
- Shreds per block: `G`
|
||||
- Block success rate: `B = (1 - S) ^ (G / N)`
|
||||
- Binomial distribution for exactly `i` results with probability of P in N trials is defined as `(N choose i) * P^i * (1 - P)^(N-i)`
|
||||
|
||||
For example:
|
||||
|
||||
* Network packet loss rate is 15%.
|
||||
* 50kpts network generates 6400 shreds per second.
|
||||
* FEC rate increases the total shres per block by the FEC ratio.
|
||||
- Network packet loss rate is 15%.
|
||||
- 50kpts network generates 6400 shreds per second.
|
||||
- FEC rate increases the total shres per block by the FEC ratio.
|
||||
|
||||
With a FEC rate: `16:4`
|
||||
* `G = 8000`
|
||||
* `P = 1 - 0.85 * 0.85 = 1 - 0.7225 = 0.2775`
|
||||
* `S = SUM of i=0 -> 4 for binomial(prob_failure = 0.2775, trials = 20, failures = i) = 0.689414`
|
||||
* `B = (1 - 0.689) ^ (8000 / 20) = 10^-203`
|
||||
|
||||
- `G = 8000`
|
||||
- `P = 1 - 0.85 * 0.85 = 1 - 0.7225 = 0.2775`
|
||||
- `S = SUM of i=0 -> 4 for binomial(prob_failure = 0.2775, trials = 20, failures = i) = 0.689414`
|
||||
- `B = (1 - 0.689) ^ (8000 / 20) = 10^-203`
|
||||
|
||||
With FEC rate of `16:16`
|
||||
* `G = 12800`
|
||||
* `S = SUM of i=0 -> 32 for binomial(prob_failure = 0.2775, trials = 64, failures = i) = 0.002132`
|
||||
* `B = (1 - 0.002132) ^ (12800 / 32) = 0.42583`
|
||||
|
||||
- `G = 12800`
|
||||
- `S = SUM of i=0 -> 32 for binomial(prob_failure = 0.2775, trials = 64, failures = i) = 0.002132`
|
||||
- `B = (1 - 0.002132) ^ (12800 / 32) = 0.42583`
|
||||
|
||||
With FEC rate of `32:32`
|
||||
* `G = 12800`
|
||||
* `S = SUM of i=0 -> 32 for binomial(prob_failure = 0.2775, trials = 64, failures = i) = 0.000048`
|
||||
* `B = (1 - 0.000048) ^ (12800 / 64) = 0.99045`
|
||||
|
||||
- `G = 12800`
|
||||
- `S = SUM of i=0 -> 32 for binomial(prob_failure = 0.2775, trials = 64, failures = i) = 0.000048`
|
||||
- `B = (1 - 0.000048) ^ (12800 / 64) = 0.99045`
|
||||
|
||||
## Neighborhoods
|
||||
|
||||
The following diagram shows how two neighborhoods in different layers interact. To cripple a neighborhood, enough nodes \(erasure codes +1\) from the neighborhood above need to fail. Since each neighborhood receives shreds from multiple nodes in a neighborhood in the upper layer, we'd need a big network failure in the upper layers to end up with incomplete data.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Secure Vote Signing
|
||||
---
|
||||
title: Secure Vote Signing
|
||||
---
|
||||
|
||||
A validator receives entries from the current leader and submits votes confirming those entries are valid. This vote submission presents a security challenge, because forged votes that violate consensus rules could be used to slash the validator's stake.
|
||||
|
||||
@@ -20,30 +22,30 @@ Currently, there is a 1:1 relationship between validators and vote signers, and
|
||||
|
||||
The vote signing service consists of a JSON RPC server and a request processor. At startup, the service starts the RPC server at a configured port and waits for validator requests. It expects the following type of requests: 1. Register a new validator node
|
||||
|
||||
* The request must contain validator's identity \(public key\)
|
||||
* The request must be signed with the validator's private key
|
||||
* The service drops the request if signature of the request cannot be
|
||||
- The request must contain validator's identity \(public key\)
|
||||
- The request must be signed with the validator's private key
|
||||
- The service drops the request if signature of the request cannot be
|
||||
|
||||
verified
|
||||
|
||||
* The service creates a new voting asymmetric key for the validator, and
|
||||
- The service creates a new voting asymmetric key for the validator, and
|
||||
|
||||
returns the public key as a response
|
||||
|
||||
* If a validator tries to register again, the service returns the public key
|
||||
- If a validator tries to register again, the service returns the public key
|
||||
|
||||
from the pre-existing keypair
|
||||
|
||||
1. Sign a vote
|
||||
|
||||
* The request must contain a voting transaction and all verification data
|
||||
* The request must be signed with the validator's private key
|
||||
* The service drops the request if signature of the request cannot be
|
||||
- The request must contain a voting transaction and all verification data
|
||||
- The request must be signed with the validator's private key
|
||||
- The service drops the request if signature of the request cannot be
|
||||
|
||||
verified
|
||||
|
||||
* The service verifies the voting data
|
||||
* The service returns a signature for the transaction
|
||||
- The service verifies the voting data
|
||||
- The service returns a signature for the transaction
|
||||
|
||||
## Validator voting
|
||||
|
||||
@@ -64,4 +66,3 @@ The validator looks up the votes submitted by all the nodes in the cluster for t
|
||||
### New Vote Signing
|
||||
|
||||
The validator creates a "new vote" transaction and sends it to the signing service using JSON RPC. The RPC request also includes the vote verification data. On success, the RPC call returns the signature for the vote. On failure, RPC call returns the failure code.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user