Bump version to 1.0.3

automerge

.gitbook.yaml

@@ -1,4 +1,4 @@
-root: ./book/src
+root: ./docs/src
 
 structure:
     readme: introduction.md

.gitignore

@@ -1,6 +1,6 @@
-/book/html/
+/docs/html/
-/book/src/tests.ok
+/docs/src/tests.ok
-/book/src/.gitbook/assets/*.svg
+/docs/src/.gitbook/assets/*.svg
 /farf/
 /solana-release/
 /solana-release.tar.bz2

CONTRIBUTING.md

@@ -45,7 +45,7 @@ $ git pull --rebase upstream master
 
 If there are no functional changes, PRs can be very large and that's no
 problem. If, however, your changes are making meaningful changes or additions,
-then about 1,000 lines of changes is about the most you should ask a Solana
+then about 1.0.3 lines of changes is about the most you should ask a Solana
 maintainer to review.
 
 ### Should I send small PRs as I develop large, new components?
@@ -224,21 +224,20 @@ Inventing new terms is allowed, but should only be done when the term is widely
 used and understood. Avoid introducing new 3-letter terms, which can be
 confused with 3-letter acronyms.
 
-[Terms currently in use](book/src/terminology.md)
+[Terms currently in use](docs/src/terminology.md)
 
 
 ## Design Proposals
 
-Solana's architecture is described by a book generated from markdown files in
+Solana's architecture is described by docs generated from markdown files in
-the `book/src/` directory, maintained by an *editor* (currently @garious). To
+the `docs/src/` directory, maintained by an *editor* (currently @garious). To
-add a design proposal, you'll need to at least propose a change the content
+add a design proposal, you'll need to include it in the
-under the [Accepted Design
+[Accepted Design Proposals](https://docs.solana.com/proposals)
-Proposals](https://docs.solana.com/book/v/master/proposals) chapter.  Here's
+section of the Solana docs.  Here's the full process:
-the full process:
 
 1. Propose a design by creating a PR that adds a markdown document to the
-   directory `book/src/` and references it from the [table of
+   `docs/src/proposals` directory and references it from the [table of
-   contents](book/src/SUMMARY.md). Add any relevant *maintainers* to the PR
+   contents](docs/src/SUMMARY.md). Add any relevant *maintainers* to the PR
    review.
 2. The PR being merged indicates your proposed change was accepted and that the
    maintainers support your plan of attack.

README.md

@@ -23,12 +23,12 @@ It's possible for a centralized database to process 710,000 transactions per sec
 
 Furthermore, and much to our surprise, it can be implemented using a mechanism that has existed in Bitcoin since day one. The Bitcoin feature is called nLocktime and it can be used to postdate transactions using block height instead of a timestamp. As a Bitcoin client, you'd use block height instead of a timestamp if you don't trust the network. Block height turns out to be an instance of what's being called a Verifiable Delay Function in cryptography circles. It's a cryptographically secure way to say time has passed. In Solana, we use a far more granular verifiable delay function, a SHA 256 hash chain, to checkpoint the ledger and coordinate consensus. With it, we implement Optimistic Concurrency Control and are now well en route towards that theoretical limit of 710,000 transactions per second.
 
-Architecture
+Documentation
 ===
 
-Before you jump into the code, review the online book [Solana: Blockchain Rebuilt for Scale](https://docs.solana.com/book/).
+Before you jump into the code, review the documentation [Solana: Blockchain Rebuilt for Scale](https://docs.solana.com).
 
-(The _latest_ development version of the online book is also [available here](https://docs.solana.com/book/v/master/).)
+(The _latest_ development version of the docs is [available here](https://docs.solana.com/v/master).)
 
 Release Binaries
 ===
@@ -121,7 +121,7 @@ $ cargo test
 Local Testnet
 ---
 
-Start your own testnet locally, instructions are in the book [Solana: Blockchain Rebuild for Scale: Getting Started](https://docs.solana.com/book/building-from-source).
+Start your own testnet locally, instructions are in the online docs [Solana: Blockchain Rebuild for Scale: Getting Started](https://docs.solana.com/building-from-source).
 
 Remote Testnets
 ---

RELEASE.md

@@ -138,7 +138,7 @@ There are three release channels that map to branches as follows:
 ### Update documentation
 TODO: Documentation update procedure is WIP as we move to gitbook
 
-Document the new recommended version by updating `book/src/running-archiver.md` and `book/src/validator-testnet.md` on the release (beta) branch to point at the `solana-install` for the upcoming release version.
+Document the new recommended version by updating `docs/src/running-archiver.md` and `docs/src/validator-testnet.md` on the release (beta) branch to point at the `solana-install` for the upcoming release version.
 
 ### Update software on devnet.solana.com
 

archiver-lib/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-archiver-lib"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana Archiver Library"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"
@@ -15,22 +15,22 @@ ed25519-dalek = "=1.0.0-pre.1"
 log = "0.4.8"
 rand = "0.6.5"
 rand_chacha = "0.1.1"
-solana-client = { path = "../client", version = "1.0.0" }
+solana-client = { path = "../client", version = "1.0.3" }
-solana-storage-program = { path = "../programs/storage", version = "1.0.0" }
+solana-storage-program = { path = "../programs/storage", version = "1.0.3" }
 thiserror = "1.0"
 serde = "1.0.104"
 serde_json = "1.0.46"
 serde_derive = "1.0.103"
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }
-solana-chacha = { path = "../chacha", version = "1.0.0" }
+solana-chacha = { path = "../chacha", version = "1.0.3" }
-solana-chacha-sys = { path = "../chacha-sys", version = "1.0.0" }
+solana-chacha-sys = { path = "../chacha-sys", version = "1.0.3" }
-solana-ledger = { path = "../ledger", version = "1.0.0" }
+solana-ledger = { path = "../ledger", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-perf = { path = "../perf", version = "1.0.0" }
+solana-perf = { path = "../perf", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
-solana-core = { path = "../core", version = "1.0.0" }
+solana-core = { path = "../core", version = "1.0.3" }
-solana-archiver-utils = { path = "../archiver-utils", version = "1.0.0" }
+solana-archiver-utils = { path = "../archiver-utils", version = "1.0.3" }
-solana-metrics = { path = "../metrics", version = "1.0.0" }
+solana-metrics = { path = "../metrics", version = "1.0.3" }
 
 [dev-dependencies]
 hex = "0.4.0"

archiver-lib/src/archiver.rs

@@ -47,7 +47,7 @@ use solana_storage_program::{
 };
 use std::{
     io::{self, ErrorKind},
-    net::{SocketAddr, UdpSocket},
+    net::{IpAddr, Ipv4Addr, SocketAddr, UdpSocket},
     path::{Path, PathBuf},
     result,
     sync::atomic::{AtomicBool, Ordering},
@@ -211,12 +211,9 @@ impl Archiver {
         let client = solana_core::gossip_service::get_client(&nodes);
 
         info!("Setting up mining account...");
-        if let Err(e) = Self::setup_mining_account(
+        if let Err(e) =
-            &client,
+            Self::setup_mining_account(&client, &keypair, &storage_keypair, client_commitment)
-            &keypair,
+        {
-            &storage_keypair,
-            client_commitment.clone(),
-        ) {
             //shutdown services before exiting
             exit.store(true, Ordering::Relaxed);
             gossip_service.join()?;
@@ -358,7 +355,7 @@ impl Archiver {
                 &cluster_info,
                 archiver_keypair,
                 storage_keypair,
-                meta.client_commitment.clone(),
+                meta.client_commitment,
             );
         }
         exit.store(true, Ordering::Relaxed);
@@ -374,7 +371,7 @@ impl Archiver {
         let client = solana_core::gossip_service::get_client(&nodes);
 
         if let Ok(Some(account)) =
-            client.get_account_with_commitment(&storage_keypair.pubkey(), client_commitment.clone())
+            client.get_account_with_commitment(&storage_keypair.pubkey(), client_commitment)
         {
             if let Ok(StorageContract::ArchiverStorage { validations, .. }) = account.state() {
                 if !validations.is_empty() {
@@ -415,7 +412,7 @@ impl Archiver {
         slot_sender: Sender<u64>,
     ) -> Result<WindowService> {
         let slots_per_segment =
-            match Self::get_segment_config(&cluster_info, meta.client_commitment.clone()) {
+            match Self::get_segment_config(&cluster_info, meta.client_commitment) {
                 Ok(slots_per_segment) => slots_per_segment,
                 Err(e) => {
                     error!("unable to get segment size configuration, exiting...");
@@ -581,7 +578,7 @@ impl Archiver {
             &keypair.pubkey(),
             &Duration::from_millis(100),
             &Duration::from_secs(5),
-            client_commitment.clone(),
+            client_commitment,
         )? == 0
         {
             return Err(ArchiverError::EmptyStorageAccountBalance);
@@ -589,16 +586,15 @@ impl Archiver {
 
         info!("checking storage account keypair...");
         // check if the storage account exists
-        let balance = client
+        let balance =
-            .poll_get_balance_with_commitment(&storage_keypair.pubkey(), client_commitment.clone());
+            client.poll_get_balance_with_commitment(&storage_keypair.pubkey(), client_commitment);
         if balance.is_err() || balance.unwrap() == 0 {
-            let blockhash =
+            let blockhash = match client.get_recent_blockhash_with_commitment(client_commitment) {
-                match client.get_recent_blockhash_with_commitment(client_commitment.clone()) {
+                Ok((blockhash, _)) => blockhash,
-                    Ok((blockhash, _)) => blockhash,
+                Err(e) => {
-                    Err(e) => {
+                    return Err(ArchiverError::TransportError(e));
-                        return Err(ArchiverError::TransportError(e));
+                }
-                    }
+            };
-                };
 
             let ix = storage_instruction::create_storage_account(
                 &keypair.pubkey(),
@@ -631,32 +627,27 @@ impl Archiver {
         // No point if we've got no storage account...
         let nodes = cluster_info.read().unwrap().tvu_peers();
         let client = solana_core::gossip_service::get_client(&nodes);
-        let storage_balance = client.poll_get_balance_with_commitment(
+        let storage_balance = client
-            &storage_keypair.pubkey(),
+            .poll_get_balance_with_commitment(&storage_keypair.pubkey(), meta.client_commitment);
-            meta.client_commitment.clone(),
-        );
         if storage_balance.is_err() || storage_balance.unwrap() == 0 {
             error!("Unable to submit mining proof, no storage account");
             return;
         }
         // ...or no lamports for fees
-        let balance = client.poll_get_balance_with_commitment(
+        let balance = client
-            &archiver_keypair.pubkey(),
+            .poll_get_balance_with_commitment(&archiver_keypair.pubkey(), meta.client_commitment);
-            meta.client_commitment.clone(),
-        );
         if balance.is_err() || balance.unwrap() == 0 {
             error!("Unable to submit mining proof, insufficient Archiver Account balance");
             return;
         }
 
-        let blockhash =
+        let blockhash = match client.get_recent_blockhash_with_commitment(meta.client_commitment) {
-            match client.get_recent_blockhash_with_commitment(meta.client_commitment.clone()) {
+            Ok((blockhash, _)) => blockhash,
-                Ok((blockhash, _)) => blockhash,
+            Err(_) => {
-                Err(_) => {
+                error!("unable to get recent blockhash, can't submit proof");
-                    error!("unable to get recent blockhash, can't submit proof");
+                return;
-                    return;
+            }
-                }
+        };
-            };
         let instruction = storage_instruction::mining_proof(
             &storage_keypair.pubkey(),
             meta.sha_state,
@@ -813,14 +804,15 @@ impl Archiver {
         blockstore: &Arc<Blockstore>,
         slots_per_segment: u64,
     ) -> Result<u64> {
+        let ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
         // Create a client which downloads from the archiver and see that it
         // can respond with shreds.
-        let start_slot = Self::get_archiver_segment_slot(archiver_info.storage_addr);
+        let start_slot = Self::get_archiver_segment_slot(ip_addr, archiver_info.storage_addr);
         info!("Archiver download: start at {}", start_slot);
 
         let exit = Arc::new(AtomicBool::new(false));
         let (s_reader, r_reader) = channel();
-        let repair_socket = Arc::new(bind_in_range(VALIDATOR_PORT_RANGE).unwrap().1);
+        let repair_socket = Arc::new(bind_in_range(ip_addr, VALIDATOR_PORT_RANGE).unwrap().1);
         let t_receiver = receiver(
             repair_socket.clone(),
             &exit,
@@ -917,8 +909,8 @@ impl Archiver {
         true
     }
 
-    fn get_archiver_segment_slot(to: SocketAddr) -> u64 {
+    fn get_archiver_segment_slot(bind_ip_addr: IpAddr, to: SocketAddr) -> u64 {
-        let (_port, socket) = bind_in_range(VALIDATOR_PORT_RANGE).unwrap();
+        let (_port, socket) = bind_in_range(bind_ip_addr, VALIDATOR_PORT_RANGE).unwrap();
         socket
             .set_read_timeout(Some(Duration::from_secs(5)))
             .unwrap();

archiver-utils/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-archiver-utils"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana Archiver Utils"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"
@@ -11,12 +11,12 @@ edition = "2018"
 [dependencies]
 log = "0.4.8"
 rand = "0.6.5"
-solana-chacha = { path = "../chacha", version = "1.0.0" }
+solana-chacha = { path = "../chacha", version = "1.0.3" }
-solana-chacha-sys = { path = "../chacha-sys", version = "1.0.0" }
+solana-chacha-sys = { path = "../chacha-sys", version = "1.0.3" }
-solana-ledger = { path = "../ledger", version = "1.0.0" }
+solana-ledger = { path = "../ledger", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-perf = { path = "../perf", version = "1.0.0" }
+solana-perf = { path = "../perf", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 
 [dev-dependencies]
 hex = "0.4.0"

archiver/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-archiver"
-version = "1.0.0"
+version = "1.0.3"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -10,11 +10,11 @@ homepage = "https://solana.com/"
 [dependencies]
 clap = "2.33.0"
 console = "0.9.2"
-solana-clap-utils = { path = "../clap-utils", version = "1.0.0" }
+solana-clap-utils = { path = "../clap-utils", version = "1.0.3" }
-solana-core = { path = "../core", version = "1.0.0" }
+solana-core = { path = "../core", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-metrics = { path = "../metrics", version = "1.0.0" }
+solana-metrics = { path = "../metrics", version = "1.0.3" }
-solana-archiver-lib = { path = "../archiver-lib", version = "1.0.0" }
+solana-archiver-lib = { path = "../archiver-lib", version = "1.0.3" }
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 

archiver/src/main.rs

@@ -13,7 +13,12 @@ use solana_core::{
     contact_info::ContactInfo,
 };
 use solana_sdk::{commitment_config::CommitmentConfig, signature::Signer};
-use std::{net::SocketAddr, path::PathBuf, process::exit, sync::Arc};
+use std::{
+    net::{IpAddr, Ipv4Addr, SocketAddr},
+    path::PathBuf,
+    process::exit,
+    sync::Arc,
+};
 
 fn main() {
     solana_logger::setup();
@@ -116,6 +121,7 @@ fn main() {
         &identity_keypair.pubkey(),
         &gossip_addr,
         VALIDATOR_PORT_RANGE,
+        IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),
     );
 
     println!(

banking-bench/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-banking-bench"
-version = "1.0.0"
+version = "1.0.3"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -10,11 +10,11 @@ homepage = "https://solana.com/"
 [dependencies]
 log = "0.4.6"
 rayon = "1.2.0"
-solana-core = { path = "../core", version = "1.0.0" }
+solana-core = { path = "../core", version = "1.0.3" }
-solana-ledger = { path = "../ledger", version = "1.0.0" }
+solana-ledger = { path = "../ledger", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-runtime = { path = "../runtime", version = "1.0.0" }
+solana-runtime = { path = "../runtime", version = "1.0.3" }
-solana-measure = { path = "../measure", version = "1.0.0" }
+solana-measure = { path = "../measure", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 rand = "0.6.5"
 crossbeam-channel = "0.3"

bench-exchange/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-bench-exchange"
-version = "1.0.0"
+version = "1.0.3"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -18,17 +18,17 @@ rand = "0.6.5"
 rayon = "1.2.0"
 serde_json = "1.0.46"
 serde_yaml = "0.8.11"
-solana-clap-utils = { path = "../clap-utils", version = "1.0.0" }
+solana-clap-utils = { path = "../clap-utils", version = "1.0.3" }
-solana-core = { path = "../core", version = "1.0.0" }
+solana-core = { path = "../core", version = "1.0.3" }
-solana-genesis = { path = "../genesis", version = "1.0.0" }
+solana-genesis = { path = "../genesis", version = "1.0.3" }
-solana-client = { path = "../client", version = "1.0.0" }
+solana-client = { path = "../client", version = "1.0.3" }
-solana-faucet = { path = "../faucet", version = "1.0.0" }
+solana-faucet = { path = "../faucet", version = "1.0.3" }
-solana-exchange-program = { path = "../programs/exchange", version = "1.0.0" }
+solana-exchange-program = { path = "../programs/exchange", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-metrics = { path = "../metrics", version = "1.0.0" }
+solana-metrics = { path = "../metrics", version = "1.0.3" }
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }
-solana-runtime = { path = "../runtime", version = "1.0.0" }
+solana-runtime = { path = "../runtime", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 
 [dev-dependencies]
-solana-local-cluster = { path = "../local-cluster", version = "1.0.0" }
+solana-local-cluster = { path = "../local-cluster", version = "1.0.3" }

bench-streamer/Cargo.toml

@@ -2,14 +2,14 @@
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-bench-streamer"
-version = "1.0.0"
+version = "1.0.3"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
 
 [dependencies]
 clap = "2.33.0"
-solana-clap-utils = { path = "../clap-utils", version = "1.0.0" }
+solana-clap-utils = { path = "../clap-utils", version = "1.0.3" }
-solana-core = { path = "../core", version = "1.0.0" }
+solana-core = { path = "../core", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }

bench-streamer/src/main.rs

@@ -67,7 +67,8 @@ fn main() -> Result<()> {
     }
 
     let mut port = 0;
-    let mut addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), 0);
+    let ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
+    let mut addr = SocketAddr::new(ip_addr, 0);
 
     let exit = Arc::new(AtomicBool::new(false));
 
@@ -75,7 +76,7 @@ fn main() -> Result<()> {
     let mut read_threads = Vec::new();
     let recycler = PacketsRecycler::default();
     for _ in 0..num_sockets {
-        let read = solana_net_utils::bind_to(port, false).unwrap();
+        let read = solana_net_utils::bind_to(ip_addr, port, false).unwrap();
         read.set_read_timeout(Some(Duration::new(1, 0))).unwrap();
 
         addr = read.local_addr().unwrap();

bench-tps/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-bench-tps"
-version = "1.0.0"
+version = "1.0.3"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -14,24 +14,24 @@ log = "0.4.8"
 rayon = "1.2.0"
 serde_json = "1.0.46"
 serde_yaml = "0.8.11"
-solana-clap-utils = { path = "../clap-utils", version = "1.0.0" }
+solana-clap-utils = { path = "../clap-utils", version = "1.0.3" }
-solana-core = { path = "../core", version = "1.0.0" }
+solana-core = { path = "../core", version = "1.0.3" }
-solana-genesis = { path = "../genesis", version = "1.0.0" }
+solana-genesis = { path = "../genesis", version = "1.0.3" }
-solana-client = { path = "../client", version = "1.0.0" }
+solana-client = { path = "../client", version = "1.0.3" }
-solana-faucet = { path = "../faucet", version = "1.0.0" }
+solana-faucet = { path = "../faucet", version = "1.0.3" }
-solana-librapay = { path = "../programs/librapay", version = "1.0.0", optional = true }
+solana-librapay = { path = "../programs/librapay", version = "1.0.3", optional = true }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-metrics = { path = "../metrics", version = "1.0.0" }
+solana-metrics = { path = "../metrics", version = "1.0.3" }
-solana-measure = { path = "../measure", version = "1.0.0" }
+solana-measure = { path = "../measure", version = "1.0.3" }
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }
-solana-runtime = { path = "../runtime", version = "1.0.0" }
+solana-runtime = { path = "../runtime", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
-solana-move-loader-program = { path = "../programs/move_loader", version = "1.0.0", optional = true }
+solana-move-loader-program = { path = "../programs/move_loader", version = "1.0.3", optional = true }
 
 [dev-dependencies]
 serial_test = "0.3.2"
 serial_test_derive = "0.4.0"
-solana-local-cluster = { path = "../local-cluster", version = "1.0.0" }
+solana-local-cluster = { path = "../local-cluster", version = "1.0.3" }
 
 [features]
 move = ["solana-librapay", "solana-move-loader-program"]

bench-tps/src/bench.rs

@@ -1059,8 +1059,8 @@ pub fn generate_and_fund_keypairs<T: 'static + Client + Send + Sync>(
     //   pay for the transaction fees in a new run.
     let enough_lamports = 8 * lamports_per_account / 10;
     if first_keypair_balance < enough_lamports || last_keypair_balance < enough_lamports {
-        let (_blockhash, fee_calculator) = get_recent_blockhash(client.as_ref());
+        let fee_rate_governor = client.get_fee_rate_governor().unwrap();
-        let max_fee = fee_calculator.max_lamports_per_signature;
+        let max_fee = fee_rate_governor.max_lamports_per_signature;
         let extra_fees = extra * max_fee;
         let total_keypairs = keypairs.len() as u64 + 1; // Add one for funding keypair
         let mut total = lamports_per_account * total_keypairs + extra_fees;
@@ -1134,7 +1134,7 @@ mod tests {
     use solana_runtime::bank::Bank;
     use solana_runtime::bank_client::BankClient;
     use solana_sdk::client::SyncClient;
-    use solana_sdk::fee_calculator::FeeCalculator;
+    use solana_sdk::fee_calculator::FeeRateGovernor;
     use solana_sdk::genesis_config::create_genesis_config;
 
     #[test]
@@ -1181,8 +1181,8 @@ mod tests {
     #[test]
     fn test_bench_tps_fund_keys_with_fees() {
         let (mut genesis_config, id) = create_genesis_config(10_000);
-        let fee_calculator = FeeCalculator::new(11, 0);
+        let fee_rate_governor = FeeRateGovernor::new(11, 0);
-        genesis_config.fee_calculator = fee_calculator;
+        genesis_config.fee_rate_governor = fee_rate_governor;
         let bank = Bank::new(&genesis_config);
         let client = Arc::new(BankClient::new(bank));
         let keypair_count = 20;

bench-tps/src/cli.rs

@@ -1,6 +1,6 @@
 use clap::{crate_description, crate_name, App, Arg, ArgMatches};
 use solana_faucet::faucet::FAUCET_PORT;
-use solana_sdk::fee_calculator::FeeCalculator;
+use solana_sdk::fee_calculator::FeeRateGovernor;
 use solana_sdk::signature::{read_keypair_file, Keypair};
 use std::{net::SocketAddr, process::exit, time::Duration};
 
@@ -43,7 +43,7 @@ impl Default for Config {
             client_ids_and_stake_file: String::new(),
             write_to_client_file: false,
             read_from_client_file: false,
-            target_lamports_per_signature: FeeCalculator::default().target_lamports_per_signature,
+            target_lamports_per_signature: FeeRateGovernor::default().target_lamports_per_signature,
             multi_client: true,
             use_move: false,
             num_lamports_per_account: NUM_LAMPORTS_PER_ACCOUNT_DEFAULT,

bench-tps/src/main.rs

@@ -3,7 +3,7 @@ use solana_bench_tps::bench::{do_bench_tps, generate_and_fund_keypairs, generate
 use solana_bench_tps::cli;
 use solana_core::gossip_service::{discover_cluster, get_client, get_multi_client};
 use solana_genesis::Base64Account;
-use solana_sdk::fee_calculator::FeeCalculator;
+use solana_sdk::fee_calculator::FeeRateGovernor;
 use solana_sdk::signature::{Keypair, Signer};
 use solana_sdk::system_program;
 use std::{collections::HashMap, fs::File, io::prelude::*, path::Path, process::exit, sync::Arc};
@@ -41,7 +41,7 @@ fn main() {
         let (keypairs, _) = generate_keypairs(&id, keypair_count as u64);
         let num_accounts = keypairs.len() as u64;
         let max_fee =
-            FeeCalculator::new(*target_lamports_per_signature, 0).max_lamports_per_signature;
+            FeeRateGovernor::new(*target_lamports_per_signature, 0).max_lamports_per_signature;
         let num_lamports_per_account = (num_accounts - 1 + NUM_SIGNATURES_FOR_TXS * max_fee)
             / num_accounts
             + num_lamports_per_account;

book/src/implemented-proposals/ed_overview/ed_validation_client_economics/ed_vce_replication_validation_transaction_fees.md

@@ -1,11 +0,0 @@
-# Replication-validation Transaction Fees
-
-**Subject to change.**
-
-As previously mentioned, validator-clients will also be responsible for validating PoReps submitted into the PoH stream by archiver-clients. In this case, validators are providing compute \(CPU/GPU\) and light storage resources to confirm that these replication proofs could only be generated by a client that is storing the referenced PoH leger block.
-
-While replication-clients are incentivized and rewarded through protocol-based rewards schedule \(see [Replication-client Economics](../ed_replication_client_economics/)\), validator-clients will be incentivized to include and validate PoReps in PoH through collection of transaction fees associated with the submitted PoReps and distribution of protocol rewards proportional to the validated PoReps. As will be described in detail in the Section 3.1, replication-client rewards are protocol-based and designed to reward based on a global data redundancy factor. I.e. the protocol will incentivize replication-client participation through rewards based on a target ledger redundancy \(e.g. 10x data redundancy\).
-
-The validation of PoReps by validation-clients is computationally more expensive than state-validation \(detail in the [Economic Sustainability](../ed_economic_sustainability.md) chapter\), thus the transaction fees are expected to be proportionally higher.
-
-There are various attack vectors available for colluding validation and replication clients, also described in detail below in [Economic Sustainability](../ed_economic_sustainability/README.md). To protect against various collusion attack vectors, for a given epoch, validator rewards are distributed across participating validation-clients in proportion to the number of validated PoReps in the epoch less the number of PoReps that mismatch the archivers challenge. The PoRep challenge game is described in [Ledger Replication](https://github.com/solana-labs/solana/blob/master/book/src/ledger-replication.md#the-porep-game). This design rewards validators proportional to the number of PoReps they process and validate, while providing negative pressure for validation-clients to submit lazy or malicious invalid votes on submitted PoReps \(note that it is computationally prohibitive to determine whether a validator-client has marked a valid PoRep as invalid\).

book/src/paper-wallet/installation.md

@@ -1,51 +0,0 @@
-# Installation Guide
-Follow this guide to setup Solana's key generation tool called `solana-keygen`
-
-{% hint style="warn" %}
-After installation, ensure your version is `0.23.1` or higher by running `solana-keygen -V`
-{% endhint %}
-
-## Download
-First, download the latest release tarball from GitHub.
-
-1. Setup download url
-
-  ```bash
-  solana_downloads=https://github.com/solana-labs/solana/releases/latest/download
-  ```
-
-2. Specify the download file based on your machine
-
-  **MacOS**
-  ```bash
-  solana_release=solana-release-x86_64-apple-darwin.tar.bz2
-  ```
-
-  **Linux**
-  ```bash
-  solana_release=solana-release-x86_64-unknown-linux-gnu.tar.bz2
-  ```
-
-3. Download
-
-  ```bash
-  curl -L -sSf -o solana-release.tar.bz2 $solana_downloads/$solana_release
-  ```
-
-## Extract
-Next, extract the tarball
-```bash
-tar xf solana-release.tar.bz2
-```
-
-## Add to "PATH"
-Now add the tool to your PATH environment variable with the following command
-```bash
-export PATH="$(pwd)/solana-release/bin:${PATH}"
-```
-
-## Check
-Finally, check that `solana-keygen` can be run by running
-```bash
-solana-keygen -V
-```

chacha-cuda/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-chacha-cuda"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana Chacha Cuda APIs"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,12 +10,12 @@ edition = "2018"
 
 [dependencies]
 log = "0.4.8"
-solana-archiver-utils = { path = "../archiver-utils", version = "1.0.0" }
+solana-archiver-utils = { path = "../archiver-utils", version = "1.0.3" }
-solana-chacha = { path = "../chacha", version = "1.0.0" }
+solana-chacha = { path = "../chacha", version = "1.0.3" }
-solana-ledger = { path = "../ledger", version = "1.0.0" }
+solana-ledger = { path = "../ledger", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-perf = { path = "../perf", version = "1.0.0" }
+solana-perf = { path = "../perf", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 
 [dev-dependencies]
 hex-literal = "0.2.1"

chacha-sys/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-chacha-sys"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana chacha-sys"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"

chacha/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-chacha"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana Chacha APIs"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"
@@ -12,11 +12,11 @@ edition = "2018"
 log = "0.4.8"
 rand = "0.6.5"
 rand_chacha = "0.1.1"
-solana-chacha-sys = { path = "../chacha-sys", version = "1.0.0" }
+solana-chacha-sys = { path = "../chacha-sys", version = "1.0.3" }
-solana-ledger = { path = "../ledger", version = "1.0.0" }
+solana-ledger = { path = "../ledger", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-perf = { path = "../perf", version = "1.0.0" }
+solana-perf = { path = "../perf", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 
 [dev-dependencies]
 hex-literal = "0.2.1"

ci/affects-files.sh

@@ -9,7 +9,7 @@
 #     ./affects-files.sh ^snap/  -- anything under the snap/ subdirectory
 #     ./affects-files.sh snap/   -- also matches foo/snap/
 # Any pattern starting with the ! character will be negated:
-#     ./affects-files.sh !^book/  -- anything *not* under the book/ subdirectory
+#     ./affects-files.sh !^docs/  -- anything *not* under the docs/ subdirectory
 #
 set -e
 cd "$(dirname "$0")"/..

ci/buildkite-secondary.yml

@@ -6,7 +6,7 @@ steps:
     timeout_in_minutes: 60
     name: "publish docker"
   - command: "ci/publish-crate.sh"
-    timeout_in_minutes: 120
+    timeout_in_minutes: 240
     name: "publish crate"
     branches: "!master"
   - command: "ci/publish-bpf-sdk.sh"
@@ -15,6 +15,6 @@ steps:
   - command: "ci/publish-tarball.sh"
     timeout_in_minutes: 60
     name: "publish tarball"
-  - command: "ci/publish-book.sh"
+  - command: "ci/publish-docs.sh"
     timeout_in_minutes: 15
-    name: "publish book"
+    name: "publish docs"

ci/docker-rust-nightly/Dockerfile

@@ -1,4 +1,4 @@
-FROM solanalabs/rust:1.41.0
+FROM solanalabs/rust:1.41.1
 ARG date
 
 RUN set -x \

ci/docker-rust/Dockerfile

@@ -1,6 +1,6 @@
 # Note: when the rust version is changed also modify
 # ci/rust-version.sh to pick up the new image tag
-FROM rust:1.41.0
+FROM rust:1.41.1
 
 # Add Google Protocol Buffers for Libra's metrics library.
 ENV PROTOC_VERSION 3.8.0

ci/publish-docs.sh

@@ -11,10 +11,10 @@ if [[ -n $CI_BRANCH ]]; then
     set -x
     (
       . ci/rust-version.sh stable
-      ci/docker-run.sh "$rust_stable_docker_image" make -Cbook -B svg
+      ci/docker-run.sh "$rust_stable_docker_image" make -C docs
     )
     # make a local commit for the svgs
-    git add -A -f book/src/.gitbook/assets/.
+    git add -A -f docs/src/.gitbook/assets/.
     if ! git diff-index --quiet HEAD; then
       git config user.email maintainers@solana.com
       git config user.name "$me"

ci/rust-version.sh

@@ -16,13 +16,13 @@
 if [[ -n $RUST_STABLE_VERSION ]]; then
   stable_version="$RUST_STABLE_VERSION"
 else
-  stable_version=1.41.0
+  stable_version=1.41.1
 fi
 
 if [[ -n $RUST_NIGHTLY_VERSION ]]; then
   nightly_version="$RUST_NIGHTLY_VERSION"
 else
-  nightly_version=2020-02-06
+  nightly_version=2020-02-27
 fi
 
 

ci/test-checks.sh

@@ -25,7 +25,7 @@ _ cargo +"$rust_stable" audit --version
 _ cargo +"$rust_stable" audit --ignore RUSTSEC-2020-0002
 _ ci/nits.sh
 _ ci/order-crates-for-publishing.py
-_ book/build.sh
+_ docs/build.sh
 _ ci/check-ssh-keys.sh
 
 {

ci/test-stable.sh

@@ -13,12 +13,12 @@ annotate() {
 # Run the appropriate test based on entrypoint
 testName=$(basename "$0" .sh)
 
-# Skip if only the book has been modified
+# Skip if only the docs have been modified
 ci/affects-files.sh \
-  \!^book/ \
+  \!^docs/ \
 || {
   annotate --style info \
-    "Skipped $testName as only book files were modified"
+    "Skipped $testName as only docs/ files were modified"
   exit 0
 }
 

clap-utils/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-clap-utils"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana utilities for the clap"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"
@@ -11,8 +11,8 @@ edition = "2018"
 [dependencies]
 clap = "2.33.0"
 rpassword = "4.0"
-solana-remote-wallet = { path = "../remote-wallet", version = "1.0.0" }
+solana-remote-wallet = { path = "../remote-wallet", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 tiny-bip39 = "0.7.0"
 url = "2.1.0"
 chrono = "0.4"

clap-utils/src/input_parsers.rs

@@ -119,8 +119,8 @@ pub fn derivation_of(matches: &ArgMatches<'_>, name: &str) -> Option<DerivationP
     matches.value_of(name).map(|derivation_str| {
         let derivation_str = derivation_str.replace("'", "");
         let mut parts = derivation_str.split('/');
-        let account = parts.next().unwrap().parse::<u16>().unwrap();
+        let account = parts.next().map(|account| account.parse::<u32>().unwrap());
-        let change = parts.next().map(|change| change.parse::<u16>().unwrap());
+        let change = parts.next().map(|change| change.parse::<u32>().unwrap());
         DerivationPath { account, change }
     })
 }
@@ -308,7 +308,7 @@ mod tests {
         assert_eq!(
             derivation_of(&matches, "single"),
             Some(DerivationPath {
-                account: 2,
+                account: Some(2),
                 change: Some(3)
             })
         );
@@ -319,7 +319,7 @@ mod tests {
         assert_eq!(
             derivation_of(&matches, "single"),
             Some(DerivationPath {
-                account: 2,
+                account: Some(2),
                 change: None
             })
         );
@@ -330,7 +330,7 @@ mod tests {
         assert_eq!(
             derivation_of(&matches, "single"),
             Some(DerivationPath {
-                account: 2,
+                account: Some(2),
                 change: Some(3)
             })
         );

clap-utils/src/input_validators.rs

@@ -1,6 +1,7 @@
 use crate::keypair::{parse_keypair_path, KeypairUrl, ASK_KEYWORD};
 use chrono::DateTime;
 use solana_sdk::{
+    clock::Slot,
     hash::Hash,
     pubkey::Pubkey,
     signature::{read_keypair_file, Signature},
@@ -93,6 +94,12 @@ pub fn is_url(string: String) -> Result<(), String> {
     }
 }
 
+pub fn is_slot(slot: String) -> Result<(), String> {
+    slot.parse::<Slot>()
+        .map(|_| ())
+        .map_err(|e| format!("{:?}", e))
+}
+
 pub fn is_port(port: String) -> Result<(), String> {
     port.parse::<u16>()
         .map(|_| ())
@@ -142,7 +149,7 @@ pub fn is_derivation(value: String) -> Result<(), String> {
     let mut parts = value.split('/');
     let account = parts.next().unwrap();
     account
-        .parse::<u16>()
+        .parse::<u32>()
         .map_err(|e| {
             format!(
                 "Unable to parse derivation, provided: {}, err: {:?}",
@@ -151,7 +158,7 @@ pub fn is_derivation(value: String) -> Result<(), String> {
         })
         .and_then(|_| {
             if let Some(change) = parts.next() {
-                change.parse::<u16>().map_err(|e| {
+                change.parse::<u32>().map_err(|e| {
                     format!(
                         "Unable to parse derivation, provided: {}, err: {:?}",
                         change, e
@@ -172,11 +179,12 @@ mod tests {
     fn test_is_derivation() {
         assert_eq!(is_derivation("2".to_string()), Ok(()));
         assert_eq!(is_derivation("0".to_string()), Ok(()));
+        assert_eq!(is_derivation("65537".to_string()), Ok(()));
         assert_eq!(is_derivation("0/2".to_string()), Ok(()));
         assert_eq!(is_derivation("0'/2'".to_string()), Ok(()));
         assert!(is_derivation("a".to_string()).is_err());
-        assert!(is_derivation("65537".to_string()).is_err());
+        assert!(is_derivation("4294967296".to_string()).is_err());
         assert!(is_derivation("a/b".to_string()).is_err());
-        assert!(is_derivation("0/65537".to_string()).is_err());
+        assert!(is_derivation("0/4294967296".to_string()).is_err());
     }
 }

clap-utils/src/keypair.rs

@@ -39,7 +39,7 @@ pub fn parse_keypair_path(path: &str) -> KeypairUrl {
     } else if path == ASK_KEYWORD {
         KeypairUrl::Ask
     } else if path.starts_with("usb://") {
-        KeypairUrl::Usb(path.split_at(6).1.to_string())
+        KeypairUrl::Usb(path.to_string())
     } else if let Ok(pubkey) = Pubkey::from_str(path) {
         KeypairUrl::Pubkey(pubkey)
     } else {
@@ -86,6 +86,7 @@ pub fn signer_from_path(
                     path,
                     derivation_of(matches, "derivation_path"),
                     wallet_manager,
+                    matches.is_present("confirm_key"),
                 )?))
             } else {
                 Err(RemoteWalletError::NoDeviceFound.into())

cli-config/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-cli-config"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.0.0"
+version = "1.0.3"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"

cli-config/src/config.rs

@@ -18,13 +18,15 @@ lazy_static! {
 #[derive(Serialize, Deserialize, Default, Debug, PartialEq)]
 pub struct Config {
     pub url: String,
+    pub websocket_url: String,
     pub keypair_path: String,
 }
 
 impl Config {
-    pub fn new(url: &str, keypair_path: &str) -> Self {
+    pub fn new(url: &str, websocket_url: &str, keypair_path: &str) -> Self {
         Self {
             url: url.to_string(),
+            websocket_url: websocket_url.to_string(),
             keypair_path: keypair_path.to_string(),
         }
     }

cli/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-cli"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.0.0"
+version = "1.0.3"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -26,27 +26,27 @@ reqwest = { version = "0.10.1", default-features = false, features = ["blocking"
 serde = "1.0.104"
 serde_derive = "1.0.103"
 serde_json = "1.0.46"
-solana-budget-program = { path = "../programs/budget", version = "1.0.0" }
+solana-budget-program = { path = "../programs/budget", version = "1.0.3" }
-solana-clap-utils = { path = "../clap-utils", version = "1.0.0" }
+solana-clap-utils = { path = "../clap-utils", version = "1.0.3" }
-solana-cli-config = { path = "../cli-config", version = "1.0.0" }
+solana-cli-config = { path = "../cli-config", version = "1.0.3" }
-solana-client = { path = "../client", version = "1.0.0" }
+solana-client = { path = "../client", version = "1.0.3" }
-solana-config-program = { path = "../programs/config", version = "1.0.0" }
+solana-config-program = { path = "../programs/config", version = "1.0.3" }
-solana-faucet = { path = "../faucet", version = "1.0.0" }
+solana-faucet = { path = "../faucet", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }
-solana-remote-wallet = { path = "../remote-wallet", version = "1.0.0" }
+solana-remote-wallet = { path = "../remote-wallet", version = "1.0.3" }
-solana-runtime = { path = "../runtime", version = "1.0.0" }
+solana-runtime = { path = "../runtime", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
-solana-stake-program = { path = "../programs/stake", version = "1.0.0" }
+solana-stake-program = { path = "../programs/stake", version = "1.0.3" }
-solana-storage-program = { path = "../programs/storage", version = "1.0.0" }
+solana-storage-program = { path = "../programs/storage", version = "1.0.3" }
-solana-vote-program = { path = "../programs/vote", version = "1.0.0" }
+solana-vote-program = { path = "../programs/vote", version = "1.0.3" }
-solana-vote-signer = { path = "../vote-signer", version = "1.0.0" }
+solana-vote-signer = { path = "../vote-signer", version = "1.0.3" }
 titlecase = "1.1.0"
 url = "2.1.1"
 
 [dev-dependencies]
-solana-core = { path = "../core", version = "1.0.0" }
+solana-core = { path = "../core", version = "1.0.3" }
-solana-budget-program = { path = "../programs/budget", version = "1.0.0" }
+solana-budget-program = { path = "../programs/budget", version = "1.0.3" }
 tempfile = "3.1.0"
 
 [[bin]]

cli/src/cli.rs

@@ -40,7 +40,10 @@ use solana_sdk::{
     system_instruction::{self, create_address_with_seed, SystemError, MAX_ADDRESS_SEED_LEN},
     transaction::{Transaction, TransactionError},
 };
-use solana_stake_program::stake_state::{Lockup, StakeAuthorize};
+use solana_stake_program::{
+    stake_instruction::LockupArgs,
+    stake_state::{Lockup, StakeAuthorize},
+};
 use solana_storage_program::storage_instruction::StorageAccountType;
 use solana_vote_program::vote_state::VoteAuthorize;
 use std::{
@@ -52,6 +55,7 @@ use std::{
     time::Duration,
     {error, fmt},
 };
+use url::Url;
 
 pub type CliSigners = Vec<Box<dyn Signer>>;
 pub type SignerIndex = usize;
@@ -163,6 +167,7 @@ pub enum CliCommand {
     // Cluster Query Commands
     Catchup {
         node_pubkey: Pubkey,
+        node_json_rpc_url: Option<String>,
     },
     ClusterVersion,
     CreateAddressWithSeed {
@@ -185,9 +190,7 @@ pub enum CliCommand {
         commitment_config: CommitmentConfig,
     },
     LeaderSchedule,
-    LiveSlots {
+    LiveSlots,
-        url: String,
-    },
     Ping {
         lamports: u64,
         interval: Duration,
@@ -206,6 +209,7 @@ pub enum CliCommand {
     },
     ShowValidators {
         use_lamports_unit: bool,
+        commitment_config: CommitmentConfig,
     },
     // Nonce commands
     AuthorizeNonceAccount {
@@ -303,7 +307,7 @@ pub enum CliCommand {
     },
     StakeSetLockup {
         stake_account_pubkey: Pubkey,
-        lockup: Lockup,
+        lockup: LockupArgs,
         custodian: SignerIndex,
         sign_only: bool,
         blockhash_query: BlockhashQuery,
@@ -351,6 +355,7 @@ pub enum CliCommand {
     ShowVoteAccount {
         pubkey: Pubkey,
         use_lamports_unit: bool,
+        commitment_config: CommitmentConfig,
     },
     VoteAuthorize {
         vote_account_pubkey: Pubkey,
@@ -435,9 +440,16 @@ impl From<Box<dyn error::Error>> for CliError {
     }
 }
 
+pub enum SettingType {
+    Explicit,
+    Computed,
+    SystemDefault,
+}
+
 pub struct CliConfig<'a> {
     pub command: CliCommand,
     pub json_rpc_url: String,
+    pub websocket_url: String,
     pub signers: Vec<&'a dyn Signer>,
     pub keypair_path: String,
     pub derivation_path: Option<DerivationPath>,
@@ -446,16 +458,97 @@ pub struct CliConfig<'a> {
 }
 
 impl CliConfig<'_> {
-    pub fn default_keypair_path() -> String {
+    fn default_keypair_path() -> String {
         let mut keypair_path = dirs::home_dir().expect("home directory");
         keypair_path.extend(&[".config", "solana", "id.json"]);
         keypair_path.to_str().unwrap().to_string()
     }
 
-    pub fn default_json_rpc_url() -> String {
+    fn default_json_rpc_url() -> String {
         "http://127.0.0.1:8899".to_string()
     }
 
+    fn default_websocket_url() -> String {
+        Self::compute_ws_url(&Self::default_json_rpc_url())
+    }
+
+    fn compute_ws_url(rpc_url: &str) -> String {
+        let rpc_url: Option<Url> = rpc_url.parse().ok();
+        if rpc_url.is_none() {
+            return "".to_string();
+        }
+        let rpc_url = rpc_url.unwrap();
+        let is_secure = rpc_url.scheme().to_ascii_lowercase() == "https";
+        let mut ws_url = rpc_url.clone();
+        ws_url
+            .set_scheme(if is_secure { "wss" } else { "ws" })
+            .expect("unable to set scheme");
+        let ws_port = match rpc_url.port() {
+            Some(port) => port + 1,
+            None => {
+                if is_secure {
+                    8901
+                } else {
+                    8900
+                }
+            }
+        };
+        ws_url.set_port(Some(ws_port)).expect("unable to set port");
+        ws_url.to_string()
+    }
+
+    fn first_nonempty_setting(
+        settings: std::vec::Vec<(SettingType, String)>,
+    ) -> (SettingType, String) {
+        settings
+            .into_iter()
+            .find(|(_, value)| value != "")
+            .expect("no nonempty setting")
+    }
+
+    pub fn compute_websocket_url_setting(
+        websocket_cmd_url: &str,
+        websocket_cfg_url: &str,
+        json_rpc_cmd_url: &str,
+        json_rpc_cfg_url: &str,
+    ) -> (SettingType, String) {
+        Self::first_nonempty_setting(vec![
+            (SettingType::Explicit, websocket_cmd_url.to_string()),
+            (SettingType::Explicit, websocket_cfg_url.to_string()),
+            (
+                SettingType::Computed,
+                Self::compute_ws_url(json_rpc_cmd_url),
+            ),
+            (
+                SettingType::Computed,
+                Self::compute_ws_url(json_rpc_cfg_url),
+            ),
+            (SettingType::SystemDefault, Self::default_websocket_url()),
+        ])
+    }
+
+    pub fn compute_json_rpc_url_setting(
+        json_rpc_cmd_url: &str,
+        json_rpc_cfg_url: &str,
+    ) -> (SettingType, String) {
+        Self::first_nonempty_setting(vec![
+            (SettingType::Explicit, json_rpc_cmd_url.to_string()),
+            (SettingType::Explicit, json_rpc_cfg_url.to_string()),
+            (SettingType::SystemDefault, Self::default_json_rpc_url()),
+        ])
+    }
+
+    pub fn compute_keypair_path_setting(
+        keypair_cmd_path: &str,
+        keypair_cfg_path: &str,
+    ) -> (SettingType, String) {
+        Self::first_nonempty_setting(vec![
+            (SettingType::Explicit, keypair_cmd_path.to_string()),
+            (SettingType::Explicit, keypair_cfg_path.to_string()),
+            (SettingType::SystemDefault, Self::default_keypair_path()),
+        ])
+    }
+
     pub(crate) fn pubkey(&self) -> Result<Pubkey, SignerError> {
         if !self.signers.is_empty() {
             self.signers[0].try_pubkey()
@@ -475,6 +568,7 @@ impl Default for CliConfig<'_> {
                 use_lamports_unit: false,
             },
             json_rpc_url: Self::default_json_rpc_url(),
+            websocket_url: Self::default_websocket_url(),
             signers: Vec::new(),
             keypair_path: Self::default_keypair_path(),
             derivation_path: None,
@@ -514,7 +608,10 @@ pub fn parse_command(
             signers: vec![],
         }),
         ("ping", Some(matches)) => parse_cluster_ping(matches, default_signer_path, wallet_manager),
-        ("live-slots", Some(matches)) => parse_live_slots(matches),
+        ("live-slots", Some(_matches)) => Ok(CliCommandInfo {
+            command: CliCommand::LiveSlots,
+            signers: vec![],
+        }),
         ("block-production", Some(matches)) => parse_show_block_production(matches),
         ("gossip", Some(_matches)) => Ok(CliCommandInfo {
             command: CliCommand::ShowGossip,
@@ -620,7 +717,7 @@ pub fn parse_command(
         ),
         ("vote-account", Some(matches)) => parse_vote_get_account_command(matches),
         // Wallet Commands
-        ("address", Some(_matches)) => Ok(CliCommandInfo {
+        ("address", Some(matches)) => Ok(CliCommandInfo {
             command: CliCommand::Address,
             signers: vec![signer_from_path(
                 matches,
@@ -1454,7 +1551,10 @@ pub fn process_command(config: &CliConfig) -> ProcessResult {
         CliCommand::Address => Ok(format!("{}", config.pubkey()?)),
 
         // Return software version of solana-cli and cluster entrypoint node
-        CliCommand::Catchup { node_pubkey } => process_catchup(&rpc_client, node_pubkey),
+        CliCommand::Catchup {
+            node_pubkey,
+            node_json_rpc_url,
+        } => process_catchup(&rpc_client, node_pubkey, node_json_rpc_url),
         CliCommand::ClusterVersion => process_cluster_version(&rpc_client),
         CliCommand::CreateAddressWithSeed {
             from_pubkey,
@@ -1465,16 +1565,16 @@ pub fn process_command(config: &CliConfig) -> ProcessResult {
         CliCommand::GetBlockTime { slot } => process_get_block_time(&rpc_client, *slot),
         CliCommand::GetGenesisHash => process_get_genesis_hash(&rpc_client),
         CliCommand::GetEpochInfo { commitment_config } => {
-            process_get_epoch_info(&rpc_client, commitment_config)
+            process_get_epoch_info(&rpc_client, *commitment_config)
         }
         CliCommand::GetSlot { commitment_config } => {
-            process_get_slot(&rpc_client, commitment_config)
+            process_get_slot(&rpc_client, *commitment_config)
         }
         CliCommand::GetTransactionCount { commitment_config } => {
-            process_get_transaction_count(&rpc_client, commitment_config)
+            process_get_transaction_count(&rpc_client, *commitment_config)
         }
         CliCommand::LeaderSchedule => process_leader_schedule(&rpc_client),
-        CliCommand::LiveSlots { url } => process_live_slots(&url),
+        CliCommand::LiveSlots => process_live_slots(&config.websocket_url),
         CliCommand::Ping {
             lamports,
             interval,
@@ -1488,7 +1588,7 @@ pub fn process_command(config: &CliConfig) -> ProcessResult {
             interval,
             count,
             timeout,
-            commitment_config,
+            *commitment_config,
         ),
         CliCommand::ShowBlockProduction { epoch, slot_limit } => {
             process_show_block_production(&rpc_client, config, *epoch, *slot_limit)
@@ -1502,9 +1602,10 @@ pub fn process_command(config: &CliConfig) -> ProcessResult {
             *use_lamports_unit,
             vote_account_pubkeys.as_deref(),
         ),
-        CliCommand::ShowValidators { use_lamports_unit } => {
+        CliCommand::ShowValidators {
-            process_show_validators(&rpc_client, *use_lamports_unit)
+            use_lamports_unit,
-        }
+            commitment_config,
+        } => process_show_validators(&rpc_client, *use_lamports_unit, *commitment_config),
 
         // Nonce Commands
 
@@ -1816,11 +1917,13 @@ pub fn process_command(config: &CliConfig) -> ProcessResult {
         CliCommand::ShowVoteAccount {
             pubkey: vote_account_pubkey,
             use_lamports_unit,
+            commitment_config,
         } => process_show_vote_account(
             &rpc_client,
             config,
             &vote_account_pubkey,
             *use_lamports_unit,
+            *commitment_config,
         ),
         CliCommand::VoteAuthorize {
             vote_account_pubkey,
@@ -2064,7 +2167,16 @@ pub fn app<'ab, 'v>(name: &str, about: &'ab str, version: &'v str) -> App<'ab, '
         .about(about)
         .version(version)
         .setting(AppSettings::SubcommandRequiredElseHelp)
-        .subcommand(SubCommand::with_name("address").about("Get your public key"))
+        .subcommand(
+            SubCommand::with_name("address")
+                .about("Get your public key")
+                .arg(
+                    Arg::with_name("confirm_key")
+                        .long("confirm-key")
+                        .takes_value(false)
+                        .help("Confirm key on device; only relevant if using remote wallet"),
+                ),
+        )
         .cluster_query_subcommands()
         .nonce_subcommands()
         .stake_subcommands()
@@ -2374,7 +2486,7 @@ mod tests {
     };
     use solana_sdk::{
         account::Account,
-        nonce_state::{Meta as NonceMeta, NonceState},
+        nonce,
         pubkey::Pubkey,
         signature::{keypair_from_seed, read_keypair_file, write_keypair_file, Presigner},
         system_program,
@@ -3153,18 +3265,16 @@ mod tests {
 
         // Nonced pay
         let blockhash = Hash::default();
+        let data =
+            nonce::state::Versions::new_current(nonce::State::Initialized(nonce::state::Data {
+                authority: config.signers[0].pubkey(),
+                blockhash,
+                fee_calculator: FeeCalculator::default(),
+            }));
         let nonce_response = json!(Response {
             context: RpcResponseContext { slot: 1 },
             value: json!(RpcAccount::encode(
-                Account::new_data(
+                Account::new_data(1, &data, &system_program::ID,).unwrap()
-                    1,
-                    &NonceState::Initialized(
-                        NonceMeta::new(&config.signers[0].pubkey()),
-                        blockhash
-                    ),
-                    &system_program::ID,
-                )
-                .unwrap()
             )),
         });
         let mut mocks = HashMap::new();
@@ -3184,15 +3294,16 @@ mod tests {
         let bob_keypair = Keypair::new();
         let bob_pubkey = bob_keypair.pubkey();
         let blockhash = Hash::default();
+        let data =
+            nonce::state::Versions::new_current(nonce::State::Initialized(nonce::state::Data {
+                authority: bob_pubkey,
+                blockhash,
+                fee_calculator: FeeCalculator::default(),
+            }));
         let nonce_authority_response = json!(Response {
             context: RpcResponseContext { slot: 1 },
             value: json!(RpcAccount::encode(
-                Account::new_data(
+                Account::new_data(1, &data, &system_program::ID,).unwrap()
-                    1,
-                    &NonceState::Initialized(NonceMeta::new(&bob_pubkey), blockhash),
-                    &system_program::ID,
-                )
-                .unwrap()
             )),
         });
         let mut mocks = HashMap::new();

cli/src/cluster_query.rs

@@ -59,6 +59,14 @@ impl ClusterQuerySubCommands for App<'_, '_> {
                         .validator(is_pubkey_or_keypair)
                         .required(true)
                         .help("Identity pubkey of the validator"),
+                )
+                .arg(
+                    Arg::with_name("node_json_rpc_url")
+                        .index(2)
+                        .value_name("URL")
+                        .takes_value(true)
+                        .validator(is_url)
+                        .help("JSON RPC URL for validator, which is useful for validators with a private RPC service")
                 ),
         )
         .subcommand(
@@ -170,16 +178,7 @@ impl ClusterQuerySubCommands for App<'_, '_> {
         )
         .subcommand(
             SubCommand::with_name("live-slots")
-                .about("Show information about the current slot progression")
+                .about("Show information about the current slot progression"),
-                .arg(
-                    Arg::with_name("websocket_url")
-                        .short("w")
-                        .long("ws")
-                        .value_name("URL")
-                        .takes_value(true)
-                        .default_value("ws://127.0.0.1:8900")
-                        .help("WebSocket URL for PubSub RPC connection"),
-                ),
         )
         .subcommand(
             SubCommand::with_name("block-production")
@@ -226,6 +225,14 @@ impl ClusterQuerySubCommands for App<'_, '_> {
             SubCommand::with_name("validators")
                 .about("Show summary information about the current validators")
                 .alias("show-validators")
+                .arg(
+                    Arg::with_name("confirmed")
+                        .long("confirmed")
+                        .takes_value(false)
+                        .help(
+                            "Return information at maximum-lockout commitment level",
+                        ),
+                )
                 .arg(
                     Arg::with_name("lamports")
                         .long("lamports")
@@ -238,8 +245,12 @@ impl ClusterQuerySubCommands for App<'_, '_> {
 
 pub fn parse_catchup(matches: &ArgMatches<'_>) -> Result<CliCommandInfo, CliError> {
     let node_pubkey = pubkey_of(matches, "node_pubkey").unwrap();
+    let node_json_rpc_url = value_t!(matches, "node_json_rpc_url", String).ok();
     Ok(CliCommandInfo {
-        command: CliCommand::Catchup { node_pubkey },
+        command: CliCommand::Catchup {
+            node_pubkey,
+            node_json_rpc_url,
+        },
         signers: vec![],
     })
 }
@@ -279,14 +290,6 @@ pub fn parse_cluster_ping(
     })
 }
 
-pub fn parse_live_slots(matches: &ArgMatches<'_>) -> Result<CliCommandInfo, CliError> {
-    let url: String = value_t_or_exit!(matches, "websocket_url", String);
-    Ok(CliCommandInfo {
-        command: CliCommand::LiveSlots { url },
-        signers: vec![],
-    })
-}
-
 pub fn parse_get_block_time(matches: &ArgMatches<'_>) -> Result<CliCommandInfo, CliError> {
     let slot = value_t_or_exit!(matches, "slot", u64);
     Ok(CliCommandInfo {
@@ -346,9 +349,17 @@ pub fn parse_show_stakes(matches: &ArgMatches<'_>) -> Result<CliCommandInfo, Cli
 
 pub fn parse_show_validators(matches: &ArgMatches<'_>) -> Result<CliCommandInfo, CliError> {
     let use_lamports_unit = matches.is_present("lamports");
+    let commitment_config = if matches.is_present("confirmed") {
+        CommitmentConfig::default()
+    } else {
+        CommitmentConfig::recent()
+    };
 
     Ok(CliCommandInfo {
-        command: CliCommand::ShowValidators { use_lamports_unit },
+        command: CliCommand::ShowValidators {
+            use_lamports_unit,
+            commitment_config,
+        },
         signers: vec![],
     })
 }
@@ -362,20 +373,42 @@ fn new_spinner_progress_bar() -> ProgressBar {
     progress_bar
 }
 
-pub fn process_catchup(rpc_client: &RpcClient, node_pubkey: &Pubkey) -> ProcessResult {
+pub fn process_catchup(
+    rpc_client: &RpcClient,
+    node_pubkey: &Pubkey,
+    node_json_rpc_url: &Option<String>,
+) -> ProcessResult {
     let cluster_nodes = rpc_client.get_cluster_nodes()?;
 
-    let rpc_addr = cluster_nodes
+    let node_client = if let Some(node_json_rpc_url) = node_json_rpc_url {
-        .iter()
+        RpcClient::new(node_json_rpc_url.to_string())
-        .find(|contact_info| contact_info.pubkey == node_pubkey.to_string())
+    } else {
-        .ok_or_else(|| format!("Contact information not found for {}", node_pubkey))?
+        RpcClient::new_socket(
-        .rpc
+            cluster_nodes
-        .ok_or_else(|| format!("RPC service not found for {}", node_pubkey))?;
+                .iter()
+                .find(|contact_info| contact_info.pubkey == node_pubkey.to_string())
+                .ok_or_else(|| format!("Contact information not found for {}", node_pubkey))?
+                .rpc
+                .ok_or_else(|| format!("RPC service not found for {}", node_pubkey))?,
+        )
+    };
+
+    let reported_node_pubkey = node_client.get_identity()?;
+    if reported_node_pubkey != *node_pubkey {
+        return Err(format!(
+            "The identity reported by node RPC URL does not match.  Expected: {:?}.  Reported: {:?}",
+            node_pubkey, reported_node_pubkey
+        )
+        .into());
+    }
+
+    if rpc_client.get_identity()? == *node_pubkey {
+        return Err("Both RPC URLs reference the same node, unable to monitor for catchup.  Try a different --url".into());
+    }
 
     let progress_bar = new_spinner_progress_bar();
     progress_bar.set_message("Connecting...");
 
-    let node_client = RpcClient::new_socket(rpc_addr);
     let mut previous_rpc_slot = std::u64::MAX;
     let mut previous_slot_distance = 0;
     let sleep_interval = 5;
@@ -483,7 +516,7 @@ fn slot_to_human_time(slot: Slot) -> String {
 
 pub fn process_get_epoch_info(
     rpc_client: &RpcClient,
-    commitment_config: &CommitmentConfig,
+    commitment_config: CommitmentConfig,
 ) -> ProcessResult {
     let epoch_info = rpc_client.get_epoch_info_with_commitment(commitment_config.clone())?;
     println!();
@@ -529,7 +562,7 @@ pub fn process_get_genesis_hash(rpc_client: &RpcClient) -> ProcessResult {
 
 pub fn process_get_slot(
     rpc_client: &RpcClient,
-    commitment_config: &CommitmentConfig,
+    commitment_config: CommitmentConfig,
 ) -> ProcessResult {
     let slot = rpc_client.get_slot_with_commitment(commitment_config.clone())?;
     Ok(slot.to_string())
@@ -718,7 +751,7 @@ pub fn process_show_block_production(
 
 pub fn process_get_transaction_count(
     rpc_client: &RpcClient,
-    commitment_config: &CommitmentConfig,
+    commitment_config: CommitmentConfig,
 ) -> ProcessResult {
     let transaction_count =
         rpc_client.get_transaction_count_with_commitment(commitment_config.clone())?;
@@ -732,7 +765,7 @@ pub fn process_ping(
     interval: &Duration,
     count: &Option<u64>,
     timeout: &Duration,
-    commitment_config: &CommitmentConfig,
+    commitment_config: CommitmentConfig,
 ) -> ProcessResult {
     let to = Keypair::new().pubkey();
 
@@ -855,11 +888,15 @@ pub fn process_ping(
 
 pub fn process_live_slots(url: &str) -> ProcessResult {
     let exit = Arc::new(AtomicBool::new(false));
-    let exit_clone = exit.clone();
 
+    // Disable Ctrl+C handler as sometimes the PubsubClient shutdown can stall.  Also it doesn't
+    // really matter that the shutdown is clean because the process is terminating.
+    /*
+    let exit_clone = exit.clone();
     ctrlc::set_handler(move || {
         exit_clone.store(true, Ordering::Relaxed);
     })?;
+    */
 
     let mut current: Option<SlotInfoMessage> = None;
     let mut message = "".to_string();
@@ -869,6 +906,12 @@ pub fn process_live_slots(url: &str) -> ProcessResult {
     let (mut client, receiver) = PubsubClient::slot_subscribe(url)?;
     slot_progress.set_message("Connected.");
 
+    let spacer = "|";
+    slot_progress.println(spacer);
+
+    let mut last_root = std::u64::MAX;
+    let mut last_root_update = Instant::now();
+    let mut slots_per_second = std::f64::NAN;
     loop {
         if exit.load(Ordering::Relaxed) {
             eprintln!("{}", message);
@@ -878,7 +921,27 @@ pub fn process_live_slots(url: &str) -> ProcessResult {
 
         match receiver.recv() {
             Ok(new_info) => {
-                message = format!("{:?}", new_info).to_owned();
+                if last_root == std::u64::MAX {
+                    last_root = new_info.root;
+                    last_root_update = Instant::now();
+                }
+                if last_root_update.elapsed().as_secs() >= 5 {
+                    let root = new_info.root;
+                    slots_per_second =
+                        (root - last_root) as f64 / last_root_update.elapsed().as_secs() as f64;
+                    last_root_update = Instant::now();
+                    last_root = root;
+                }
+
+                message = if slots_per_second.is_nan() {
+                    format!("{:?}", new_info)
+                } else {
+                    format!(
+                        "{:?} | root slot advancing at {:.2} slots/second",
+                        new_info, slots_per_second
+                    )
+                }
+                .to_owned();
                 slot_progress.set_message(&message);
 
                 if let Some(previous) = current {
@@ -891,11 +954,19 @@ pub fn process_live_slots(url: &str) -> ProcessResult {
                     //
                     if slot_delta != root_delta {
                         let prev_root = format!(
-                            "|<- {} <- … <- {} <- {}",
+                            "|<--- {} <- … <- {} <- {}   (prev)",
                             previous.root, previous.parent, previous.slot
-                        )
+                        );
-                        .to_owned();
                         slot_progress.println(&prev_root);
+
+                        let new_root = format!(
+                            "|  '- {} <- … <- {} <- {}   (next)",
+                            new_info.root, new_info.parent, new_info.slot
+                        );
+
+                        slot_progress.println(prev_root);
+                        slot_progress.println(new_root);
+                        slot_progress.println(spacer);
                     }
                 }
                 current = Some(new_info);
@@ -985,9 +1056,13 @@ pub fn process_show_stakes(
     Ok("".to_string())
 }
 
-pub fn process_show_validators(rpc_client: &RpcClient, use_lamports_unit: bool) -> ProcessResult {
+pub fn process_show_validators(
-    let epoch_info = rpc_client.get_epoch_info()?;
+    rpc_client: &RpcClient,
-    let vote_accounts = rpc_client.get_vote_accounts()?;
+    use_lamports_unit: bool,
+    commitment_config: CommitmentConfig,
+) -> ProcessResult {
+    let epoch_info = rpc_client.get_epoch_info_with_commitment(commitment_config)?;
+    let vote_accounts = rpc_client.get_vote_accounts_with_commitment(commitment_config)?;
     let total_active_stake = vote_accounts
         .current
         .iter()

cli/src/display.rs

@@ -1,3 +1,4 @@
+use crate::cli::SettingType;
 use console::style;
 use solana_sdk::transaction::Transaction;
 
@@ -11,17 +12,19 @@ pub fn println_name_value(name: &str, value: &str) {
     println!("{} {}", style(name).bold(), styled_value);
 }
 
-pub fn println_name_value_or(name: &str, value: &str, default_value: &str) {
+pub fn println_name_value_or(name: &str, value: &str, setting_type: SettingType) {
-    if value == "" {
+    let description = match setting_type {
-        println!(
+        SettingType::Explicit => "",
-            "{} {} {}",
+        SettingType::Computed => "(computed)",
-            style(name).bold(),
+        SettingType::SystemDefault => "(default)",
-            style(default_value),
-            style("(default)").italic()
-        );
-    } else {
-        println!("{} {}", style(name).bold(), style(value));
     };
+
+    println!(
+        "{} {} {}",
+        style(name).bold(),
+        style(value),
+        style(description).italic(),
+    );
 }
 
 pub fn println_signers(tx: &Transaction) {

cli/src/main.rs

@@ -20,29 +20,31 @@ fn parse_settings(matches: &ArgMatches<'_>) -> Result<bool, Box<dyn error::Error
             ("get", Some(subcommand_matches)) => {
                 if let Some(config_file) = matches.value_of("config_file") {
                     let config = Config::load(config_file).unwrap_or_default();
+
+                    let (url_setting_type, json_rpc_url) =
+                        CliConfig::compute_json_rpc_url_setting("", &config.url);
+                    let (ws_setting_type, websocket_url) = CliConfig::compute_websocket_url_setting(
+                        "",
+                        &config.websocket_url,
+                        "",
+                        &config.url,
+                    );
+                    let (keypair_setting_type, keypair_path) =
+                        CliConfig::compute_keypair_path_setting("", &config.keypair_path);
+
                     if let Some(field) = subcommand_matches.value_of("specific_setting") {
-                        let (field_name, value, default_value) = match field {
+                        let (field_name, value, setting_type) = match field {
-                            "url" => ("RPC URL", config.url, CliConfig::default_json_rpc_url()),
+                            "json_rpc_url" => ("RPC URL", json_rpc_url, url_setting_type),
-                            "keypair" => (
+                            "websocket_url" => ("WS URL", websocket_url, ws_setting_type),
-                                "Key Path",
+                            "keypair" => ("Key Path", keypair_path, keypair_setting_type),
-                                config.keypair_path,
-                                CliConfig::default_keypair_path(),
-                            ),
                             _ => unreachable!(),
                         };
-                        println_name_value_or(&format!("{}:", field_name), &value, &default_value);
+                        println_name_value_or(&format!("{}:", field_name), &value, setting_type);
                     } else {
                         println_name_value("Config File:", config_file);
-                        println_name_value_or(
+                        println_name_value_or("RPC URL:", &json_rpc_url, url_setting_type);
-                            "RPC URL:",
+                        println_name_value_or("WS URL:", &websocket_url, ws_setting_type);
-                            &config.url,
+                        println_name_value_or("Keypair Path:", &keypair_path, keypair_setting_type);
-                            &CliConfig::default_json_rpc_url(),
-                        );
-                        println_name_value_or(
-                            "Keypair Path:",
-                            &config.keypair_path,
-                            &CliConfig::default_keypair_path(),
-                        );
                     }
                 } else {
                     println!(
@@ -58,13 +60,29 @@ fn parse_settings(matches: &ArgMatches<'_>) -> Result<bool, Box<dyn error::Error
                     if let Some(url) = subcommand_matches.value_of("json_rpc_url") {
                         config.url = url.to_string();
                     }
+                    if let Some(url) = subcommand_matches.value_of("websocket_url") {
+                        config.websocket_url = url.to_string();
+                    }
                     if let Some(keypair) = subcommand_matches.value_of("keypair") {
                         config.keypair_path = keypair.to_string();
                     }
                     config.save(config_file)?;
+
+                    let (url_setting_type, json_rpc_url) =
+                        CliConfig::compute_json_rpc_url_setting("", &config.url);
+                    let (ws_setting_type, websocket_url) = CliConfig::compute_websocket_url_setting(
+                        "",
+                        &config.websocket_url,
+                        "",
+                        &config.url,
+                    );
+                    let (keypair_setting_type, keypair_path) =
+                        CliConfig::compute_keypair_path_setting("", &config.keypair_path);
+
                     println_name_value("Config File:", config_file);
-                    println_name_value("RPC URL:", &config.url);
+                    println_name_value_or("RPC URL:", &json_rpc_url, url_setting_type);
-                    println_name_value("Keypair Path:", &config.keypair_path);
+                    println_name_value_or("WS URL:", &websocket_url, ws_setting_type);
+                    println_name_value_or("Keypair Path:", &keypair_path, keypair_setting_type);
                 } else {
                     println!(
                         "{} Either provide the `--config` arg or ensure home directory exists to use the default config location",
@@ -89,22 +107,20 @@ pub fn parse_args<'a>(
     } else {
         Config::default()
     };
-    let json_rpc_url = if let Some(url) = matches.value_of("json_rpc_url") {
+    let (_, json_rpc_url) = CliConfig::compute_json_rpc_url_setting(
-        url.to_string()
+        matches.value_of("json_rpc_url").unwrap_or(""),
-    } else if config.url != "" {
+        &config.url,
-        config.url
+    );
-    } else {
+    let (_, websocket_url) = CliConfig::compute_websocket_url_setting(
-        let default = CliConfig::default();
+        matches.value_of("websocket_url").unwrap_or(""),
-        default.json_rpc_url
+        &config.websocket_url,
-    };
+        matches.value_of("json_rpc_url").unwrap_or(""),
-
+        &config.url,
-    let default_signer_path = if matches.is_present("keypair") {
+    );
-        matches.value_of("keypair").unwrap().to_string()
+    let (_, default_signer_path) = CliConfig::compute_keypair_path_setting(
-    } else if config.keypair_path != "" {
+        matches.value_of("keypair").unwrap_or(""),
-        config.keypair_path
+        &config.keypair_path,
-    } else {
+    );
-        CliConfig::default_keypair_path()
-    };
 
     let CliCommandInfo { command, signers } =
         parse_command(&matches, &default_signer_path, wallet_manager.as_ref())?;
@@ -113,6 +129,7 @@ pub fn parse_args<'a>(
         CliConfig {
             command,
             json_rpc_url,
+            websocket_url,
             signers: vec![],
             keypair_path: default_signer_path,
             derivation_path: derivation_of(matches, "derivation_path"),
@@ -154,6 +171,15 @@ fn main() -> Result<(), Box<dyn error::Error>> {
             .validator(is_url)
             .help("JSON RPC URL for the solana cluster"),
     )
+    .arg(
+        Arg::with_name("websocket_url")
+            .long("ws")
+            .value_name("URL")
+            .takes_value(true)
+            .global(true)
+            .validator(is_url)
+            .help("WebSocket URL for the solana cluster"),
+    )
     .arg(
         Arg::with_name("keypair")
             .short("k")
@@ -198,7 +224,7 @@ fn main() -> Result<(), Box<dyn error::Error>> {
                             .index(1)
                             .value_name("CONFIG_FIELD")
                             .takes_value(true)
-                            .possible_values(&["url", "keypair"])
+                            .possible_values(&["json_rpc_url", "websocket_url", "keypair"])
                             .help("Return a specific config setting"),
                     ),
             )
@@ -207,7 +233,7 @@ fn main() -> Result<(), Box<dyn error::Error>> {
                     .about("Set a config setting")
                     .group(
                         ArgGroup::with_name("config_settings")
-                            .args(&["json_rpc_url", "keypair"])
+                            .args(&["json_rpc_url", "websocket_url", "keypair"])
                             .multiple(true)
                             .required(true),
                     ),

cli/src/nonce.rs

@@ -14,7 +14,7 @@ use solana_sdk::{
     account_utils::StateMut,
     hash::Hash,
     message::Message,
-    nonce_state::{Meta, NonceState},
+    nonce::{self, state::Versions, State},
     pubkey::Pubkey,
     system_instruction::{
         advance_nonce_account, authorize_nonce_account, create_address_with_seed,
@@ -363,22 +363,20 @@ pub fn check_nonce_account(
     if nonce_account.owner != system_program::ID {
         return Err(CliError::InvalidNonce(CliNonceError::InvalidAccountOwner).into());
     }
-    let nonce_state: NonceState = nonce_account
+    let nonce_state = StateMut::<Versions>::state(nonce_account)
-        .state()
+        .map(|v| v.convert_to_current())
         .map_err(|_| Box::new(CliError::InvalidNonce(CliNonceError::InvalidAccountData)))?;
     match nonce_state {
-        NonceState::Initialized(meta, hash) => {
+        State::Initialized(ref data) => {
-            if &hash != nonce_hash {
+            if &data.blockhash != nonce_hash {
                 Err(CliError::InvalidNonce(CliNonceError::InvalidHash).into())
-            } else if nonce_authority != &meta.nonce_authority {
+            } else if nonce_authority != &data.authority {
                 Err(CliError::InvalidNonce(CliNonceError::InvalidAuthority).into())
             } else {
                 Ok(())
             }
         }
-        NonceState::Uninitialized => {
+        State::Uninitialized => Err(CliError::InvalidNonce(CliNonceError::InvalidState).into()),
-            Err(CliError::InvalidNonce(CliNonceError::InvalidState).into())
-        }
     }
 }
 
@@ -429,7 +427,7 @@ pub fn process_create_nonce_account(
 
     if let Ok(nonce_account) = rpc_client.get_account(&nonce_account_address) {
         let err_msg = if nonce_account.owner == system_program::id()
-            && StateMut::<NonceState>::state(&nonce_account).is_ok()
+            && StateMut::<Versions>::state(&nonce_account).is_ok()
         {
             format!("Nonce account {} already exists", nonce_account_address)
         } else {
@@ -441,7 +439,7 @@ pub fn process_create_nonce_account(
         return Err(CliError::BadParameter(err_msg).into());
     }
 
-    let minimum_balance = rpc_client.get_minimum_balance_for_rent_exemption(NonceState::size())?;
+    let minimum_balance = rpc_client.get_minimum_balance_for_rent_exemption(State::size())?;
     if lamports < minimum_balance {
         return Err(CliError::BadParameter(format!(
             "need at least {} lamports for nonce account to be rent exempt, provided lamports: {}",
@@ -495,9 +493,10 @@ pub fn process_get_nonce(rpc_client: &RpcClient, nonce_account_pubkey: &Pubkey)
         ))
         .into());
     }
-    match nonce_account.state() {
+    let nonce_state = StateMut::<Versions>::state(&nonce_account).map(|v| v.convert_to_current());
-        Ok(NonceState::Uninitialized) => Ok("Nonce account is uninitialized".to_string()),
+    match nonce_state {
-        Ok(NonceState::Initialized(_, hash)) => Ok(format!("{:?}", hash)),
+        Ok(State::Uninitialized) => Ok("Nonce account is uninitialized".to_string()),
+        Ok(State::Initialized(ref data)) => Ok(format!("{:?}", data.blockhash)),
         Err(err) => Err(CliError::RpcRequestError(format!(
             "Account data could not be deserialized to nonce state: {:?}",
             err
@@ -554,7 +553,7 @@ pub fn process_show_nonce_account(
         ))
         .into());
     }
-    let print_account = |data: Option<(Meta, Hash)>| {
+    let print_account = |data: Option<&nonce::state::Data>| {
         println!(
             "Balance: {}",
             build_balance_message(nonce_account.lamports, use_lamports_unit, true)
@@ -562,26 +561,32 @@ pub fn process_show_nonce_account(
         println!(
             "Minimum Balance Required: {}",
             build_balance_message(
-                rpc_client.get_minimum_balance_for_rent_exemption(NonceState::size())?,
+                rpc_client.get_minimum_balance_for_rent_exemption(State::size())?,
                 use_lamports_unit,
                 true
             )
         );
         match data {
-            Some((meta, hash)) => {
+            Some(ref data) => {
-                println!("Nonce: {}", hash);
+                println!("Nonce: {}", data.blockhash);
-                println!("Authority: {}", meta.nonce_authority);
+                println!(
+                    "Fee: {} lamports per signature",
+                    data.fee_calculator.lamports_per_signature
+                );
+                println!("Authority: {}", data.authority);
             }
             None => {
                 println!("Nonce: uninitialized");
+                println!("Fees: uninitialized");
                 println!("Authority: uninitialized");
             }
         }
         Ok("".to_string())
     };
-    match nonce_account.state() {
+    let nonce_state = StateMut::<Versions>::state(&nonce_account).map(|v| v.convert_to_current());
-        Ok(NonceState::Uninitialized) => print_account(None),
+    match nonce_state {
-        Ok(NonceState::Initialized(meta, hash)) => print_account(Some((meta, hash))),
+        Ok(State::Uninitialized) => print_account(None),
+        Ok(State::Initialized(ref data)) => print_account(Some(data)),
         Err(err) => Err(CliError::RpcRequestError(format!(
             "Account data could not be deserialized to nonce state: {:?}",
             err
@@ -626,8 +631,9 @@ mod tests {
     use crate::cli::{app, parse_command};
     use solana_sdk::{
         account::Account,
+        fee_calculator::FeeCalculator,
         hash::hash,
-        nonce_state::{Meta as NonceMeta, NonceState},
+        nonce::{self, State},
         signature::{read_keypair_file, write_keypair, Keypair, Signer},
         system_program,
     };
@@ -903,18 +909,15 @@ mod tests {
     fn test_check_nonce_account() {
         let blockhash = Hash::default();
         let nonce_pubkey = Pubkey::new_rand();
-        let valid = Account::new_data(
+        let data = Versions::new_current(State::Initialized(nonce::state::Data {
-            1,
+            authority: nonce_pubkey,
-            &NonceState::Initialized(NonceMeta::new(&nonce_pubkey), blockhash),
+            blockhash,
-            &system_program::ID,
+            fee_calculator: FeeCalculator::default(),
-        );
+        }));
+        let valid = Account::new_data(1, &data, &system_program::ID);
         assert!(check_nonce_account(&valid.unwrap(), &nonce_pubkey, &blockhash).is_ok());
 
-        let invalid_owner = Account::new_data(
+        let invalid_owner = Account::new_data(1, &data, &Pubkey::new(&[1u8; 32]));
-            1,
-            &NonceState::Initialized(NonceMeta::new(&nonce_pubkey), blockhash),
-            &Pubkey::new(&[1u8; 32]),
-        );
         assert_eq!(
             check_nonce_account(&invalid_owner.unwrap(), &nonce_pubkey, &blockhash),
             Err(Box::new(CliError::InvalidNonce(
@@ -930,21 +933,23 @@ mod tests {
             ))),
         );
 
-        let invalid_hash = Account::new_data(
+        let data = Versions::new_current(State::Initialized(nonce::state::Data {
-            1,
+            authority: nonce_pubkey,
-            &NonceState::Initialized(NonceMeta::new(&nonce_pubkey), hash(b"invalid")),
+            blockhash: hash(b"invalid"),
-            &system_program::ID,
+            fee_calculator: FeeCalculator::default(),
-        );
+        }));
+        let invalid_hash = Account::new_data(1, &data, &system_program::ID);
         assert_eq!(
             check_nonce_account(&invalid_hash.unwrap(), &nonce_pubkey, &blockhash),
             Err(Box::new(CliError::InvalidNonce(CliNonceError::InvalidHash))),
         );
 
-        let invalid_authority = Account::new_data(
+        let data = Versions::new_current(State::Initialized(nonce::state::Data {
-            1,
+            authority: Pubkey::new_rand(),
-            &NonceState::Initialized(NonceMeta::new(&Pubkey::new_rand()), blockhash),
+            blockhash,
-            &system_program::ID,
+            fee_calculator: FeeCalculator::default(),
-        );
+        }));
+        let invalid_authority = Account::new_data(1, &data, &system_program::ID);
         assert_eq!(
             check_nonce_account(&invalid_authority.unwrap(), &nonce_pubkey, &blockhash),
             Err(Box::new(CliError::InvalidNonce(
@@ -952,7 +957,8 @@ mod tests {
             ))),
         );
 
-        let invalid_state = Account::new_data(1, &NonceState::Uninitialized, &system_program::ID);
+        let data = Versions::new_current(State::Uninitialized);
+        let invalid_state = Account::new_data(1, &data, &system_program::ID);
         assert_eq!(
             check_nonce_account(&invalid_state.unwrap(), &nonce_pubkey, &blockhash),
             Err(Box::new(CliError::InvalidNonce(

cli/src/offline.rs

@@ -201,7 +201,7 @@ mod tests {
     fn test_blockhashspec_get_blockhash_fee_calc() {
         let test_blockhash = hash(&[0u8]);
         let rpc_blockhash = hash(&[1u8]);
-        let rpc_fee_calc = FeeCalculator::new(42, 42);
+        let rpc_fee_calc = FeeCalculator::new(42);
         let get_recent_blockhash_response = json!(Response {
             context: RpcResponseContext { slot: 1 },
             value: json!((

cli/src/stake.rs

@@ -7,7 +7,8 @@ use crate::{
     nonce::{check_nonce_account, nonce_arg, NONCE_ARG, NONCE_AUTHORITY_ARG},
     offline::*,
 };
-use clap::{App, Arg, ArgMatches, SubCommand};
+use chrono::{DateTime, NaiveDateTime, SecondsFormat, Utc};
+use clap::{App, Arg, ArgGroup, ArgMatches, SubCommand};
 use console::style;
 use solana_clap_utils::{input_parsers::*, input_validators::*, offline::*, ArgConstant};
 use solana_client::rpc_client::RpcClient;
@@ -24,7 +25,7 @@ use solana_sdk::{
     transaction::Transaction,
 };
 use solana_stake_program::{
-    stake_instruction::{self, StakeError},
+    stake_instruction::{self, LockupArgs, StakeError},
     stake_state::{Authorized, Lockup, Meta, StakeAuthorize, StakeState},
 };
 use solana_vote_program::vote_state::VoteState;
@@ -364,6 +365,9 @@ impl StakeSubCommands for App<'_, '_> {
                         .validator(is_pubkey_or_keypair)
                         .help("Identity of the new lockup custodian (can withdraw before lockup expires)")
                 )
+                .group(ArgGroup::with_name("lockup_details")
+                    .args(&["lockup_epoch", "lockup_date", "new_custodian"])
+                    .required(true))
                 .arg(
                     Arg::with_name("custodian")
                         .long("custodian")
@@ -672,9 +676,9 @@ pub fn parse_stake_set_lockup(
     wallet_manager: Option<&Arc<RemoteWalletManager>>,
 ) -> Result<CliCommandInfo, CliError> {
     let stake_account_pubkey = pubkey_of(matches, "stake_account_pubkey").unwrap();
-    let epoch = value_of(matches, "lockup_epoch").unwrap_or(0);
+    let epoch = value_of(matches, "lockup_epoch");
-    let unix_timestamp = unix_timestamp_from_rfc3339_datetime(matches, "lockup_date").unwrap_or(0);
+    let unix_timestamp = unix_timestamp_from_rfc3339_datetime(matches, "lockup_date");
-    let new_custodian = pubkey_of(matches, "new_custodian").unwrap_or_default();
+    let new_custodian = pubkey_of(matches, "new_custodian");
 
     let sign_only = matches.is_present(SIGN_ONLY_ARG.name);
     let blockhash_query = BlockhashQuery::new_from_matches(matches);
@@ -695,7 +699,7 @@ pub fn parse_stake_set_lockup(
     Ok(CliCommandInfo {
         command: CliCommand::StakeSetLockup {
             stake_account_pubkey,
-            lockup: Lockup {
+            lockup: LockupArgs {
                 custodian: new_custodian,
                 epoch,
                 unix_timestamp,
@@ -1155,7 +1159,7 @@ pub fn process_stake_set_lockup(
     rpc_client: &RpcClient,
     config: &CliConfig,
     stake_account_pubkey: &Pubkey,
-    lockup: &mut Lockup,
+    lockup: &mut LockupArgs,
     custodian: SignerIndex,
     sign_only: bool,
     blockhash_query: &BlockhashQuery,
@@ -1166,10 +1170,7 @@ pub fn process_stake_set_lockup(
     let (recent_blockhash, fee_calculator) =
         blockhash_query.get_blockhash_fee_calculator(rpc_client)?;
     let custodian = config.signers[custodian];
-    // If new custodian is not explicitly set, default to current custodian
+
-    if lockup.custodian == Pubkey::default() {
-        lockup.custodian = custodian.pubkey();
-    }
     let ixs = vec![stake_instruction::set_lockup(
         stake_account_pubkey,
         lockup,
@@ -1215,6 +1216,12 @@ pub fn print_stake_state(stake_lamports: u64, stake_state: &StakeState, use_lamp
         println!("Authorized Withdrawer: {}", authorized.withdrawer);
     }
     fn show_lockup(lockup: &Lockup) {
+        println!(
+            "Lockup Timestamp: {} (UnixTimestamp: {})",
+            DateTime::<Utc>::from_utc(NaiveDateTime::from_timestamp(lockup.unix_timestamp, 0), Utc)
+                .to_rfc3339_opts(SecondsFormat::Secs, true),
+            lockup.unix_timestamp
+        );
         println!("Lockup Epoch: {}", lockup.epoch);
         println!("Lockup Custodian: {}", lockup.custodian);
     }

cli/src/vote.rs

@@ -9,6 +9,7 @@ use solana_client::rpc_client::RpcClient;
 use solana_remote_wallet::remote_wallet::RemoteWalletManager;
 use solana_sdk::{
     account::Account,
+    commitment_config::CommitmentConfig,
     message::Message,
     pubkey::Pubkey,
     system_instruction::{create_address_with_seed, SystemError},
@@ -158,6 +159,14 @@ impl VoteSubCommands for App<'_, '_> {
             SubCommand::with_name("vote-account")
                 .about("Show the contents of a vote account")
                 .alias("show-vote-account")
+                .arg(
+                    Arg::with_name("confirmed")
+                        .long("confirmed")
+                        .takes_value(false)
+                        .help(
+                            "Return information at maximum-lockout commitment level",
+                        ),
+                )
                 .arg(
                     Arg::with_name("vote_account_pubkey")
                         .index(1)
@@ -267,10 +276,16 @@ pub fn parse_vote_get_account_command(
 ) -> Result<CliCommandInfo, CliError> {
     let vote_account_pubkey = pubkey_of(matches, "vote_account_pubkey").unwrap();
     let use_lamports_unit = matches.is_present("lamports");
+    let commitment_config = if matches.is_present("confirmed") {
+        CommitmentConfig::default()
+    } else {
+        CommitmentConfig::recent()
+    };
     Ok(CliCommandInfo {
         command: CliCommand::ShowVoteAccount {
             pubkey: vote_account_pubkey,
             use_lamports_unit,
+            commitment_config,
         },
         signers: vec![],
     })
@@ -423,8 +438,14 @@ pub fn process_vote_update_validator(
 fn get_vote_account(
     rpc_client: &RpcClient,
     vote_account_pubkey: &Pubkey,
+    commitment_config: CommitmentConfig,
 ) -> Result<(Account, VoteState), Box<dyn std::error::Error>> {
-    let vote_account = rpc_client.get_account(vote_account_pubkey)?;
+    let vote_account = rpc_client
+        .get_account_with_commitment(vote_account_pubkey, commitment_config)?
+        .value
+        .ok_or_else(|| {
+            CliError::RpcRequestError(format!("{:?} account does not exist", vote_account_pubkey))
+        })?;
 
     if vote_account.owner != solana_vote_program::id() {
         return Err(CliError::RpcRequestError(format!(
@@ -447,8 +468,10 @@ pub fn process_show_vote_account(
     _config: &CliConfig,
     vote_account_pubkey: &Pubkey,
     use_lamports_unit: bool,
+    commitment_config: CommitmentConfig,
 ) -> ProcessResult {
-    let (vote_account, vote_state) = get_vote_account(rpc_client, vote_account_pubkey)?;
+    let (vote_account, vote_state) =
+        get_vote_account(rpc_client, vote_account_pubkey, commitment_config)?;
 
     let epoch_schedule = rpc_client.get_epoch_schedule()?;
 
@@ -457,7 +480,7 @@ pub fn process_show_vote_account(
         build_balance_message(vote_account.lamports, use_lamports_unit, true)
     );
     println!("Validator Identity: {}", vote_state.node_pubkey);
-    println!("Authorized Voter: {}", vote_state.authorized_voter);
+    println!("Authorized Voter: {:?}", vote_state.authorized_voters());
     println!(
         "Authorized Withdrawer: {}",
         vote_state.authorized_withdrawer

cli/tests/deploy.rs

@@ -1,7 +1,7 @@
 use serde_json::Value;
 use solana_cli::cli::{process_command, CliCommand, CliConfig};
 use solana_client::rpc_client::RpcClient;
-use solana_core::validator::new_validator_for_tests;
+use solana_core::validator::TestValidator;
 use solana_faucet::faucet::run_local_faucet;
 use solana_sdk::{bpf_loader, pubkey::Pubkey, signature::Keypair};
 use std::{
@@ -22,7 +22,13 @@ fn test_cli_deploy_program() {
     pathbuf.push("noop");
     pathbuf.set_extension("so");
 
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
 
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);

cli/tests/nonce.rs

@@ -1,5 +1,6 @@
 use solana_cli::cli::{process_command, request_and_confirm_airdrop, CliCommand, CliConfig};
 use solana_client::rpc_client::RpcClient;
+use solana_core::validator::TestValidator;
 use solana_faucet::faucet::run_local_faucet;
 use solana_sdk::{
     hash::Hash,
@@ -10,9 +11,6 @@ use solana_sdk::{
 };
 use std::{fs::remove_dir_all, sync::mpsc::channel, thread::sleep, time::Duration};
 
-#[cfg(test)]
-use solana_core::validator::new_validator_for_tests;
-
 fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
     (0..5).for_each(|tries| {
         let balance = client.retry_get_balance(pubkey, 1).unwrap().unwrap();
@@ -28,7 +26,13 @@ fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
 
 #[test]
 fn test_nonce() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -44,7 +48,13 @@ fn test_nonce() {
 
 #[test]
 fn test_nonce_with_seed() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -66,7 +76,13 @@ fn test_nonce_with_seed() {
 
 #[test]
 fn test_nonce_with_authority() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();

cli/tests/pay.rs

@@ -6,19 +6,17 @@ use solana_cli::{
     offline::{parse_sign_only_reply_string, BlockhashQuery},
 };
 use solana_client::rpc_client::RpcClient;
+use solana_core::validator::TestValidator;
 use solana_faucet::faucet::run_local_faucet;
 use solana_sdk::{
     account_utils::StateMut,
     fee_calculator::FeeCalculator,
-    nonce_state::NonceState,
+    nonce,
     pubkey::Pubkey,
     signature::{Keypair, Signer},
 };
 use std::{fs::remove_dir_all, sync::mpsc::channel, thread::sleep, time::Duration};
 
-#[cfg(test)]
-use solana_core::validator::new_validator_for_tests;
-
 fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
     (0..5).for_each(|tries| {
         let balance = client.retry_get_balance(pubkey, 1).unwrap().unwrap();
@@ -34,7 +32,13 @@ fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
 
 #[test]
 fn test_cli_timestamp_tx() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let bob_pubkey = Pubkey::new_rand();
 
     let (sender, receiver) = channel();
@@ -113,7 +117,13 @@ fn test_cli_timestamp_tx() {
 
 #[test]
 fn test_cli_witness_tx() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let bob_pubkey = Pubkey::new_rand();
 
     let (sender, receiver) = channel();
@@ -187,7 +197,13 @@ fn test_cli_witness_tx() {
 
 #[test]
 fn test_cli_cancel_tx() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let bob_pubkey = Pubkey::new_rand();
 
     let (sender, receiver) = channel();
@@ -255,7 +271,13 @@ fn test_cli_cancel_tx() {
 
 #[test]
 fn test_offline_pay_tx() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let bob_pubkey = Pubkey::new_rand();
 
     let (sender, receiver) = channel();
@@ -336,7 +358,13 @@ fn test_offline_pay_tx() {
 fn test_nonced_pay_tx() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -349,7 +377,7 @@ fn test_nonced_pay_tx() {
     config.signers = vec![&default_signer];
 
     let minimum_nonce_balance = rpc_client
-        .get_minimum_balance_for_rent_exemption(NonceState::size())
+        .get_minimum_balance_for_rent_exemption(nonce::State::size())
         .unwrap();
 
     request_and_confirm_airdrop(
@@ -381,9 +409,11 @@ fn test_nonced_pay_tx() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -403,9 +433,11 @@ fn test_nonced_pay_tx() {
 
     // Verify that nonce has been used
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     match nonce_state {
-        NonceState::Initialized(_meta, hash) => assert_ne!(hash, nonce_hash),
+        nonce::State::Initialized(ref data) => assert_ne!(data.blockhash, nonce_hash),
         _ => assert!(false, "Nonce is not initialized"),
     }
 

cli/tests/request_airdrop.rs

@@ -1,13 +1,19 @@
 use solana_cli::cli::{process_command, CliCommand, CliConfig};
 use solana_client::rpc_client::RpcClient;
-use solana_core::validator::new_validator_for_tests;
+use solana_core::validator::TestValidator;
 use solana_faucet::faucet::run_local_faucet;
 use solana_sdk::signature::Keypair;
 use std::{fs::remove_dir_all, sync::mpsc::channel};
 
 #[test]
 fn test_cli_request_airdrop() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();

cli/tests/stake.rs

@@ -4,22 +4,21 @@ use solana_cli::{
     offline::{parse_sign_only_reply_string, BlockhashQuery},
 };
 use solana_client::rpc_client::RpcClient;
+use solana_core::validator::{TestValidator, TestValidatorOptions};
 use solana_faucet::faucet::run_local_faucet;
 use solana_sdk::{
     account_utils::StateMut,
     fee_calculator::FeeCalculator,
-    nonce_state::NonceState,
+    nonce,
     pubkey::Pubkey,
     signature::{keypair_from_seed, Keypair, Signer},
     system_instruction::create_address_with_seed,
 };
-use solana_stake_program::stake_state::{Lockup, StakeAuthorize, StakeState};
+use solana_stake_program::{
-use std::{fs::remove_dir_all, sync::mpsc::channel, thread::sleep, time::Duration};
+    stake_instruction::LockupArgs,
-
+    stake_state::{Lockup, StakeAuthorize, StakeState},
-#[cfg(test)]
-use solana_core::validator::{
-    new_validator_for_tests, new_validator_for_tests_ex, new_validator_for_tests_with_vote_pubkey,
 };
+use std::{fs::remove_dir_all, sync::mpsc::channel, thread::sleep, time::Duration};
 
 fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
     (0..5).for_each(|tries| {
@@ -36,7 +35,13 @@ fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
 
 #[test]
 fn test_stake_delegation_force() {
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -124,8 +129,14 @@ fn test_stake_delegation_force() {
 fn test_seed_stake_delegation_and_deactivation() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path, vote_pubkey) =
+    let TestValidator {
-        new_validator_for_tests_with_vote_pubkey();
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        vote_pubkey,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -206,8 +217,14 @@ fn test_seed_stake_delegation_and_deactivation() {
 fn test_stake_delegation_and_deactivation() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path, vote_pubkey) =
+    let TestValidator {
-        new_validator_for_tests_with_vote_pubkey();
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        vote_pubkey,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -284,8 +301,14 @@ fn test_stake_delegation_and_deactivation() {
 fn test_offline_stake_delegation_and_deactivation() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path, vote_pubkey) =
+    let TestValidator {
-        new_validator_for_tests_with_vote_pubkey();
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        vote_pubkey,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -414,8 +437,14 @@ fn test_offline_stake_delegation_and_deactivation() {
 fn test_nonced_stake_delegation_and_deactivation() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path, vote_pubkey) =
+    let TestValidator {
-        new_validator_for_tests_with_vote_pubkey();
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        vote_pubkey,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -428,7 +457,7 @@ fn test_nonced_stake_delegation_and_deactivation() {
     config.json_rpc_url = format!("http://{}:{}", leader_data.rpc.ip(), leader_data.rpc.port());
 
     let minimum_nonce_balance = rpc_client
-        .get_minimum_balance_for_rent_exemption(NonceState::size())
+        .get_minimum_balance_for_rent_exemption(nonce::State::size())
         .unwrap();
 
     request_and_confirm_airdrop(
@@ -471,9 +500,11 @@ fn test_nonced_stake_delegation_and_deactivation() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -494,9 +525,11 @@ fn test_nonced_stake_delegation_and_deactivation() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -520,7 +553,13 @@ fn test_nonced_stake_delegation_and_deactivation() {
 fn test_stake_authorize() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -665,7 +704,7 @@ fn test_stake_authorize() {
 
     // Create nonce account
     let minimum_nonce_balance = rpc_client
-        .get_minimum_balance_for_rent_exemption(NonceState::size())
+        .get_minimum_balance_for_rent_exemption(nonce::State::size())
         .unwrap();
     let nonce_account = Keypair::new();
     config.signers = vec![&default_signer, &nonce_account];
@@ -679,9 +718,11 @@ fn test_stake_authorize() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -728,9 +769,11 @@ fn test_stake_authorize() {
     };
     assert_eq!(current_authority, online_authority_pubkey);
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let new_nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
     assert_ne!(nonce_hash, new_nonce_hash);
@@ -744,8 +787,16 @@ fn test_stake_authorize_with_fee_payer() {
     solana_logger::setup();
     const SIG_FEE: u64 = 42;
 
-    let (server, leader_data, alice, ledger_path, _voter) =
+    let TestValidator {
-        new_validator_for_tests_ex(SIG_FEE, 42_000);
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run_with_options(TestValidatorOptions {
+        fees: SIG_FEE,
+        bootstrap_validator_lamports: 42_000,
+    });
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -867,7 +918,16 @@ fn test_stake_authorize_with_fee_payer() {
 fn test_stake_split() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path, _voter) = new_validator_for_tests_ex(1, 42_000);
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run_with_options(TestValidatorOptions {
+        fees: 1,
+        bootstrap_validator_lamports: 42_000,
+    });
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -930,7 +990,7 @@ fn test_stake_split() {
 
     // Create nonce account
     let minimum_nonce_balance = rpc_client
-        .get_minimum_balance_for_rent_exemption(NonceState::size())
+        .get_minimum_balance_for_rent_exemption(nonce::State::size())
         .unwrap();
     let nonce_account = keypair_from_seed(&[1u8; 32]).unwrap();
     config.signers = vec![&default_signer, &nonce_account];
@@ -945,9 +1005,11 @@ fn test_stake_split() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -1003,7 +1065,16 @@ fn test_stake_split() {
 fn test_stake_set_lockup() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path, _voter) = new_validator_for_tests_ex(1, 42_000);
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run_with_options(TestValidatorOptions {
+        fees: 1,
+        bootstrap_validator_lamports: 42_000,
+    });
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -1070,10 +1141,10 @@ fn test_stake_set_lockup() {
     );
 
     // Online set lockup
-    let mut lockup = Lockup {
+    let lockup = LockupArgs {
-        unix_timestamp: 1581534570,
+        unix_timestamp: Some(1581534570),
-        epoch: 200,
+        epoch: Some(200),
-        custodian: Pubkey::default(),
+        custodian: None,
     };
     config.signers.pop();
     config.command = CliCommand::StakeSetLockup {
@@ -1093,17 +1164,21 @@ fn test_stake_set_lockup() {
         StakeState::Initialized(meta) => meta.lockup,
         _ => panic!("Unexpected stake state!"),
     };
-    lockup.custodian = config.signers[0].pubkey(); // Default new_custodian is config.signers[0]
+    assert_eq!(
-    assert_eq!(current_lockup, lockup);
+        current_lockup.unix_timestamp,
+        lockup.unix_timestamp.unwrap()
+    );
+    assert_eq!(current_lockup.epoch, lockup.epoch.unwrap());
+    assert_eq!(current_lockup.custodian, config.signers[0].pubkey());
 
     // Set custodian to another pubkey
     let online_custodian = Keypair::new();
     let online_custodian_pubkey = online_custodian.pubkey();
 
-    let lockup = Lockup {
+    let lockup = LockupArgs {
-        unix_timestamp: 1581534571,
+        unix_timestamp: Some(1581534571),
-        epoch: 201,
+        epoch: Some(201),
-        custodian: online_custodian_pubkey,
+        custodian: Some(online_custodian_pubkey),
     };
     config.command = CliCommand::StakeSetLockup {
         stake_account_pubkey,
@@ -1117,10 +1192,10 @@ fn test_stake_set_lockup() {
     };
     process_command(&config).unwrap();
 
-    let mut lockup = Lockup {
+    let lockup = LockupArgs {
-        unix_timestamp: 1581534572,
+        unix_timestamp: Some(1581534572),
-        epoch: 202,
+        epoch: Some(202),
-        custodian: Pubkey::default(),
+        custodian: None,
     };
     config.signers = vec![&default_signer, &online_custodian];
     config.command = CliCommand::StakeSetLockup {
@@ -1140,14 +1215,18 @@ fn test_stake_set_lockup() {
         StakeState::Initialized(meta) => meta.lockup,
         _ => panic!("Unexpected stake state!"),
     };
-    lockup.custodian = online_custodian_pubkey; // Default new_custodian is designated custodian
+    assert_eq!(
-    assert_eq!(current_lockup, lockup);
+        current_lockup.unix_timestamp,
+        lockup.unix_timestamp.unwrap()
+    );
+    assert_eq!(current_lockup.epoch, lockup.epoch.unwrap());
+    assert_eq!(current_lockup.custodian, online_custodian_pubkey);
 
     // Set custodian to offline pubkey
-    let lockup = Lockup {
+    let lockup = LockupArgs {
-        unix_timestamp: 1581534573,
+        unix_timestamp: Some(1581534573),
-        epoch: 203,
+        epoch: Some(203),
-        custodian: offline_pubkey,
+        custodian: Some(offline_pubkey),
     };
     config.command = CliCommand::StakeSetLockup {
         stake_account_pubkey,
@@ -1163,7 +1242,7 @@ fn test_stake_set_lockup() {
 
     // Create nonce account
     let minimum_nonce_balance = rpc_client
-        .get_minimum_balance_for_rent_exemption(NonceState::size())
+        .get_minimum_balance_for_rent_exemption(nonce::State::size())
         .unwrap();
     let nonce_account = keypair_from_seed(&[1u8; 32]).unwrap();
     let nonce_account_pubkey = nonce_account.pubkey();
@@ -1179,17 +1258,19 @@ fn test_stake_set_lockup() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account_pubkey).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
     // Nonced offline set lockup
-    let lockup = Lockup {
+    let lockup = LockupArgs {
-        unix_timestamp: 1581534576,
+        unix_timestamp: Some(1581534576),
-        epoch: 222,
+        epoch: Some(222),
-        custodian: offline_pubkey,
+        custodian: None,
     };
     config_offline.command = CliCommand::StakeSetLockup {
         stake_account_pubkey,
@@ -1222,7 +1303,12 @@ fn test_stake_set_lockup() {
         StakeState::Initialized(meta) => meta.lockup,
         _ => panic!("Unexpected stake state!"),
     };
-    assert_eq!(current_lockup, lockup);
+    assert_eq!(
+        current_lockup.unix_timestamp,
+        lockup.unix_timestamp.unwrap()
+    );
+    assert_eq!(current_lockup.epoch, lockup.epoch.unwrap());
+    assert_eq!(current_lockup.custodian, offline_pubkey);
 
     server.close().unwrap();
     remove_dir_all(ledger_path).unwrap();
@@ -1232,7 +1318,13 @@ fn test_stake_set_lockup() {
 fn test_offline_nonced_create_stake_account_and_withdraw() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let (sender, receiver) = channel();
     run_local_faucet(alice, sender, None);
     let faucet_addr = receiver.recv().unwrap();
@@ -1267,7 +1359,7 @@ fn test_offline_nonced_create_stake_account_and_withdraw() {
 
     // Create nonce account
     let minimum_nonce_balance = rpc_client
-        .get_minimum_balance_for_rent_exemption(NonceState::size())
+        .get_minimum_balance_for_rent_exemption(nonce::State::size())
         .unwrap();
     let nonce_account = keypair_from_seed(&[3u8; 32]).unwrap();
     let nonce_pubkey = nonce_account.pubkey();
@@ -1282,9 +1374,11 @@ fn test_offline_nonced_create_stake_account_and_withdraw() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -1330,9 +1424,11 @@ fn test_offline_nonced_create_stake_account_and_withdraw() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -1371,9 +1467,11 @@ fn test_offline_nonced_create_stake_account_and_withdraw() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 

cli/tests/transfer.rs

@@ -4,19 +4,17 @@ use solana_cli::{
     offline::{parse_sign_only_reply_string, BlockhashQuery},
 };
 use solana_client::rpc_client::RpcClient;
+use solana_core::validator::{TestValidator, TestValidatorOptions};
 use solana_faucet::faucet::run_local_faucet;
 use solana_sdk::{
     account_utils::StateMut,
     fee_calculator::FeeCalculator,
-    nonce_state::NonceState,
+    nonce,
     pubkey::Pubkey,
     signature::{keypair_from_seed, Keypair, Signer},
 };
 use std::{fs::remove_dir_all, sync::mpsc::channel, thread::sleep, time::Duration};
 
-#[cfg(test)]
-use solana_core::validator::new_validator_for_tests_ex;
-
 fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
     (0..5).for_each(|tries| {
         let balance = client.retry_get_balance(pubkey, 1).unwrap().unwrap();
@@ -32,7 +30,16 @@ fn check_balance(expected_balance: u64, client: &RpcClient, pubkey: &Pubkey) {
 
 #[test]
 fn test_transfer() {
-    let (server, leader_data, mint_keypair, ledger_path, _) = new_validator_for_tests_ex(1, 42_000);
+    let TestValidator {
+        server,
+        leader_data,
+        alice: mint_keypair,
+        ledger_path,
+        ..
+    } = TestValidator::run_with_options(TestValidatorOptions {
+        fees: 1,
+        bootstrap_validator_lamports: 42_000,
+    });
 
     let (sender, receiver) = channel();
     run_local_faucet(mint_keypair, sender, None);
@@ -113,7 +120,7 @@ fn test_transfer() {
     // Create nonce account
     let nonce_account = keypair_from_seed(&[3u8; 32]).unwrap();
     let minimum_nonce_balance = rpc_client
-        .get_minimum_balance_for_rent_exemption(NonceState::size())
+        .get_minimum_balance_for_rent_exemption(nonce::State::size())
         .unwrap();
     config.signers = vec![&default_signer, &nonce_account];
     config.command = CliCommand::CreateNonceAccount {
@@ -127,9 +134,11 @@ fn test_transfer() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 
@@ -149,9 +158,11 @@ fn test_transfer() {
     check_balance(49_976 - minimum_nonce_balance, &rpc_client, &sender_pubkey);
     check_balance(30, &rpc_client, &recipient_pubkey);
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let new_nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
     assert_ne!(nonce_hash, new_nonce_hash);
@@ -168,9 +179,11 @@ fn test_transfer() {
 
     // Fetch nonce hash
     let account = rpc_client.get_account(&nonce_account.pubkey()).unwrap();
-    let nonce_state: NonceState = account.state().unwrap();
+    let nonce_state = StateMut::<nonce::state::Versions>::state(&account)
+        .unwrap()
+        .convert_to_current();
     let nonce_hash = match nonce_state {
-        NonceState::Initialized(_meta, hash) => hash,
+        nonce::State::Initialized(ref data) => data.blockhash,
         _ => panic!("Nonce is not initialized"),
     };
 

client/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-client"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana Client"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"
@@ -18,8 +18,8 @@ reqwest = { version = "0.10.1", default-features = false, features = ["blocking"
 serde = "1.0.104"
 serde_derive = "1.0.103"
 serde_json = "1.0.46"
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
 thiserror = "1.0"
 tungstenite = "0.10.1"
 url = "2.1.1"
@@ -28,4 +28,4 @@ url = "2.1.1"
 assert_matches = "1.3.0"
 jsonrpc-core = "14.0.5"
 jsonrpc-http-server = "14.0.6"
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }

client/src/mock_rpc_client_request.rs

@@ -6,7 +6,7 @@ use crate::{
 };
 use serde_json::{Number, Value};
 use solana_sdk::{
-    fee_calculator::FeeCalculator,
+    fee_calculator::{FeeCalculator, FeeRateGovernor},
     instruction::InstructionError,
     transaction::{self, TransactionError},
 };
@@ -71,6 +71,10 @@ impl GenericRpcClientRequest for MockRpcClientRequest {
                     serde_json::to_value(FeeCalculator::default()).unwrap(),
                 ),
             })?,
+            RpcRequest::GetFeeRateGovernor => serde_json::to_value(Response {
+                context: RpcResponseContext { slot: 1 },
+                value: serde_json::to_value(FeeRateGovernor::default()).unwrap(),
+            })?,
             RpcRequest::GetSignatureStatus => {
                 let response: Option<transaction::Result<()>> = if self.url == "account_in_use" {
                     Some(Err(TransactionError::AccountInUse))

client/src/rpc_client.rs

@@ -6,8 +6,8 @@ use crate::{
     rpc_request::RpcRequest,
     rpc_response::{
         Response, RpcAccount, RpcBlockhashFeeCalculator, RpcConfirmedBlock, RpcContactInfo,
-        RpcEpochInfo, RpcKeyedAccount, RpcLeaderSchedule, RpcResponse, RpcVersionInfo,
+        RpcEpochInfo, RpcFeeRateGovernor, RpcIdentity, RpcKeyedAccount, RpcLeaderSchedule,
-        RpcVoteAccountStatus,
+        RpcResponse, RpcVersionInfo, RpcVoteAccountStatus,
     },
 };
 use bincode::serialize;
@@ -18,7 +18,7 @@ use solana_sdk::{
     clock::{Slot, UnixTimestamp, DEFAULT_TICKS_PER_SECOND, DEFAULT_TICKS_PER_SLOT},
     commitment_config::CommitmentConfig,
     epoch_schedule::EpochSchedule,
-    fee_calculator::FeeCalculator,
+    fee_calculator::{FeeCalculator, FeeRateGovernor},
     hash::Hash,
     inflation::Inflation,
     pubkey::Pubkey,
@@ -165,9 +165,16 @@ impl RpcClient {
     }
 
     pub fn get_vote_accounts(&self) -> io::Result<RpcVoteAccountStatus> {
+        self.get_vote_accounts_with_commitment(CommitmentConfig::default())
+    }
+
+    pub fn get_vote_accounts_with_commitment(
+        &self,
+        commitment_config: CommitmentConfig,
+    ) -> io::Result<RpcVoteAccountStatus> {
         let response = self
             .client
-            .send(&RpcRequest::GetVoteAccounts, Value::Null, 0)
+            .send(&RpcRequest::GetVoteAccounts, json!([commitment_config]), 0)
             .map_err(|err| {
                 io::Error::new(
                     io::ErrorKind::Other,
@@ -349,6 +356,34 @@ impl RpcClient {
         })
     }
 
+    pub fn get_identity(&self) -> io::Result<Pubkey> {
+        let response = self
+            .client
+            .send(&RpcRequest::GetIdentity, Value::Null, 0)
+            .map_err(|err| {
+                io::Error::new(
+                    io::ErrorKind::Other,
+                    format!("GetIdentity request failure: {:?}", err),
+                )
+            })?;
+
+        serde_json::from_value(response)
+            .map_err(|err| {
+                io::Error::new(
+                    io::ErrorKind::Other,
+                    format!("GetIdentity failure: {:?}", err),
+                )
+            })
+            .and_then(|rpc_identity: RpcIdentity| {
+                rpc_identity.identity.parse::<Pubkey>().map_err(|err| {
+                    io::Error::new(
+                        io::ErrorKind::Other,
+                        format!("GetIdentity invalid pubkey failure: {:?}", err),
+                    )
+                })
+            })
+    }
+
     pub fn get_inflation(&self) -> io::Result<Inflation> {
         let response = self
             .client
@@ -804,6 +839,31 @@ impl RpcClient {
         })
     }
 
+    pub fn get_fee_rate_governor(&self) -> RpcResponse<FeeRateGovernor> {
+        let response = self
+            .client
+            .send(&RpcRequest::GetFeeRateGovernor, Value::Null, 0)
+            .map_err(|e| {
+                io::Error::new(
+                    io::ErrorKind::Other,
+                    format!("GetFeeRateGovernor request failure: {:?}", e),
+                )
+            })?;
+        let Response {
+            context,
+            value: RpcFeeRateGovernor { fee_rate_governor },
+        } = serde_json::from_value::<Response<RpcFeeRateGovernor>>(response).map_err(|e| {
+            io::Error::new(
+                io::ErrorKind::Other,
+                format!("GetFeeRateGovernor parse failure: {:?}", e),
+            )
+        })?;
+        Ok(Response {
+            context,
+            value: fee_rate_governor,
+        })
+    }
+
     pub fn get_new_blockhash(&self, blockhash: &Hash) -> io::Result<(Hash, FeeCalculator)> {
         let mut num_retries = 0;
         let start = Instant::now();

client/src/rpc_request.rs

@@ -15,11 +15,13 @@ pub enum RpcRequest {
     GetEpochInfo,
     GetEpochSchedule,
     GetGenesisHash,
+    GetIdentity,
     GetInflation,
     GetLeaderSchedule,
     GetNumBlocksSinceSignatureConfirmation,
     GetProgramAccounts,
     GetRecentBlockhash,
+    GetFeeRateGovernor,
     GetSignatureStatus,
     GetSlot,
     GetSlotLeader,
@@ -54,6 +56,7 @@ impl RpcRequest {
             RpcRequest::GetEpochInfo => "getEpochInfo",
             RpcRequest::GetEpochSchedule => "getEpochSchedule",
             RpcRequest::GetGenesisHash => "getGenesisHash",
+            RpcRequest::GetIdentity => "getIdentity",
             RpcRequest::GetInflation => "getInflation",
             RpcRequest::GetLeaderSchedule => "getLeaderSchedule",
             RpcRequest::GetNumBlocksSinceSignatureConfirmation => {
@@ -61,6 +64,7 @@ impl RpcRequest {
             }
             RpcRequest::GetProgramAccounts => "getProgramAccounts",
             RpcRequest::GetRecentBlockhash => "getRecentBlockhash",
+            RpcRequest::GetFeeRateGovernor => "getFeeRateGovernor",
             RpcRequest::GetSignatureStatus => "getSignatureStatus",
             RpcRequest::GetSlot => "getSlot",
             RpcRequest::GetSlotLeader => "getSlotLeader",
@@ -138,6 +142,10 @@ mod tests {
         let request = test_request.build_request_json(1, Value::Null);
         assert_eq!(request["method"], "getRecentBlockhash");
 
+        let test_request = RpcRequest::GetFeeRateGovernor;
+        let request = test_request.build_request_json(1, Value::Null);
+        assert_eq!(request["method"], "getFeeRateGovernor");
+
         let test_request = RpcRequest::GetSlot;
         let request = test_request.build_request_json(1, Value::Null);
         assert_eq!(request["method"], "getSlot");

client/src/rpc_response.rs

@@ -4,7 +4,7 @@ use jsonrpc_core::Result as JsonResult;
 use solana_sdk::{
     account::Account,
     clock::{Epoch, Slot},
-    fee_calculator::FeeCalculator,
+    fee_calculator::{FeeCalculator, FeeRateGovernor},
     message::MessageHeader,
     pubkey::Pubkey,
     transaction::{Result, Transaction},
@@ -152,6 +152,12 @@ pub struct RpcBlockhashFeeCalculator {
     pub fee_calculator: FeeCalculator,
 }
 
+#[derive(Serialize, Deserialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct RpcFeeRateGovernor {
+    pub fee_rate_governor: FeeRateGovernor,
+}
+
 #[derive(Serialize, Deserialize, Clone, Debug)]
 #[serde(rename_all = "camelCase")]
 pub struct RpcKeyedAccount {
@@ -235,6 +241,13 @@ pub struct RpcVersionInfo {
     pub solana_core: String,
 }
 
+#[derive(Serialize, Deserialize, Clone, Debug)]
+#[serde(rename_all = "kebab-case")]
+pub struct RpcIdentity {
+    /// The current node identity pubkey
+    pub identity: String,
+}
+
 #[derive(Serialize, Deserialize, Clone, Debug)]
 #[serde(rename_all = "camelCase")]
 pub struct RpcVoteAccountStatus {

client/src/thin_client.rs

@@ -11,7 +11,7 @@ use solana_sdk::{
     client::{AsyncClient, Client, SyncClient},
     clock::MAX_PROCESSING_AGE,
     commitment_config::CommitmentConfig,
-    fee_calculator::FeeCalculator,
+    fee_calculator::{FeeCalculator, FeeRateGovernor},
     hash::Hash,
     instruction::Instruction,
     message::Message,
@@ -26,7 +26,7 @@ use solana_sdk::{
 };
 use std::{
     io,
-    net::{SocketAddr, UdpSocket},
+    net::{IpAddr, Ipv4Addr, SocketAddr, UdpSocket},
     sync::{
         atomic::{AtomicBool, AtomicUsize, Ordering},
         RwLock,
@@ -445,6 +445,11 @@ impl SyncClient for ThinClient {
         }
     }
 
+    fn get_fee_rate_governor(&self) -> TransportResult<FeeRateGovernor> {
+        let fee_rate_governor = self.rpc_client().get_fee_rate_governor()?;
+        Ok(fee_rate_governor.value)
+    }
+
     fn get_signature_status(
         &self,
         signature: &Signature,
@@ -598,7 +603,8 @@ impl AsyncClient for ThinClient {
 }
 
 pub fn create_client((rpc, tpu): (SocketAddr, SocketAddr), range: (u16, u16)) -> ThinClient {
-    let (_, transactions_socket) = solana_net_utils::bind_in_range(range).unwrap();
+    let (_, transactions_socket) =
+        solana_net_utils::bind_in_range(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), range).unwrap();
     ThinClient::new(rpc, tpu, transactions_socket)
 }
 
@@ -607,7 +613,8 @@ pub fn create_client_with_timeout(
     range: (u16, u16),
     timeout: Duration,
 ) -> ThinClient {
-    let (_, transactions_socket) = solana_net_utils::bind_in_range(range).unwrap();
+    let (_, transactions_socket) =
+        solana_net_utils::bind_in_range(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), range).unwrap();
     ThinClient::new_socket_with_timeout(rpc, tpu, transactions_socket, timeout)
 }
 

core/Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "solana-core"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.0.0"
+version = "1.0.3"
 documentation = "https://docs.rs/solana"
 homepage = "https://solana.com/"
 readme = "../README.md"
@@ -25,6 +25,7 @@ fs_extra = "1.1.0"
 indexmap = "1.3"
 itertools = "0.8.2"
 jsonrpc-core = "14.0.5"
+jsonrpc-core-client = { version = "14.0.5", features = ["ws"] }
 jsonrpc-derive = "14.0.5"
 jsonrpc-http-server = "14.0.6"
 jsonrpc-pubsub = "14.0.6"
@@ -36,29 +37,30 @@ num-traits = "0.2"
 rand = "0.6.5"
 rand_chacha = "0.1.1"
 rayon = "1.2.0"
+regex = "1.3.4"
 serde = "1.0.104"
 serde_derive = "1.0.103"
 serde_json = "1.0.46"
-solana-budget-program = { path = "../programs/budget", version = "1.0.0" }
+solana-budget-program = { path = "../programs/budget", version = "1.0.3" }
-solana-clap-utils = { path = "../clap-utils", version = "1.0.0" }
+solana-clap-utils = { path = "../clap-utils", version = "1.0.3" }
-solana-client = { path = "../client", version = "1.0.0" }
+solana-client = { path = "../client", version = "1.0.3" }
-solana-faucet = { path = "../faucet", version = "1.0.0" }
+solana-faucet = { path = "../faucet", version = "1.0.3" }
 ed25519-dalek = "=1.0.0-pre.1"
-solana-ledger = { path = "../ledger", version = "1.0.0" }
+solana-ledger = { path = "../ledger", version = "1.0.3" }
-solana-logger = { path = "../logger", version = "1.0.0" }
+solana-logger = { path = "../logger", version = "1.0.3" }
-solana-merkle-tree = { path = "../merkle-tree", version = "1.0.0" }
+solana-merkle-tree = { path = "../merkle-tree", version = "1.0.3" }
-solana-metrics = { path = "../metrics", version = "1.0.0" }
+solana-metrics = { path = "../metrics", version = "1.0.3" }
-solana-measure = { path = "../measure", version = "1.0.0" }
+solana-measure = { path = "../measure", version = "1.0.3" }
-solana-net-utils = { path = "../net-utils", version = "1.0.0" }
+solana-net-utils = { path = "../net-utils", version = "1.0.3" }
-solana-chacha-cuda = { path = "../chacha-cuda", version = "1.0.0" }
+solana-chacha-cuda = { path = "../chacha-cuda", version = "1.0.3" }
-solana-perf = { path = "../perf", version = "1.0.0" }
+solana-perf = { path = "../perf", version = "1.0.3" }
-solana-runtime = { path = "../runtime", version = "1.0.0" }
+solana-runtime = { path = "../runtime", version = "1.0.3" }
-solana-sdk = { path = "../sdk", version = "1.0.0" }
+solana-sdk = { path = "../sdk", version = "1.0.3" }
-solana-stake-program = { path = "../programs/stake", version = "1.0.0" }
+solana-stake-program = { path = "../programs/stake", version = "1.0.3" }
-solana-storage-program = { path = "../programs/storage", version = "1.0.0" }
+solana-storage-program = { path = "../programs/storage", version = "1.0.3" }
-solana-vote-program = { path = "../programs/vote", version = "1.0.0" }
+solana-vote-program = { path = "../programs/vote", version = "1.0.3" }
-solana-vote-signer = { path = "../vote-signer", version = "1.0.0" }
+solana-vote-signer = { path = "../vote-signer", version = "1.0.3" }
-solana-sys-tuner = { path = "../sys-tuner", version = "1.0.0" }
+solana-sys-tuner = { path = "../sys-tuner", version = "1.0.3" }
 sys-info = "0.5.9"
 tempfile = "3.1.0"
 thiserror = "1.0"
@@ -66,7 +68,7 @@ tokio = "0.1"
 tokio-codec = "0.1"
 tokio-fs = "0.1"
 tokio-io = "0.1"
-solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "1.0.0" }
+solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "1.0.3" }
 trees = "0.2.1"
 
 [dev-dependencies]

core/src/cluster_info.rs

@@ -84,6 +84,9 @@ const MAX_GOSSIP_TRAFFIC: usize = 128_000_000 / PACKET_DATA_SIZE;
 const NUM_BITS_PER_BYTE: u64 = 8;
 const MIN_SIZE_TO_COMPRESS_GZIP: u64 = 64;
 
+/// Keep the number of snapshot hashes a node publishes under MAX_PROTOCOL_PAYLOAD_SIZE
+pub const MAX_SNAPSHOT_HASHES: usize = 16;
+
 #[derive(Debug, PartialEq, Eq)]
 pub enum ClusterInfoError {
     NoPeers,
@@ -441,6 +444,14 @@ impl ClusterInfo {
     }
 
     pub fn push_snapshot_hashes(&mut self, snapshot_hashes: Vec<(Slot, Hash)>) {
+        if snapshot_hashes.len() > MAX_SNAPSHOT_HASHES {
+            warn!(
+                "snapshot_hashes too large, ignored: {}",
+                snapshot_hashes.len()
+            );
+            return;
+        }
+
         let now = timestamp();
         let entry = CrdsValue::new_signed(
             CrdsData::SnapshotHash(SnapshotHash::new(self.id(), snapshot_hashes, now)),
@@ -1059,7 +1070,7 @@ impl ClusterInfo {
     }
 
     /// Splits a Vec of CrdsValues into a nested Vec, trying to make sure that
-    /// each Vec is no larger than `PROTOCOL_PAYLOAD_SIZE`
+    /// each Vec is no larger than `MAX_PROTOCOL_PAYLOAD_SIZE`
     /// Note: some messages cannot be contained within that size so in the worst case this returns
     /// N nested Vecs with 1 item each.
     fn split_gossip_messages(msgs: Vec<CrdsValue>) -> Vec<Vec<CrdsValue>> {
@@ -1624,8 +1635,9 @@ impl ClusterInfo {
         id: &Pubkey,
         gossip_addr: &SocketAddr,
     ) -> (ContactInfo, UdpSocket, Option<TcpListener>) {
+        let bind_ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
         let (port, (gossip_socket, ip_echo)) =
-            Node::get_gossip_port(gossip_addr, VALIDATOR_PORT_RANGE);
+            Node::get_gossip_port(gossip_addr, VALIDATOR_PORT_RANGE, bind_ip_addr);
         let contact_info = Self::gossip_contact_info(id, SocketAddr::new(gossip_addr.ip(), port));
 
         (contact_info, gossip_socket, Some(ip_echo))
@@ -1633,7 +1645,8 @@ impl ClusterInfo {
 
     /// A Node with dummy ports to spy on gossip via pull requests
     pub fn spy_node(id: &Pubkey) -> (ContactInfo, UdpSocket, Option<TcpListener>) {
-        let (_, gossip_socket) = bind_in_range(VALIDATOR_PORT_RANGE).unwrap();
+        let bind_ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
+        let (_, gossip_socket) = bind_in_range(bind_ip_addr, VALIDATOR_PORT_RANGE).unwrap();
         let contact_info = Self::spy_contact_info(id);
 
         (contact_info, gossip_socket, None)
@@ -1748,16 +1761,18 @@ impl Node {
         }
     }
     pub fn new_localhost_with_pubkey(pubkey: &Pubkey) -> Self {
+        let bind_ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
         let tpu = UdpSocket::bind("127.0.0.1:0").unwrap();
-        let (gossip_port, (gossip, ip_echo)) = bind_common_in_range((1024, 65535)).unwrap();
+        let (gossip_port, (gossip, ip_echo)) =
+            bind_common_in_range(bind_ip_addr, (1024, 65535)).unwrap();
         let gossip_addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), gossip_port);
         let tvu = UdpSocket::bind("127.0.0.1:0").unwrap();
         let tvu_forwards = UdpSocket::bind("127.0.0.1:0").unwrap();
         let tpu_forwards = UdpSocket::bind("127.0.0.1:0").unwrap();
         let repair = UdpSocket::bind("127.0.0.1:0").unwrap();
-        let rpc_port = find_available_port_in_range((1024, 65535)).unwrap();
+        let rpc_port = find_available_port_in_range(bind_ip_addr, (1024, 65535)).unwrap();
         let rpc_addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), rpc_port);
-        let rpc_pubsub_port = find_available_port_in_range((1024, 65535)).unwrap();
+        let rpc_pubsub_port = find_available_port_in_range(bind_ip_addr, (1024, 65535)).unwrap();
         let rpc_pubsub_addr =
             SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), rpc_pubsub_port);
 
@@ -1800,45 +1815,52 @@ impl Node {
     fn get_gossip_port(
         gossip_addr: &SocketAddr,
         port_range: PortRange,
+        bind_ip_addr: IpAddr,
     ) -> (u16, (UdpSocket, TcpListener)) {
         if gossip_addr.port() != 0 {
             (
                 gossip_addr.port(),
-                bind_common(gossip_addr.port(), false).unwrap_or_else(|e| {
+                bind_common(bind_ip_addr, gossip_addr.port(), false).unwrap_or_else(|e| {
                     panic!("gossip_addr bind_to port {}: {}", gossip_addr.port(), e)
                 }),
             )
         } else {
-            bind_common_in_range(port_range).expect("Failed to bind")
+            bind_common_in_range(bind_ip_addr, port_range).expect("Failed to bind")
         }
     }
-    fn bind(port_range: PortRange) -> (u16, UdpSocket) {
+    fn bind(bind_ip_addr: IpAddr, port_range: PortRange) -> (u16, UdpSocket) {
-        bind_in_range(port_range).expect("Failed to bind")
+        bind_in_range(bind_ip_addr, port_range).expect("Failed to bind")
     }
+
     pub fn new_with_external_ip(
         pubkey: &Pubkey,
         gossip_addr: &SocketAddr,
         port_range: PortRange,
+        bind_ip_addr: IpAddr,
     ) -> Node {
-        let (gossip_port, (gossip, ip_echo)) = Self::get_gossip_port(gossip_addr, port_range);
+        let (gossip_port, (gossip, ip_echo)) =
+            Self::get_gossip_port(gossip_addr, port_range, bind_ip_addr);
 
-        let (tvu_port, tvu_sockets) = multi_bind_in_range(port_range, 8).expect("tvu multi_bind");
+        let (tvu_port, tvu_sockets) =
+            multi_bind_in_range(bind_ip_addr, port_range, 8).expect("tvu multi_bind");
 
         let (tvu_forwards_port, tvu_forwards_sockets) =
-            multi_bind_in_range(port_range, 8).expect("tvu_forwards multi_bind");
+            multi_bind_in_range(bind_ip_addr, port_range, 8).expect("tvu_forwards multi_bind");
 
-        let (tpu_port, tpu_sockets) = multi_bind_in_range(port_range, 32).expect("tpu multi_bind");
+        let (tpu_port, tpu_sockets) =
+            multi_bind_in_range(bind_ip_addr, port_range, 32).expect("tpu multi_bind");
 
         let (tpu_forwards_port, tpu_forwards_sockets) =
-            multi_bind_in_range(port_range, 8).expect("tpu_forwards multi_bind");
+            multi_bind_in_range(bind_ip_addr, port_range, 8).expect("tpu_forwards multi_bind");
 
         let (_, retransmit_sockets) =
-            multi_bind_in_range(port_range, 8).expect("retransmit multi_bind");
+            multi_bind_in_range(bind_ip_addr, port_range, 8).expect("retransmit multi_bind");
 
-        let (repair_port, repair) = Self::bind(port_range);
+        let (repair_port, repair) = Self::bind(bind_ip_addr, port_range);
-        let (serve_repair_port, serve_repair) = Self::bind(port_range);
+        let (serve_repair_port, serve_repair) = Self::bind(bind_ip_addr, port_range);
 
-        let (_, broadcast) = multi_bind_in_range(port_range, 4).expect("broadcast multi_bind");
+        let (_, broadcast) =
+            multi_bind_in_range(bind_ip_addr, port_range, 4).expect("broadcast multi_bind");
 
         let info = ContactInfo {
             id: *pubkey,
@@ -1878,9 +1900,10 @@ impl Node {
         pubkey: &Pubkey,
         gossip_addr: &SocketAddr,
         port_range: PortRange,
+        bind_ip_addr: IpAddr,
     ) -> Node {
-        let mut new = Self::new_with_external_ip(pubkey, gossip_addr, port_range);
+        let mut new = Self::new_with_external_ip(pubkey, gossip_addr, port_range, bind_ip_addr);
-        let (storage_port, storage_socket) = Self::bind(port_range);
+        let (storage_port, storage_socket) = Self::bind(bind_ip_addr, port_range);
 
         new.info.storage_addr = SocketAddr::new(gossip_addr.ip(), storage_port);
         new.sockets.storage = Some(storage_socket);
@@ -2014,6 +2037,7 @@ mod tests {
             &Pubkey::new_rand(),
             &socketaddr!(ip, 0),
             VALIDATOR_PORT_RANGE,
+            IpAddr::V4(ip),
         );
 
         check_node_sockets(&node, IpAddr::V4(ip), VALIDATOR_PORT_RANGE);
@@ -2023,7 +2047,7 @@ mod tests {
     fn new_with_external_ip_test_gossip() {
         let ip = IpAddr::V4(Ipv4Addr::from(0));
         let port = {
-            bind_in_range(VALIDATOR_PORT_RANGE)
+            bind_in_range(ip, VALIDATOR_PORT_RANGE)
                 .expect("Failed to bind")
                 .0
         };
@@ -2031,6 +2055,7 @@ mod tests {
             &Pubkey::new_rand(),
             &socketaddr!(0, port),
             VALIDATOR_PORT_RANGE,
+            ip,
         );
 
         check_node_sockets(&node, ip, VALIDATOR_PORT_RANGE);
@@ -2045,6 +2070,7 @@ mod tests {
             &Pubkey::new_rand(),
             &socketaddr!(ip, 0),
             VALIDATOR_PORT_RANGE,
+            IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),
         );
 
         let ip = IpAddr::V4(ip);

core/src/commitment.rs

@@ -233,7 +233,7 @@ mod tests {
     use crate::genesis_utils::{create_genesis_config, GenesisConfigInfo};
     use solana_sdk::pubkey::Pubkey;
     use solana_stake_program::stake_state;
-    use solana_vote_program::vote_state;
+    use solana_vote_program::vote_state::{self, VoteStateVersions};
 
     #[test]
     fn test_block_commitment() {
@@ -446,13 +446,15 @@ mod tests {
         let mut vote_state1 = VoteState::from(&vote_account1).unwrap();
         vote_state1.process_slot_vote_unchecked(3);
         vote_state1.process_slot_vote_unchecked(5);
-        vote_state1.to(&mut vote_account1).unwrap();
+        let versioned = VoteStateVersions::Current(Box::new(vote_state1));
+        VoteState::to(&versioned, &mut vote_account1).unwrap();
         bank.store_account(&pk1, &vote_account1);
 
         let mut vote_state2 = VoteState::from(&vote_account2).unwrap();
         vote_state2.process_slot_vote_unchecked(9);
         vote_state2.process_slot_vote_unchecked(10);
-        vote_state2.to(&mut vote_account2).unwrap();
+        let versioned = VoteStateVersions::Current(Box::new(vote_state2));
+        VoteState::to(&versioned, &mut vote_account2).unwrap();
         bank.store_account(&pk2, &vote_account2);
 
         let commitment = AggregateCommitmentService::aggregate_commitment(&ancestors, &bank);

core/src/consensus.rs

@@ -483,7 +483,10 @@ pub mod test {
         signature::{Keypair, Signer},
         transaction::Transaction,
     };
-    use solana_vote_program::{vote_instruction, vote_state::Vote};
+    use solana_vote_program::{
+        vote_instruction,
+        vote_state::{Vote, VoteStateVersions},
+    };
     use std::collections::{HashMap, VecDeque};
     use std::sync::RwLock;
     use std::{thread::sleep, time::Duration};
@@ -510,7 +513,7 @@ pub mod test {
             my_keypairs: &ValidatorVoteKeypairs,
             progress: &mut HashMap<u64, ForkProgress>,
             tower: &mut Tower,
-        ) -> VoteResult {
+        ) -> Vec<VoteFailures> {
             let node = self
                 .find_node_and_update_simulation(vote_slot)
                 .expect("Vote to simulate must be for a slot in the tree");
@@ -585,6 +588,7 @@ pub mod test {
                 .values()
                 .cloned()
                 .collect();
+
             ReplayStage::compute_bank_stats(
                 &my_pubkey,
                 &ancestors,
@@ -592,7 +596,6 @@ pub mod test {
                 tower,
                 progress,
             );
-            ReplayStage::select_fork(&frozen_banks, tower, progress);
 
             let bank = bank_forks
                 .read()
@@ -606,12 +609,15 @@ pub mod test {
                 .expect("Slot for vote must exist in progress map");
             info!("Checking vote: {}", vote_slot);
             info!("lockouts: {:?}", fork_progress.fork_stats.stake_lockouts);
-            if fork_progress.fork_stats.is_locked_out && !fork_progress.fork_stats.vote_threshold {
+            let mut failures = vec![];
-                return VoteResult::FailedAllChecks(vote_slot);
+            if fork_progress.fork_stats.is_locked_out {
-            } else if fork_progress.fork_stats.is_locked_out {
+                failures.push(VoteFailures::LockedOut(vote_slot));
-                return VoteResult::LockedOut(vote_slot);
+            }
-            } else if !fork_progress.fork_stats.vote_threshold {
+            if !fork_progress.fork_stats.vote_threshold {
-                return VoteResult::FailedThreshold(vote_slot);
+                failures.push(VoteFailures::FailedThreshold(vote_slot));
+            }
+            if !failures.is_empty() {
+                return failures;
             }
             let vote = tower.new_vote_from_bank(&bank, &my_vote_pubkey).0;
             if let Some(new_root) = tower.record_bank_vote(vote) {
@@ -621,7 +627,7 @@ pub mod test {
             // Mark the vote for this bank under this node's pubkey so it will be
             // integrated into any future child banks
             cluster_votes.entry(my_pubkey).or_default().push(vote_slot);
-            VoteResult::Ok
+            vec![]
         }
 
         // Find a node representing the given slot
@@ -666,11 +672,9 @@ pub mod test {
     }
 
     #[derive(PartialEq, Debug)]
-    pub(crate) enum VoteResult {
+    pub(crate) enum VoteFailures {
         LockedOut(u64),
         FailedThreshold(u64),
-        FailedAllChecks(u64),
-        Ok,
     }
 
     // Setup BankForks with bank 0 and all the validator accounts
@@ -706,9 +710,11 @@ pub mod test {
             for slot in *votes {
                 vote_state.process_slot_vote_unchecked(*slot);
             }
-            vote_state
+            VoteState::serialize(
-                .serialize(&mut account.data)
+                &VoteStateVersions::Current(Box::new(vote_state)),
-                .expect("serialize state");
+                &mut account.data,
+            )
+            .expect("serialize state");
             stakes.push((Pubkey::new_rand(), (*lamports, account)));
         }
         stakes
@@ -782,9 +788,8 @@ pub mod test {
 
         let mut cluster_votes = HashMap::new();
         for vote in votes {
-            assert_eq!(
+            assert!(voting_simulator
-                VoteResult::Ok,
+                .simulate_vote(
-                voting_simulator.simulate_vote(
                     vote,
                     &bank_forks,
                     &mut cluster_votes,
@@ -793,7 +798,7 @@ pub mod test {
                     &mut progress,
                     &mut tower,
                 )
-            );
+                .is_empty());
         }
 
         for i in 0..5 {
@@ -856,8 +861,8 @@ pub mod test {
         let mut tower = Tower::new_with_key(&node_pubkey);
         for vote in &votes {
             // All these votes should be ok
-            assert_eq!(
+            assert!(voting_simulator
-                voting_simulator.simulate_vote(
+                .simulate_vote(
                     *vote,
                     &bank_forks,
                     &mut cluster_votes,
@@ -865,15 +870,14 @@ pub mod test {
                     keypairs.get(&node_pubkey).unwrap(),
                     &mut progress,
                     &mut tower,
-                ),
+                )
-                VoteResult::Ok
+                .is_empty());
-            );
         }
 
         // Try to come back to main fork
         let next_unlocked_slot = 110;
-        assert_eq!(
+        assert!(voting_simulator
-            voting_simulator.simulate_vote(
+            .simulate_vote(
                 next_unlocked_slot,
                 &bank_forks,
                 &mut cluster_votes,
@@ -881,9 +885,8 @@ pub mod test {
                 keypairs.get(&node_pubkey).unwrap(),
                 &mut progress,
                 &mut tower,
-            ),
+            )
-            VoteResult::Ok
+            .is_empty());
-        );
 
         info!("local tower: {:#?}", tower.lockouts.votes);
         let vote_accounts = bank_forks

core/src/gossip_service.rs

@@ -9,7 +9,7 @@ use solana_ledger::bank_forks::BankForks;
 use solana_perf::recycler::Recycler;
 use solana_sdk::pubkey::Pubkey;
 use solana_sdk::signature::{Keypair, Signer};
-use std::net::{SocketAddr, TcpListener, UdpSocket};
+use std::net::{IpAddr, Ipv4Addr, SocketAddr, TcpListener, UdpSocket};
 use std::sync::atomic::{AtomicBool, Ordering};
 use std::sync::mpsc::channel;
 use std::sync::{Arc, RwLock};
@@ -163,7 +163,11 @@ pub fn get_multi_client(nodes: &[ContactInfo]) -> (ThinClient, usize) {
         .collect();
     let rpc_addrs: Vec<_> = addrs.iter().map(|addr| addr.0).collect();
     let tpu_addrs: Vec<_> = addrs.iter().map(|addr| addr.1).collect();
-    let (_, transactions_socket) = solana_net_utils::bind_in_range(VALIDATOR_PORT_RANGE).unwrap();
+    let (_, transactions_socket) = solana_net_utils::bind_in_range(
+        IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),
+        VALIDATOR_PORT_RANGE,
+    )
+    .unwrap();
     let num_nodes = tpu_addrs.len();
     (
         ThinClient::new_from_addrs(rpc_addrs, tpu_addrs, transactions_socket),

core/src/local_vote_signer_service.rs

@@ -15,9 +15,12 @@ pub struct LocalVoteSignerService {
 impl LocalVoteSignerService {
     #[allow(clippy::new_ret_no_self)]
     pub fn new(port_range: PortRange) -> (Self, SocketAddr) {
-        let addr = solana_net_utils::find_available_port_in_range(port_range)
+        let ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
-            .map(|port| SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), port))
+        let addr = SocketAddr::new(
-            .expect("Failed to find an available port for local vote signer service");
+            ip_addr,
+            solana_net_utils::find_available_port_in_range(ip_addr, port_range)
+                .expect("Failed to find an available port for local vote signer service"),
+        );
         let exit = Arc::new(AtomicBool::new(false));
         let thread_exit = exit.clone();
         let thread = Builder::new()

core/src/poh_recorder.rs

@@ -28,8 +28,8 @@ use std::sync::{Arc, Mutex};
 use std::time::Instant;
 use thiserror::Error;
 
-const GRACE_TICKS_FACTOR: u64 = 2;
+pub const GRACE_TICKS_FACTOR: u64 = 2;
-const MAX_GRACE_SLOTS: u64 = 2;
+pub const MAX_GRACE_SLOTS: u64 = 2;
 
 #[derive(Error, Debug, Clone)]
 pub enum PohRecorderError {
@@ -85,6 +85,7 @@ impl PohRecorder {
                 bank.slot(),
                 &bank,
                 Some(&self.blockstore),
+                GRACE_TICKS_FACTOR * MAX_GRACE_SLOTS,
             );
             assert_eq!(self.ticks_per_slot, bank.ticks_per_slot());
             let (leader_first_tick_height, leader_last_tick_height, grace_ticks) =
@@ -151,6 +152,17 @@ impl PohRecorder {
         })
     }
 
+    fn prev_slot_was_mine(&self, current_slot: Slot) -> bool {
+        if let Some(leader_id) = self
+            .leader_schedule_cache
+            .slot_leader_at(current_slot.saturating_sub(1), None)
+        {
+            leader_id == self.id
+        } else {
+            false
+        }
+    }
+
     fn reached_leader_tick(&self, leader_first_tick_height: u64) -> bool {
         let target_tick_height = leader_first_tick_height.saturating_sub(1);
         let ideal_target_tick_height = target_tick_height.saturating_sub(self.grace_ticks);
@@ -160,7 +172,8 @@ impl PohRecorder {
         self.tick_height >= target_tick_height
             || self.start_tick_height + self.grace_ticks == leader_first_tick_height
             || (self.tick_height >= ideal_target_tick_height
-                && !self.received_any_previous_leader_data(current_slot))
+                && (self.prev_slot_was_mine(current_slot)
+                    || !self.received_any_previous_leader_data(current_slot)))
     }
 
     /// returns if leader slot has been reached, how many grace ticks were afforded,
@@ -1130,6 +1143,7 @@ mod tests {
             let GenesisConfigInfo { genesis_config, .. } = create_genesis_config(2);
             let bank = Arc::new(Bank::new(&genesis_config));
             let prev_hash = bank.last_blockhash();
+            let leader_schedule_cache = Arc::new(LeaderScheduleCache::new_from_bank(&bank));
             let (mut poh_recorder, _entry_receiver) = PohRecorder::new(
                 0,
                 prev_hash,
@@ -1138,10 +1152,12 @@ mod tests {
                 bank.ticks_per_slot(),
                 &Pubkey::default(),
                 &Arc::new(blockstore),
-                &Arc::new(LeaderScheduleCache::new_from_bank(&bank)),
+                &leader_schedule_cache,
                 &Arc::new(PohConfig::default()),
             );
 
+            let bootstrap_validator_id = leader_schedule_cache.slot_leader_at(0, None).unwrap();
+
             assert_eq!(poh_recorder.reached_leader_tick(0), true);
 
             let grace_ticks = bank.ticks_per_slot() * MAX_GRACE_SLOTS;
@@ -1170,6 +1186,11 @@ mod tests {
                 poh_recorder.reached_leader_tick(new_tick_height + grace_ticks),
                 false
             );
+
+            // From the bootstrap validator's perspective, it should have reached
+            // the tick
+            poh_recorder.id = bootstrap_validator_id;
+            assert!(poh_recorder.reached_leader_tick(new_tick_height + grace_ticks));
         }
     }
 

core/src/repair_service.rs

@@ -12,7 +12,8 @@ use solana_ledger::{
 use solana_sdk::clock::DEFAULT_SLOTS_PER_EPOCH;
 use solana_sdk::{clock::Slot, epoch_schedule::EpochSchedule, pubkey::Pubkey};
 use std::{
-    collections::BTreeSet,
+    collections::{BTreeSet, HashSet},
+    iter::Iterator,
     net::UdpSocket,
     ops::Bound::{Included, Unbounded},
     sync::atomic::{AtomicBool, Ordering},
@@ -209,10 +210,8 @@ impl RepairService {
         // TODO: Incorporate gossip to determine priorities for repair?
 
         // Try to resolve orphans in blockstore
-        let mut orphans = blockstore.get_orphans(Some(MAX_ORPHANS));
+        let orphans = blockstore.orphans_iterator(root + 1).unwrap();
-        orphans.retain(|x| *x > root);
+        Self::generate_repairs_for_orphans(orphans, &mut repairs);
-
-        Self::generate_repairs_for_orphans(&orphans[..], &mut repairs);
         Ok(repairs)
     }
 
@@ -240,8 +239,11 @@ impl RepairService {
         }
     }
 
-    fn generate_repairs_for_orphans(orphans: &[u64], repairs: &mut Vec<RepairType>) {
+    fn generate_repairs_for_orphans(
-        repairs.extend(orphans.iter().map(|h| RepairType::Orphan(*h)));
+        orphans: impl Iterator<Item = u64>,
+        repairs: &mut Vec<RepairType>,
+    ) {
+        repairs.extend(orphans.take(MAX_ORPHANS).map(RepairType::Orphan));
     }
 
     /// Repairs any fork starting at the input slot
@@ -317,7 +319,7 @@ impl RepairService {
         );
     }
 
-    // Update the gossiped structure used for the "Repairmen" repair protocol. See book
+    // Update the gossiped structure used for the "Repairmen" repair protocol. See docs
     // for details.
     fn update_epoch_slots(
         id: Pubkey,
@@ -402,6 +404,20 @@ impl RepairService {
         }
     }
 
+    #[allow(dead_code)]
+    fn find_incomplete_slots(blockstore: &Blockstore, root: Slot) -> HashSet<Slot> {
+        blockstore
+            .live_slots_iterator(root)
+            .filter_map(|(slot, slot_meta)| {
+                if !slot_meta.is_full() {
+                    Some(slot)
+                } else {
+                    None
+                }
+            })
+            .collect()
+    }
+
     pub fn join(self) -> thread::Result<()> {
         self.t_repair.join()
     }
@@ -914,4 +930,63 @@ mod test {
         );
         assert_eq!(stash.len(), 2);
     }
+
+    #[test]
+    fn test_find_incomplete_slots() {
+        let blockstore_path = get_tmp_ledger_path!();
+        {
+            let blockstore = Blockstore::open(&blockstore_path).unwrap();
+            let num_entries_per_slot = 100;
+            let (mut shreds, _) = make_slot_entries(0, 0, num_entries_per_slot);
+            assert!(shreds.len() > 1);
+            let (shreds4, _) = make_slot_entries(4, 0, num_entries_per_slot);
+            shreds.extend(shreds4);
+            blockstore.insert_shreds(shreds, None, false).unwrap();
+
+            // Nothing is incomplete
+            assert!(RepairService::find_incomplete_slots(&blockstore, 0).is_empty());
+
+            // Insert a slot 5 that chains to an incomplete orphan slot 3
+            let (shreds5, _) = make_slot_entries(5, 3, num_entries_per_slot);
+            blockstore.insert_shreds(shreds5, None, false).unwrap();
+            assert_eq!(
+                RepairService::find_incomplete_slots(&blockstore, 0),
+                vec![3].into_iter().collect()
+            );
+
+            // Insert another incomplete orphan slot 2 that is the parent of slot 3.
+            // Both should be incomplete
+            let (shreds3, _) = make_slot_entries(3, 2, num_entries_per_slot);
+            blockstore
+                .insert_shreds(shreds3[1..].to_vec(), None, false)
+                .unwrap();
+            assert_eq!(
+                RepairService::find_incomplete_slots(&blockstore, 0),
+                vec![2, 3].into_iter().collect()
+            );
+
+            // Insert a incomplete slot 6 that chains to the root 0,
+            // should also be incomplete
+            let (shreds6, _) = make_slot_entries(6, 0, num_entries_per_slot);
+            blockstore
+                .insert_shreds(shreds6[1..].to_vec(), None, false)
+                .unwrap();
+            assert_eq!(
+                RepairService::find_incomplete_slots(&blockstore, 0),
+                vec![2, 3, 6].into_iter().collect()
+            );
+
+            // Complete slot 3, should no longer be marked incomplete
+            blockstore
+                .insert_shreds(shreds3[..].to_vec(), None, false)
+                .unwrap();
+
+            assert_eq!(
+                RepairService::find_incomplete_slots(&blockstore, 0),
+                vec![2, 6].into_iter().collect()
+            );
+        }
+
+        Blockstore::destroy(&blockstore_path).expect("Expected successful database destruction");
+    }
 }

core/src/replay_stage.rs

@@ -4,7 +4,7 @@ use crate::{
     cluster_info::ClusterInfo,
     commitment::{AggregateCommitmentService, BlockCommitmentCache, CommitmentAggregationData},
     consensus::{StakeLockout, Tower},
-    poh_recorder::PohRecorder,
+    poh_recorder::{PohRecorder, GRACE_TICKS_FACTOR, MAX_GRACE_SLOTS},
     result::Result,
     rewards_recorder_service::RewardsRecorderSender,
     rpc_subscriptions::RpcSubscriptions,
@@ -252,146 +252,138 @@ impl ReplayStage {
                     );
 
                     let ancestors = Arc::new(bank_forks.read().unwrap().ancestors());
-                    loop {
+                    let start = allocated.get();
-                        let start = allocated.get();
+                    let mut frozen_banks: Vec<_> = bank_forks
-                        let mut frozen_banks: Vec<_> = bank_forks
+                        .read()
-                            .read()
+                        .unwrap()
-                            .unwrap()
+                        .frozen_banks()
-                            .frozen_banks()
+                        .values()
-                            .values()
+                        .cloned()
-                            .cloned()
+                        .collect();
-                            .collect();
+                    let newly_computed_slot_stats = Self::compute_bank_stats(
-                        let newly_computed_slot_stats = Self::compute_bank_stats(
+                        &my_pubkey,
-                            &my_pubkey,
+                        &ancestors,
-                            &ancestors,
+                        &mut frozen_banks,
-                            &mut frozen_banks,
+                        &tower,
+                        &mut progress,
+                    );
+                    for slot in newly_computed_slot_stats {
+                        let fork_stats = &progress.get(&slot).unwrap().fork_stats;
+                        let confirmed_forks = Self::confirm_forks(
                             &tower,
-                            &mut progress,
+                            &fork_stats.stake_lockouts,
+                            fork_stats.total_staked,
+                            &progress,
+                            &bank_forks,
                         );
-                        for slot in newly_computed_slot_stats {
-                            let fork_stats = &progress.get(&slot).unwrap().fork_stats;
-                            let confirmed_forks = Self::confirm_forks(
-                                &tower,
-                                &fork_stats.stake_lockouts,
-                                fork_stats.total_staked,
-                                &progress,
-                                &bank_forks,
-                            );
 
-                            for slot in confirmed_forks {
+                        for slot in confirmed_forks {
-                                progress
+                            progress
-                                    .get_mut(&slot)
+                                .get_mut(&slot)
-                                    .unwrap()
+                                .unwrap()
-                                    .fork_stats
+                                .fork_stats
-                                    .confirmation_reported = true;
+                                .confirmation_reported = true;
-                            }
-                        }
-
-                        let vote_bank = Self::select_fork(&frozen_banks, &tower, &mut progress);
-                        datapoint_debug!(
-                            "replay_stage-memory",
-                            ("select_fork", (allocated.get() - start) as i64, i64),
-                        );
-                        if vote_bank.is_none() {
-                            break;
-                        }
-                        let bank = vote_bank.unwrap();
-                        let (is_locked_out, vote_threshold, fork_weight, total_staked) = {
-                            let fork_stats = &progress.get(&bank.slot()).unwrap().fork_stats;
-                            (
-                                fork_stats.is_locked_out,
-                                fork_stats.vote_threshold,
-                                fork_stats.weight,
-                                fork_stats.total_staked,
-                            )
-                        };
-                        let mut done = false;
-                        let mut vote_bank_slot = None;
-                        let start = allocated.get();
-                        if !is_locked_out && vote_threshold {
-                            info!("voting: {} {}", bank.slot(), fork_weight);
-                            subscriptions.notify_subscribers(bank.slot(), &bank_forks);
-                            if let Some(votable_leader) =
-                                leader_schedule_cache.slot_leader_at(bank.slot(), Some(&bank))
-                            {
-                                Self::log_leader_change(
-                                    &my_pubkey,
-                                    bank.slot(),
-                                    &mut current_leader,
-                                    &votable_leader,
-                                );
-                            }
-                            vote_bank_slot = Some(bank.slot());
-                            Self::handle_votable_bank(
-                                &bank,
-                                &bank_forks,
-                                &mut tower,
-                                &mut progress,
-                                &vote_account,
-                                &voting_keypair,
-                                &cluster_info,
-                                &blockstore,
-                                &leader_schedule_cache,
-                                &root_bank_sender,
-                                total_staked,
-                                &lockouts_sender,
-                                &snapshot_package_sender,
-                                &latest_root_senders,
-                            )?;
-                        }
-                        datapoint_debug!(
-                            "replay_stage-memory",
-                            ("votable_bank", (allocated.get() - start) as i64, i64),
-                        );
-                        let start = allocated.get();
-                        if last_reset != bank.last_blockhash() {
-                            Self::reset_poh_recorder(
-                                &my_pubkey,
-                                &blockstore,
-                                &bank,
-                                &poh_recorder,
-                                &leader_schedule_cache,
-                            );
-                            last_reset = bank.last_blockhash();
-                            tpu_has_bank = false;
-                            info!(
-                                "vote bank: {:?} reset bank: {}",
-                                vote_bank_slot,
-                                bank.slot()
-                            );
-                            if !partition && vote_bank_slot != Some(bank.slot()) {
-                                warn!(
-                                    "PARTITION DETECTED waiting to join fork: {} last vote: {:?}",
-                                    bank.slot(),
-                                    tower.last_vote()
-                                );
-                                inc_new_counter_info!("replay_stage-partition_detected", 1);
-                                datapoint_info!(
-                                    "replay_stage-partition",
-                                    ("slot", bank.slot() as i64, i64)
-                                );
-                                partition = true;
-                            } else if partition && vote_bank_slot == Some(bank.slot()) {
-                                warn!(
-                                    "PARTITION resolved fork: {} last vote: {:?}",
-                                    bank.slot(),
-                                    tower.last_vote()
-                                );
-                                partition = false;
-                                inc_new_counter_info!("replay_stage-partition_resolved", 1);
-                            }
-                        } else {
-                            done = true;
-                        }
-                        datapoint_debug!(
-                            "replay_stage-memory",
-                            ("reset_bank", (allocated.get() - start) as i64, i64),
-                        );
-                        if done {
-                            break;
                         }
                     }
 
+                    let vote_bank = Self::select_fork(&frozen_banks, &tower, &progress);
+                    datapoint_debug!(
+                        "replay_stage-memory",
+                        ("select_fork", (allocated.get() - start) as i64, i64),
+                    );
+                    if vote_bank.is_none() {
+                        break;
+                    }
+                    let bank = vote_bank.unwrap();
+                    let (is_locked_out, vote_threshold, fork_weight, total_staked) = {
+                        let fork_stats = &progress.get(&bank.slot()).unwrap().fork_stats;
+                        (
+                            fork_stats.is_locked_out,
+                            fork_stats.vote_threshold,
+                            fork_stats.weight,
+                            fork_stats.total_staked,
+                        )
+                    };
+                    let mut vote_bank_slot = None;
+                    let start = allocated.get();
+                    if !is_locked_out && vote_threshold {
+                        info!("voting: {} {}", bank.slot(), fork_weight);
+                        subscriptions.notify_subscribers(bank.slot(), &bank_forks);
+                        if let Some(votable_leader) =
+                            leader_schedule_cache.slot_leader_at(bank.slot(), Some(&bank))
+                        {
+                            Self::log_leader_change(
+                                &my_pubkey,
+                                bank.slot(),
+                                &mut current_leader,
+                                &votable_leader,
+                            );
+                        }
+                        vote_bank_slot = Some(bank.slot());
+                        Self::handle_votable_bank(
+                            &bank,
+                            &bank_forks,
+                            &mut tower,
+                            &mut progress,
+                            &vote_account,
+                            &voting_keypair,
+                            &cluster_info,
+                            &blockstore,
+                            &leader_schedule_cache,
+                            &root_bank_sender,
+                            total_staked,
+                            &lockouts_sender,
+                            &snapshot_package_sender,
+                            &latest_root_senders,
+                        )?;
+                    }
+                    datapoint_debug!(
+                        "replay_stage-memory",
+                        ("votable_bank", (allocated.get() - start) as i64, i64),
+                    );
+                    let start = allocated.get();
+                    if last_reset != bank.last_blockhash() {
+                        Self::reset_poh_recorder(
+                            &my_pubkey,
+                            &blockstore,
+                            &bank,
+                            &poh_recorder,
+                            &leader_schedule_cache,
+                        );
+                        last_reset = bank.last_blockhash();
+                        tpu_has_bank = false;
+                        info!(
+                            "vote bank: {:?} reset bank: {}",
+                            vote_bank_slot,
+                            bank.slot()
+                        );
+                        if !partition && vote_bank_slot != Some(bank.slot()) {
+                            warn!(
+                                "PARTITION DETECTED waiting to join fork: {} last vote: {:?}",
+                                bank.slot(),
+                                tower.last_vote()
+                            );
+                            inc_new_counter_info!("replay_stage-partition_detected", 1);
+                            datapoint_info!(
+                                "replay_stage-partition",
+                                ("slot", bank.slot() as i64, i64)
+                            );
+                            partition = true;
+                        } else if partition && vote_bank_slot == Some(bank.slot()) {
+                            warn!(
+                                "PARTITION resolved fork: {} last vote: {:?}",
+                                bank.slot(),
+                                tower.last_vote()
+                            );
+                            partition = false;
+                            inc_new_counter_info!("replay_stage-partition_resolved", 1);
+                        }
+                    }
+                    datapoint_debug!(
+                        "replay_stage-memory",
+                        ("reset_bank", (allocated.get() - start) as i64, i64),
+                    );
+
                     let start = allocated.get();
                     if !tpu_has_bank {
                         Self::maybe_start_leader(
@@ -690,6 +682,7 @@ impl ReplayStage {
             bank.slot(),
             &bank,
             Some(blockstore),
+            GRACE_TICKS_FACTOR * MAX_GRACE_SLOTS,
         );
         poh_recorder
             .lock()
@@ -831,6 +824,7 @@ impl ReplayStage {
                 stats.stake_lockouts = stake_lockouts;
                 stats.block_height = bank.block_height();
                 stats.computed = true;
+                new_stats.push(stats.slot);
             }
             stats.vote_threshold = tower.check_vote_stake_threshold(
                 bank.slot(),
@@ -840,7 +834,6 @@ impl ReplayStage {
             stats.is_locked_out = tower.is_locked_out(bank.slot(), &ancestors);
             stats.has_voted = tower.has_voted(bank.slot());
             stats.is_recent = tower.is_recent(bank.slot());
-            new_stats.push(stats.slot);
         }
         new_stats
     }
@@ -848,7 +841,7 @@ impl ReplayStage {
     pub(crate) fn select_fork(
         frozen_banks: &[Arc<Bank>],
         tower: &Tower,
-        progress: &mut HashMap<u64, ForkProgress>,
+        progress: &HashMap<u64, ForkProgress>,
     ) -> Option<Arc<Bank>> {
         let tower_start = Instant::now();
         let num_frozen_banks = frozen_banks.len();
@@ -971,6 +964,7 @@ impl ReplayStage {
         bank: Arc<Bank>,
         slot_full_senders: &[Sender<(u64, Pubkey)>],
     ) {
+        info!("bank frozen: {}", bank.slot());
         bank.freeze();
         slot_full_senders.iter().for_each(|sender| {
             if let Err(e) = sender.send((bank.slot(), *bank.collector_id())) {
@@ -1055,7 +1049,7 @@ pub(crate) mod tests {
     use super::*;
     use crate::{
         commitment::BlockCommitment,
-        consensus::test::{initialize_state, VoteResult, VoteSimulator},
+        consensus::test::{initialize_state, VoteSimulator},
         consensus::Tower,
         genesis_utils::{create_genesis_config, create_genesis_config_with_leader},
         replay_stage::ReplayStage,
@@ -1087,7 +1081,10 @@ pub(crate) mod tests {
         transaction::TransactionError,
     };
     use solana_stake_program::stake_state;
-    use solana_vote_program::vote_state::{self, Vote, VoteState};
+    use solana_vote_program::{
+        vote_state::{self, Vote, VoteState, VoteStateVersions},
+        vote_transaction,
+    };
     use std::{
         fs::remove_dir_all,
         iter,
@@ -1122,7 +1119,8 @@ pub(crate) mod tests {
             let mut vote_account = bank.get_account(&pubkey).unwrap();
             let mut vote_state = VoteState::from(&vote_account).unwrap();
             vote_state.process_slot_vote_unchecked(slot);
-            vote_state.to(&mut vote_account).unwrap();
+            let versioned = VoteStateVersions::Current(Box::new(vote_state));
+            VoteState::to(&versioned, &mut vote_account).unwrap();
             bank.store_account(&pubkey, &vote_account);
         }
 
@@ -1298,7 +1296,7 @@ pub(crate) mod tests {
                     &mut fork_progresses[i],
                 );
                 let response =
-                    ReplayStage::select_fork(&frozen_banks, &towers[i], &mut fork_progresses[i]);
+                    ReplayStage::select_fork(&frozen_banks, &towers[i], &fork_progresses[i]);
 
                 if response.is_none() {
                     None
@@ -1706,7 +1704,8 @@ pub(crate) mod tests {
             let mut leader_vote_account = bank.get_account(&pubkey).unwrap();
             let mut vote_state = VoteState::from(&leader_vote_account).unwrap();
             vote_state.process_slot_vote_unchecked(bank.slot());
-            vote_state.to(&mut leader_vote_account).unwrap();
+            let versioned = VoteStateVersions::Current(Box::new(vote_state));
+            VoteState::to(&versioned, &mut leader_vote_account).unwrap();
             bank.store_account(&pubkey, &leader_vote_account);
         }
 
@@ -1914,6 +1913,119 @@ pub(crate) mod tests {
         Blockstore::destroy(&ledger_path).unwrap();
     }
 
+    #[test]
+    fn test_compute_bank_stats_confirmed() {
+        let node_keypair = Keypair::new();
+        let vote_keypair = Keypair::new();
+        let stake_keypair = Keypair::new();
+        let node_pubkey = node_keypair.pubkey();
+        let mut keypairs = HashMap::new();
+        keypairs.insert(
+            node_pubkey,
+            ValidatorVoteKeypairs::new(node_keypair, vote_keypair, stake_keypair),
+        );
+
+        let (bank_forks, mut progress) = initialize_state(&keypairs);
+        let bank0 = bank_forks.get(0).unwrap().clone();
+        let my_keypairs = keypairs.get(&node_pubkey).unwrap();
+        let vote_tx = vote_transaction::new_vote_transaction(
+            vec![0],
+            bank0.hash(),
+            bank0.last_blockhash(),
+            &my_keypairs.node_keypair,
+            &my_keypairs.vote_keypair,
+            &my_keypairs.vote_keypair,
+        );
+
+        let bank_forks = RwLock::new(bank_forks);
+        let bank1 = Bank::new_from_parent(&bank0, &node_pubkey, 1);
+        bank1.process_transaction(&vote_tx).unwrap();
+        bank1.freeze();
+
+        // Test confirmations
+        let ancestors = bank_forks.read().unwrap().ancestors();
+        let mut frozen_banks: Vec<_> = bank_forks
+            .read()
+            .unwrap()
+            .frozen_banks()
+            .values()
+            .cloned()
+            .collect();
+        let tower = Tower::new_for_tests(0, 0.67);
+        let newly_computed = ReplayStage::compute_bank_stats(
+            &node_pubkey,
+            &ancestors,
+            &mut frozen_banks,
+            &tower,
+            &mut progress,
+        );
+        assert_eq!(newly_computed, vec![0]);
+        // The only vote is in bank 1, and bank_forks does not currently contain
+        // bank 1, so no slot should be confirmed.
+        {
+            let fork_progress = progress.get(&0).unwrap();
+            let confirmed_forks = ReplayStage::confirm_forks(
+                &tower,
+                &fork_progress.fork_stats.stake_lockouts,
+                fork_progress.fork_stats.total_staked,
+                &progress,
+                &bank_forks,
+            );
+
+            assert!(confirmed_forks.is_empty())
+        }
+
+        // Insert the bank that contains a vote for slot 0, which confirms slot 0
+        bank_forks.write().unwrap().insert(bank1);
+        progress.insert(1, ForkProgress::new(bank0.last_blockhash()));
+        let ancestors = bank_forks.read().unwrap().ancestors();
+        let mut frozen_banks: Vec<_> = bank_forks
+            .read()
+            .unwrap()
+            .frozen_banks()
+            .values()
+            .cloned()
+            .collect();
+        let newly_computed = ReplayStage::compute_bank_stats(
+            &node_pubkey,
+            &ancestors,
+            &mut frozen_banks,
+            &tower,
+            &mut progress,
+        );
+
+        assert_eq!(newly_computed, vec![1]);
+        {
+            let fork_progress = progress.get(&1).unwrap();
+            let confirmed_forks = ReplayStage::confirm_forks(
+                &tower,
+                &fork_progress.fork_stats.stake_lockouts,
+                fork_progress.fork_stats.total_staked,
+                &progress,
+                &bank_forks,
+            );
+            assert_eq!(confirmed_forks, vec![0]);
+        }
+
+        let ancestors = bank_forks.read().unwrap().ancestors();
+        let mut frozen_banks: Vec<_> = bank_forks
+            .read()
+            .unwrap()
+            .frozen_banks()
+            .values()
+            .cloned()
+            .collect();
+        let newly_computed = ReplayStage::compute_bank_stats(
+            &node_pubkey,
+            &ancestors,
+            &mut frozen_banks,
+            &tower,
+            &mut progress,
+        );
+        // No new stats should have been computed
+        assert!(newly_computed.is_empty());
+    }
+
     #[test]
     fn test_child_bank_heavier() {
         let node_keypair = Keypair::new();
@@ -1937,8 +2049,8 @@ pub(crate) mod tests {
         let mut cluster_votes: HashMap<Pubkey, Vec<Slot>> = HashMap::new();
         let votes: Vec<Slot> = vec![0, 2];
         for vote in &votes {
-            assert_eq!(
+            assert!(voting_simulator
-                voting_simulator.simulate_vote(
+                .simulate_vote(
                     *vote,
                     &bank_forks,
                     &mut cluster_votes,
@@ -1946,9 +2058,8 @@ pub(crate) mod tests {
                     keypairs.get(&node_pubkey).unwrap(),
                     &mut progress,
                     &mut tower,
-                ),
+                )
-                VoteResult::Ok
+                .is_empty());
-            );
         }
 
         let mut frozen_banks: Vec<_> = bank_forks

core/src/retransmit_stage.rs

@@ -284,6 +284,7 @@ mod tests {
     use solana_ledger::create_new_tmp_ledger;
     use solana_net_utils::find_available_port_in_range;
     use solana_sdk::pubkey::Pubkey;
+    use std::net::{IpAddr, Ipv4Addr};
 
     #[test]
     fn test_skip_repair() {
@@ -300,11 +301,12 @@ mod tests {
         let bank_forks = Arc::new(RwLock::new(bank_forks));
 
         let mut me = ContactInfo::new_localhost(&Pubkey::new_rand(), 0);
-        let port = find_available_port_in_range((8000, 10000)).unwrap();
+        let ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
+        let port = find_available_port_in_range(ip_addr, (8000, 10000)).unwrap();
         let me_retransmit = UdpSocket::bind(format!("127.0.0.1:{}", port)).unwrap();
         // need to make sure tvu and tpu are valid addresses
         me.tvu_forwards = me_retransmit.local_addr().unwrap();
-        let port = find_available_port_in_range((8000, 10000)).unwrap();
+        let port = find_available_port_in_range(ip_addr, (8000, 10000)).unwrap();
         me.tvu = UdpSocket::bind(format!("127.0.0.1:{}", port))
             .unwrap()
             .local_addr()

core/src/rpc.rs

@@ -9,9 +9,9 @@ use jsonrpc_core::{Error, Metadata, Result};
 use jsonrpc_derive::rpc;
 use solana_client::rpc_response::{
     Response, RpcAccount, RpcBlockCommitment, RpcBlockhashFeeCalculator, RpcConfirmedBlock,
-    RpcContactInfo, RpcEpochInfo, RpcKeyedAccount, RpcLeaderSchedule, RpcResponseContext,
+    RpcContactInfo, RpcEpochInfo, RpcFeeRateGovernor, RpcIdentity, RpcKeyedAccount,
-    RpcSignatureConfirmation, RpcStorageTurn, RpcTransactionEncoding, RpcVersionInfo,
+    RpcLeaderSchedule, RpcResponseContext, RpcSignatureConfirmation, RpcStorageTurn,
-    RpcVoteAccountInfo, RpcVoteAccountStatus,
+    RpcTransactionEncoding, RpcVersionInfo, RpcVoteAccountInfo, RpcVoteAccountStatus,
 };
 use solana_faucet::faucet::request_airdrop_transaction;
 use solana_ledger::{
@@ -49,6 +49,7 @@ fn new_response<T>(bank: &Bank, value: T) -> RpcResponse<T> {
 pub struct JsonRpcConfig {
     pub enable_validator_exit: bool,
     pub enable_get_confirmed_block: bool,
+    pub identity_pubkey: Pubkey,
     pub faucet_addr: Option<SocketAddr>,
 }
 
@@ -164,6 +165,17 @@ impl JsonRpcRequestProcessor {
         )
     }
 
+    fn get_fee_rate_governor(&self) -> RpcResponse<RpcFeeRateGovernor> {
+        let bank = &*self.bank(None);
+        let fee_rate_governor = bank.get_fee_rate_governor();
+        new_response(
+            bank,
+            RpcFeeRateGovernor {
+                fee_rate_governor: fee_rate_governor.clone(),
+            },
+        )
+    }
+
     pub fn confirm_transaction(
         &self,
         signature: Result<Signature>,
@@ -491,6 +503,9 @@ pub trait RpcSol {
         commitment: Option<CommitmentConfig>,
     ) -> RpcResponse<RpcBlockhashFeeCalculator>;
 
+    #[rpc(meta, name = "getFeeRateGovernor")]
+    fn get_fee_rate_governor(&self, meta: Self::Metadata) -> RpcResponse<RpcFeeRateGovernor>;
+
     #[rpc(meta, name = "getSignatureStatus")]
     fn get_signature_status(
         &self,
@@ -580,6 +595,9 @@ pub trait RpcSol {
         commitment: Option<CommitmentConfig>,
     ) -> Result<Option<RpcSignatureConfirmation>>;
 
+    #[rpc(meta, name = "getIdentity")]
+    fn get_identity(&self, meta: Self::Metadata) -> Result<RpcIdentity>;
+
     #[rpc(meta, name = "getVersion")]
     fn get_version(&self, meta: Self::Metadata) -> Result<RpcVersionInfo>;
 
@@ -813,6 +831,14 @@ impl RpcSol for RpcSolImpl {
             .get_recent_blockhash(commitment)
     }
 
+    fn get_fee_rate_governor(&self, meta: Self::Metadata) -> RpcResponse<RpcFeeRateGovernor> {
+        debug!("get_fee_rate_governor rpc request received");
+        meta.request_processor
+            .read()
+            .unwrap()
+            .get_fee_rate_governor()
+    }
+
     fn get_signature_status(
         &self,
         meta: Self::Metadata,
@@ -1054,6 +1080,18 @@ impl RpcSol for RpcSolImpl {
         meta.request_processor.read().unwrap().validator_exit()
     }
 
+    fn get_identity(&self, meta: Self::Metadata) -> Result<RpcIdentity> {
+        Ok(RpcIdentity {
+            identity: meta
+                .request_processor
+                .read()
+                .unwrap()
+                .config
+                .identity_pubkey
+                .to_string(),
+        })
+    }
+
     fn get_version(&self, _: Self::Metadata) -> Result<RpcVersionInfo> {
         Ok(RpcVersionInfo {
             solana_core: solana_clap_utils::version!().to_string(),
@@ -1247,6 +1285,7 @@ pub mod tests {
         let request_processor = Arc::new(RwLock::new(JsonRpcRequestProcessor::new(
             JsonRpcConfig {
                 enable_get_confirmed_block: true,
+                identity_pubkey: *pubkey,
                 ..JsonRpcConfig::default()
             },
             bank_forks.clone(),
@@ -1770,8 +1809,32 @@ pub mod tests {
             "value":{
                 "blockhash": blockhash.to_string(),
                 "feeCalculator": {
-                    "burnPercent": DEFAULT_BURN_PERCENT,
                     "lamportsPerSignature": 0,
+                }
+            }},
+            "id": 1
+        });
+        let expected: Response =
+            serde_json::from_value(expected).expect("expected response deserialization");
+        let result: Response = serde_json::from_str(&res.expect("actual response"))
+            .expect("actual response deserialization");
+        assert_eq!(expected, result);
+    }
+
+    #[test]
+    fn test_rpc_get_fee_rate_governor() {
+        let bob_pubkey = Pubkey::new_rand();
+        let RpcHandler { io, meta, .. } = start_rpc_handler_with_tx(&bob_pubkey);
+
+        let req = format!(r#"{{"jsonrpc":"2.0","id":1,"method":"getFeeRateGovernor"}}"#);
+        let res = io.handle_request_sync(&req, meta);
+        let expected = json!({
+            "jsonrpc": "2.0",
+            "result": {
+            "context":{"slot":0},
+            "value":{
+                "feeRateGovernor": {
+                    "burnPercent": DEFAULT_BURN_PERCENT,
                     "maxLamportsPerSignature": 0,
                     "minLamportsPerSignature": 0,
                     "targetLamportsPerSignature": 0,
@@ -1950,6 +2013,27 @@ pub mod tests {
         assert_eq!(exit.load(Ordering::Relaxed), true);
     }
 
+    #[test]
+    fn test_rpc_get_identity() {
+        let bob_pubkey = Pubkey::new_rand();
+        let RpcHandler { io, meta, .. } = start_rpc_handler_with_tx(&bob_pubkey);
+
+        let req = format!(r#"{{"jsonrpc":"2.0","id":1,"method":"getIdentity"}}"#);
+        let res = io.handle_request_sync(&req, meta);
+        let expected = json!({
+            "jsonrpc": "2.0",
+            "result": {
+                "identity": bob_pubkey.to_string()
+            },
+            "id": 1
+        });
+        let expected: Response =
+            serde_json::from_value(expected).expect("expected response deserialization");
+        let result: Response = serde_json::from_str(&res.expect("actual response"))
+            .expect("actual response deserialization");
+        assert_eq!(expected, result);
+    }
+
     #[test]
     fn test_rpc_get_version() {
         let bob_pubkey = Pubkey::new_rand();

core/src/rpc_pubsub.rs

@@ -12,7 +12,7 @@ use std::sync::{atomic, Arc};
 //   https://github.com/paritytech/jsonrpc/blob/2d38e6424d8461cdf72e78425ce67d51af9c6586/derive/src/lib.rs#L204
 // Once https://github.com/paritytech/jsonrpc/issues/418 is resolved, try to remove this clippy allow
 #[allow(clippy::needless_return)]
-#[rpc(server)]
+#[rpc]
 pub trait RpcSolPubSub {
     type Metadata;
 

core/src/rpc_pubsub_service.rs

@@ -37,6 +37,8 @@ impl PubSubService {
                         });
                         session
                 })
+                .max_connections(1000) // Arbitrary, default of 100 is too low
+                .max_payload(10 * 1024 * 1024 + 1024) // max account size (10MB) + extra (1K)
                 .start(&pubsub_addr);
 
                 if let Err(e) = server {

core/src/rpc_service.rs

@@ -9,7 +9,12 @@ use jsonrpc_http_server::{
     hyper, AccessControlAllowOrigin, CloseHandle, DomainsValidation, RequestMiddleware,
     RequestMiddlewareAction, ServerBuilder,
 };
-use solana_ledger::{bank_forks::BankForks, blockstore::Blockstore};
+use regex::Regex;
+use solana_ledger::{
+    bank_forks::{BankForks, SnapshotConfig},
+    blockstore::Blockstore,
+    snapshot_utils,
+};
 use solana_sdk::hash::Hash;
 use std::{
     net::SocketAddr,
@@ -28,13 +33,28 @@ pub struct JsonRpcService {
     close_handle: Option<CloseHandle>,
 }
 
-#[derive(Default)]
 struct RpcRequestMiddleware {
     ledger_path: PathBuf,
+    snapshot_archive_path_regex: Regex,
+    snapshot_config: Option<SnapshotConfig>,
 }
+
 impl RpcRequestMiddleware {
-    pub fn new(ledger_path: PathBuf) -> Self {
+    pub fn new(ledger_path: PathBuf, snapshot_config: Option<SnapshotConfig>) -> Self {
-        Self { ledger_path }
+        Self {
+            ledger_path,
+            snapshot_archive_path_regex: Regex::new(r"/snapshot-\d+-[[:alnum:]]+\.tar\.bz2$")
+                .unwrap(),
+            snapshot_config,
+        }
+    }
+
+    fn redirect(location: &str) -> hyper::Response<hyper::Body> {
+        hyper::Response::builder()
+            .status(hyper::StatusCode::SEE_OTHER)
+            .header(hyper::header::LOCATION, location)
+            .body(hyper::Body::from(String::from(location)))
+            .unwrap()
     }
 
     fn not_found() -> hyper::Response<hyper::Body> {
@@ -51,9 +71,25 @@ impl RpcRequestMiddleware {
             .unwrap()
     }
 
-    fn get(&self, filename: &str) -> RequestMiddlewareAction {
+    fn is_get_path(&self, path: &str) -> bool {
-        info!("get {}", filename);
+        match path {
-        let filename = self.ledger_path.join(filename);
+            "/genesis.tar.bz2" => true,
+            _ => {
+                if self.snapshot_config.is_some() {
+                    self.snapshot_archive_path_regex.is_match(path)
+                } else {
+                    false
+                }
+            }
+        }
+    }
+
+    fn get(&self, path: &str) -> RequestMiddlewareAction {
+        let filename = self.ledger_path.join(
+            path.split_at(1).1, // Drop leading '/' from path
+        );
+        info!("get {} -> {:?}", path, filename);
+
         RequestMiddlewareAction::Respond {
             should_validate_hosts: true,
             response: Box::new(
@@ -73,13 +109,40 @@ impl RpcRequestMiddleware {
 impl RequestMiddleware for RpcRequestMiddleware {
     fn on_request(&self, request: hyper::Request<hyper::Body>) -> RequestMiddlewareAction {
         trace!("request uri: {}", request.uri());
-        match request.uri().path() {
+
-            "/snapshot.tar.bz2" => self.get("snapshot.tar.bz2"),
+        if let Some(ref snapshot_config) = self.snapshot_config {
-            "/genesis.tar.bz2" => self.get("genesis.tar.bz2"),
+            if request.uri().path() == "/snapshot.tar.bz2" {
-            _ => RequestMiddlewareAction::Proceed {
+                // Convenience redirect to the latest snapshot
+                return RequestMiddlewareAction::Respond {
+                    should_validate_hosts: true,
+                    response: Box::new(jsonrpc_core::futures::future::ok(
+                        if let Some((snapshot_archive, _)) =
+                            snapshot_utils::get_highest_snapshot_archive_path(
+                                &snapshot_config.snapshot_package_output_path,
+                            )
+                        {
+                            RpcRequestMiddleware::redirect(&format!(
+                                "/{}",
+                                snapshot_archive
+                                    .file_name()
+                                    .unwrap_or_else(|| std::ffi::OsStr::new(""))
+                                    .to_str()
+                                    .unwrap_or(&"")
+                            ))
+                        } else {
+                            RpcRequestMiddleware::not_found()
+                        },
+                    )),
+                };
+            }
+        }
+        if self.is_get_path(request.uri().path()) {
+            self.get(request.uri().path())
+        } else {
+            RequestMiddlewareAction::Proceed {
                 should_continue_on_invalid_cors: false,
                 request,
-            },
+            }
         }
     }
 }
@@ -89,6 +152,7 @@ impl JsonRpcService {
     pub fn new(
         rpc_addr: SocketAddr,
         config: JsonRpcConfig,
+        snapshot_config: Option<SnapshotConfig>,
         bank_forks: Arc<RwLock<BankForks>>,
         block_commitment_cache: Arc<RwLock<BlockCommitmentCache>>,
         blockstore: Arc<Blockstore>,
@@ -132,7 +196,7 @@ impl JsonRpcService {
                             AccessControlAllowOrigin::Any,
                         ]))
                         .cors_max_age(86400)
-                        .request_middleware(RpcRequestMiddleware::new(ledger_path))
+                        .request_middleware(RpcRequestMiddleware::new(ledger_path, snapshot_config))
                         .start_http(&rpc_addr);
                 if let Err(e) = server {
                     warn!("JSON RPC service unavailable error: {:?}. \nAlso, check that port {} is not already in use by another application", e, rpc_addr.port());
@@ -198,9 +262,10 @@ mod tests {
         let cluster_info = Arc::new(RwLock::new(ClusterInfo::new_with_invalid_keypair(
             ContactInfo::default(),
         )));
+        let ip_addr = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));
         let rpc_addr = SocketAddr::new(
-            IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),
+            ip_addr,
-            solana_net_utils::find_available_port_in_range((10000, 65535)).unwrap(),
+            solana_net_utils::find_available_port_in_range(ip_addr, (10000, 65535)).unwrap(),
         );
         let bank_forks = Arc::new(RwLock::new(BankForks::new(bank.slot(), bank)));
         let block_commitment_cache = Arc::new(RwLock::new(BlockCommitmentCache::default()));
@@ -209,6 +274,7 @@ mod tests {
         let mut rpc_service = JsonRpcService::new(
             rpc_addr,
             JsonRpcConfig::default(),
+            None,
             bank_forks,
             block_commitment_cache,
             Arc::new(blockstore),
@@ -234,4 +300,36 @@ mod tests {
         rpc_service.exit();
         rpc_service.join().unwrap();
     }
+
+    #[test]
+    fn test_is_get_path() {
+        let rrm = RpcRequestMiddleware::new(PathBuf::from("/"), None);
+        let rrm_with_snapshot_config = RpcRequestMiddleware::new(
+            PathBuf::from("/"),
+            Some(SnapshotConfig {
+                snapshot_interval_slots: 0,
+                snapshot_package_output_path: PathBuf::from("/"),
+                snapshot_path: PathBuf::from("/"),
+            }),
+        );
+
+        assert!(rrm.is_get_path("/genesis.tar.bz2"));
+        assert!(!rrm.is_get_path("genesis.tar.bz2"));
+
+        assert!(!rrm.is_get_path("/snapshot.tar.bz2")); // This is a redirect
+
+        assert!(
+            !rrm.is_get_path("/snapshot-100-AvFf9oS8A8U78HdjT9YG2sTTThLHJZmhaMn2g8vkWYnr.tar.bz2")
+        );
+        assert!(rrm_with_snapshot_config
+            .is_get_path("/snapshot-100-AvFf9oS8A8U78HdjT9YG2sTTThLHJZmhaMn2g8vkWYnr.tar.bz2"));
+
+        assert!(!rrm.is_get_path(
+            "/snapshot-notaslotnumber-AvFf9oS8A8U78HdjT9YG2sTTThLHJZmhaMn2g8vkWYnr.tar.bz2"
+        ));
+
+        assert!(!rrm.is_get_path("/"));
+        assert!(!rrm.is_get_path(".."));
+        assert!(!rrm.is_get_path("🎣"));
+    }
 }

core/src/rpc_subscriptions.rs

@@ -17,14 +17,16 @@ use std::thread::{Builder, JoinHandle};
 use std::time::Duration;
 use std::{
     collections::{HashMap, HashSet},
+    iter,
     sync::{Arc, Mutex, RwLock},
 };
+use tokio::runtime::{Builder as RuntimeBuilder, Runtime, TaskExecutor};
 
 const RECEIVE_DELAY_MILLIS: u64 = 100;
 
 pub type Confirmations = usize;
 
-#[derive(Serialize, Clone, Copy, Debug)]
+#[derive(Serialize, Deserialize, Clone, Copy, Debug)]
 pub struct SlotInfo {
     pub slot: Slot,
     pub parent: Slot,
@@ -103,19 +105,20 @@ where
     found
 }
 
-fn check_confirmations_and_notify<K, S, F, N, X>(
+fn check_confirmations_and_notify<K, S, B, F, X>(
     subscriptions: &HashMap<K, HashMap<SubscriptionId, (Sink<S>, Confirmations)>>,
     hashmap_key: &K,
     current_slot: Slot,
     bank_forks: &Arc<RwLock<BankForks>>,
-    bank_method: F,
+    bank_method: B,
-    notify: N,
+    filter_results: F,
+    notifier: &RpcNotifier,
 ) -> HashSet<SubscriptionId>
 where
     K: Eq + Hash + Clone + Copy,
     S: Clone + Serialize,
-    F: Fn(&Bank, &K) -> X,
+    B: Fn(&Bank, &K) -> X,
-    N: Fn(X, &Sink<S>, u64) -> bool,
+    F: Fn(X, u64) -> Box<dyn Iterator<Item = S>>,
     X: Clone + Serialize,
 {
     let current_ancestors = bank_forks
@@ -149,8 +152,9 @@ where
                     .get(desired_slot[0])
                     .unwrap()
                     .clone();
-                let result = bank_method(&desired_bank, hashmap_key);
+                let results = bank_method(&desired_bank, hashmap_key);
-                if notify(result, &sink, root) {
+                for result in filter_results(results, root) {
+                    notifier.notify(result, sink);
                     notified_set.insert(bank_sub_id.clone());
                 }
             }
@@ -159,41 +163,49 @@ where
     notified_set
 }
 
-fn notify_account(result: Option<(Account, Slot)>, sink: &Sink<RpcAccount>, root: Slot) -> bool {
+struct RpcNotifier(TaskExecutor);
+
+impl RpcNotifier {
+    fn notify<T>(&self, value: T, sink: &Sink<T>)
+    where
+        T: serde::Serialize,
+    {
+        self.0
+            .spawn(sink.notify(Ok(value)).map(|_| ()).map_err(|_| ()));
+    }
+}
+
+fn filter_account_result(
+    result: Option<(Account, Slot)>,
+    root: Slot,
+) -> Box<dyn Iterator<Item = RpcAccount>> {
     if let Some((account, fork)) = result {
         if fork >= root {
-            sink.notify(Ok(RpcAccount::encode(account))).wait().unwrap();
+            return Box::new(iter::once(RpcAccount::encode(account)));
-            return true;
         }
     }
-    false
+    Box::new(iter::empty())
 }
 
-fn notify_signature<S>(result: Option<S>, sink: &Sink<S>, _root: Slot) -> bool
+fn filter_signature_result<S>(result: Option<S>, _root: Slot) -> Box<dyn Iterator<Item = S>>
 where
-    S: Clone + Serialize,
+    S: 'static + Clone + Serialize,
 {
-    if let Some(result) = result {
+    Box::new(result.into_iter())
-        sink.notify(Ok(result)).wait().unwrap();
-        return true;
-    }
-    false
 }
 
-fn notify_program(
+fn filter_program_results(
     accounts: Vec<(Pubkey, Account)>,
-    sink: &Sink<RpcKeyedAccount>,
     _root: Slot,
-) -> bool {
+) -> Box<dyn Iterator<Item = RpcKeyedAccount>> {
-    for (pubkey, account) in accounts.iter() {
+    Box::new(
-        sink.notify(Ok(RpcKeyedAccount {
+        accounts
-            pubkey: pubkey.to_string(),
+            .into_iter()
-            account: RpcAccount::encode(account.clone()),
+            .map(|(pubkey, account)| RpcKeyedAccount {
-        }))
+                pubkey: pubkey.to_string(),
-        .wait()
+                account: RpcAccount::encode(account),
-        .unwrap();
+            }),
-    }
+    )
-    !accounts.is_empty()
 }
 
 pub struct RpcSubscriptions {
@@ -203,6 +215,7 @@ pub struct RpcSubscriptions {
     slot_subscriptions: Arc<RpcSlotSubscriptions>,
     notification_sender: Arc<Mutex<Sender<NotificationEntry>>>,
     t_cleanup: Option<JoinHandle<()>>,
+    notifier_runtime: Option<Runtime>,
     exit: Arc<AtomicBool>,
 }
 
@@ -239,11 +252,19 @@ impl RpcSubscriptions {
         let signature_subscriptions_clone = signature_subscriptions.clone();
         let slot_subscriptions_clone = slot_subscriptions.clone();
 
+        let notifier_runtime = RuntimeBuilder::new()
+            .core_threads(1)
+            .name_prefix("solana-rpc-notifier-")
+            .build()
+            .unwrap();
+
+        let notifier = RpcNotifier(notifier_runtime.executor());
         let t_cleanup = Builder::new()
             .name("solana-rpc-notifications".to_string())
             .spawn(move || {
                 Self::process_notifications(
                     exit_clone,
+                    notifier,
                     notification_receiver,
                     account_subscriptions_clone,
                     program_subscriptions_clone,
@@ -259,6 +280,7 @@ impl RpcSubscriptions {
             signature_subscriptions,
             slot_subscriptions,
             notification_sender,
+            notifier_runtime: Some(notifier_runtime),
             t_cleanup: Some(t_cleanup),
             exit: exit.clone(),
         }
@@ -269,6 +291,7 @@ impl RpcSubscriptions {
         current_slot: Slot,
         bank_forks: &Arc<RwLock<BankForks>>,
         account_subscriptions: Arc<RpcAccountSubscriptions>,
+        notifier: &RpcNotifier,
     ) {
         let subscriptions = account_subscriptions.read().unwrap();
         check_confirmations_and_notify(
@@ -277,7 +300,8 @@ impl RpcSubscriptions {
             current_slot,
             bank_forks,
             Bank::get_account_modified_since_parent,
-            notify_account,
+            filter_account_result,
+            notifier,
         );
     }
 
@@ -286,6 +310,7 @@ impl RpcSubscriptions {
         current_slot: Slot,
         bank_forks: &Arc<RwLock<BankForks>>,
         program_subscriptions: Arc<RpcProgramSubscriptions>,
+        notifier: &RpcNotifier,
     ) {
         let subscriptions = program_subscriptions.read().unwrap();
         check_confirmations_and_notify(
@@ -294,7 +319,8 @@ impl RpcSubscriptions {
             current_slot,
             bank_forks,
             Bank::get_program_accounts_modified_since_parent,
-            notify_program,
+            filter_program_results,
+            notifier,
         );
     }
 
@@ -303,6 +329,7 @@ impl RpcSubscriptions {
         current_slot: Slot,
         bank_forks: &Arc<RwLock<BankForks>>,
         signature_subscriptions: Arc<RpcSignatureSubscriptions>,
+        notifier: &RpcNotifier,
     ) {
         let mut subscriptions = signature_subscriptions.write().unwrap();
         let notified_ids = check_confirmations_and_notify(
@@ -311,7 +338,8 @@ impl RpcSubscriptions {
             current_slot,
             bank_forks,
             Bank::get_signature_status,
-            notify_signature,
+            filter_signature_result,
+            notifier,
         );
         if let Some(subscription_ids) = subscriptions.get_mut(signature) {
             subscription_ids.retain(|k, _| !notified_ids.contains(k));
@@ -408,6 +436,7 @@ impl RpcSubscriptions {
 
     fn process_notifications(
         exit: Arc<AtomicBool>,
+        notifier: RpcNotifier,
         notification_receiver: Receiver<NotificationEntry>,
         account_subscriptions: Arc<RpcAccountSubscriptions>,
         program_subscriptions: Arc<RpcProgramSubscriptions>,
@@ -423,7 +452,7 @@ impl RpcSubscriptions {
                     NotificationEntry::Slot(slot_info) => {
                         let subscriptions = slot_subscriptions.read().unwrap();
                         for (_, sink) in subscriptions.iter() {
-                            sink.notify(Ok(slot_info)).wait().unwrap();
+                            notifier.notify(slot_info, sink);
                         }
                     }
                     NotificationEntry::Bank((current_slot, bank_forks)) => {
@@ -437,6 +466,7 @@ impl RpcSubscriptions {
                                 current_slot,
                                 &bank_forks,
                                 account_subscriptions.clone(),
+                                &notifier,
                             );
                         }
 
@@ -450,6 +480,7 @@ impl RpcSubscriptions {
                                 current_slot,
                                 &bank_forks,
                                 program_subscriptions.clone(),
+                                &notifier,
                             );
                         }
 
@@ -463,6 +494,7 @@ impl RpcSubscriptions {
                                 current_slot,
                                 &bank_forks,
                                 signature_subscriptions.clone(),
+                                &notifier,
                             );
                         }
                     }
@@ -479,6 +511,12 @@ impl RpcSubscriptions {
     }
 
     fn shutdown(&mut self) -> std::thread::Result<()> {
+        if let Some(runtime) = self.notifier_runtime.take() {
+            info!("RPC Notifier runtime - shutting down");
+            let _ = runtime.shutdown_now().wait();
+            info!("RPC Notifier runtime - shut down");
+        }
+
         if self.t_cleanup.is_some() {
             info!("RPC Notification thread - shutting down");
             self.exit.store(true, Ordering::Relaxed);

core/src/snapshot_packager_service.rs

@@ -1,4 +1,4 @@
-use crate::cluster_info::ClusterInfo;
+use crate::cluster_info::{ClusterInfo, MAX_SNAPSHOT_HASHES};
 use solana_ledger::{
     snapshot_package::SnapshotPackageReceiver, snapshot_utils::archive_snapshot_package,
 };
@@ -16,8 +16,6 @@ pub struct SnapshotPackagerService {
     t_snapshot_packager: JoinHandle<()>,
 }
 
-const MAX_SNAPSHOT_HASHES: usize = 24;
-
 impl SnapshotPackagerService {
     pub fn new(
         snapshot_package_receiver: SnapshotPackageReceiver,
@@ -37,24 +35,24 @@ impl SnapshotPackagerService {
 
                     match snapshot_package_receiver.recv_timeout(Duration::from_secs(1)) {
                         Ok(mut snapshot_package) => {
-                            hashes.push((snapshot_package.root, snapshot_package.hash));
                             // Only package the latest
                             while let Ok(new_snapshot_package) =
                                 snapshot_package_receiver.try_recv()
                             {
                                 snapshot_package = new_snapshot_package;
-                                hashes.push((snapshot_package.root, snapshot_package.hash));
                             }
                             if let Err(err) = archive_snapshot_package(&snapshot_package) {
                                 warn!("Failed to create snapshot archive: {}", err);
+                            } else {
+                                hashes.push((snapshot_package.root, snapshot_package.hash));
+                                while hashes.len() > MAX_SNAPSHOT_HASHES {
+                                    hashes.remove(0);
+                                }
+                                cluster_info
+                                    .write()
+                                    .unwrap()
+                                    .push_snapshot_hashes(hashes.clone());
                             }
-                            while hashes.len() > MAX_SNAPSHOT_HASHES {
-                                hashes.remove(0);
-                            }
-                            cluster_info
-                                .write()
-                                .unwrap()
-                                .push_snapshot_hashes(hashes.clone());
                         }
                         Err(RecvTimeoutError::Disconnected) => break,
                         Err(RecvTimeoutError::Timeout) => (),
@@ -154,8 +152,10 @@ mod tests {
         }
 
         // Create a packageable snapshot
-        let output_tar_path =
+        let output_tar_path = snapshot_utils::get_snapshot_archive_path(
-            snapshot_utils::get_snapshot_archive_path(&snapshot_package_output_path);
+            &snapshot_package_output_path,
+            &(42, Hash::default()),
+        );
         let snapshot_package = SnapshotPackage::new(
             5,
             vec![],

core/src/streamer.rs

@@ -133,7 +133,7 @@ pub fn responder(name: &'static str, sock: Arc<UdpSocket>, r: PacketReceiver) ->
                 match e {
                     StreamerError::RecvTimeoutError(RecvTimeoutError::Disconnected) => break,
                     StreamerError::RecvTimeoutError(RecvTimeoutError::Timeout) => (),
-                    _ => warn!("{} responder error: {:?}", name, e),
+                    _ => info!("{} responder error: {:?}", name, e),
                 }
             }
         })

core/src/transaction_status_service.rs

@@ -1,7 +1,10 @@
 use crossbeam_channel::{Receiver, RecvTimeoutError};
 use solana_client::rpc_response::RpcTransactionStatus;
 use solana_ledger::{blockstore::Blockstore, blockstore_processor::TransactionStatusBatch};
-use solana_runtime::bank::{Bank, HashAgeKind};
+use solana_runtime::{
+    bank::{Bank, HashAgeKind},
+    nonce_utils,
+};
 use std::{
     sync::{
         atomic::{AtomicBool, Ordering},
@@ -59,14 +62,13 @@ impl TransactionStatusService {
             .zip(balances.post_balances)
         {
             if Bank::can_commit(&status) && !transaction.signatures.is_empty() {
-                let fee_hash = if let Some(HashAgeKind::DurableNonce(_, _)) = hash_age_kind {
+                let fee_calculator = match hash_age_kind {
-                    bank.last_blockhash()
+                    Some(HashAgeKind::DurableNonce(_, account)) => {
-                } else {
+                        nonce_utils::fee_calculator_of(&account)
-                    transaction.message().recent_blockhash
+                    }
-                };
+                    _ => bank.get_fee_calculator(&transaction.message().recent_blockhash),
-                let fee_calculator = bank
+                }
-                    .get_fee_calculator(&fee_hash)
+                .expect("FeeCalculator must exist");
-                    .expect("FeeCalculator must exist");
                 let fee = fee_calculator.calculate_fee(transaction.message());
                 blockstore
                     .write_transaction_status(

core/src/validator.rs

@@ -6,7 +6,7 @@ use crate::{
     commitment::BlockCommitmentCache,
     contact_info::ContactInfo,
     gossip_service::{discover_cluster, GossipService},
-    poh_recorder::PohRecorder,
+    poh_recorder::{PohRecorder, GRACE_TICKS_FACTOR, MAX_GRACE_SLOTS},
     poh_service::PohService,
     rewards_recorder_service::RewardsRecorderService,
     rpc::JsonRpcConfig,
@@ -35,6 +35,7 @@ use solana_metrics::datapoint_info;
 use solana_runtime::bank::Bank;
 use solana_sdk::{
     clock::{Slot, DEFAULT_SLOTS_PER_TURN},
+    epoch_schedule::MAX_LEADER_SCHEDULE_EPOCH_OFFSET,
     genesis_config::GenesisConfig,
     hash::Hash,
     pubkey::Pubkey,
@@ -43,6 +44,7 @@ use solana_sdk::{
     timing::timestamp,
 };
 use std::{
+    collections::HashSet,
     net::{IpAddr, Ipv4Addr, SocketAddr},
     path::{Path, PathBuf},
     process,
@@ -70,8 +72,9 @@ pub struct ValidatorConfig {
     pub broadcast_stage_type: BroadcastStageType,
     pub enable_partition: Option<Arc<AtomicBool>>,
     pub fixed_leader_schedule: Option<FixedSchedule>,
-    pub wait_for_supermajority: bool,
+    pub wait_for_supermajority: Option<Slot>,
     pub new_hard_forks: Option<Vec<Slot>>,
+    pub trusted_validators: Option<HashSet<Pubkey>>, // None = trust all
 }
 
 impl Default for ValidatorConfig {
@@ -92,8 +95,9 @@ impl Default for ValidatorConfig {
             broadcast_stage_type: BroadcastStageType::Standard,
             enable_partition: None,
             fixed_leader_schedule: None,
-            wait_for_supermajority: false,
+            wait_for_supermajority: None,
             new_hard_forks: None,
+            trusted_validators: None,
         }
     }
 }
@@ -238,6 +242,7 @@ impl Validator {
                 JsonRpcService::new(
                     SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), rpc_port),
                     config.rpc_config.clone(),
+                    config.snapshot_config.clone(),
                     bank_forks.clone(),
                     block_commitment_cache.clone(),
                     blockstore.clone(),
@@ -305,7 +310,13 @@ impl Validator {
             bank.tick_height(),
             bank.last_blockhash(),
             bank.slot(),
-            leader_schedule_cache.next_leader_slot(&id, bank.slot(), &bank, Some(&blockstore)),
+            leader_schedule_cache.next_leader_slot(
+                &id,
+                bank.slot(),
+                &bank,
+                Some(&blockstore),
+                GRACE_TICKS_FACTOR * MAX_GRACE_SLOTS,
+            ),
             bank.ticks_per_slot(),
             &id,
             &blockstore,
@@ -345,9 +356,7 @@ impl Validator {
         }
 
         if let Some(snapshot_hash) = snapshot_hash {
-            if let Some(ref trusted_validators) =
+            if let Some(ref trusted_validators) = config.trusted_validators {
-                config.snapshot_config.as_ref().unwrap().trusted_validators
-            {
                 let mut trusted = false;
                 for _ in 0..10 {
                     trusted = cluster_info
@@ -372,12 +381,6 @@ impl Validator {
                     process::exit(1);
                 }
             }
-
-            // If the node was loaded from a snapshot, advertise it in gossip
-            cluster_info
-                .write()
-                .unwrap()
-                .push_snapshot_hashes(vec![snapshot_hash]);
         }
 
         wait_for_supermajority(config, &bank, &cluster_info);
@@ -562,6 +565,14 @@ fn new_banks_from_blockstore(
         error!("Failed to load genesis from {:?}: {}", blockstore_path, err);
         process::exit(1);
     });
+
+    // This needs to be limited otherwise the state in the VoteAccount data
+    // grows too large
+    let leader_schedule_slot_offset = genesis_config.epoch_schedule.leader_schedule_slot_offset;
+    let slots_per_epoch = genesis_config.epoch_schedule.slots_per_epoch;
+    let leader_epoch_offset = (leader_schedule_slot_offset + slots_per_epoch - 1) / slots_per_epoch;
+    assert!(leader_epoch_offset <= MAX_LEADER_SCHEDULE_EPOCH_OFFSET);
+
     let genesis_hash = genesis_config.hash();
     info!("genesis hash: {}", genesis_hash);
 
@@ -620,7 +631,7 @@ fn wait_for_supermajority(
     bank: &Arc<Bank>,
     cluster_info: &Arc<RwLock<ClusterInfo>>,
 ) {
-    if !config.wait_for_supermajority {
+    if config.wait_for_supermajority != Some(bank.slot()) {
         return;
     }
 
@@ -639,74 +650,94 @@ fn wait_for_supermajority(
     }
 }
 
-pub fn new_validator_for_tests() -> (Validator, ContactInfo, Keypair, PathBuf) {
+pub struct TestValidator {
-    let (node, contact_info, mint_keypair, ledger_path, _vote_pubkey) =
+    pub server: Validator,
-        new_validator_for_tests_with_vote_pubkey();
+    pub leader_data: ContactInfo,
-    (node, contact_info, mint_keypair, ledger_path)
+    pub alice: Keypair,
+    pub ledger_path: PathBuf,
+    pub genesis_hash: Hash,
+    pub vote_pubkey: Pubkey,
 }
 
-pub fn new_validator_for_tests_with_vote_pubkey(
+pub struct TestValidatorOptions {
-) -> (Validator, ContactInfo, Keypair, PathBuf, Pubkey) {
+    pub fees: u64,
-    use crate::genesis_utils::BOOTSTRAP_VALIDATOR_LAMPORTS;
+    pub bootstrap_validator_lamports: u64,
-    new_validator_for_tests_ex(0, BOOTSTRAP_VALIDATOR_LAMPORTS)
 }
 
-pub fn new_validator_for_tests_ex(
+impl Default for TestValidatorOptions {
-    fees: u64,
+    fn default() -> Self {
-    bootstrap_validator_lamports: u64,
+        use crate::genesis_utils::BOOTSTRAP_VALIDATOR_LAMPORTS;
-) -> (Validator, ContactInfo, Keypair, PathBuf, Pubkey) {
+        TestValidatorOptions {
-    use crate::genesis_utils::{create_genesis_config_with_leader_ex, GenesisConfigInfo};
+            fees: 0,
-    use solana_sdk::fee_calculator::FeeCalculator;
+            bootstrap_validator_lamports: BOOTSTRAP_VALIDATOR_LAMPORTS,
+        }
+    }
+}
 
-    let node_keypair = Arc::new(Keypair::new());
+impl TestValidator {
-    let node = Node::new_localhost_with_pubkey(&node_keypair.pubkey());
+    pub fn run() -> Self {
-    let contact_info = node.info.clone();
+        Self::run_with_options(TestValidatorOptions::default())
+    }
 
-    let GenesisConfigInfo {
+    pub fn run_with_options(options: TestValidatorOptions) -> Self {
-        mut genesis_config,
+        use crate::genesis_utils::{create_genesis_config_with_leader_ex, GenesisConfigInfo};
-        mint_keypair,
+        use solana_sdk::fee_calculator::FeeRateGovernor;
-        voting_keypair,
-    } = create_genesis_config_with_leader_ex(
-        1_000_000,
-        &contact_info.id,
-        42,
-        bootstrap_validator_lamports,
-    );
-    genesis_config
-        .native_instruction_processors
-        .push(solana_budget_program!());
 
-    genesis_config.rent.lamports_per_byte_year = 1;
+        let TestValidatorOptions {
-    genesis_config.rent.exemption_threshold = 1.0;
+            fees,
-    genesis_config.fee_calculator = FeeCalculator::new(fees, 0);
+            bootstrap_validator_lamports,
+        } = options;
+        let node_keypair = Arc::new(Keypair::new());
+        let node = Node::new_localhost_with_pubkey(&node_keypair.pubkey());
+        let contact_info = node.info.clone();
 
-    let (ledger_path, _blockhash) = create_new_tmp_ledger!(&genesis_config);
+        let GenesisConfigInfo {
+            mut genesis_config,
+            mint_keypair,
+            voting_keypair,
+        } = create_genesis_config_with_leader_ex(
+            1_000_000,
+            &contact_info.id,
+            42,
+            bootstrap_validator_lamports,
+        );
+        genesis_config
+            .native_instruction_processors
+            .push(solana_budget_program!());
 
-    let leader_voting_keypair = Arc::new(voting_keypair);
+        genesis_config.rent.lamports_per_byte_year = 1;
-    let storage_keypair = Arc::new(Keypair::new());
+        genesis_config.rent.exemption_threshold = 1.0;
-    let config = ValidatorConfig {
+        genesis_config.fee_rate_governor = FeeRateGovernor::new(fees, 0);
-        rpc_ports: Some((node.info.rpc.port(), node.info.rpc_pubsub.port())),
+
-        ..ValidatorConfig::default()
+        let (ledger_path, blockhash) = create_new_tmp_ledger!(&genesis_config);
-    };
+
-    let node = Validator::new(
+        let leader_voting_keypair = Arc::new(voting_keypair);
-        node,
+        let storage_keypair = Arc::new(Keypair::new());
-        &node_keypair,
+        let config = ValidatorConfig {
-        &ledger_path,
+            rpc_ports: Some((node.info.rpc.port(), node.info.rpc_pubsub.port())),
-        &leader_voting_keypair.pubkey(),
+            ..ValidatorConfig::default()
-        &leader_voting_keypair,
+        };
-        &storage_keypair,
+        let node = Validator::new(
-        None,
+            node,
-        true,
+            &node_keypair,
-        &config,
+            &ledger_path,
-    );
+            &leader_voting_keypair.pubkey(),
-    discover_cluster(&contact_info.gossip, 1).expect("Node startup failed");
+            &leader_voting_keypair,
-    (
+            &storage_keypair,
-        node,
+            None,
-        contact_info,
+            true,
-        mint_keypair,
+            &config,
-        ledger_path,
+        );
-        leader_voting_keypair.pubkey(),
+        discover_cluster(&contact_info.gossip, 1).expect("Node startup failed");
-    )
+        TestValidator {
+            server: node,
+            leader_data: contact_info,
+            alice: mint_keypair,
+            ledger_path,
+            genesis_hash: blockhash,
+            vote_pubkey: leader_voting_keypair.pubkey(),
+        }
+    }
 }
 
 fn report_target_features() {

core/tests/bank_forks.rs

@@ -55,7 +55,6 @@ mod tests {
             snapshot_interval_slots,
             snapshot_package_output_path: PathBuf::from(snapshot_output_path.path()),
             snapshot_path: PathBuf::from(snapshot_dir.path()),
-            trusted_validators: None,
         };
         bank_forks.set_snapshot_config(Some(snapshot_config.clone()));
         SnapshotTestConfig {
@@ -68,13 +67,19 @@ mod tests {
         }
     }
 
-    fn restore_from_snapshot(old_bank_forks: &BankForks, account_paths: Vec<PathBuf>) {
+    fn restore_from_snapshot(
+        old_bank_forks: &BankForks,
+        old_last_slot: Slot,
+        account_paths: Vec<PathBuf>,
+    ) {
         let (snapshot_path, snapshot_package_output_path) = old_bank_forks
             .snapshot_config
             .as_ref()
             .map(|c| (&c.snapshot_path, &c.snapshot_package_output_path))
             .unwrap();
 
+        let old_last_bank = old_bank_forks.get(old_last_slot).unwrap();
+
         let deserialized_bank = snapshot_utils::bank_from_archive(
             &account_paths,
             &old_bank_forks
@@ -82,7 +87,10 @@ mod tests {
                 .as_ref()
                 .unwrap()
                 .snapshot_path,
-            snapshot_utils::get_snapshot_archive_path(snapshot_package_output_path),
+            snapshot_utils::get_snapshot_archive_path(
+                snapshot_package_output_path,
+                &(old_last_bank.slot(), old_last_bank.get_accounts_hash()),
+            ),
         )
         .unwrap();
 
@@ -140,18 +148,20 @@ mod tests {
             slot_snapshot_paths
                 .last()
                 .expect("no snapshots found in path"),
-            snapshot_utils::get_snapshot_archive_path(
-                &snapshot_config.snapshot_package_output_path,
-            ),
             &snapshot_config.snapshot_path,
             &last_bank.src.roots(),
+            &snapshot_config.snapshot_package_output_path,
             storages,
         )
         .unwrap();
 
         snapshot_utils::archive_snapshot_package(&snapshot_package).unwrap();
 
-        restore_from_snapshot(bank_forks, vec![accounts_dir.path().to_path_buf()]);
+        restore_from_snapshot(
+            bank_forks,
+            last_slot,
+            vec![accounts_dir.path().to_path_buf()],
+        );
     }
 
     #[test]
@@ -222,9 +232,8 @@ mod tests {
         let saved_snapshots_dir = TempDir::new().unwrap();
         let saved_accounts_dir = TempDir::new().unwrap();
         let saved_slot = 4;
-        let saved_tar = snapshot_config
+        let mut saved_archive_path = None;
-            .snapshot_package_output_path
+
-            .join(saved_slot.to_string());
         for forks in 0..MAX_CACHE_ENTRIES + 2 {
             let bank = Bank::new_from_parent(
                 &bank_forks[forks as u64],
@@ -236,6 +245,7 @@ mod tests {
             let tx = system_transaction::transfer(&mint_keypair, &key1, 1, genesis_config.hash());
             assert_eq!(bank.process_transaction(&tx), Ok(()));
             bank.squash();
+            let accounts_hash = bank.update_accounts_hash();
             bank_forks.insert(bank);
 
             let package_sender = {
@@ -250,14 +260,7 @@ mod tests {
             };
 
             bank_forks
-                .generate_snapshot(
+                .generate_snapshot(slot, &vec![], &package_sender)
-                    slot,
-                    &vec![],
-                    &package_sender,
-                    snapshot_config
-                        .snapshot_package_output_path
-                        .join(slot.to_string()),
-                )
                 .unwrap();
 
             if slot == saved_slot as u64 {
@@ -283,6 +286,11 @@ mod tests {
                     &options,
                 )
                 .unwrap();
+
+                saved_archive_path = Some(snapshot_utils::get_snapshot_archive_path(
+                    &snapshot_config.snapshot_package_output_path,
+                    &(slot, accounts_hash),
+                ));
             }
         }
 
@@ -320,7 +328,7 @@ mod tests {
             .join()
             .expect("SnapshotPackagerService exited with error");
 
-        // Check the tar we cached the state for earlier was generated correctly
+        // Check the archive we cached the state for earlier was generated correctly
 
         // before we compare, stick an empty status_cache in this dir so that the package comparision works
         // This is needed since the status_cache is added by the packager and is not collected from
@@ -339,7 +347,7 @@ mod tests {
         .unwrap();
 
         snapshot_utils::verify_snapshot_archive(
-            saved_tar,
+            saved_archive_path.unwrap(),
             saved_snapshots_dir.path(),
             saved_accounts_dir
                 .path()

core/tests/client.rs

@@ -4,7 +4,7 @@ use solana_client::{
 };
 use solana_core::{
     rpc_pubsub_service::PubSubService, rpc_subscriptions::RpcSubscriptions,
-    validator::new_validator_for_tests,
+    validator::TestValidator,
 };
 use solana_sdk::{
     commitment_config::CommitmentConfig, pubkey::Pubkey, rpc_port, signature::Signer,
@@ -26,7 +26,13 @@ use systemstat::Ipv4Addr;
 fn test_rpc_client() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let bob_pubkey = Pubkey::new_rand();
 
     let client = RpcClient::new_socket(leader_data.rpc);

core/tests/fork-selection.rs

@@ -1,6 +1,6 @@
 //! Fork Selection Simulation
 //!
-//! Description of the algorithm can be found in [book/src/fork-selection.md](book/src/fork-selection.md).
+//! Description of the algorithm can be found in [docs/src/fork-selection.md](docs/src/fork-selection.md).
 //!
 //! A test library function exists for configuring networks.
 //! ```

core/tests/rpc.rs

@@ -1,21 +1,38 @@
 use bincode::serialize;
+use jsonrpc_core::futures::{
+    future::{self, Future},
+    stream::Stream,
+};
+use jsonrpc_core_client::transports::ws;
 use log::*;
 use reqwest::{self, header::CONTENT_TYPE};
 use serde_json::{json, Value};
 use solana_client::rpc_client::get_rpc_request_str;
-use solana_core::validator::new_validator_for_tests;
+use solana_core::{rpc_pubsub::gen_client::Client as PubsubClient, validator::TestValidator};
-use solana_sdk::hash::Hash;
+use solana_sdk::{hash::Hash, pubkey::Pubkey, system_transaction, transaction};
-use solana_sdk::pubkey::Pubkey;
+use std::{
-use solana_sdk::system_transaction;
+    collections::HashSet,
-use std::fs::remove_dir_all;
+    fs::remove_dir_all,
-use std::thread::sleep;
+    net::UdpSocket,
-use std::time::Duration;
+    sync::mpsc::channel,
+    sync::{Arc, Mutex},
+    thread::sleep,
+    time::Duration,
+    time::SystemTime,
+};
+use tokio::runtime::Runtime;
 
 #[test]
 fn test_rpc_send_tx() {
     solana_logger::setup();
 
-    let (server, leader_data, alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let bob_pubkey = Pubkey::new_rand();
 
     let client = reqwest::blocking::Client::new();
@@ -100,7 +117,12 @@ fn test_rpc_send_tx() {
 fn test_rpc_invalid_requests() {
     solana_logger::setup();
 
-    let (server, leader_data, _alice, ledger_path) = new_validator_for_tests();
+    let TestValidator {
+        server,
+        leader_data,
+        ledger_path,
+        ..
+    } = TestValidator::run();
     let bob_pubkey = Pubkey::new_rand();
 
     // test invalid get_balance request
@@ -166,3 +188,82 @@ fn test_rpc_invalid_requests() {
     server.close().unwrap();
     remove_dir_all(ledger_path).unwrap();
 }
+
+#[test]
+fn test_rpc_subscriptions() {
+    solana_logger::setup();
+
+    let TestValidator {
+        server,
+        leader_data,
+        alice,
+        ledger_path,
+        genesis_hash,
+        ..
+    } = TestValidator::run();
+
+    // Create transaction signatures to subscribe to
+    let transactions_socket = UdpSocket::bind("0.0.0.0:0").unwrap();
+    let mut signature_set: HashSet<String> = (0..1000)
+        .map(|_| {
+            let tx = system_transaction::transfer(&alice, &Pubkey::new_rand(), 1, genesis_hash);
+            transactions_socket
+                .send_to(&bincode::serialize(&tx).unwrap(), leader_data.tpu)
+                .unwrap();
+            tx.signatures[0].to_string()
+        })
+        .collect();
+
+    // Create the pub sub runtime
+    let mut rt = Runtime::new().unwrap();
+    let rpc_pubsub_url = format!("ws://{}/", leader_data.rpc_pubsub);
+    let (sender, receiver) = channel::<(String, transaction::Result<()>)>();
+    let sender = Arc::new(Mutex::new(sender));
+
+    rt.spawn({
+        let connect = ws::try_connect::<PubsubClient>(&rpc_pubsub_url).unwrap();
+        let signature_set = signature_set.clone();
+        connect
+            .and_then(move |client| {
+                for sig in signature_set {
+                    let sender = sender.clone();
+                    tokio::spawn(
+                        client
+                            .signature_subscribe(sig.clone(), None)
+                            .and_then(move |sig_stream| {
+                                sig_stream.for_each(move |result| {
+                                    sender.lock().unwrap().send((sig.clone(), result)).unwrap();
+                                    future::ok(())
+                                })
+                            })
+                            .map_err(|err| {
+                                eprintln!("sig sub err: {:#?}", err);
+                            }),
+                    );
+                }
+                future::ok(())
+            })
+            .map_err(|_| ())
+    });
+
+    // Wait for all signature subscriptions
+    let now = SystemTime::now();
+    let timeout = Duration::from_secs(5);
+    while !signature_set.is_empty() {
+        assert!(now.elapsed().unwrap() < timeout);
+        match receiver.recv_timeout(Duration::from_secs(1)) {
+            Ok((sig, result)) => {
+                assert!(result.is_ok());
+                assert!(signature_set.remove(&sig));
+            }
+            Err(_err) => {
+                eprintln!("unexpected receive timeout");
+                assert!(false)
+            }
+        }
+    }
+
+    rt.shutdown_now().wait().unwrap();
+    server.close().unwrap();
+    remove_dir_all(ledger_path).unwrap();
+}

crate-features/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-crate-features"
-version = "1.0.0"
+version = "1.0.3"
 description = "Solana Crate Features"
 authors = ["Solana Maintainers <maintainers@solana.com>"]
 repository = "https://github.com/solana-labs/solana"

docs/README.md

@@ -1,7 +1,7 @@
 Building the Solana book
 ---
 
-Install the book's dependencies, build, and test the book:
+Install dependencies, build, and test the docs:
 
 ```bash
 $ ./build.sh
@@ -19,7 +19,7 @@ Render markdown as HTML:
 $ make build
 ```
 
-Render and view the book:
+Render and view the docs:
 
 ```bash
 $ make open