Add RecentItems metrics (#20484) (#20490)

* Cost Model to limit transactions which are not parallelizeable (#16694)

* * Add following to banking_stage:
  1. CostModel as immutable ref shared between threads, to provide estimated cost for transactions.
  2. CostTracker which is shared between threads, tracks transaction costs for each block.

* replace hard coded program ID with id() calls

* Add Account Access Cost as part of TransactionCost. Account Access cost are weighted differently between read and write, signed and non-signed.

* Establish instruction_execution_cost_table, add function to update or insert instruction cost, unit tested. It is read-only for now; it allows Replay to insert realtime instruction execution costs to the table.

* add test for cost_tracker atomically try_add operation, serves as safety guard for future changes

* check cost against local copy of cost_tracker, return transactions that would exceed limit as unprocessed transaction to be buffered; only apply bank processed transactions cost to tracker;

* bencher to new banking_stage with max cost limit to allow cost model being hit consistently during bench iterations

* replay stage feed back program cost (#17731)

* replay stage feeds back realtime per-program execution cost to cost model;

* program cost execution table is initialized into empty table, no longer populated with hardcoded numbers;

* changed cost unit to microsecond, using value collected from mainnet;

* add ExecuteCostTable with fixed capacity for security concern, when its limit is reached, programs with old age AND less occurrence will be pushed out to make room for new programs.

* investigate system performance test degradation  (#17919)

* Add stats and counter around cost model ops, mainly:
- calculate transaction cost
- check transaction can fit in a block
- update block cost tracker after transactions are added to block
- replay_stage to update/insert execution cost to table

* Change mutex on cost_tracker to RwLock

* removed cloning cost_tracker for local use, as the metrics show clone is very expensive.

* acquire and hold locks for block of TXs, instead of acquire and release per transaction;

* remove redundant would_fit check from cost_tracker update execution path

* refactor cost checking with less frequent lock acquiring

* avoid many Transaction_cost heap allocation when calculate cost, which
is in the hot path - executed per transaction.

* create hashmap with new_capacity to reduce runtime heap realloc.

* code review changes: categorize stats, replace explicit drop calls, concisely initiate to default

* address potential deadlock by acquiring locks one at time

* Persist cost table to blockstore (#18123)

* Add `ProgramCosts` Column Family to blockstore, implement LedgerColumn; add `delete_cf` to Rocks
* Add ProgramCosts to compaction excluding list alone side with TransactionStatusIndex in one place: `excludes_from_compaction()`

* Write cost table to blockstore after `replay_stage` replayed active banks; add stats to measure persist time
* Deletes program from `ProgramCosts` in blockstore when they are removed from cost_table in memory
* Only try to persist to blockstore when cost_table is changed.
* Restore cost table during validator startup

* Offload `cost_model` related operations from replay main thread to dedicated service thread, add channel to send execute_timings between these threads;
* Move `cost_update_service` to its own module; replay_stage is now decoupled from cost_model.

* log warning when channel send fails (#18391)

* Aggregate cost_model into cost_tracker (#18374)

* * aggregate cost_model into cost_tracker, decouple it from banking_stage to prevent accidental deadlock. * Simplified code, removed unused functions

* review fixes

* update ledger tool to restore cost table from blockstore (#18489)

* update ledger tool to restore cost model from blockstore when compute-slot-cost

* Move initialize_cost_table into cost_model, so the function can be tested and shared between validator and ledger-tool

* refactor and simplify a test

* manually fix merge conflicts

* Per-program id timings (#17554)

* more manual fixing

* solve a merge conflict

* featurize cost model

* more merge fix

* cost model uses compute_unit to replace microsecond as cost unit
(#18934)

* Reject blocks for costs above the max block cost (#18994)

* Update block max cost limit to fix performance regession (#19276)

* replace function with const var for better readability (#19285)

* Add few more metrics data points (#19624)

* periodically report sigverify_stage stats (#19674)

* manual merge

* cost model nits (#18528)

* Accumulate consumed units (#18714)

* tx wide compute budget (#18631)

* more manual merge

* ignore zerorize drop security

* - update const cost values with data collected by #19627
- update cost calculation to closely proposed fee schedule #16984

* add transaction cost histogram metrics (#20350)

* rebase to 1.7.15

* add tx count and thread id to stats (#20451)
each stat reports and resets when slot changes

* remove cost_model feature_set

* ignore vote transactions from cost model

Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Jeff Biseda <jbiseda@gmail.com>
Co-authored-by: Jack May <jack@solana.com>

.travis.yml

@@ -61,6 +61,12 @@ jobs:
     - <<: *release-artifacts
       name: "Windows release artifacts"
       os: windows
+      install:
+        - choco install openssl
+        - export OPENSSL_DIR="C:\Program Files\OpenSSL-Win64"
+        - source ci/rust-version.sh
+        - PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
+        - readlink -f .
     #  Linux release artifacts are still built by ci/buildkite-secondary.yml
     #- <<: *release-artifacts
     #  name: "Linux release artifacts"

Cargo.toml

@@ -43,6 +43,7 @@ members = [
     "poh-bench",
     "program-test",
     "programs/bpf_loader",
+    "programs/compute-budget",
     "programs/config",
     "programs/exchange",
     "programs/failure",

account-decoder/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-account-decoder"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana account decoder"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -19,9 +19,9 @@ lazy_static = "1.4.0"
 serde = "1.0.122"
 serde_derive = "1.0.103"
 serde_json = "1.0.56"
-solana-config-program = { path = "../programs/config", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-config-program = { path = "../programs/config", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 spl-token-v2-0 = { package = "spl-token", version = "=3.2.0", features = ["no-entrypoint"] }
 thiserror = "1.0"
 zstd = "0.5.1"

accounts-bench/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-accounts-bench"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -11,11 +11,11 @@ publish = false
 [dependencies]
 log = "0.4.11"
 rayon = "1.5.0"
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 rand = "0.7.0"
 clap = "2.33.1"
 crossbeam-channel = "0.4"

accounts-cluster-bench/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-accounts-cluster-bench"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -13,24 +13,24 @@ clap = "2.33.1"
 log = "0.4.11"
 rand = "0.7.0"
 rayon = "1.4.1"
-solana-account-decoder = { path = "../account-decoder", version = "=1.7.13" }
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-core = { path = "../core", version = "=1.7.13" }
-solana-faucet = { path = "../faucet", version = "=1.7.13" }
-solana-gossip = { path = "../gossip", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-transaction-status = { path = "../transaction-status", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-account-decoder = { path = "../account-decoder", version = "=1.8.0" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-faucet = { path = "../faucet", version = "=1.8.0" }
+solana-gossip = { path = "../gossip", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-transaction-status = { path = "../transaction-status", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 spl-token-v2-0 = { package = "spl-token", version = "=3.2.0", features = ["no-entrypoint"] }
 
 [dev-dependencies]
-solana-local-cluster = { path = "../local-cluster", version = "=1.7.13" }
+solana-local-cluster = { path = "../local-cluster", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

banking-bench/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-banking-bench"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -14,18 +14,18 @@ crossbeam-channel = "0.4"
 log = "0.4.11"
 rand = "0.7.0"
 rayon = "1.5.0"
-solana-core = { path = "../core", version = "=1.7.13" }
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-gossip = { path = "../gossip", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-perf = { path = "../perf", version = "=1.7.13" }
-solana-poh = { path = "../poh", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-gossip = { path = "../gossip", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-perf = { path = "../perf", version = "=1.8.0" }
+solana-poh = { path = "../poh", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

banking-bench/src/main.rs

@@ -4,7 +4,7 @@ use crossbeam_channel::unbounded;
 use log::*;
 use rand::{thread_rng, Rng};
 use rayon::prelude::*;
-use solana_core::banking_stage::BankingStage;
+use solana_core::{banking_stage::BankingStage, cost_model::CostModel, cost_tracker::CostTracker};
 use solana_gossip::{cluster_info::ClusterInfo, cluster_info::Node};
 use solana_ledger::{
     blockstore::Blockstore,
@@ -27,7 +27,7 @@ use solana_sdk::{
 };
 use solana_streamer::socket::SocketAddrSpace;
 use std::{
-    sync::{atomic::Ordering, mpsc::Receiver, Arc, Mutex},
+    sync::{atomic::Ordering, mpsc::Receiver, Arc, Mutex, RwLock},
     thread::sleep,
     time::{Duration, Instant},
 };
@@ -231,6 +231,9 @@ fn main() {
             vote_receiver,
             None,
             replay_vote_sender,
+            Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                CostModel::default(),
+            ))))),
         );
         poh_recorder.lock().unwrap().set_bank(&bank);
 

banks-client/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-banks-client"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana banks client"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -15,16 +15,16 @@ borsh = "0.9.0"
 borsh-derive = "0.9.0"
 futures = "0.3"
 mio = "0.7.6"
-solana-banks-interface = { path = "../banks-interface", version = "=1.7.13" }
-solana-program = { path = "../sdk/program", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
+solana-banks-interface = { path = "../banks-interface", version = "=1.8.0" }
+solana-program = { path = "../sdk/program", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
 tarpc = { version = "0.24.1", features = ["full"] }
 tokio = { version = "1", features = ["full"] }
 tokio-serde = { version = "0.8", features = ["bincode"] }
 
 [dev-dependencies]
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-banks-server = { path = "../banks-server", version = "=1.7.13" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-banks-server = { path = "../banks-server", version = "=1.8.0" }
 
 [lib]
 crate-type = ["lib"]

banks-interface/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-banks-interface"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana banks RPC interface"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -12,7 +12,7 @@ edition = "2018"
 [dependencies]
 mio = "0.7.6"
 serde = { version = "1.0.122", features = ["derive"] }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
 tarpc = { version = "0.24.1", features = ["full"] }
 
 [dev-dependencies]

banks-server/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-banks-server"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana banks server"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -14,10 +14,10 @@ bincode = "1.3.1"
 futures = "0.3"
 log = "0.4.11"
 mio = "0.7.6"
-solana-banks-interface = { path = "../banks-interface", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
+solana-banks-interface = { path = "../banks-interface", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
 tarpc = { version = "0.24.1", features = ["full"] }
 tokio = { version = "1", features = ["full"] }
 tokio-serde = { version = "0.8", features = ["bincode"] }

bench-exchange/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-bench-exchange"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -18,23 +18,23 @@ rand = "0.7.0"
 rayon = "1.5.0"
 serde_json = "1.0.56"
 serde_yaml = "0.8.13"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-core = { path = "../core", version = "=1.7.13" }
-solana-genesis = { path = "../genesis", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-exchange-program = { path = "../programs/exchange", version = "=1.7.13" }
-solana-faucet = { path = "../faucet", version = "=1.7.13" }
-solana-gossip = { path = "../gossip", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-genesis = { path = "../genesis", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-exchange-program = { path = "../programs/exchange", version = "=1.8.0" }
+solana-faucet = { path = "../faucet", version = "=1.8.0" }
+solana-gossip = { path = "../gossip", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 
 [dev-dependencies]
-solana-local-cluster = { path = "../local-cluster", version = "=1.7.13" }
+solana-local-cluster = { path = "../local-cluster", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

bench-streamer/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-bench-streamer"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -10,11 +10,11 @@ publish = false
 
 [dependencies]
 clap = "2.33.1"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

bench-tps/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-bench-tps"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -15,24 +15,24 @@ log = "0.4.11"
 rayon = "1.5.0"
 serde_json = "1.0.56"
 serde_yaml = "0.8.13"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-core = { path = "../core", version = "=1.7.13" }
-solana-genesis = { path = "../genesis", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-faucet = { path = "../faucet", version = "=1.7.13" }
-solana-gossip = { path = "../gossip", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-genesis = { path = "../genesis", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-faucet = { path = "../faucet", version = "=1.8.0" }
+solana-gossip = { path = "../gossip", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 
 [dev-dependencies]
 serial_test = "0.4.0"
-solana-local-cluster = { path = "../local-cluster", version = "=1.7.13" }
+solana-local-cluster = { path = "../local-cluster", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

ci/do-audit.sh

@@ -45,5 +45,6 @@ cargo_audit_ignores=(
   # Blocked on jsonrpc removing dependency on unmaintained `websocket`
   # https://github.com/paritytech/jsonrpc/issues/605
   --ignore RUSTSEC-2021-0079
+
 )
 scripts/cargo-for-all-lock-files.sh stable audit "${cargo_audit_ignores[@]}"

clap-utils/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-clap-utils"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana utilities for the clap"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -12,8 +12,8 @@ edition = "2018"
 [dependencies]
 clap = "2.33.0"
 rpassword = "4.0"
-solana-remote-wallet = { path = "../remote-wallet", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
+solana-remote-wallet = { path = "../remote-wallet", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
 thiserror = "1.0.21"
 tiny-bip39 = "0.8.1"
 uriparse = "0.6.3"

cli-config/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-cli-config"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"

cli-output/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-cli-output"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -20,12 +20,12 @@ indicatif = "0.15.0"
 serde = "1.0.122"
 serde_derive = "1.0.103"
 serde_json = "1.0.56"
-solana-account-decoder = { path = "../account-decoder", version = "=1.7.13" }
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-transaction-status = { path = "../transaction-status", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-account-decoder = { path = "../account-decoder", version = "=1.8.0" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-transaction-status = { path = "../transaction-status", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 spl-memo = { version = "=3.0.1", features = ["no-entrypoint"] }
 
 [package.metadata.docs.rs]

cli/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-cli"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -29,30 +29,30 @@ reqwest = { version = "0.11.2", default-features = false, features = ["blocking"
 serde = "1.0.122"
 serde_derive = "1.0.103"
 serde_json = "1.0.56"
-solana-account-decoder = { path = "../account-decoder", version = "=1.7.13" }
-solana-bpf-loader-program = { path = "../programs/bpf_loader", version = "=1.7.13" }
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-cli-config = { path = "../cli-config", version = "=1.7.13" }
-solana-cli-output = { path = "../cli-output", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-config-program = { path = "../programs/config", version = "=1.7.13" }
-solana-faucet = { path = "../faucet", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
+solana-account-decoder = { path = "../account-decoder", version = "=1.8.0" }
+solana-bpf-loader-program = { path = "../programs/bpf_loader", version = "=1.8.0" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-cli-config = { path = "../cli-config", version = "=1.8.0" }
+solana-cli-output = { path = "../cli-output", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-config-program = { path = "../programs/config", version = "=1.8.0" }
+solana-faucet = { path = "../faucet", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
 solana_rbpf = "=0.2.11"
-solana-remote-wallet = { path = "../remote-wallet", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-transaction-status = { path = "../transaction-status", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-remote-wallet = { path = "../remote-wallet", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-transaction-status = { path = "../transaction-status", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 spl-memo = { version = "=3.0.1", features = ["no-entrypoint"] }
 thiserror = "1.0.21"
 tiny-bip39 = "0.8.1"
 url = "2.1.1"
 
 [dev-dependencies]
-solana-core = { path = "../core", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
 tempfile = "3.1.0"
 
 [[bin]]

cli/src/program.rs

@@ -2018,7 +2018,13 @@ fn complete_partial_program_init(
         return Err("Buffer account is already executable".into());
     }
     if account.owner != *loader_id && !system_program::check_id(&account.owner) {
-        return Err("Buffer account is already owned by another account".into());
+        return Err("Buffer account passed is already in use by another program".into());
+    }
+    if !account.data.is_empty() && account.data.len() < account_data_len {
+        return Err(
+            "Buffer account passed is not large enough, may have been for a different deploy?"
+                .into(),
+        );
     }
 
     if account.data.is_empty() && system_program::check_id(&account.owner) {
@@ -2029,24 +2035,24 @@ fn complete_partial_program_init(
         if account.owner != *loader_id {
             instructions.push(system_instruction::assign(elf_pubkey, loader_id));
         }
-    }
-    if account.lamports < minimum_balance {
-        let balance = minimum_balance - account.lamports;
-        instructions.push(system_instruction::transfer(
-            payer_pubkey,
-            elf_pubkey,
-            balance,
-        ));
-        balance_needed = balance;
-    } else if account.lamports > minimum_balance
-        && system_program::check_id(&account.owner)
-        && !allow_excessive_balance
-    {
-        return Err(format!(
-            "Buffer account has a balance: {:?}; it may already be in use",
-            Sol(account.lamports)
-        )
-        .into());
+        if account.lamports < minimum_balance {
+            let balance = minimum_balance - account.lamports;
+            instructions.push(system_instruction::transfer(
+                payer_pubkey,
+                elf_pubkey,
+                balance,
+            ));
+            balance_needed = balance;
+        } else if account.lamports > minimum_balance
+            && system_program::check_id(&account.owner)
+            && !allow_excessive_balance
+        {
+            return Err(format!(
+                "Buffer account has a balance: {:?}; it may already be in use",
+                Sol(account.lamports)
+            )
+            .into());
+        }
     }
     Ok((instructions, balance_needed))
 }

cli/src/stake.rs

@@ -39,13 +39,11 @@ use solana_sdk::{
     stake::{
         self,
         instruction::{self as stake_instruction, LockupArgs, StakeError},
-        state::{Authorized, Lockup, Meta, StakeAuthorize, StakeState},
+        state::{Authorized, Lockup, Meta, StakeActivationStatus, StakeAuthorize, StakeState},
     },
+    stake_history::StakeHistory,
     system_instruction::SystemError,
-    sysvar::{
-        clock,
-        stake_history::{self, StakeHistory},
-    },
+    sysvar::{clock, stake_history},
     transaction::Transaction,
 };
 use solana_vote_program::vote_state::VoteState;
@@ -2020,7 +2018,11 @@ pub fn build_stake_state(
             stake,
         ) => {
             let current_epoch = clock.epoch;
-            let (active_stake, activating_stake, deactivating_stake) = stake
+            let StakeActivationStatus {
+                effective,
+                activating,
+                deactivating,
+            } = stake
                 .delegation
                 .stake_activating_and_deactivating(current_epoch, Some(stake_history));
             let lockup = if lockup.is_in_force(clock, None) {
@@ -2055,9 +2057,9 @@ pub fn build_stake_state(
                 use_lamports_unit,
                 current_epoch,
                 rent_exempt_reserve: Some(*rent_exempt_reserve),
-                active_stake: u64_some_if_not_zero(active_stake),
-                activating_stake: u64_some_if_not_zero(activating_stake),
-                deactivating_stake: u64_some_if_not_zero(deactivating_stake),
+                active_stake: u64_some_if_not_zero(effective),
+                activating_stake: u64_some_if_not_zero(activating),
+                deactivating_stake: u64_some_if_not_zero(deactivating),
                 ..CliStakeState::default()
             }
         }

cli/tests/fixtures/build.sh

@@ -5,3 +5,4 @@ cd "$(dirname "$0")"
 
 make -C ../../../programs/bpf/c/
 cp ../../../programs/bpf/c/out/noop.so .
+cat noop.so noop.so noop.so > noop_large.so

cli/tests/program.rs

@@ -22,11 +22,11 @@ use std::{env, fs::File, io::Read, path::PathBuf, str::FromStr};
 fn test_cli_program_deploy_non_upgradeable() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -37,7 +37,7 @@ fn test_cli_program_deploy_non_upgradeable() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let minimum_balance_for_rent_exemption = rpc_client
@@ -55,7 +55,7 @@ fn test_cli_program_deploy_non_upgradeable() {
     process_command(&config).unwrap();
 
     config.command = CliCommand::Deploy {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         address: None,
         use_deprecated_loader: false,
         allow_excessive_balance: false,
@@ -75,7 +75,7 @@ fn test_cli_program_deploy_non_upgradeable() {
     assert_eq!(account0.lamports, minimum_balance_for_rent_exemption);
     assert_eq!(account0.owner, bpf_loader::id());
     assert!(account0.executable);
-    let mut file = File::open(pathbuf.to_str().unwrap().to_string()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap().to_string()).unwrap();
     let mut elf = Vec::new();
     file.read_to_end(&mut elf).unwrap();
     assert_eq!(account0.data, elf);
@@ -84,7 +84,7 @@ fn test_cli_program_deploy_non_upgradeable() {
     let custom_address_keypair = Keypair::new();
     config.signers = vec![&keypair, &custom_address_keypair];
     config.command = CliCommand::Deploy {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         address: Some(1),
         use_deprecated_loader: false,
         allow_excessive_balance: false,
@@ -111,7 +111,7 @@ fn test_cli_program_deploy_non_upgradeable() {
     process_command(&config).unwrap();
     config.signers = vec![&keypair, &custom_address_keypair];
     config.command = CliCommand::Deploy {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         address: Some(1),
         use_deprecated_loader: false,
         allow_excessive_balance: false,
@@ -120,7 +120,7 @@ fn test_cli_program_deploy_non_upgradeable() {
 
     // Use forcing parameter to deploy to account with excess balance
     config.command = CliCommand::Deploy {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         address: Some(1),
         use_deprecated_loader: false,
         allow_excessive_balance: true,
@@ -139,11 +139,11 @@ fn test_cli_program_deploy_non_upgradeable() {
 fn test_cli_program_deploy_no_authority() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -154,7 +154,7 @@ fn test_cli_program_deploy_no_authority() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -181,7 +181,7 @@ fn test_cli_program_deploy_no_authority() {
     // Deploy a program
     config.signers = vec![&keypair, &upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: None,
         buffer_signer_index: None,
@@ -206,7 +206,7 @@ fn test_cli_program_deploy_no_authority() {
     // Attempt to upgrade the program
     config.signers = vec![&keypair, &upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: Some(program_id),
         buffer_signer_index: None,
@@ -223,11 +223,11 @@ fn test_cli_program_deploy_no_authority() {
 fn test_cli_program_deploy_with_authority() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -238,7 +238,7 @@ fn test_cli_program_deploy_with_authority() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -266,7 +266,7 @@ fn test_cli_program_deploy_with_authority() {
     let program_keypair = Keypair::new();
     config.signers = vec![&keypair, &upgrade_authority, &program_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: Some(2),
         program_pubkey: Some(program_keypair.pubkey()),
         buffer_signer_index: None,
@@ -313,7 +313,7 @@ fn test_cli_program_deploy_with_authority() {
     // Deploy the upgradeable program
     config.signers = vec![&keypair, &upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: None,
         buffer_signer_index: None,
@@ -354,7 +354,7 @@ fn test_cli_program_deploy_with_authority() {
     // Upgrade the program
     config.signers = vec![&keypair, &upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: Some(program_pubkey),
         buffer_signer_index: None,
@@ -408,7 +408,7 @@ fn test_cli_program_deploy_with_authority() {
     // Upgrade with new authority
     config.signers = vec![&keypair, &new_upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: Some(program_pubkey),
         buffer_signer_index: None,
@@ -482,7 +482,7 @@ fn test_cli_program_deploy_with_authority() {
     // Upgrade with no authority
     config.signers = vec![&keypair, &new_upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: Some(program_pubkey),
         buffer_signer_index: None,
@@ -497,7 +497,7 @@ fn test_cli_program_deploy_with_authority() {
     // deploy with finality
     config.signers = vec![&keypair, &new_upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: None,
         buffer_signer_index: None,
@@ -556,11 +556,11 @@ fn test_cli_program_deploy_with_authority() {
 fn test_cli_program_close_program() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -571,7 +571,7 @@ fn test_cli_program_close_program() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -599,7 +599,7 @@ fn test_cli_program_close_program() {
     let program_keypair = Keypair::new();
     config.signers = vec![&keypair, &upgrade_authority, &program_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: Some(2),
         program_pubkey: Some(program_keypair.pubkey()),
         buffer_signer_index: None,
@@ -638,11 +638,17 @@ fn test_cli_program_close_program() {
 fn test_cli_program_write_buffer() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
+
+    let mut noop_large_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_large_path.push("tests");
+    noop_large_path.push("fixtures");
+    noop_large_path.push("noop_large");
+    noop_large_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -653,7 +659,7 @@ fn test_cli_program_write_buffer() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -681,7 +687,7 @@ fn test_cli_program_write_buffer() {
     // Write a buffer with default params
     config.signers = vec![&keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: None,
         buffer_pubkey: None,
         buffer_authority_signer_index: None,
@@ -715,7 +721,7 @@ fn test_cli_program_write_buffer() {
     let buffer_keypair = Keypair::new();
     config.signers = vec![&keypair, &buffer_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: Some(1),
         buffer_pubkey: Some(buffer_keypair.pubkey()),
         buffer_authority_signer_index: None,
@@ -776,7 +782,7 @@ fn test_cli_program_write_buffer() {
     let authority_keypair = Keypair::new();
     config.signers = vec![&keypair, &buffer_keypair, &authority_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: Some(1),
         buffer_pubkey: Some(buffer_keypair.pubkey()),
         buffer_authority_signer_index: Some(2),
@@ -813,7 +819,7 @@ fn test_cli_program_write_buffer() {
     let authority_keypair = Keypair::new();
     config.signers = vec![&keypair, &buffer_keypair, &authority_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: None,
         buffer_pubkey: None,
         buffer_authority_signer_index: Some(2),
@@ -885,7 +891,7 @@ fn test_cli_program_write_buffer() {
     // Write a buffer with default params
     config.signers = vec![&keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: None,
         buffer_pubkey: None,
         buffer_authority_signer_index: None,
@@ -919,17 +925,47 @@ fn test_cli_program_write_buffer() {
         pre_lamports + minimum_balance_for_buffer,
         recipient_account.lamports
     );
+
+    // write small buffer then attempt to deploy larger program
+    let buffer_keypair = Keypair::new();
+    config.signers = vec![&keypair, &buffer_keypair];
+    config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
+        program_location: noop_path.to_str().unwrap().to_string(),
+        buffer_signer_index: Some(1),
+        buffer_pubkey: Some(buffer_keypair.pubkey()),
+        buffer_authority_signer_index: None,
+        max_len: None, //Some(max_len),
+    });
+    process_command(&config).unwrap();
+    config.signers = vec![&keypair, &buffer_keypair];
+    config.command = CliCommand::Program(ProgramCliCommand::Deploy {
+        program_location: Some(noop_large_path.to_str().unwrap().to_string()),
+        program_signer_index: None,
+        program_pubkey: None,
+        buffer_signer_index: Some(1),
+        buffer_pubkey: Some(buffer_keypair.pubkey()),
+        allow_excessive_balance: false,
+        upgrade_authority_signer_index: 1,
+        is_final: true,
+        max_len: None,
+    });
+    config.output_format = OutputFormat::JsonCompact;
+    let error = process_command(&config).unwrap_err();
+    assert_eq!(
+        error.to_string(),
+        "Buffer account passed is not large enough, may have been for a different deploy?"
+    );
 }
 
 #[test]
 fn test_cli_program_set_buffer_authority() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -940,7 +976,7 @@ fn test_cli_program_set_buffer_authority() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -964,7 +1000,7 @@ fn test_cli_program_set_buffer_authority() {
     let buffer_keypair = Keypair::new();
     config.signers = vec![&keypair, &buffer_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: Some(1),
         buffer_pubkey: Some(buffer_keypair.pubkey()),
         buffer_authority_signer_index: None,
@@ -1039,11 +1075,11 @@ fn test_cli_program_set_buffer_authority() {
 fn test_cli_program_mismatch_buffer_authority() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -1054,7 +1090,7 @@ fn test_cli_program_mismatch_buffer_authority() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -1079,7 +1115,7 @@ fn test_cli_program_mismatch_buffer_authority() {
     let buffer_keypair = Keypair::new();
     config.signers = vec![&keypair, &buffer_keypair, &buffer_authority];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: Some(1),
         buffer_pubkey: Some(buffer_keypair.pubkey()),
         buffer_authority_signer_index: Some(2),
@@ -1097,7 +1133,7 @@ fn test_cli_program_mismatch_buffer_authority() {
     let upgrade_authority = Keypair::new();
     config.signers = vec![&keypair, &upgrade_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: None,
         buffer_signer_index: None,
@@ -1112,7 +1148,7 @@ fn test_cli_program_mismatch_buffer_authority() {
     // Attempt to deploy matched authority
     config.signers = vec![&keypair, &buffer_authority];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: None,
         program_pubkey: None,
         buffer_signer_index: None,
@@ -1129,11 +1165,11 @@ fn test_cli_program_mismatch_buffer_authority() {
 fn test_cli_program_show() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -1144,7 +1180,7 @@ fn test_cli_program_show() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -1172,7 +1208,7 @@ fn test_cli_program_show() {
     let authority_keypair = Keypair::new();
     config.signers = vec![&keypair, &buffer_keypair, &authority_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: Some(1),
         buffer_pubkey: Some(buffer_keypair.pubkey()),
         buffer_authority_signer_index: Some(2),
@@ -1227,7 +1263,7 @@ fn test_cli_program_show() {
     let program_keypair = Keypair::new();
     config.signers = vec![&keypair, &authority_keypair, &program_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::Deploy {
-        program_location: Some(pathbuf.to_str().unwrap().to_string()),
+        program_location: Some(noop_path.to_str().unwrap().to_string()),
         program_signer_index: Some(2),
         program_pubkey: Some(program_keypair.pubkey()),
         buffer_signer_index: None,
@@ -1314,11 +1350,11 @@ fn test_cli_program_show() {
 fn test_cli_program_dump() {
     solana_logger::setup();
 
-    let mut pathbuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    pathbuf.push("tests");
-    pathbuf.push("fixtures");
-    pathbuf.push("noop");
-    pathbuf.set_extension("so");
+    let mut noop_path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    noop_path.push("tests");
+    noop_path.push("fixtures");
+    noop_path.push("noop");
+    noop_path.set_extension("so");
 
     let mint_keypair = Keypair::new();
     let mint_pubkey = mint_keypair.pubkey();
@@ -1329,7 +1365,7 @@ fn test_cli_program_dump() {
     let rpc_client =
         RpcClient::new_with_commitment(test_validator.rpc_url(), CommitmentConfig::processed());
 
-    let mut file = File::open(pathbuf.to_str().unwrap()).unwrap();
+    let mut file = File::open(noop_path.to_str().unwrap()).unwrap();
     let mut program_data = Vec::new();
     file.read_to_end(&mut program_data).unwrap();
     let max_len = program_data.len();
@@ -1357,7 +1393,7 @@ fn test_cli_program_dump() {
     let authority_keypair = Keypair::new();
     config.signers = vec![&keypair, &buffer_keypair, &authority_keypair];
     config.command = CliCommand::Program(ProgramCliCommand::WriteBuffer {
-        program_location: pathbuf.to_str().unwrap().to_string(),
+        program_location: noop_path.to_str().unwrap().to_string(),
         buffer_signer_index: Some(1),
         buffer_pubkey: Some(buffer_keypair.pubkey()),
         buffer_authority_signer_index: Some(2),

client/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-client"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Client"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -24,14 +24,14 @@ semver = "0.11.0"
 serde = "1.0.122"
 serde_derive = "1.0.103"
 serde_json = "1.0.56"
-solana-account-decoder = { path = "../account-decoder", version = "=1.7.13" }
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-faucet = { path = "../faucet", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-transaction-status = { path = "../transaction-status", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-account-decoder = { path = "../account-decoder", version = "=1.8.0" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-faucet = { path = "../faucet", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-transaction-status = { path = "../transaction-status", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 thiserror = "1.0"
 tokio = { version = "1", features = ["full"] }
 tungstenite = "0.10.1"
@@ -40,7 +40,7 @@ url = "2.1.1"
 [dev-dependencies]
 assert_matches = "1.3.0"
 jsonrpc-http-server = "18.0.0"
-solana-logger = { path = "../logger", version = "=1.7.13" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

core/Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "solana-core"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 homepage = "https://solana.com/"
 documentation = "https://docs.rs/solana-core"
 readme = "../README.md"
@@ -27,13 +27,14 @@ ed25519-dalek = "=1.0.1"
 fs_extra = "1.2.0"
 flate2 = "1.0"
 indexmap = { version = "1.5", features = ["rayon"] }
-itertools = "0.9.0"
 libc = "0.2.81"
-log = "0.4.11"
-lru = "0.6.1"
 miow = "0.2.2"
 net2 = "0.2.37"
 num-traits = "0.2"
+histogram = "0.6.9"
+itertools = "0.10.1"
+log = "0.4.14"
+lru = "0.6.6"
 rand = "0.7.0"
 rand_chacha = "0.2.2"
 rand_core = "0.6.2"
@@ -43,33 +44,33 @@ retain_mut = "0.1.2"
 serde = "1.0.122"
 serde_bytes = "0.11"
 serde_derive = "1.0.103"
-solana-account-decoder = { path = "../account-decoder", version = "=1.7.13" }
-solana-banks-server = { path = "../banks-server", version = "=1.7.13" }
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-gossip = { path = "../gossip", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-merkle-tree = { path = "../merkle-tree", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-perf = { path = "../perf", version = "=1.7.13" }
-solana-poh = { path = "../poh", version = "=1.7.13" }
-solana-program-test = { path = "../program-test", version = "=1.7.13" }
-solana-rpc = { path = "../rpc", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-frozen-abi = { path = "../frozen-abi", version = "=1.7.13" }
-solana-frozen-abi-macro = { path = "../frozen-abi/macro", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-transaction-status = { path = "../transaction-status", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-account-decoder = { path = "../account-decoder", version = "=1.8.0" }
+solana-banks-server = { path = "../banks-server", version = "=1.8.0" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-gossip = { path = "../gossip", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-merkle-tree = { path = "../merkle-tree", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-perf = { path = "../perf", version = "=1.8.0" }
+solana-poh = { path = "../poh", version = "=1.8.0" }
+solana-program-test = { path = "../program-test", version = "=1.8.0" }
+solana-rpc = { path = "../rpc", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-frozen-abi = { path = "../frozen-abi", version = "=1.8.0" }
+solana-frozen-abi-macro = { path = "../frozen-abi/macro", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-transaction-status = { path = "../transaction-status", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 spl-token-v2-0 = { package = "spl-token", version = "=3.2.0", features = ["no-entrypoint"] }
 tempfile = "3.1.0"
 thiserror = "1.0"
-solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.7.13" }
+solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.8.0" }
 trees = "0.2.1"
 
 [dev-dependencies]
@@ -82,8 +83,8 @@ num_cpus = "1.13.0"
 reqwest = { version = "0.11.2", default-features = false, features = ["blocking", "rustls-tls", "json"] }
 serde_json = "1.0.56"
 serial_test = "0.4.0"
-solana-stake-program = { path = "../programs/stake", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-stake-program = { path = "../programs/stake", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 symlink = "0.1.0"
 systemstat = "0.1.5"
 tokio = { version = "1", features = ["full"] }

core/benches/banking_stage.rs

@@ -8,6 +8,9 @@ use log::*;
 use rand::{thread_rng, Rng};
 use rayon::prelude::*;
 use solana_core::banking_stage::{BankingStage, BankingStageStats};
+use solana_core::cost_model::CostModel;
+use solana_core::cost_tracker::CostTracker;
+use solana_core::cost_tracker_stats::CostTrackerStats;
 use solana_gossip::cluster_info::ClusterInfo;
 use solana_gossip::cluster_info::Node;
 use solana_ledger::blockstore_processor::process_entries;
@@ -33,7 +36,7 @@ use solana_streamer::socket::SocketAddrSpace;
 use std::collections::VecDeque;
 use std::sync::atomic::Ordering;
 use std::sync::mpsc::Receiver;
-use std::sync::Arc;
+use std::sync::{Arc, RwLock};
 use std::time::{Duration, Instant};
 use test::Bencher;
 
@@ -92,6 +95,10 @@ fn bench_consume_buffered(bencher: &mut Bencher) {
                 None::<Box<dyn Fn()>>,
                 &BankingStageStats::default(),
                 &recorder,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::new(std::u64::MAX, std::u64::MAX),
+                ))))),
+                &mut CostTrackerStats::default(),
             );
         });
 
@@ -218,6 +225,9 @@ fn bench_banking(bencher: &mut Bencher, tx_type: TransactionType) {
             vote_receiver,
             None,
             s,
+            Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                CostModel::new(std::u64::MAX, std::u64::MAX),
+            ))))),
         );
         poh_recorder.lock().unwrap().set_bank(&bank);
 

core/benches/sigverify_stage.rs

@@ -18,6 +18,44 @@ use std::sync::mpsc::channel;
 use std::time::{Duration, Instant};
 use test::Bencher;
 
+#[bench]
+fn bench_packet_discard(bencher: &mut Bencher) {
+    solana_logger::setup();
+    let len = 30 * 1000;
+    let chunk_size = 1024;
+    let tx = test_tx();
+    let mut batches = to_packets_chunked(&vec![tx; len], chunk_size);
+
+    let mut total = 0;
+
+    let ips: Vec<_> = (0..10_000)
+        .into_iter()
+        .map(|_| {
+            let mut addr = [0u16; 8];
+            thread_rng().fill(&mut addr);
+            addr
+        })
+        .collect();
+
+    for batch in batches.iter_mut() {
+        total += batch.packets.len();
+        for p in batch.packets.iter_mut() {
+            let ip_index = thread_rng().gen_range(0, ips.len());
+            p.meta.addr = ips[ip_index];
+        }
+    }
+    info!("total packets: {}", total);
+
+    bencher.iter(move || {
+        SigVerifyStage::discard_excess_packets(&mut batches, 10_000);
+        for batch in batches.iter_mut() {
+            for p in batch.packets.iter_mut() {
+                p.meta.discard = false;
+            }
+        }
+    });
+}
+
 #[bench]
 fn bench_sigverify_stage(bencher: &mut Bencher) {
     solana_logger::setup();

core/src/banking_stage.rs

@@ -1,7 +1,9 @@
 //! The `banking_stage` processes Transaction messages. It is intended to be used
 //! to contruct a software pipeline. The stage uses all available CPU cores and
 //! can do its processing in parallel with signature verification on the GPU.
-use crate::packet_hasher::PacketHasher;
+use crate::{
+    cost_tracker::CostTracker, cost_tracker_stats::CostTrackerStats, packet_hasher::PacketHasher,
+};
 use crossbeam_channel::{Receiver as CrossbeamReceiver, RecvTimeoutError};
 use itertools::Itertools;
 use lru::LruCache;
@@ -52,7 +54,7 @@ use std::{
     net::{SocketAddr, UdpSocket},
     ops::DerefMut,
     sync::atomic::{AtomicU64, AtomicUsize, Ordering},
-    sync::{Arc, Mutex},
+    sync::{Arc, Mutex, RwLock},
     thread::{self, Builder, JoinHandle},
     time::Duration,
     time::Instant,
@@ -93,6 +95,9 @@ pub struct BankingStageStats {
     current_buffered_packet_batches_count: AtomicUsize,
     rebuffered_packets_count: AtomicUsize,
     consumed_buffered_packets_count: AtomicUsize,
+    reset_cost_tracker_count: AtomicUsize,
+    cost_tracker_check_count: AtomicUsize,
+    cost_forced_retry_transactions_count: AtomicUsize,
 
     // Timing
     consume_buffered_packets_elapsed: AtomicU64,
@@ -101,7 +106,11 @@ pub struct BankingStageStats {
     filter_pending_packets_elapsed: AtomicU64,
     packet_duplicate_check_elapsed: AtomicU64,
     packet_conversion_elapsed: AtomicU64,
+    unprocessed_packet_conversion_elapsed: AtomicU64,
     transaction_processing_elapsed: AtomicU64,
+    cost_tracker_update_elapsed: AtomicU64,
+    cost_tracker_clone_elapsed: AtomicU64,
+    cost_tracker_check_elapsed: AtomicU64,
 }
 
 impl BankingStageStats {
@@ -165,6 +174,22 @@ impl BankingStageStats {
                         .swap(0, Ordering::Relaxed) as i64,
                     i64
                 ),
+                (
+                    "reset_cost_tracker_count",
+                    self.reset_cost_tracker_count.swap(0, Ordering::Relaxed) as i64,
+                    i64
+                ),
+                (
+                    "cost_tracker_check_count",
+                    self.cost_tracker_check_count.swap(0, Ordering::Relaxed) as i64,
+                    i64
+                ),
+                (
+                    "cost_forced_retry_transactions_count",
+                    self.cost_forced_retry_transactions_count
+                        .swap(0, Ordering::Relaxed) as i64,
+                    i64
+                ),
                 (
                     "consume_buffered_packets_elapsed",
                     self.consume_buffered_packets_elapsed
@@ -199,12 +224,33 @@ impl BankingStageStats {
                     self.packet_conversion_elapsed.swap(0, Ordering::Relaxed) as i64,
                     i64
                 ),
+                (
+                    "unprocessed_packet_conversion_elapsed",
+                    self.unprocessed_packet_conversion_elapsed
+                        .swap(0, Ordering::Relaxed) as i64,
+                    i64
+                ),
                 (
                     "transaction_processing_elapsed",
                     self.transaction_processing_elapsed
                         .swap(0, Ordering::Relaxed) as i64,
                     i64
                 ),
+                (
+                    "cost_tracker_update_elapsed",
+                    self.cost_tracker_update_elapsed.swap(0, Ordering::Relaxed) as i64,
+                    i64
+                ),
+                (
+                    "cost_tracker_clone_elapsed",
+                    self.cost_tracker_clone_elapsed.swap(0, Ordering::Relaxed) as i64,
+                    i64
+                ),
+                (
+                    "cost_tracker_check_elapsed",
+                    self.cost_tracker_check_elapsed.swap(0, Ordering::Relaxed) as i64,
+                    i64
+                ),
             );
         }
     }
@@ -241,6 +287,7 @@ impl BankingStage {
         verified_vote_receiver: CrossbeamReceiver<Vec<Packets>>,
         transaction_status_sender: Option<TransactionStatusSender>,
         gossip_vote_sender: ReplayVoteSender,
+        cost_tracker: Arc<RwLock<CostTracker>>,
     ) -> Self {
         Self::new_num_threads(
             cluster_info,
@@ -251,6 +298,7 @@ impl BankingStage {
             Self::num_threads(),
             transaction_status_sender,
             gossip_vote_sender,
+            cost_tracker,
         )
     }
 
@@ -263,6 +311,7 @@ impl BankingStage {
         num_threads: u32,
         transaction_status_sender: Option<TransactionStatusSender>,
         gossip_vote_sender: ReplayVoteSender,
+        cost_tracker: Arc<RwLock<CostTracker>>,
     ) -> Self {
         let batch_limit = TOTAL_BUFFERED_PACKETS / ((num_threads - 1) as usize * PACKETS_PER_BATCH);
         // Single thread to generate entries from many banks.
@@ -298,6 +347,7 @@ impl BankingStage {
                 let gossip_vote_sender = gossip_vote_sender.clone();
                 let duplicates = duplicates.clone();
                 let data_budget = data_budget.clone();
+                let cost_tracker = cost_tracker.clone();
                 Builder::new()
                     .name("solana-banking-stage-tx".to_string())
                     .spawn(move || {
@@ -314,6 +364,7 @@ impl BankingStage {
                             gossip_vote_sender,
                             &duplicates,
                             &data_budget,
+                            &cost_tracker,
                         );
                     })
                     .unwrap()
@@ -371,6 +422,25 @@ impl BankingStage {
         has_more_unprocessed_transactions
     }
 
+    fn reset_cost_tracker_if_new_bank(
+        cost_tracker: &Arc<RwLock<CostTracker>>,
+        bank: Arc<Bank>,
+        banking_stage_stats: &BankingStageStats,
+        cost_tracker_stats: &mut CostTrackerStats,
+    ) {
+        if cost_tracker
+            .write()
+            .unwrap()
+            .reset_if_new_bank(bank.slot(), cost_tracker_stats)
+        {
+            // only increase counter when bank changed
+            banking_stage_stats
+                .reset_cost_tracker_count
+                .fetch_add(1, Ordering::Relaxed);
+        }
+    }
+
+    #[allow(clippy::too_many_arguments)]
     pub fn consume_buffered_packets(
         my_pubkey: &Pubkey,
         max_tx_ingestion_ns: u128,
@@ -381,6 +451,8 @@ impl BankingStage {
         test_fn: Option<impl Fn()>,
         banking_stage_stats: &BankingStageStats,
         recorder: &TransactionRecorder,
+        cost_tracker: &Arc<RwLock<CostTracker>>,
+        cost_tracker_stats: &mut CostTrackerStats,
     ) {
         let mut rebuffered_packets_len = 0;
         let mut new_tx_count = 0;
@@ -398,6 +470,9 @@ impl BankingStage {
                     original_unprocessed_indexes,
                     my_pubkey,
                     *next_leader,
+                    cost_tracker,
+                    banking_stage_stats,
+                    cost_tracker_stats,
                 );
                 Self::update_buffered_packets_with_new_unprocessed(
                     original_unprocessed_indexes,
@@ -406,6 +481,12 @@ impl BankingStage {
             } else {
                 let bank_start = poh_recorder.lock().unwrap().bank_start();
                 if let Some((bank, bank_creation_time)) = bank_start {
+                    Self::reset_cost_tracker_if_new_bank(
+                        cost_tracker,
+                        bank.clone(),
+                        banking_stage_stats,
+                        cost_tracker_stats,
+                    );
                     let (processed, verified_txs_len, new_unprocessed_indexes) =
                         Self::process_packets_transactions(
                             &bank,
@@ -416,6 +497,8 @@ impl BankingStage {
                             transaction_status_sender.clone(),
                             gossip_vote_sender,
                             banking_stage_stats,
+                            cost_tracker,
+                            cost_tracker_stats,
                         );
                     if processed < verified_txs_len
                         || !Bank::should_bank_still_be_processing_txs(
@@ -519,6 +602,8 @@ impl BankingStage {
         banking_stage_stats: &BankingStageStats,
         recorder: &TransactionRecorder,
         data_budget: &DataBudget,
+        cost_tracker: &Arc<RwLock<CostTracker>>,
+        cost_tracker_stats: &mut CostTrackerStats,
     ) -> BufferedPacketsDecision {
         let bank_start;
         let (
@@ -529,6 +614,14 @@ impl BankingStage {
         ) = {
             let poh = poh_recorder.lock().unwrap();
             bank_start = poh.bank_start();
+            if let Some((ref bank, _)) = bank_start {
+                Self::reset_cost_tracker_if_new_bank(
+                    cost_tracker,
+                    bank.clone(),
+                    banking_stage_stats,
+                    cost_tracker_stats,
+                );
+            };
             (
                 poh.leader_after_n_slots(FORWARD_TRANSACTIONS_TO_LEADER_AT_SLOT_OFFSET),
                 PohRecorder::get_bank_still_processing_txs(&bank_start),
@@ -559,6 +652,8 @@ impl BankingStage {
                     None::<Box<dyn Fn()>>,
                     banking_stage_stats,
                     recorder,
+                    cost_tracker,
+                    cost_tracker_stats,
                 );
             }
             BufferedPacketsDecision::Forward => {
@@ -638,11 +733,13 @@ impl BankingStage {
         gossip_vote_sender: ReplayVoteSender,
         duplicates: &Arc<Mutex<(LruCache<u64, ()>, PacketHasher)>>,
         data_budget: &DataBudget,
+        cost_tracker: &Arc<RwLock<CostTracker>>,
     ) {
         let recorder = poh_recorder.lock().unwrap().recorder();
         let socket = UdpSocket::bind("0.0.0.0:0").unwrap();
         let mut buffered_packets = VecDeque::with_capacity(batch_limit);
         let banking_stage_stats = BankingStageStats::new(id);
+        let mut cost_tracker_stats = CostTrackerStats::new(id, 0);
         loop {
             while !buffered_packets.is_empty() {
                 let decision = Self::process_buffered_packets(
@@ -657,6 +754,8 @@ impl BankingStage {
                     &banking_stage_stats,
                     &recorder,
                     data_budget,
+                    cost_tracker,
+                    &mut cost_tracker_stats,
                 );
                 if matches!(decision, BufferedPacketsDecision::Hold)
                     || matches!(decision, BufferedPacketsDecision::ForwardAndHold)
@@ -691,6 +790,8 @@ impl BankingStage {
                 &banking_stage_stats,
                 duplicates,
                 &recorder,
+                cost_tracker,
+                &mut cost_tracker_stats,
             ) {
                 Ok(()) | Err(RecvTimeoutError::Timeout) => (),
                 Err(RecvTimeoutError::Disconnected) => break,
@@ -935,12 +1036,12 @@ impl BankingStage {
     ) -> (usize, Vec<usize>) {
         let mut chunk_start = 0;
         let mut unprocessed_txs = vec![];
+
         while chunk_start != transactions.len() {
             let chunk_end = std::cmp::min(
                 transactions.len(),
                 chunk_start + MAX_NUM_TRANSACTIONS_PER_BATCH,
             );
-
             let (result, retryable_txs_in_chunk) = Self::process_and_record_transactions(
                 bank,
                 &transactions[chunk_start..chunk_end],
@@ -1023,13 +1124,21 @@ impl BankingStage {
     // This function deserializes packets into transactions, computes the blake3 hash of transaction messages,
     // and verifies secp256k1 instructions. A list of valid transactions are returned with their message hashes
     // and packet indexes.
+    // Also returned is packet indexes for transaction should be retried due to cost limits.
+    #[allow(clippy::needless_collect)]
     fn transactions_from_packets(
         msgs: &Packets,
         transaction_indexes: &[usize],
         libsecp256k1_0_5_upgrade_enabled: bool,
         votes_only: bool,
-    ) -> (Vec<HashedTransaction<'static>>, Vec<usize>) {
-        transaction_indexes
+        cost_tracker: &Arc<RwLock<CostTracker>>,
+        banking_stage_stats: &BankingStageStats,
+        demote_program_write_locks: bool,
+        cost_tracker_stats: &mut CostTrackerStats,
+    ) -> (Vec<HashedTransaction<'static>>, Vec<usize>, Vec<usize>) {
+        let mut retryable_transaction_packet_indexes: Vec<usize> = vec![];
+
+        let verified_transactions_with_packet_indexes: Vec<_> = transaction_indexes
             .iter()
             .filter_map(|tx_index| {
                 let p = &msgs.packets[*tx_index];
@@ -1040,14 +1149,68 @@ impl BankingStage {
                 let tx: Transaction = limited_deserialize(&p.data[0..p.meta.size]).ok()?;
                 tx.verify_precompiles(libsecp256k1_0_5_upgrade_enabled)
                     .ok()?;
-                let message_bytes = Self::packet_message(p)?;
-                let message_hash = Message::hash_raw_message(message_bytes);
-                Some((
-                    HashedTransaction::new(Cow::Owned(tx), message_hash),
-                    tx_index,
-                ))
+
+                Some((tx, *tx_index))
             })
-            .unzip()
+            .collect();
+        banking_stage_stats.cost_tracker_check_count.fetch_add(
+            verified_transactions_with_packet_indexes.len(),
+            Ordering::Relaxed,
+        );
+
+        let mut cost_tracker_check_time = Measure::start("cost_tracker_check_time");
+        let filtered_transactions_with_packet_indexes: Vec<_> = {
+            let cost_tracker_readonly = cost_tracker.read().unwrap();
+            verified_transactions_with_packet_indexes
+                .into_iter()
+                .filter_map(|(tx, tx_index)| {
+                    // put transaction into retry queue if it wouldn't fit
+                    // into current bank
+                    let is_vote = &msgs.packets[tx_index].meta.is_simple_vote_tx;
+
+                    // excluding vote TX from cost_model, for now
+                    if !is_vote
+                        && cost_tracker_readonly
+                            .would_transaction_fit(
+                                &tx,
+                                demote_program_write_locks,
+                                cost_tracker_stats,
+                            )
+                            .is_err()
+                    {
+                        debug!("transaction {:?} would exceed limit", tx);
+                        retryable_transaction_packet_indexes.push(tx_index);
+                        return None;
+                    }
+                    Some((tx, tx_index))
+                })
+                .collect()
+        };
+        cost_tracker_check_time.stop();
+
+        let (filtered_transactions, filter_transaction_packet_indexes) =
+            filtered_transactions_with_packet_indexes
+                .into_iter()
+                .filter_map(|(tx, tx_index)| {
+                    let p = &msgs.packets[tx_index];
+                    let message_bytes = Self::packet_message(p)?;
+                    let message_hash = Message::hash_raw_message(message_bytes);
+                    Some((
+                        HashedTransaction::new(Cow::Owned(tx), message_hash),
+                        tx_index,
+                    ))
+                })
+                .unzip();
+
+        banking_stage_stats
+            .cost_tracker_check_elapsed
+            .fetch_add(cost_tracker_check_time.as_us(), Ordering::Relaxed);
+
+        (
+            filtered_transactions,
+            filter_transaction_packet_indexes,
+            retryable_transaction_packet_indexes,
+        )
     }
 
     /// This function filters pending packets that are still valid
@@ -1089,6 +1252,7 @@ impl BankingStage {
         Self::filter_valid_transaction_indexes(&results, transaction_to_packet_indexes)
     }
 
+    #[allow(clippy::too_many_arguments)]
     fn process_packets_transactions(
         bank: &Arc<Bank>,
         bank_creation_time: &Instant,
@@ -1098,20 +1262,32 @@ impl BankingStage {
         transaction_status_sender: Option<TransactionStatusSender>,
         gossip_vote_sender: &ReplayVoteSender,
         banking_stage_stats: &BankingStageStats,
+        cost_tracker: &Arc<RwLock<CostTracker>>,
+        cost_tracker_stats: &mut CostTrackerStats,
     ) -> (usize, usize, Vec<usize>) {
         let mut packet_conversion_time = Measure::start("packet_conversion");
-        let (transactions, transaction_to_packet_indexes) = Self::transactions_from_packets(
-            msgs,
-            &packet_indexes,
-            bank.libsecp256k1_0_5_upgrade_enabled(),
-            bank.vote_only_bank(),
-        );
+        let (transactions, transaction_to_packet_indexes, retryable_packet_indexes) =
+            Self::transactions_from_packets(
+                msgs,
+                &packet_indexes,
+                bank.libsecp256k1_0_5_upgrade_enabled(),
+                bank.vote_only_bank(),
+                cost_tracker,
+                banking_stage_stats,
+                bank.demote_program_write_locks(),
+                cost_tracker_stats,
+            );
         packet_conversion_time.stop();
+        inc_new_counter_info!("banking_stage-packet_conversion", 1);
 
+        banking_stage_stats
+            .cost_forced_retry_transactions_count
+            .fetch_add(retryable_packet_indexes.len(), Ordering::Relaxed);
         debug!(
-            "bank: {} filtered transactions {}",
+            "bank: {} filtered transactions {} cost limited transactions {}",
             bank.slot(),
-            transactions.len()
+            transactions.len(),
+            retryable_packet_indexes.len()
         );
 
         let tx_len = transactions.len();
@@ -1126,11 +1302,27 @@ impl BankingStage {
             gossip_vote_sender,
         );
         process_tx_time.stop();
-
         let unprocessed_tx_count = unprocessed_tx_indexes.len();
+        inc_new_counter_info!(
+            "banking_stage-unprocessed_transactions",
+            unprocessed_tx_count
+        );
+
+        // applying cost of processed transactions to shared cost_tracker
+        let mut cost_tracking_time = Measure::start("cost_tracking_time");
+        transactions.iter().enumerate().for_each(|(index, tx)| {
+            if unprocessed_tx_indexes.iter().all(|&i| i != index) {
+                cost_tracker.write().unwrap().add_transaction_cost(
+                    tx.transaction(),
+                    bank.demote_program_write_locks(),
+                    cost_tracker_stats,
+                );
+            }
+        });
+        cost_tracking_time.stop();
 
         let mut filter_pending_packets_time = Measure::start("filter_pending_packets_time");
-        let filtered_unprocessed_packet_indexes = Self::filter_pending_packets_from_pending_txs(
+        let mut filtered_unprocessed_packet_indexes = Self::filter_pending_packets_from_pending_txs(
             bank,
             &transactions,
             &transaction_to_packet_indexes,
@@ -1143,12 +1335,19 @@ impl BankingStage {
             unprocessed_tx_count.saturating_sub(filtered_unprocessed_packet_indexes.len())
         );
 
+        // combine cost-related unprocessed transactions with bank determined unprocessed for
+        // buffering
+        filtered_unprocessed_packet_indexes.extend(retryable_packet_indexes);
+
         banking_stage_stats
             .packet_conversion_elapsed
             .fetch_add(packet_conversion_time.as_us(), Ordering::Relaxed);
         banking_stage_stats
             .transaction_processing_elapsed
             .fetch_add(process_tx_time.as_us(), Ordering::Relaxed);
+        banking_stage_stats
+            .cost_tracker_update_elapsed
+            .fetch_add(cost_tracking_time.as_us(), Ordering::Relaxed);
         banking_stage_stats
             .filter_pending_packets_elapsed
             .fetch_add(filter_pending_packets_time.as_us(), Ordering::Relaxed);
@@ -1162,6 +1361,9 @@ impl BankingStage {
         transaction_indexes: &[usize],
         my_pubkey: &Pubkey,
         next_leader: Option<Pubkey>,
+        cost_tracker: &Arc<RwLock<CostTracker>>,
+        banking_stage_stats: &BankingStageStats,
+        cost_tracker_stats: &mut CostTrackerStats,
     ) -> Vec<usize> {
         // Check if we are the next leader. If so, let's not filter the packets
         // as we'll filter it again while processing the packets.
@@ -1172,27 +1374,43 @@ impl BankingStage {
             }
         }
 
-        let (transactions, transaction_to_packet_indexes) = Self::transactions_from_packets(
-            msgs,
-            &transaction_indexes,
-            bank.libsecp256k1_0_5_upgrade_enabled(),
-            bank.vote_only_bank(),
-        );
+        let mut unprocessed_packet_conversion_time =
+            Measure::start("unprocessed_packet_conversion");
+        let (transactions, transaction_to_packet_indexes, retry_packet_indexes) =
+            Self::transactions_from_packets(
+                msgs,
+                &transaction_indexes,
+                bank.libsecp256k1_0_5_upgrade_enabled(),
+                bank.vote_only_bank(),
+                cost_tracker,
+                banking_stage_stats,
+                bank.demote_program_write_locks(),
+                cost_tracker_stats,
+            );
+        unprocessed_packet_conversion_time.stop();
 
         let tx_count = transaction_to_packet_indexes.len();
 
         let unprocessed_tx_indexes = (0..transactions.len()).collect_vec();
-        let filtered_unprocessed_packet_indexes = Self::filter_pending_packets_from_pending_txs(
+        let mut filtered_unprocessed_packet_indexes = Self::filter_pending_packets_from_pending_txs(
             bank,
             &transactions,
             &transaction_to_packet_indexes,
             &unprocessed_tx_indexes,
         );
 
+        filtered_unprocessed_packet_indexes.extend(retry_packet_indexes);
+
         inc_new_counter_info!(
             "banking_stage-dropped_tx_before_forwarding",
             tx_count.saturating_sub(filtered_unprocessed_packet_indexes.len())
         );
+        banking_stage_stats
+            .unprocessed_packet_conversion_elapsed
+            .fetch_add(
+                unprocessed_packet_conversion_time.as_us(),
+                Ordering::Relaxed,
+            );
 
         filtered_unprocessed_packet_indexes
     }
@@ -1228,6 +1446,8 @@ impl BankingStage {
         banking_stage_stats: &BankingStageStats,
         duplicates: &Arc<Mutex<(LruCache<u64, ()>, PacketHasher)>>,
         recorder: &TransactionRecorder,
+        cost_tracker: &Arc<RwLock<CostTracker>>,
+        cost_tracker_stats: &mut CostTrackerStats,
     ) -> Result<(), RecvTimeoutError> {
         let mut recv_time = Measure::start("process_packets_recv");
         let mms = verified_receiver.recv_timeout(recv_timeout)?;
@@ -1268,6 +1488,12 @@ impl BankingStage {
                 continue;
             }
             let (bank, bank_creation_time) = bank_start.unwrap();
+            Self::reset_cost_tracker_if_new_bank(
+                cost_tracker,
+                bank.clone(),
+                banking_stage_stats,
+                cost_tracker_stats,
+            );
 
             let (processed, verified_txs_len, unprocessed_indexes) =
                 Self::process_packets_transactions(
@@ -1279,6 +1505,8 @@ impl BankingStage {
                     transaction_status_sender.clone(),
                     gossip_vote_sender,
                     banking_stage_stats,
+                    cost_tracker,
+                    cost_tracker_stats,
                 );
 
             new_tx_count += processed;
@@ -1310,6 +1538,9 @@ impl BankingStage {
                         &packet_indexes,
                         my_pubkey,
                         next_leader,
+                        cost_tracker,
+                        banking_stage_stats,
+                        cost_tracker_stats,
                     );
                     Self::push_unprocessed(
                         buffered_packets,
@@ -1464,6 +1695,7 @@ where
 #[cfg(test)]
 mod tests {
     use super::*;
+    use crate::cost_model::CostModel;
     use crossbeam_channel::unbounded;
     use itertools::Itertools;
     use solana_gossip::{cluster_info::Node, contact_info::ContactInfo};
@@ -1536,6 +1768,9 @@ mod tests {
                 gossip_verified_vote_receiver,
                 None,
                 vote_forward_sender,
+                Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
             );
             drop(verified_sender);
             drop(gossip_verified_vote_sender);
@@ -1584,6 +1819,9 @@ mod tests {
                 verified_gossip_vote_receiver,
                 None,
                 vote_forward_sender,
+                Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
             );
             trace!("sending bank");
             drop(verified_sender);
@@ -1656,6 +1894,9 @@ mod tests {
                 gossip_verified_vote_receiver,
                 None,
                 gossip_vote_sender,
+                Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
             );
 
             // fund another account so we can send 2 good transactions in a single batch.
@@ -1806,6 +2047,9 @@ mod tests {
                     3,
                     None,
                     gossip_vote_sender,
+                    Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                        CostModel::default(),
+                    ))))),
                 );
 
                 // wait for banking_stage to eat the packets
@@ -2627,6 +2871,10 @@ mod tests {
                 None::<Box<dyn Fn()>>,
                 &BankingStageStats::default(),
                 &recorder,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
+                &mut CostTrackerStats::default(),
             );
             assert_eq!(buffered_packets[0].1.len(), num_conflicting_transactions);
             // When the poh recorder has a bank, should process all non conflicting buffered packets.
@@ -2643,6 +2891,10 @@ mod tests {
                     None::<Box<dyn Fn()>>,
                     &BankingStageStats::default(),
                     &recorder,
+                    &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                        CostModel::default(),
+                    ))))),
+                    &mut CostTrackerStats::default(),
                 );
                 if num_expected_unprocessed == 0 {
                     assert!(buffered_packets.is_empty())
@@ -2708,6 +2960,10 @@ mod tests {
                         test_fn,
                         &BankingStageStats::default(),
                         &recorder,
+                        &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                            CostModel::default(),
+                        ))))),
+                        &mut CostTrackerStats::default(),
                     );
 
                     // Check everything is correct. All indexes after `interrupted_iteration`
@@ -2956,21 +3212,33 @@ mod tests {
                 make_test_packets(vec![transfer_tx.clone(), transfer_tx.clone()], vote_indexes);
 
             let mut votes_only = false;
-            let (txs, tx_packet_index) = BankingStage::transactions_from_packets(
+            let (txs, tx_packet_index, _) = BankingStage::transactions_from_packets(
                 &packets,
                 &packet_indexes,
                 false,
                 votes_only,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
+                &BankingStageStats::default(),
+                false,
+                &mut CostTrackerStats::default(),
             );
             assert_eq!(2, txs.len());
             assert_eq!(vec![0, 1], tx_packet_index);
 
             votes_only = true;
-            let (txs, tx_packet_index) = BankingStage::transactions_from_packets(
+            let (txs, tx_packet_index, _) = BankingStage::transactions_from_packets(
                 &packets,
                 &packet_indexes,
                 false,
                 votes_only,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
+                &BankingStageStats::default(),
+                false,
+                &mut CostTrackerStats::default(),
             );
             assert_eq!(0, txs.len());
             assert_eq!(0, tx_packet_index.len());
@@ -2985,21 +3253,33 @@ mod tests {
             );
 
             let mut votes_only = false;
-            let (txs, tx_packet_index) = BankingStage::transactions_from_packets(
+            let (txs, tx_packet_index, _) = BankingStage::transactions_from_packets(
                 &packets,
                 &packet_indexes,
                 false,
                 votes_only,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
+                &BankingStageStats::default(),
+                false,
+                &mut CostTrackerStats::default(),
             );
             assert_eq!(3, txs.len());
             assert_eq!(vec![0, 1, 2], tx_packet_index);
 
             votes_only = true;
-            let (txs, tx_packet_index) = BankingStage::transactions_from_packets(
+            let (txs, tx_packet_index, _) = BankingStage::transactions_from_packets(
                 &packets,
                 &packet_indexes,
                 false,
                 votes_only,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
+                &BankingStageStats::default(),
+                false,
+                &mut CostTrackerStats::default(),
             );
             assert_eq!(2, txs.len());
             assert_eq!(vec![0, 2], tx_packet_index);
@@ -3014,21 +3294,33 @@ mod tests {
             );
 
             let mut votes_only = false;
-            let (txs, tx_packet_index) = BankingStage::transactions_from_packets(
+            let (txs, tx_packet_index, _) = BankingStage::transactions_from_packets(
                 &packets,
                 &packet_indexes,
                 false,
                 votes_only,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
+                &BankingStageStats::default(),
+                false,
+                &mut CostTrackerStats::default(),
             );
             assert_eq!(3, txs.len());
             assert_eq!(vec![0, 1, 2], tx_packet_index);
 
             votes_only = true;
-            let (txs, tx_packet_index) = BankingStage::transactions_from_packets(
+            let (txs, tx_packet_index, _) = BankingStage::transactions_from_packets(
                 &packets,
                 &packet_indexes,
                 false,
                 votes_only,
+                &Arc::new(RwLock::new(CostTracker::new(Arc::new(RwLock::new(
+                    CostModel::default(),
+                ))))),
+                &BankingStageStats::default(),
+                false,
+                &mut CostTrackerStats::default(),
             );
             assert_eq!(3, txs.len());
             assert_eq!(vec![0, 1, 2], tx_packet_index);

core/src/cost_model.rs

@@ -0,0 +1,519 @@
+//! 'cost_model` provides service to estimate a transaction's cost
+//! following proposed fee schedule #16984; Relevant cluster cost
+//! measuring is described by #19627
+//!
+//! The main function is `calculate_cost` which returns &TransactionCost.
+//!
+use crate::execute_cost_table::ExecuteCostTable;
+use log::*;
+use solana_ledger::block_cost_limits::*;
+use solana_sdk::{pubkey::Pubkey, transaction::Transaction};
+use std::collections::HashMap;
+
+const MAX_WRITABLE_ACCOUNTS: usize = 256;
+
+#[derive(Debug, Clone)]
+pub enum CostModelError {
+    /// transaction that would fail sanitize, cost model is not able to process
+    /// such transaction.
+    InvalidTransaction,
+
+    /// would exceed block max limit
+    WouldExceedBlockMaxLimit,
+
+    /// would exceed account max limit
+    WouldExceedAccountMaxLimit,
+}
+
+#[derive(Default, Debug)]
+pub struct TransactionCost {
+    pub writable_accounts: Vec<Pubkey>,
+    pub signature_cost: u64,
+    pub write_lock_cost: u64,
+    pub data_bytes_cost: u64,
+    pub execution_cost: u64,
+}
+
+impl TransactionCost {
+    pub fn new_with_capacity(capacity: usize) -> Self {
+        Self {
+            writable_accounts: Vec::with_capacity(capacity),
+            ..Self::default()
+        }
+    }
+
+    pub fn reset(&mut self) {
+        self.writable_accounts.clear();
+        self.signature_cost = 0;
+        self.write_lock_cost = 0;
+        self.data_bytes_cost = 0;
+        self.execution_cost = 0;
+    }
+
+    pub fn sum(&self) -> u64 {
+        self.signature_cost + self.write_lock_cost + self.data_bytes_cost + self.execution_cost
+    }
+}
+
+#[derive(Debug)]
+pub struct CostModel {
+    account_cost_limit: u64,
+    block_cost_limit: u64,
+    instruction_execution_cost_table: ExecuteCostTable,
+
+    // reusable variables
+    transaction_cost: TransactionCost,
+}
+
+impl Default for CostModel {
+    fn default() -> Self {
+        CostModel::new(MAX_WRITABLE_ACCOUNT_UNITS, MAX_BLOCK_UNITS)
+    }
+}
+
+impl CostModel {
+    pub fn new(chain_max: u64, block_max: u64) -> Self {
+        Self {
+            account_cost_limit: chain_max,
+            block_cost_limit: block_max,
+            instruction_execution_cost_table: ExecuteCostTable::default(),
+            transaction_cost: TransactionCost::new_with_capacity(MAX_WRITABLE_ACCOUNTS),
+        }
+    }
+
+    pub fn get_account_cost_limit(&self) -> u64 {
+        self.account_cost_limit
+    }
+
+    pub fn get_block_cost_limit(&self) -> u64 {
+        self.block_cost_limit
+    }
+
+    pub fn initialize_cost_table(&mut self, cost_table: &[(Pubkey, u64)]) {
+        cost_table
+            .iter()
+            .map(|(key, cost)| (key, cost))
+            .chain(BUILT_IN_INSTRUCTION_COSTS.iter())
+            .for_each(|(program_id, cost)| {
+                match self
+                    .instruction_execution_cost_table
+                    .upsert(program_id, *cost)
+                {
+                    Some(c) => {
+                        debug!(
+                            "initiating cost table, instruction {:?} has cost {}",
+                            program_id, c
+                        );
+                    }
+                    None => {
+                        debug!(
+                            "initiating cost table, failed for instruction {:?}",
+                            program_id
+                        );
+                    }
+                }
+            });
+        debug!(
+            "restored cost model instruction cost table from blockstore, current values: {:?}",
+            self.get_instruction_cost_table()
+        );
+    }
+
+    pub fn calculate_cost(
+        &mut self,
+        transaction: &Transaction,
+        demote_program_write_locks: bool,
+    ) -> &TransactionCost {
+        self.transaction_cost.reset();
+
+        self.transaction_cost.signature_cost = self.get_signature_cost(transaction);
+        self.get_write_lock_cost(transaction, demote_program_write_locks);
+        self.transaction_cost.data_bytes_cost = self.get_data_bytes_cost(transaction);
+        self.transaction_cost.execution_cost = self.get_transaction_cost(transaction);
+
+        debug!(
+            "transaction {:?} has cost {:?}",
+            transaction, self.transaction_cost
+        );
+        &self.transaction_cost
+    }
+
+    pub fn upsert_instruction_cost(
+        &mut self,
+        program_key: &Pubkey,
+        cost: u64,
+    ) -> Result<u64, &'static str> {
+        self.instruction_execution_cost_table
+            .upsert(program_key, cost);
+        match self.instruction_execution_cost_table.get_cost(program_key) {
+            Some(cost) => Ok(*cost),
+            None => Err("failed to upsert to ExecuteCostTable"),
+        }
+    }
+
+    pub fn get_instruction_cost_table(&self) -> &HashMap<Pubkey, u64> {
+        self.instruction_execution_cost_table.get_cost_table()
+    }
+
+    fn get_signature_cost(&self, transaction: &Transaction) -> u64 {
+        transaction.signatures.len() as u64 * SIGNATURE_COST
+    }
+
+    fn get_write_lock_cost(&mut self, transaction: &Transaction, demote_program_write_locks: bool) {
+        let message = transaction.message();
+        message.account_keys.iter().enumerate().for_each(|(i, k)| {
+            let is_writable = message.is_writable(i, demote_program_write_locks);
+
+            if is_writable {
+                self.transaction_cost.writable_accounts.push(*k);
+                self.transaction_cost.write_lock_cost += WRITE_LOCK_UNITS;
+            }
+        });
+    }
+
+    fn get_data_bytes_cost(&self, transaction: &Transaction) -> u64 {
+        let mut data_bytes_cost: u64 = 0;
+        transaction.message().instructions.iter().for_each(|ix| {
+            data_bytes_cost += ix.data.len() as u64 / DATA_BYTES_UNITS;
+        });
+        data_bytes_cost
+    }
+
+    fn get_transaction_cost(&self, transaction: &Transaction) -> u64 {
+        let mut cost: u64 = 0;
+
+        for instruction in &transaction.message().instructions {
+            let program_id =
+                transaction.message().account_keys[instruction.program_id_index as usize];
+            let instruction_cost = self.find_instruction_cost(&program_id);
+            trace!(
+                "instruction {:?} has cost of {}",
+                instruction,
+                instruction_cost
+            );
+            cost = cost.saturating_add(instruction_cost);
+        }
+        cost
+    }
+
+    fn find_instruction_cost(&self, program_key: &Pubkey) -> u64 {
+        match self.instruction_execution_cost_table.get_cost(program_key) {
+            Some(cost) => *cost,
+            None => {
+                let default_value = self.instruction_execution_cost_table.get_mode();
+                debug!(
+                    "Program key {:?} does not have assigned cost, using mode {}",
+                    program_key, default_value
+                );
+                default_value
+            }
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use solana_runtime::{
+        bank::Bank,
+        genesis_utils::{create_genesis_config, GenesisConfigInfo},
+    };
+    use solana_sdk::{
+        bpf_loader,
+        hash::Hash,
+        instruction::CompiledInstruction,
+        message::Message,
+        signature::{Keypair, Signer},
+        system_instruction::{self},
+        system_program, system_transaction,
+    };
+    use std::{
+        str::FromStr,
+        sync::{Arc, RwLock},
+        thread::{self, JoinHandle},
+    };
+
+    fn test_setup() -> (Keypair, Hash) {
+        solana_logger::setup();
+        let GenesisConfigInfo {
+            genesis_config,
+            mint_keypair,
+            ..
+        } = create_genesis_config(10);
+        let bank = Arc::new(Bank::new_no_wallclock_throttle(&genesis_config));
+        let start_hash = bank.last_blockhash();
+        (mint_keypair, start_hash)
+    }
+
+    #[test]
+    fn test_cost_model_instruction_cost() {
+        let mut testee = CostModel::default();
+
+        let known_key = Pubkey::from_str("known11111111111111111111111111111111111111").unwrap();
+        testee.upsert_instruction_cost(&known_key, 100).unwrap();
+        // find cost for known programs
+        assert_eq!(100, testee.find_instruction_cost(&known_key));
+
+        testee
+            .upsert_instruction_cost(&bpf_loader::id(), 1999)
+            .unwrap();
+        assert_eq!(1999, testee.find_instruction_cost(&bpf_loader::id()));
+
+        // unknown program is assigned with default cost
+        assert_eq!(
+            testee.instruction_execution_cost_table.get_mode(),
+            testee.find_instruction_cost(
+                &Pubkey::from_str("unknown111111111111111111111111111111111111").unwrap()
+            )
+        );
+    }
+
+    #[test]
+    fn test_cost_model_simple_transaction() {
+        let (mint_keypair, start_hash) = test_setup();
+
+        let keypair = Keypair::new();
+        let simple_transaction =
+            system_transaction::transfer(&mint_keypair, &keypair.pubkey(), 2, start_hash);
+        debug!(
+            "system_transaction simple_transaction {:?}",
+            simple_transaction
+        );
+
+        // expected cost for one system transfer instructions
+        let expected_cost = 8;
+
+        let mut testee = CostModel::default();
+        testee
+            .upsert_instruction_cost(&system_program::id(), expected_cost)
+            .unwrap();
+        assert_eq!(
+            expected_cost,
+            testee.get_transaction_cost(&simple_transaction)
+        );
+    }
+
+    #[test]
+    fn test_cost_model_transaction_many_transfer_instructions() {
+        let (mint_keypair, start_hash) = test_setup();
+
+        let key1 = solana_sdk::pubkey::new_rand();
+        let key2 = solana_sdk::pubkey::new_rand();
+        let instructions =
+            system_instruction::transfer_many(&mint_keypair.pubkey(), &[(key1, 1), (key2, 1)]);
+        let message = Message::new(&instructions, Some(&mint_keypair.pubkey()));
+        let tx = Transaction::new(&[&mint_keypair], message, start_hash);
+        debug!("many transfer transaction {:?}", tx);
+
+        // expected cost for two system transfer instructions
+        let program_cost = 8;
+        let expected_cost = program_cost * 2;
+
+        let mut testee = CostModel::default();
+        testee
+            .upsert_instruction_cost(&system_program::id(), program_cost)
+            .unwrap();
+        assert_eq!(expected_cost, testee.get_transaction_cost(&tx));
+    }
+
+    #[test]
+    fn test_cost_model_message_many_different_instructions() {
+        let (mint_keypair, start_hash) = test_setup();
+
+        // construct a transaction with multiple random instructions
+        let key1 = solana_sdk::pubkey::new_rand();
+        let key2 = solana_sdk::pubkey::new_rand();
+        let prog1 = solana_sdk::pubkey::new_rand();
+        let prog2 = solana_sdk::pubkey::new_rand();
+        let instructions = vec![
+            CompiledInstruction::new(3, &(), vec![0, 1]),
+            CompiledInstruction::new(4, &(), vec![0, 2]),
+        ];
+        let tx = Transaction::new_with_compiled_instructions(
+            &[&mint_keypair],
+            &[key1, key2],
+            start_hash,
+            vec![prog1, prog2],
+            instructions,
+        );
+        debug!("many random transaction {:?}", tx);
+
+        let testee = CostModel::default();
+        let result = testee.get_transaction_cost(&tx);
+
+        // expected cost for two random/unknown program is
+        let expected_cost = testee.instruction_execution_cost_table.get_mode() * 2;
+        assert_eq!(expected_cost, result);
+    }
+
+    #[test]
+    fn test_cost_model_sort_message_accounts_by_type() {
+        // construct a transaction with two random instructions with same signer
+        let signer1 = Keypair::new();
+        let signer2 = Keypair::new();
+        let key1 = Pubkey::new_unique();
+        let key2 = Pubkey::new_unique();
+        let prog1 = Pubkey::new_unique();
+        let prog2 = Pubkey::new_unique();
+        let instructions = vec![
+            CompiledInstruction::new(4, &(), vec![0, 2]),
+            CompiledInstruction::new(5, &(), vec![1, 3]),
+        ];
+        let tx = Transaction::new_with_compiled_instructions(
+            &[&signer1, &signer2],
+            &[key1, key2],
+            Hash::new_unique(),
+            vec![prog1, prog2],
+            instructions,
+        );
+
+        let mut cost_model = CostModel::default();
+        let tx_cost = cost_model.calculate_cost(&tx, /*demote_program_write_locks=*/ true);
+        assert_eq!(2 + 2, tx_cost.writable_accounts.len());
+        assert_eq!(signer1.pubkey(), tx_cost.writable_accounts[0]);
+        assert_eq!(signer2.pubkey(), tx_cost.writable_accounts[1]);
+        assert_eq!(key1, tx_cost.writable_accounts[2]);
+        assert_eq!(key2, tx_cost.writable_accounts[3]);
+    }
+
+    #[test]
+    fn test_cost_model_insert_instruction_cost() {
+        let key1 = Pubkey::new_unique();
+        let cost1 = 100;
+
+        let mut cost_model = CostModel::default();
+        // Using default cost for unknown instruction
+        assert_eq!(
+            cost_model.instruction_execution_cost_table.get_mode(),
+            cost_model.find_instruction_cost(&key1)
+        );
+
+        // insert instruction cost to table
+        assert!(cost_model.upsert_instruction_cost(&key1, cost1).is_ok());
+
+        // now it is known insturction with known cost
+        assert_eq!(cost1, cost_model.find_instruction_cost(&key1));
+    }
+
+    #[test]
+    fn test_cost_model_calculate_cost() {
+        let (mint_keypair, start_hash) = test_setup();
+        let tx =
+            system_transaction::transfer(&mint_keypair, &Keypair::new().pubkey(), 2, start_hash);
+
+        let expected_account_cost = WRITE_LOCK_UNITS * 2;
+        let expected_execution_cost = 8;
+
+        let mut cost_model = CostModel::default();
+        cost_model
+            .upsert_instruction_cost(&system_program::id(), expected_execution_cost)
+            .unwrap();
+        let tx_cost = cost_model.calculate_cost(&tx, /*demote_program_write_locks=*/ true);
+        assert_eq!(expected_account_cost, tx_cost.write_lock_cost);
+        assert_eq!(expected_execution_cost, tx_cost.execution_cost);
+        assert_eq!(2, tx_cost.writable_accounts.len());
+    }
+
+    #[test]
+    fn test_cost_model_update_instruction_cost() {
+        let key1 = Pubkey::new_unique();
+        let cost1 = 100;
+        let cost2 = 200;
+        let updated_cost = (cost1 + cost2) / 2;
+
+        let mut cost_model = CostModel::default();
+
+        // insert instruction cost to table
+        assert!(cost_model.upsert_instruction_cost(&key1, cost1).is_ok());
+        assert_eq!(cost1, cost_model.find_instruction_cost(&key1));
+
+        // update instruction cost
+        assert!(cost_model.upsert_instruction_cost(&key1, cost2).is_ok());
+        assert_eq!(updated_cost, cost_model.find_instruction_cost(&key1));
+    }
+
+    #[test]
+    fn test_cost_model_can_be_shared_concurrently_with_rwlock() {
+        let (mint_keypair, start_hash) = test_setup();
+        // construct a transaction with multiple random instructions
+        let key1 = solana_sdk::pubkey::new_rand();
+        let key2 = solana_sdk::pubkey::new_rand();
+        let prog1 = solana_sdk::pubkey::new_rand();
+        let prog2 = solana_sdk::pubkey::new_rand();
+        let instructions = vec![
+            CompiledInstruction::new(3, &(), vec![0, 1]),
+            CompiledInstruction::new(4, &(), vec![0, 2]),
+        ];
+        let tx = Arc::new(Transaction::new_with_compiled_instructions(
+            &[&mint_keypair],
+            &[key1, key2],
+            start_hash,
+            vec![prog1, prog2],
+            instructions,
+        ));
+
+        let number_threads = 10;
+        let expected_account_cost = WRITE_LOCK_UNITS * 3;
+        let cost1 = 100;
+        let cost2 = 200;
+        // execution cost can be either 2 * Default (before write) or cost1+cost2 (after write)
+
+        let cost_model: Arc<RwLock<CostModel>> = Arc::new(RwLock::new(CostModel::default()));
+
+        let thread_handlers: Vec<JoinHandle<()>> = (0..number_threads)
+            .map(|i| {
+                let cost_model = cost_model.clone();
+                let tx = tx.clone();
+
+                if i == 5 {
+                    thread::spawn(move || {
+                        let mut cost_model = cost_model.write().unwrap();
+                        assert!(cost_model.upsert_instruction_cost(&prog1, cost1).is_ok());
+                        assert!(cost_model.upsert_instruction_cost(&prog2, cost2).is_ok());
+                    })
+                } else {
+                    thread::spawn(move || {
+                        let mut cost_model = cost_model.write().unwrap();
+                        let tx_cost = cost_model
+                            .calculate_cost(&tx, /*demote_program_write_locks=*/ true);
+                        assert_eq!(3, tx_cost.writable_accounts.len());
+                        assert_eq!(expected_account_cost, tx_cost.write_lock_cost);
+                    })
+                }
+            })
+            .collect();
+
+        for th in thread_handlers {
+            th.join().unwrap();
+        }
+    }
+
+    #[test]
+    fn test_initialize_cost_table() {
+        // build cost table
+        let cost_table = vec![
+            (Pubkey::new_unique(), 10),
+            (Pubkey::new_unique(), 20),
+            (Pubkey::new_unique(), 30),
+        ];
+
+        // init cost model
+        let mut cost_model = CostModel::default();
+        cost_model.initialize_cost_table(&cost_table);
+
+        // verify
+        for (id, cost) in cost_table.iter() {
+            assert_eq!(*cost, cost_model.find_instruction_cost(id));
+        }
+
+        // verify built-in programs
+        assert!(cost_model
+            .instruction_execution_cost_table
+            .get_cost(&system_program::id())
+            .is_some());
+        assert!(cost_model
+            .instruction_execution_cost_table
+            .get_cost(&solana_vote_program::id())
+            .is_some());
+    }
+}

core/src/cost_tracker.rs

@@ -0,0 +1,482 @@
+//! `cost_tracker` keeps tracking transaction cost per chained accounts as well as for entire block
+//! It aggregates `cost_model`, which provides service of calculating transaction cost.
+//! The main functions are:
+//! - would_transaction_fit(&tx), immutable function to test if `tx` would fit into current block
+//! - add_transaction_cost(&tx), mutable function to accumulate `tx` cost to tracker.
+//!
+use crate::cost_model::{CostModel, TransactionCost};
+use crate::cost_tracker_stats::CostTrackerStats;
+use solana_sdk::{clock::Slot, pubkey::Pubkey, transaction::Transaction};
+use std::{
+    collections::HashMap,
+    sync::{Arc, RwLock},
+};
+
+const WRITABLE_ACCOUNTS_PER_BLOCK: usize = 512;
+
+#[derive(Debug)]
+pub struct CostTracker {
+    cost_model: Arc<RwLock<CostModel>>,
+    account_cost_limit: u64,
+    block_cost_limit: u64,
+    current_bank_slot: Slot,
+    cost_by_writable_accounts: HashMap<Pubkey, u64>,
+    block_cost: u64,
+}
+
+impl CostTracker {
+    pub fn new(cost_model: Arc<RwLock<CostModel>>) -> Self {
+        let (account_cost_limit, block_cost_limit) = {
+            let cost_model = cost_model.read().unwrap();
+            (
+                cost_model.get_account_cost_limit(),
+                cost_model.get_block_cost_limit(),
+            )
+        };
+        assert!(account_cost_limit <= block_cost_limit);
+        Self {
+            cost_model,
+            account_cost_limit,
+            block_cost_limit,
+            current_bank_slot: 0,
+            cost_by_writable_accounts: HashMap::with_capacity(WRITABLE_ACCOUNTS_PER_BLOCK),
+            block_cost: 0,
+        }
+    }
+
+    pub fn would_transaction_fit(
+        &self,
+        transaction: &Transaction,
+        demote_program_write_locks: bool,
+        stats: &mut CostTrackerStats,
+    ) -> Result<(), &'static str> {
+        let mut cost_model = self.cost_model.write().unwrap();
+        let tx_cost = cost_model.calculate_cost(transaction, demote_program_write_locks);
+        self.would_fit(&tx_cost.writable_accounts, &tx_cost.sum(), stats)
+    }
+
+    pub fn add_transaction_cost(
+        &mut self,
+        transaction: &Transaction,
+        demote_program_write_locks: bool,
+        stats: &mut CostTrackerStats,
+    ) {
+        let mut cost_model = self.cost_model.write().unwrap();
+        let tx_cost = cost_model.calculate_cost(transaction, demote_program_write_locks);
+        let cost = tx_cost.sum();
+        for account_key in tx_cost.writable_accounts.iter() {
+            *self
+                .cost_by_writable_accounts
+                .entry(*account_key)
+                .or_insert(0) += cost;
+        }
+        self.block_cost += cost;
+
+        stats.transaction_count += 1;
+        stats.block_cost += cost;
+    }
+
+    pub fn reset_if_new_bank(&mut self, slot: Slot, stats: &mut CostTrackerStats) -> bool {
+        // report stats when slot changes
+        if slot != stats.bank_slot {
+            stats.report();
+            *stats = CostTrackerStats::new(stats.id, slot);
+        }
+
+        if slot != self.current_bank_slot {
+            self.current_bank_slot = slot;
+            self.cost_by_writable_accounts.clear();
+            self.block_cost = 0;
+
+            true
+        } else {
+            false
+        }
+    }
+
+    pub fn try_add(
+        &mut self,
+        transaction_cost: &TransactionCost,
+        stats: &mut CostTrackerStats,
+    ) -> Result<u64, &'static str> {
+        let cost = transaction_cost.sum();
+        self.would_fit(&transaction_cost.writable_accounts, &cost, stats)?;
+
+        self.add_transaction(&transaction_cost.writable_accounts, &cost);
+        Ok(self.block_cost)
+    }
+
+    fn would_fit(
+        &self,
+        keys: &[Pubkey],
+        cost: &u64,
+        stats: &mut CostTrackerStats,
+    ) -> Result<(), &'static str> {
+        stats.transaction_cost_histogram.increment(*cost).unwrap();
+
+        // check against the total package cost
+        if self.block_cost + cost > self.block_cost_limit {
+            return Err("would exceed block cost limit");
+        }
+
+        // check if the transaction itself is more costly than the account_cost_limit
+        if *cost > self.account_cost_limit {
+            return Err("Transaction is too expansive, exceeds account cost limit");
+        }
+
+        // check each account against account_cost_limit,
+        for account_key in keys.iter() {
+            match self.cost_by_writable_accounts.get(&account_key) {
+                Some(chained_cost) => {
+                    stats
+                        .writable_accounts_cost_histogram
+                        .increment(*chained_cost)
+                        .unwrap();
+
+                    if chained_cost + cost > self.account_cost_limit {
+                        return Err("would exceed account cost limit");
+                    } else {
+                        continue;
+                    }
+                }
+                None => continue,
+            }
+        }
+
+        Ok(())
+    }
+
+    fn add_transaction(&mut self, keys: &[Pubkey], cost: &u64) {
+        for account_key in keys.iter() {
+            *self
+                .cost_by_writable_accounts
+                .entry(*account_key)
+                .or_insert(0) += cost;
+        }
+        self.block_cost += cost;
+    }
+}
+
+// CostStats can be collected by util, such as ledger_tool
+#[derive(Default, Debug)]
+pub struct CostStats {
+    pub bank_slot: Slot,
+    pub total_cost: u64,
+    pub number_of_accounts: usize,
+    pub costliest_account: Pubkey,
+    pub costliest_account_cost: u64,
+}
+
+impl CostTracker {
+    pub fn get_stats(&self) -> CostStats {
+        let mut stats = CostStats {
+            bank_slot: self.current_bank_slot,
+            total_cost: self.block_cost,
+            number_of_accounts: self.cost_by_writable_accounts.len(),
+            costliest_account: Pubkey::default(),
+            costliest_account_cost: 0,
+        };
+
+        for (key, cost) in self.cost_by_writable_accounts.iter() {
+            if cost > &stats.costliest_account_cost {
+                stats.costliest_account = *key;
+                stats.costliest_account_cost = *cost;
+            }
+        }
+
+        stats
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use solana_runtime::{
+        bank::Bank,
+        genesis_utils::{create_genesis_config, GenesisConfigInfo},
+    };
+    use solana_sdk::{
+        hash::Hash,
+        signature::{Keypair, Signer},
+        system_transaction,
+        transaction::Transaction,
+    };
+    use std::{cmp, sync::Arc};
+
+    fn test_setup() -> (Keypair, Hash) {
+        solana_logger::setup();
+        let GenesisConfigInfo {
+            genesis_config,
+            mint_keypair,
+            ..
+        } = create_genesis_config(10);
+        let bank = Arc::new(Bank::new_no_wallclock_throttle(&genesis_config));
+        let start_hash = bank.last_blockhash();
+        (mint_keypair, start_hash)
+    }
+
+    fn build_simple_transaction(
+        mint_keypair: &Keypair,
+        start_hash: &Hash,
+    ) -> (Transaction, Vec<Pubkey>, u64) {
+        let keypair = Keypair::new();
+        let simple_transaction =
+            system_transaction::transfer(&mint_keypair, &keypair.pubkey(), 2, *start_hash);
+
+        (simple_transaction, vec![mint_keypair.pubkey()], 5)
+    }
+
+    #[test]
+    fn test_cost_tracker_initialization() {
+        let testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(10, 11))));
+        assert_eq!(10, testee.account_cost_limit);
+        assert_eq!(11, testee.block_cost_limit);
+        assert_eq!(0, testee.cost_by_writable_accounts.len());
+        assert_eq!(0, testee.block_cost);
+    }
+
+    #[test]
+    fn test_cost_tracker_ok_add_one() {
+        let (mint_keypair, start_hash) = test_setup();
+        let (_tx, keys, cost) = build_simple_transaction(&mint_keypair, &start_hash);
+
+        // build testee to have capacity for one simple transaction
+        let mut testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(cost, cost))));
+        assert!(testee
+            .would_fit(&keys, &cost, &mut CostTrackerStats::default())
+            .is_ok());
+        testee.add_transaction(&keys, &cost);
+        assert_eq!(cost, testee.block_cost);
+    }
+
+    #[test]
+    fn test_cost_tracker_ok_add_two_same_accounts() {
+        let (mint_keypair, start_hash) = test_setup();
+        // build two transactions with same signed account
+        let (_tx1, keys1, cost1) = build_simple_transaction(&mint_keypair, &start_hash);
+        let (_tx2, keys2, cost2) = build_simple_transaction(&mint_keypair, &start_hash);
+
+        // build testee to have capacity for two simple transactions, with same accounts
+        let mut testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(
+            cost1 + cost2,
+            cost1 + cost2,
+        ))));
+        {
+            assert!(testee
+                .would_fit(&keys1, &cost1, &mut CostTrackerStats::default())
+                .is_ok());
+            testee.add_transaction(&keys1, &cost1);
+        }
+        {
+            assert!(testee
+                .would_fit(&keys2, &cost2, &mut CostTrackerStats::default())
+                .is_ok());
+            testee.add_transaction(&keys2, &cost2);
+        }
+        assert_eq!(cost1 + cost2, testee.block_cost);
+        assert_eq!(1, testee.cost_by_writable_accounts.len());
+    }
+
+    #[test]
+    fn test_cost_tracker_ok_add_two_diff_accounts() {
+        let (mint_keypair, start_hash) = test_setup();
+        // build two transactions with diff accounts
+        let (_tx1, keys1, cost1) = build_simple_transaction(&mint_keypair, &start_hash);
+        let second_account = Keypair::new();
+        let (_tx2, keys2, cost2) = build_simple_transaction(&second_account, &start_hash);
+
+        // build testee to have capacity for two simple transactions, with same accounts
+        let mut testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(
+            cmp::max(cost1, cost2),
+            cost1 + cost2,
+        ))));
+        {
+            assert!(testee
+                .would_fit(&keys1, &cost1, &mut CostTrackerStats::default())
+                .is_ok());
+            testee.add_transaction(&keys1, &cost1);
+        }
+        {
+            assert!(testee
+                .would_fit(&keys2, &cost2, &mut CostTrackerStats::default())
+                .is_ok());
+            testee.add_transaction(&keys2, &cost2);
+        }
+        assert_eq!(cost1 + cost2, testee.block_cost);
+        assert_eq!(2, testee.cost_by_writable_accounts.len());
+    }
+
+    #[test]
+    fn test_cost_tracker_chain_reach_limit() {
+        let (mint_keypair, start_hash) = test_setup();
+        // build two transactions with same signed account
+        let (_tx1, keys1, cost1) = build_simple_transaction(&mint_keypair, &start_hash);
+        let (_tx2, keys2, cost2) = build_simple_transaction(&mint_keypair, &start_hash);
+
+        // build testee to have capacity for two simple transactions, but not for same accounts
+        let mut testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(
+            cmp::min(cost1, cost2),
+            cost1 + cost2,
+        ))));
+        // should have room for first transaction
+        {
+            assert!(testee
+                .would_fit(&keys1, &cost1, &mut CostTrackerStats::default())
+                .is_ok());
+            testee.add_transaction(&keys1, &cost1);
+        }
+        // but no more sapce on the same chain (same signer account)
+        {
+            assert!(testee
+                .would_fit(&keys2, &cost2, &mut CostTrackerStats::default())
+                .is_err());
+        }
+    }
+
+    #[test]
+    fn test_cost_tracker_reach_limit() {
+        let (mint_keypair, start_hash) = test_setup();
+        // build two transactions with diff accounts
+        let (_tx1, keys1, cost1) = build_simple_transaction(&mint_keypair, &start_hash);
+        let second_account = Keypair::new();
+        let (_tx2, keys2, cost2) = build_simple_transaction(&second_account, &start_hash);
+
+        // build testee to have capacity for each chain, but not enough room for both transactions
+        let mut testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(
+            cmp::max(cost1, cost2),
+            cost1 + cost2 - 1,
+        ))));
+        // should have room for first transaction
+        {
+            assert!(testee
+                .would_fit(&keys1, &cost1, &mut CostTrackerStats::default())
+                .is_ok());
+            testee.add_transaction(&keys1, &cost1);
+        }
+        // but no more room for package as whole
+        {
+            assert!(testee
+                .would_fit(&keys2, &cost2, &mut CostTrackerStats::default())
+                .is_err());
+        }
+    }
+
+    #[test]
+    fn test_cost_tracker_reset() {
+        let (mint_keypair, start_hash) = test_setup();
+        // build two transactions with same signed account
+        let (_tx1, keys1, cost1) = build_simple_transaction(&mint_keypair, &start_hash);
+        let (_tx2, keys2, cost2) = build_simple_transaction(&mint_keypair, &start_hash);
+
+        // build testee to have capacity for two simple transactions, but not for same accounts
+        let mut testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(
+            cmp::min(cost1, cost2),
+            cost1 + cost2,
+        ))));
+        // should have room for first transaction
+        {
+            assert!(testee
+                .would_fit(&keys1, &cost1, &mut CostTrackerStats::default())
+                .is_ok());
+            testee.add_transaction(&keys1, &cost1);
+            assert_eq!(1, testee.cost_by_writable_accounts.len());
+            assert_eq!(cost1, testee.block_cost);
+        }
+        // but no more sapce on the same chain (same signer account)
+        {
+            assert!(testee
+                .would_fit(&keys2, &cost2, &mut CostTrackerStats::default())
+                .is_err());
+        }
+        // reset the tracker
+        {
+            testee.reset_if_new_bank(100, &mut CostTrackerStats::default());
+            assert_eq!(0, testee.cost_by_writable_accounts.len());
+            assert_eq!(0, testee.block_cost);
+        }
+        //now the second transaction can be added
+        {
+            assert!(testee
+                .would_fit(&keys2, &cost2, &mut CostTrackerStats::default())
+                .is_ok());
+        }
+    }
+
+    #[test]
+    fn test_cost_tracker_try_add_is_atomic() {
+        let acct1 = Pubkey::new_unique();
+        let acct2 = Pubkey::new_unique();
+        let acct3 = Pubkey::new_unique();
+        let cost = 100;
+        let account_max = cost * 2;
+        let block_max = account_max * 3; // for three accts
+
+        let mut testee = CostTracker::new(Arc::new(RwLock::new(CostModel::new(
+            account_max,
+            block_max,
+        ))));
+
+        // case 1: a tx writes to 3 accounts, should success, we will have:
+        // | acct1 | $cost |
+        // | acct2 | $cost |
+        // | acct2 | $cost |
+        // and block_cost = $cost
+        {
+            let tx_cost = TransactionCost {
+                writable_accounts: vec![acct1, acct2, acct3],
+                execution_cost: cost,
+                ..TransactionCost::default()
+            };
+            assert!(testee
+                .try_add(&tx_cost, &mut CostTrackerStats::default())
+                .is_ok());
+            let stat = testee.get_stats();
+            assert_eq!(cost, stat.total_cost);
+            assert_eq!(3, stat.number_of_accounts);
+            assert_eq!(cost, stat.costliest_account_cost);
+        }
+
+        // case 2: add tx writes to acct2 with $cost, should succeed, result to
+        // | acct1 | $cost |
+        // | acct2 | $cost * 2 |
+        // | acct2 | $cost |
+        // and block_cost = $cost * 2
+        {
+            let tx_cost = TransactionCost {
+                writable_accounts: vec![acct2],
+                execution_cost: cost,
+                ..TransactionCost::default()
+            };
+            assert!(testee
+                .try_add(&tx_cost, &mut CostTrackerStats::default())
+                .is_ok());
+            let stat = testee.get_stats();
+            assert_eq!(cost * 2, stat.total_cost);
+            assert_eq!(3, stat.number_of_accounts);
+            assert_eq!(cost * 2, stat.costliest_account_cost);
+            assert_eq!(acct2, stat.costliest_account);
+        }
+
+        // case 3: add tx writes to [acct1, acct2], acct2 exceeds limit, should failed atomically,
+        // we shoudl still have:
+        // | acct1 | $cost |
+        // | acct2 | $cost |
+        // | acct2 | $cost |
+        // and block_cost = $cost
+        {
+            let tx_cost = TransactionCost {
+                writable_accounts: vec![acct1, acct2],
+                execution_cost: cost,
+                ..TransactionCost::default()
+            };
+            assert!(testee
+                .try_add(&tx_cost, &mut CostTrackerStats::default())
+                .is_err());
+            let stat = testee.get_stats();
+            assert_eq!(cost * 2, stat.total_cost);
+            assert_eq!(3, stat.number_of_accounts);
+            assert_eq!(cost * 2, stat.costliest_account_cost);
+            assert_eq!(acct2, stat.costliest_account);
+        }
+    }
+}

core/src/cost_tracker_stats.rs

@@ -0,0 +1,75 @@
+//! The Stats is not thread safe, each thread should have its own
+//! instance of stat with `id`; Stat reports and reset for each slot.
+#[derive(Debug, Default)]
+pub struct CostTrackerStats {
+    pub id: u32,
+    pub transaction_cost_histogram: histogram::Histogram,
+    pub writable_accounts_cost_histogram: histogram::Histogram,
+    pub transaction_count: u64,
+    pub block_cost: u64,
+    pub bank_slot: u64,
+}
+
+impl CostTrackerStats {
+    pub fn new(id: u32, bank_slot: u64) -> Self {
+        CostTrackerStats {
+            id,
+            bank_slot,
+            ..CostTrackerStats::default()
+        }
+    }
+
+    pub fn report(&self) {
+        datapoint_info!(
+            "cost_tracker_stats",
+            ("id", self.id as i64, i64),
+            (
+                "transaction_cost_unit_min",
+                self.transaction_cost_histogram.minimum().unwrap_or(0),
+                i64
+            ),
+            (
+                "transaction_cost_unit_max",
+                self.transaction_cost_histogram.maximum().unwrap_or(0),
+                i64
+            ),
+            (
+                "transaction_cost_unit_mean",
+                self.transaction_cost_histogram.mean().unwrap_or(0),
+                i64
+            ),
+            (
+                "transaction_cost_unit_2nd_std",
+                self.transaction_cost_histogram
+                    .percentile(95.0)
+                    .unwrap_or(0),
+                i64
+            ),
+            (
+                "writable_accounts_cost_min",
+                self.writable_accounts_cost_histogram.minimum().unwrap_or(0),
+                i64
+            ),
+            (
+                "writable_accounts_cost_max",
+                self.writable_accounts_cost_histogram.maximum().unwrap_or(0),
+                i64
+            ),
+            (
+                "writable_accounts_cost_mean",
+                self.writable_accounts_cost_histogram.mean().unwrap_or(0),
+                i64
+            ),
+            (
+                "writable_accounts_cost_2nd_std",
+                self.writable_accounts_cost_histogram
+                    .percentile(95.0)
+                    .unwrap_or(0),
+                i64
+            ),
+            ("transaction_count", self.transaction_count as i64, i64),
+            ("block_cost", self.block_cost as i64, i64),
+            ("bank_slot", self.bank_slot as i64, i64),
+        );
+    }
+}

core/src/cost_update_service.rs

@@ -0,0 +1,292 @@
+//! this service receives instruction ExecuteTimings from replay_stage,
+//! update cost_model which is shared with banking_stage to optimize
+//! packing transactions into block; it also triggers persisting cost
+//! table to blockstore.
+
+use crate::cost_model::CostModel;
+use solana_ledger::blockstore::Blockstore;
+use solana_measure::measure::Measure;
+use solana_runtime::bank::ExecuteTimings;
+use solana_sdk::timing::timestamp;
+use std::{
+    sync::{
+        atomic::{AtomicBool, Ordering},
+        mpsc::Receiver,
+        Arc, RwLock,
+    },
+    thread::{self, Builder, JoinHandle},
+    time::Duration,
+};
+
+#[derive(Default)]
+pub struct CostUpdateServiceTiming {
+    last_print: u64,
+    update_cost_model_count: u64,
+    update_cost_model_elapsed: u64,
+    persist_cost_table_elapsed: u64,
+}
+
+impl CostUpdateServiceTiming {
+    fn update(
+        &mut self,
+        update_cost_model_count: u64,
+        update_cost_model_elapsed: u64,
+        persist_cost_table_elapsed: u64,
+    ) {
+        self.update_cost_model_count += update_cost_model_count;
+        self.update_cost_model_elapsed += update_cost_model_elapsed;
+        self.persist_cost_table_elapsed += persist_cost_table_elapsed;
+
+        let now = timestamp();
+        let elapsed_ms = now - self.last_print;
+        if elapsed_ms > 1000 {
+            datapoint_info!(
+                "cost-update-service-stats",
+                ("total_elapsed_us", elapsed_ms * 1000, i64),
+                (
+                    "update_cost_model_count",
+                    self.update_cost_model_count as i64,
+                    i64
+                ),
+                (
+                    "update_cost_model_elapsed",
+                    self.update_cost_model_elapsed as i64,
+                    i64
+                ),
+                (
+                    "persist_cost_table_elapsed",
+                    self.persist_cost_table_elapsed as i64,
+                    i64
+                ),
+            );
+
+            *self = CostUpdateServiceTiming::default();
+            self.last_print = now;
+        }
+    }
+}
+
+pub type CostUpdateReceiver = Receiver<ExecuteTimings>;
+
+pub struct CostUpdateService {
+    thread_hdl: JoinHandle<()>,
+}
+
+impl CostUpdateService {
+    #[allow(clippy::new_ret_no_self)]
+    pub fn new(
+        exit: Arc<AtomicBool>,
+        blockstore: Arc<Blockstore>,
+        cost_model: Arc<RwLock<CostModel>>,
+        cost_update_receiver: CostUpdateReceiver,
+    ) -> Self {
+        let thread_hdl = Builder::new()
+            .name("solana-cost-update-service".to_string())
+            .spawn(move || {
+                Self::service_loop(exit, blockstore, cost_model, cost_update_receiver);
+            })
+            .unwrap();
+
+        Self { thread_hdl }
+    }
+
+    pub fn join(self) -> thread::Result<()> {
+        self.thread_hdl.join()
+    }
+
+    fn service_loop(
+        exit: Arc<AtomicBool>,
+        blockstore: Arc<Blockstore>,
+        cost_model: Arc<RwLock<CostModel>>,
+        cost_update_receiver: CostUpdateReceiver,
+    ) {
+        let mut cost_update_service_timing = CostUpdateServiceTiming::default();
+        let mut dirty: bool;
+        let mut update_count: u64;
+        let wait_timer = Duration::from_millis(100);
+
+        loop {
+            if exit.load(Ordering::Relaxed) {
+                break;
+            }
+
+            dirty = false;
+            update_count = 0_u64;
+            let mut update_cost_model_time = Measure::start("update_cost_model_time");
+            for cost_update in cost_update_receiver.try_iter() {
+                dirty |= Self::update_cost_model(&cost_model, &cost_update);
+                update_count += 1;
+            }
+            update_cost_model_time.stop();
+
+            let mut persist_cost_table_time = Measure::start("persist_cost_table_time");
+            if dirty {
+                Self::persist_cost_table(&blockstore, &cost_model);
+            }
+            persist_cost_table_time.stop();
+
+            cost_update_service_timing.update(
+                update_count,
+                update_cost_model_time.as_us(),
+                persist_cost_table_time.as_us(),
+            );
+
+            thread::sleep(wait_timer);
+        }
+    }
+
+    fn update_cost_model(cost_model: &RwLock<CostModel>, execute_timings: &ExecuteTimings) -> bool {
+        let mut dirty = false;
+        {
+            let mut cost_model_mutable = cost_model.write().unwrap();
+            for (program_id, timing) in &execute_timings.details.per_program_timings {
+                if timing.count < 1 {
+                    continue;
+                }
+                let units = timing.accumulated_units / timing.count as u64;
+                match cost_model_mutable.upsert_instruction_cost(program_id, units) {
+                    Ok(c) => {
+                        debug!(
+                            "after replayed into bank, instruction {:?} has averaged cost {}",
+                            program_id, c
+                        );
+                        dirty = true;
+                    }
+                    Err(err) => {
+                        debug!(
+                        "after replayed into bank, instruction {:?} failed to update cost, err: {}",
+                        program_id, err
+                    );
+                    }
+                }
+            }
+        }
+        debug!(
+           "after replayed into bank, updated cost model instruction cost table, current values: {:?}",
+           cost_model.read().unwrap().get_instruction_cost_table()
+        );
+        dirty
+    }
+
+    fn persist_cost_table(blockstore: &Blockstore, cost_model: &RwLock<CostModel>) {
+        let cost_model_read = cost_model.read().unwrap();
+        let cost_table = cost_model_read.get_instruction_cost_table();
+        let db_records = blockstore.read_program_costs().expect("read programs");
+
+        // delete records from blockstore if they are no longer in cost_table
+        db_records.iter().for_each(|(pubkey, _)| {
+            if cost_table.get(pubkey).is_none() {
+                blockstore
+                    .delete_program_cost(pubkey)
+                    .expect("delete old program");
+            }
+        });
+
+        for (key, cost) in cost_table.iter() {
+            blockstore
+                .write_program_cost(key, cost)
+                .expect("persist program costs to blockstore");
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use solana_runtime::message_processor::ProgramTiming;
+    use solana_sdk::pubkey::Pubkey;
+
+    #[test]
+    fn test_update_cost_model_with_empty_execute_timings() {
+        let cost_model = Arc::new(RwLock::new(CostModel::default()));
+        let empty_execute_timings = ExecuteTimings::default();
+        CostUpdateService::update_cost_model(&cost_model, &empty_execute_timings);
+
+        assert_eq!(
+            0,
+            cost_model
+                .read()
+                .unwrap()
+                .get_instruction_cost_table()
+                .len()
+        );
+    }
+
+    #[test]
+    fn test_update_cost_model_with_execute_timings() {
+        let cost_model = Arc::new(RwLock::new(CostModel::default()));
+        let mut execute_timings = ExecuteTimings::default();
+
+        let program_key_1 = Pubkey::new_unique();
+        let mut expected_cost: u64;
+
+        // add new program
+        {
+            let accumulated_us: u64 = 1000;
+            let accumulated_units: u64 = 100;
+            let count: u32 = 10;
+            expected_cost = accumulated_units / count as u64;
+
+            execute_timings.details.per_program_timings.insert(
+                program_key_1,
+                ProgramTiming {
+                    accumulated_us,
+                    accumulated_units,
+                    count,
+                },
+            );
+            CostUpdateService::update_cost_model(&cost_model, &execute_timings);
+            assert_eq!(
+                1,
+                cost_model
+                    .read()
+                    .unwrap()
+                    .get_instruction_cost_table()
+                    .len()
+            );
+            assert_eq!(
+                Some(&expected_cost),
+                cost_model
+                    .read()
+                    .unwrap()
+                    .get_instruction_cost_table()
+                    .get(&program_key_1)
+            );
+        }
+
+        // update program
+        {
+            let accumulated_us: u64 = 2000;
+            let accumulated_units: u64 = 200;
+            let count: u32 = 10;
+            // to expect new cost is Average(new_value, existing_value)
+            expected_cost = ((accumulated_units / count as u64) + expected_cost) / 2;
+
+            execute_timings.details.per_program_timings.insert(
+                program_key_1,
+                ProgramTiming {
+                    accumulated_us,
+                    accumulated_units,
+                    count,
+                },
+            );
+            CostUpdateService::update_cost_model(&cost_model, &execute_timings);
+            assert_eq!(
+                1,
+                cost_model
+                    .read()
+                    .unwrap()
+                    .get_instruction_cost_table()
+                    .len()
+            );
+            assert_eq!(
+                Some(&expected_cost),
+                cost_model
+                    .read()
+                    .unwrap()
+                    .get_instruction_cost_table()
+                    .get(&program_key_1)
+            );
+        }
+    }
+}

core/src/execute_cost_table.rs

@@ -0,0 +1,279 @@
+/// ExecuteCostTable is aggregated by Cost Model, it keeps each program's
+/// average cost in its HashMap, with fixed capacity to avoid from growing
+/// unchecked.
+/// When its capacity limit is reached, it prunes old and less-used programs
+/// to make room for new ones.
+use log::*;
+use solana_sdk::pubkey::Pubkey;
+use std::{collections::HashMap, time::SystemTime};
+
+// prune is rather expensive op, free up bulk space in each operation
+// would be more efficient. PRUNE_RATIO defines the after prune table
+// size will be original_size * PRUNE_RATIO.
+const PRUNE_RATIO: f64 = 0.75;
+// with 50_000 TPS as norm, weights occurrences '100' per microsec
+const OCCURRENCES_WEIGHT: i64 = 100;
+
+const DEFAULT_CAPACITY: usize = 1024;
+
+#[derive(Debug)]
+pub struct ExecuteCostTable {
+    capacity: usize,
+    table: HashMap<Pubkey, u64>,
+    occurrences: HashMap<Pubkey, (usize, SystemTime)>,
+}
+
+impl Default for ExecuteCostTable {
+    fn default() -> Self {
+        ExecuteCostTable::new(DEFAULT_CAPACITY)
+    }
+}
+
+impl ExecuteCostTable {
+    pub fn new(cap: usize) -> Self {
+        Self {
+            capacity: cap,
+            table: HashMap::with_capacity(cap),
+            occurrences: HashMap::with_capacity(cap),
+        }
+    }
+
+    pub fn get_cost_table(&self) -> &HashMap<Pubkey, u64> {
+        &self.table
+    }
+
+    pub fn get_count(&self) -> usize {
+        self.table.len()
+    }
+
+    // instead of assigning unknown program with a configured/hard-coded cost
+    // use average or mode function to make a educated guess.
+    pub fn get_average(&self) -> u64 {
+        if self.table.is_empty() {
+            0
+        } else {
+            self.table.iter().map(|(_, value)| value).sum::<u64>() / self.get_count() as u64
+        }
+    }
+
+    pub fn get_mode(&self) -> u64 {
+        if self.occurrences.is_empty() {
+            0
+        } else {
+            let key = self
+                .occurrences
+                .iter()
+                .max_by_key(|&(_, count)| count)
+                .map(|(key, _)| key)
+                .expect("cannot find mode from cost table");
+
+            *self.table.get(&key).unwrap()
+        }
+    }
+
+    // returns None if program doesn't exist in table. In this case,
+    // client is advised to call `get_average()` or `get_mode()` to
+    // assign a 'default' value for new program.
+    pub fn get_cost(&self, key: &Pubkey) -> Option<&u64> {
+        self.table.get(&key)
+    }
+
+    pub fn upsert(&mut self, key: &Pubkey, value: u64) -> Option<u64> {
+        let need_to_add = self.table.get(key).is_none();
+        let current_size = self.get_count();
+        if current_size == self.capacity && need_to_add {
+            self.prune_to(&((current_size as f64 * PRUNE_RATIO) as usize));
+        }
+
+        let program_cost = self.table.entry(*key).or_insert(value);
+        *program_cost = (*program_cost + value) / 2;
+
+        let (count, timestamp) = self
+            .occurrences
+            .entry(*key)
+            .or_insert((0, SystemTime::now()));
+        *count += 1;
+        *timestamp = SystemTime::now();
+
+        Some(*program_cost)
+    }
+
+    // prune the old programs so the table contains `new_size` of records,
+    // where `old` is defined as weighted age, which is negatively correlated
+    // with program's age and
+    // positively correlated with how frequently the program
+    // is executed (eg. occurrence),
+    fn prune_to(&mut self, new_size: &usize) {
+        debug!(
+            "prune cost table, current size {}, new size {}",
+            self.get_count(),
+            new_size
+        );
+
+        if *new_size == self.get_count() {
+            return;
+        }
+
+        if *new_size == 0 {
+            self.table.clear();
+            self.occurrences.clear();
+            return;
+        }
+
+        let now = SystemTime::now();
+        let mut sorted_by_weighted_age: Vec<_> = self
+            .occurrences
+            .iter()
+            .map(|(key, (count, timestamp))| {
+                let age = now.duration_since(*timestamp).unwrap().as_micros();
+                let weighted_age = *count as i64 * OCCURRENCES_WEIGHT + -(age as i64);
+                (weighted_age, *key)
+            })
+            .collect();
+        sorted_by_weighted_age.sort_by(|x, y| x.0.partial_cmp(&y.0).unwrap());
+
+        for i in sorted_by_weighted_age.iter() {
+            self.table.remove(&i.1);
+            self.occurrences.remove(&i.1);
+            if *new_size == self.get_count() {
+                break;
+            }
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_execute_cost_table_prune_simple_table() {
+        solana_logger::setup();
+        let capacity: usize = 3;
+        let mut testee = ExecuteCostTable::new(capacity);
+
+        let key1 = Pubkey::new_unique();
+        let key2 = Pubkey::new_unique();
+        let key3 = Pubkey::new_unique();
+
+        testee.upsert(&key1, 1);
+        testee.upsert(&key2, 2);
+        testee.upsert(&key3, 3);
+
+        testee.prune_to(&(capacity - 1));
+
+        // the oldest, key1, should be pruned
+        assert!(testee.get_cost(&key1).is_none());
+        assert!(testee.get_cost(&key2).is_some());
+        assert!(testee.get_cost(&key2).is_some());
+    }
+
+    #[test]
+    fn test_execute_cost_table_prune_weighted_table() {
+        solana_logger::setup();
+        let capacity: usize = 3;
+        let mut testee = ExecuteCostTable::new(capacity);
+
+        let key1 = Pubkey::new_unique();
+        let key2 = Pubkey::new_unique();
+        let key3 = Pubkey::new_unique();
+
+        testee.upsert(&key1, 1);
+        testee.upsert(&key1, 1);
+        testee.upsert(&key2, 2);
+        testee.upsert(&key3, 3);
+
+        testee.prune_to(&(capacity - 1));
+
+        // the oldest, key1, has 2 counts; 2nd oldest Key2 has 1 count;
+        // expect key2 to be pruned.
+        assert!(testee.get_cost(&key1).is_some());
+        assert!(testee.get_cost(&key2).is_none());
+        assert!(testee.get_cost(&key3).is_some());
+    }
+
+    #[test]
+    fn test_execute_cost_table_upsert_within_capacity() {
+        solana_logger::setup();
+        let mut testee = ExecuteCostTable::default();
+
+        let key1 = Pubkey::new_unique();
+        let key2 = Pubkey::new_unique();
+        let cost1: u64 = 100;
+        let cost2: u64 = 110;
+
+        // query empty table
+        assert!(testee.get_cost(&key1).is_none());
+
+        // insert one record
+        testee.upsert(&key1, cost1);
+        assert_eq!(1, testee.get_count());
+        assert_eq!(cost1, testee.get_average());
+        assert_eq!(cost1, testee.get_mode());
+        assert_eq!(&cost1, testee.get_cost(&key1).unwrap());
+
+        // insert 2nd record
+        testee.upsert(&key2, cost2);
+        assert_eq!(2, testee.get_count());
+        assert_eq!((cost1 + cost2) / 2_u64, testee.get_average());
+        assert_eq!(cost2, testee.get_mode());
+        assert_eq!(&cost1, testee.get_cost(&key1).unwrap());
+        assert_eq!(&cost2, testee.get_cost(&key2).unwrap());
+
+        // update 1st record
+        testee.upsert(&key1, cost2);
+        assert_eq!(2, testee.get_count());
+        assert_eq!(((cost1 + cost2) / 2 + cost2) / 2, testee.get_average());
+        assert_eq!((cost1 + cost2) / 2, testee.get_mode());
+        assert_eq!(&((cost1 + cost2) / 2), testee.get_cost(&key1).unwrap());
+        assert_eq!(&cost2, testee.get_cost(&key2).unwrap());
+    }
+
+    #[test]
+    fn test_execute_cost_table_upsert_exceeds_capacity() {
+        solana_logger::setup();
+        let capacity: usize = 2;
+        let mut testee = ExecuteCostTable::new(capacity);
+
+        let key1 = Pubkey::new_unique();
+        let key2 = Pubkey::new_unique();
+        let key3 = Pubkey::new_unique();
+        let key4 = Pubkey::new_unique();
+        let cost1: u64 = 100;
+        let cost2: u64 = 110;
+        let cost3: u64 = 120;
+        let cost4: u64 = 130;
+
+        // insert one record
+        testee.upsert(&key1, cost1);
+        assert_eq!(1, testee.get_count());
+        assert_eq!(&cost1, testee.get_cost(&key1).unwrap());
+
+        // insert 2nd record
+        testee.upsert(&key2, cost2);
+        assert_eq!(2, testee.get_count());
+        assert_eq!(&cost1, testee.get_cost(&key1).unwrap());
+        assert_eq!(&cost2, testee.get_cost(&key2).unwrap());
+
+        // insert 3rd record, pushes out the oldest (eg 1st) record
+        testee.upsert(&key3, cost3);
+        assert_eq!(2, testee.get_count());
+        assert_eq!((cost2 + cost3) / 2_u64, testee.get_average());
+        assert_eq!(cost3, testee.get_mode());
+        assert!(testee.get_cost(&key1).is_none());
+        assert_eq!(&cost2, testee.get_cost(&key2).unwrap());
+        assert_eq!(&cost3, testee.get_cost(&key3).unwrap());
+
+        // update 2nd record, so the 3rd becomes the oldest
+        // add 4th record, pushes out 3rd key
+        testee.upsert(&key2, cost1);
+        testee.upsert(&key4, cost4);
+        assert_eq!(((cost1 + cost2) / 2 + cost4) / 2_u64, testee.get_average());
+        assert_eq!((cost1 + cost2) / 2, testee.get_mode());
+        assert_eq!(2, testee.get_count());
+        assert!(testee.get_cost(&key1).is_none());
+        assert_eq!(&((cost1 + cost2) / 2), testee.get_cost(&key2).unwrap());
+        assert!(testee.get_cost(&key3).is_none());
+        assert_eq!(&cost4, testee.get_cost(&key4).unwrap());
+    }
+}

core/src/lib.rs

@@ -19,6 +19,11 @@ pub mod cluster_slots_service;
 pub mod commitment_service;
 pub mod completed_data_sets_service;
 pub mod consensus;
+pub mod cost_model;
+pub mod cost_tracker;
+pub mod cost_tracker_stats;
+pub mod cost_update_service;
+pub mod execute_cost_table;
 pub mod fetch_stage;
 pub mod fork_choice;
 pub mod gen_keys;

core/src/progress_map.rs

@@ -114,6 +114,43 @@ impl ReplaySlotStats {
                 i64
             ),
         );
+
+        let mut per_pubkey_timings: Vec<_> = self
+            .execute_timings
+            .details
+            .per_program_timings
+            .iter()
+            .collect();
+        per_pubkey_timings.sort_by(|a, b| b.1.accumulated_us.cmp(&a.1.accumulated_us));
+        let (total_us, total_units, total_count) =
+            per_pubkey_timings
+                .iter()
+                .fold((0, 0, 0), |(sum_us, sum_units, sum_count), a| {
+                    (
+                        sum_us + a.1.accumulated_us,
+                        sum_units + a.1.accumulated_units,
+                        sum_count + a.1.count,
+                    )
+                });
+
+        for (pubkey, time) in per_pubkey_timings.iter().take(5) {
+            datapoint_info!(
+                "per_program_timings",
+                ("slot", slot as i64, i64),
+                ("pubkey", pubkey.to_string(), String),
+                ("execute_us", time.accumulated_us, i64),
+                ("accumulated_units", time.accumulated_units, i64),
+                ("count", time.count, i64)
+            );
+        }
+        datapoint_info!(
+            "per_program_timings",
+            ("slot", slot as i64, i64),
+            ("pubkey", "all", String),
+            ("execute_us", total_us, i64),
+            ("accumulated_units", total_units, i64),
+            ("count", total_count, i64)
+        );
     }
 }
 

core/src/replay_stage.rs

@@ -18,7 +18,6 @@ use crate::{
     latest_validator_votes_for_frozen_banks::LatestValidatorVotesForFrozenBanks,
     progress_map::{ForkProgress, ProgressMap, PropagatedStats},
     repair_service::DuplicateSlotsResetReceiver,
-    result::Result,
     rewards_recorder_service::RewardsRecorderSender,
     unfrozen_gossip_verified_vote_hashes::UnfrozenGossipVerifiedVoteHashes,
     voting_service::VoteOp,
@@ -41,8 +40,11 @@ use solana_rpc::{
     rpc_subscriptions::RpcSubscriptions,
 };
 use solana_runtime::{
-    accounts_background_service::AbsRequestSender, bank::Bank, bank_forks::BankForks,
-    commitment::BlockCommitmentCache, vote_sender_types::ReplayVoteSender,
+    accounts_background_service::AbsRequestSender,
+    bank::{Bank, ExecuteTimings, NewBankOptions},
+    bank_forks::BankForks,
+    commitment::BlockCommitmentCache,
+    vote_sender_types::ReplayVoteSender,
 };
 use solana_sdk::{
     clock::{Slot, MAX_PROCESSING_AGE, NUM_CONSECUTIVE_LEADER_SLOTS},
@@ -128,6 +130,7 @@ pub struct ReplayStageConfig {
     pub cache_block_meta_sender: Option<CacheBlockMetaSender>,
     pub bank_notification_sender: Option<BankNotificationSender>,
     pub wait_for_vote_to_start_leader: bool,
+    pub disable_epoch_boundary_optimization: bool,
 }
 
 #[derive(Default)]
@@ -277,7 +280,7 @@ impl ReplayTiming {
                     "process_duplicate_slots_elapsed",
                     self.process_duplicate_slots_elapsed as i64,
                     i64
-                )
+                ),
             );
 
             *self = ReplayTiming::default();
@@ -287,7 +290,7 @@ impl ReplayTiming {
 }
 
 pub struct ReplayStage {
-    t_replay: JoinHandle<Result<()>>,
+    t_replay: JoinHandle<()>,
     commitment_service: AggregateCommitmentService,
 }
 
@@ -311,6 +314,7 @@ impl ReplayStage {
         gossip_verified_vote_hash_receiver: GossipVerifiedVoteHashReceiver,
         cluster_slots_update_sender: ClusterSlotsUpdateSender,
         voting_sender: Sender<VoteOp>,
+        cost_update_sender: Sender<ExecuteTimings>,
     ) -> Self {
         let ReplayStageConfig {
             my_pubkey,
@@ -327,6 +331,7 @@ impl ReplayStage {
             cache_block_meta_sender,
             bank_notification_sender,
             wait_for_vote_to_start_leader,
+            disable_epoch_boundary_optimization,
         } = config;
 
         trace!("replay stage");
@@ -407,6 +412,7 @@ impl ReplayStage {
                         &mut unfrozen_gossip_verified_vote_hashes,
                         &mut latest_validator_votes_for_frozen_banks,
                         &cluster_slots_update_sender,
+                        &cost_update_sender,
                     );
                     replay_active_banks_time.stop();
 
@@ -689,6 +695,7 @@ impl ReplayStage {
                             &retransmit_slots_sender,
                             &mut skipped_slots_info,
                             has_new_vote_been_rooted,
+                            disable_epoch_boundary_optimization,
                         );
 
                         let poh_bank = poh_recorder.lock().unwrap().bank();
@@ -736,7 +743,6 @@ impl ReplayStage {
                         process_duplicate_slots_time.as_us(),
                     );
                 }
-                Ok(())
             })
             .unwrap();
 
@@ -1069,6 +1075,7 @@ impl ReplayStage {
         }
     }
 
+    #[allow(clippy::too_many_arguments)]
     fn maybe_start_leader(
         my_pubkey: &Pubkey,
         bank_forks: &Arc<RwLock<BankForks>>,
@@ -1079,6 +1086,7 @@ impl ReplayStage {
         retransmit_slots_sender: &RetransmitSlotsSender,
         skipped_slots_info: &mut SkippedSlotsInfo,
         has_new_vote_been_rooted: bool,
+        disable_epoch_boundary_optimization: bool,
     ) {
         // all the individual calls to poh_recorder.lock() are designed to
         // increase granularity, decrease contention
@@ -1196,7 +1204,10 @@ impl ReplayStage {
                 root_slot,
                 my_pubkey,
                 rpc_subscriptions,
-                vote_only_bank,
+                NewBankOptions {
+                    vote_only_bank,
+                    disable_epoch_boundary_optimization,
+                },
             );
 
             let tpu_bank = bank_forks.write().unwrap().insert(tpu_bank);
@@ -1679,9 +1690,11 @@ impl ReplayStage {
         unfrozen_gossip_verified_vote_hashes: &mut UnfrozenGossipVerifiedVoteHashes,
         latest_validator_votes_for_frozen_banks: &mut LatestValidatorVotesForFrozenBanks,
         cluster_slots_update_sender: &ClusterSlotsUpdateSender,
+        cost_update_sender: &Sender<ExecuteTimings>,
     ) -> bool {
         let mut did_complete_bank = false;
         let mut tx_count = 0;
+        let mut execute_timings = ExecuteTimings::default();
         let active_banks = bank_forks.read().unwrap().active_banks();
         trace!("active banks {:?}", active_banks);
 
@@ -1752,6 +1765,12 @@ impl ReplayStage {
             }
             assert_eq!(*bank_slot, bank.slot());
             if bank.is_complete() {
+                execute_timings.accumulate(&bank_progress.replay_stats.execute_timings);
+                debug!("bank {} is completed replay from blockstore, contribute to update cost with {:?}",
+                       bank.slot(),
+                       bank_progress.replay_stats.execute_timings
+                       );
+
                 bank_progress.replay_stats.report_stats(
                     bank.slot(),
                     bank_progress.replay_progress.num_entries,
@@ -1813,6 +1832,14 @@ impl ReplayStage {
                 );
             }
         }
+
+        // send accumulated excute-timings to cost_update_service
+        if !execute_timings.details.per_program_timings.is_empty() {
+            cost_update_sender
+                .send(execute_timings)
+                .unwrap_or_else(|err| warn!("cost_update_sender failed: {:?}", err));
+        }
+
         inc_new_counter_info!("replay_stage-replay_transactions", tx_count);
         did_complete_bank
     }
@@ -2452,7 +2479,7 @@ impl ReplayStage {
                     forks.root(),
                     &leader,
                     rpc_subscriptions,
-                    false,
+                    NewBankOptions::default(),
                 );
                 let empty: Vec<Pubkey> = vec![];
                 Self::update_fork_propagated_threshold_from_votes(
@@ -2479,10 +2506,10 @@ impl ReplayStage {
         root_slot: u64,
         leader: &Pubkey,
         rpc_subscriptions: &Arc<RpcSubscriptions>,
-        vote_only_bank: bool,
+        new_bank_options: NewBankOptions,
     ) -> Bank {
         rpc_subscriptions.notify_slot(slot, parent.slot(), root_slot);
-        Bank::new_from_parent_with_vote_only(parent, leader, slot, vote_only_bank)
+        Bank::new_from_parent_with_options(parent, leader, slot, new_bank_options)
     }
 
     fn record_rewards(bank: &Bank, rewards_recorder_sender: &Option<RewardsRecorderSender>) {
@@ -4918,7 +4945,6 @@ mod tests {
         );
         assert_eq!(tower.last_voted_slot().unwrap(), 1);
     }
-
     fn run_compute_and_select_forks(
         bank_forks: &RwLock<BankForks>,
         progress: &mut ProgressMap,

core/src/sigverify_stage.rs

@@ -8,18 +8,16 @@
 use crate::sigverify;
 use crossbeam_channel::{SendError, Sender as CrossbeamSender};
 use solana_measure::measure::Measure;
-use solana_metrics::datapoint_debug;
 use solana_perf::packet::Packets;
-use solana_perf::perf_libs;
 use solana_sdk::timing;
 use solana_streamer::streamer::{self, PacketReceiver, StreamerError};
+use std::collections::HashMap;
 use std::sync::mpsc::{Receiver, RecvTimeoutError};
-use std::sync::{Arc, Mutex};
 use std::thread::{self, Builder, JoinHandle};
+use std::time::Instant;
 use thiserror::Error;
 
-const RECV_BATCH_MAX_CPU: usize = 1_000;
-const RECV_BATCH_MAX_GPU: usize = 5_000;
+const MAX_SIGVERIFY_BATCH: usize = 10_000;
 
 #[derive(Error, Debug)]
 pub enum SigVerifyServiceError {
@@ -33,7 +31,7 @@ pub enum SigVerifyServiceError {
 type Result<T> = std::result::Result<T, SigVerifyServiceError>;
 
 pub struct SigVerifyStage {
-    thread_hdls: Vec<JoinHandle<()>>,
+    thread_hdl: JoinHandle<()>,
 }
 
 pub trait SigVerifier {
@@ -43,6 +41,82 @@ pub trait SigVerifier {
 #[derive(Default, Clone)]
 pub struct DisabledSigVerifier {}
 
+#[derive(Default)]
+struct SigVerifierStats {
+    recv_batches_us_hist: histogram::Histogram, // time to call recv_batch
+    verify_batches_pp_us_hist: histogram::Histogram, // per-packet time to call verify_batch
+    batches_hist: histogram::Histogram,         // number of Packets structures per verify call
+    packets_hist: histogram::Histogram,         // number of packets per verify call
+    total_batches: usize,
+    total_packets: usize,
+}
+
+impl SigVerifierStats {
+    fn report(&self) {
+        datapoint_info!(
+            "sigverify_stage-total_verify_time",
+            (
+                "recv_batches_us_90pct",
+                self.recv_batches_us_hist.percentile(90.0).unwrap_or(0),
+                i64
+            ),
+            (
+                "recv_batches_us_min",
+                self.recv_batches_us_hist.minimum().unwrap_or(0),
+                i64
+            ),
+            (
+                "recv_batches_us_max",
+                self.recv_batches_us_hist.maximum().unwrap_or(0),
+                i64
+            ),
+            (
+                "recv_batches_us_mean",
+                self.recv_batches_us_hist.mean().unwrap_or(0),
+                i64
+            ),
+            (
+                "verify_batches_pp_us_90pct",
+                self.verify_batches_pp_us_hist.percentile(90.0).unwrap_or(0),
+                i64
+            ),
+            (
+                "verify_batches_pp_us_min",
+                self.verify_batches_pp_us_hist.minimum().unwrap_or(0),
+                i64
+            ),
+            (
+                "verify_batches_pp_us_max",
+                self.verify_batches_pp_us_hist.maximum().unwrap_or(0),
+                i64
+            ),
+            (
+                "verify_batches_pp_us_mean",
+                self.verify_batches_pp_us_hist.mean().unwrap_or(0),
+                i64
+            ),
+            (
+                "batches_90pct",
+                self.batches_hist.percentile(90.0).unwrap_or(0),
+                i64
+            ),
+            ("batches_min", self.batches_hist.minimum().unwrap_or(0), i64),
+            ("batches_max", self.batches_hist.maximum().unwrap_or(0), i64),
+            ("batches_mean", self.batches_hist.mean().unwrap_or(0), i64),
+            (
+                "packets_90pct",
+                self.packets_hist.percentile(90.0).unwrap_or(0),
+                i64
+            ),
+            ("packets_min", self.packets_hist.minimum().unwrap_or(0), i64),
+            ("packets_max", self.packets_hist.maximum().unwrap_or(0), i64),
+            ("packets_mean", self.packets_hist.mean().unwrap_or(0), i64),
+            ("total_batches", self.total_batches, i64),
+            ("total_packets", self.total_packets, i64),
+        );
+    }
+}
+
 impl SigVerifier for DisabledSigVerifier {
     fn verify_batch(&self, mut batch: Vec<Packets>) -> Vec<Packets> {
         sigverify::ed25519_verify_disabled(&mut batch);
@@ -57,68 +131,103 @@ impl SigVerifyStage {
         verified_sender: CrossbeamSender<Vec<Packets>>,
         verifier: T,
     ) -> Self {
-        let thread_hdls = Self::verifier_services(packet_receiver, verified_sender, verifier);
-        Self { thread_hdls }
+        let thread_hdl = Self::verifier_services(packet_receiver, verified_sender, verifier);
+        Self { thread_hdl }
+    }
+
+    pub fn discard_excess_packets(batches: &mut Vec<Packets>, max_packets: usize) {
+        let mut received_ips = HashMap::new();
+        for (batch_index, batch) in batches.iter().enumerate() {
+            for (packet_index, packets) in batch.packets.iter().enumerate() {
+                let e = received_ips
+                    .entry(packets.meta.addr().ip())
+                    .or_insert_with(Vec::new);
+                e.push((batch_index, packet_index));
+            }
+        }
+        let mut batch_len = 0;
+        while batch_len < max_packets {
+            for (_ip, indexes) in received_ips.iter_mut() {
+                if !indexes.is_empty() {
+                    indexes.remove(0);
+                    batch_len += 1;
+                    if batch_len >= MAX_SIGVERIFY_BATCH {
+                        break;
+                    }
+                }
+            }
+        }
+        for (_addr, indexes) in received_ips {
+            for (batch_index, packet_index) in indexes {
+                batches[batch_index].packets[packet_index].meta.discard = true;
+            }
+        }
     }
 
     fn verifier<T: SigVerifier>(
-        recvr: &Arc<Mutex<PacketReceiver>>,
+        recvr: &PacketReceiver,
         sendr: &CrossbeamSender<Vec<Packets>>,
-        id: usize,
         verifier: &T,
+        stats: &mut SigVerifierStats,
     ) -> Result<()> {
-        let (batch, len, recv_time) = streamer::recv_batch(
-            &recvr.lock().expect("'recvr' lock in fn verifier"),
-            if perf_libs::api().is_some() {
-                RECV_BATCH_MAX_GPU
-            } else {
-                RECV_BATCH_MAX_CPU
-            },
-        )?;
+        let (mut batches, len, recv_time) = streamer::recv_batch(recvr)?;
 
         let mut verify_batch_time = Measure::start("sigverify_batch_time");
-        let batch_len = batch.len();
-        debug!(
-            "@{:?} verifier: verifying: {} id: {}",
-            timing::timestamp(),
-            len,
-            id
-        );
-        sendr.send(verifier.verify_batch(batch))?;
+        let batches_len = batches.len();
+        debug!("@{:?} verifier: verifying: {}", timing::timestamp(), len,);
+        if len > MAX_SIGVERIFY_BATCH {
+            Self::discard_excess_packets(&mut batches, MAX_SIGVERIFY_BATCH);
+        }
+        sendr.send(verifier.verify_batch(batches))?;
         verify_batch_time.stop();
 
         debug!(
-            "@{:?} verifier: done. batches: {} total verify time: {:?} id: {} verified: {} v/s {}",
+            "@{:?} verifier: done. batches: {} total verify time: {:?} verified: {} v/s {}",
             timing::timestamp(),
-            batch_len,
+            batches_len,
             verify_batch_time.as_ms(),
-            id,
             len,
             (len as f32 / verify_batch_time.as_s())
         );
 
         datapoint_debug!(
             "sigverify_stage-total_verify_time",
-            ("num_batches", batch_len, i64),
+            ("num_batches", batches_len, i64),
             ("num_packets", len, i64),
             ("verify_time_ms", verify_batch_time.as_ms(), i64),
             ("recv_time", recv_time, i64),
         );
 
+        stats
+            .recv_batches_us_hist
+            .increment(recv_time as u64)
+            .unwrap();
+        stats
+            .verify_batches_pp_us_hist
+            .increment(verify_batch_time.as_us() / (len as u64))
+            .unwrap();
+        stats.batches_hist.increment(batches_len as u64).unwrap();
+        stats.packets_hist.increment(len as u64).unwrap();
+        stats.total_batches += batches_len;
+        stats.total_packets += len;
+
         Ok(())
     }
 
     fn verifier_service<T: SigVerifier + 'static + Send + Clone>(
-        packet_receiver: Arc<Mutex<PacketReceiver>>,
+        packet_receiver: PacketReceiver,
         verified_sender: CrossbeamSender<Vec<Packets>>,
-        id: usize,
         verifier: &T,
     ) -> JoinHandle<()> {
         let verifier = verifier.clone();
+        let mut stats = SigVerifierStats::default();
+        let mut last_print = Instant::now();
         Builder::new()
-            .name(format!("solana-verifier-{}", id))
+            .name("solana-verifier".to_string())
             .spawn(move || loop {
-                if let Err(e) = Self::verifier(&packet_receiver, &verified_sender, id, &verifier) {
+                if let Err(e) =
+                    Self::verifier(&packet_receiver, &verified_sender, &verifier, &mut stats)
+                {
                     match e {
                         SigVerifyServiceError::Streamer(StreamerError::RecvTimeout(
                             RecvTimeoutError::Disconnected,
@@ -132,6 +241,11 @@ impl SigVerifyStage {
                         _ => error!("{:?}", e),
                     }
                 }
+                if last_print.elapsed().as_secs() > 2 {
+                    stats.report();
+                    stats = SigVerifierStats::default();
+                    last_print = Instant::now();
+                }
             })
             .unwrap()
     }
@@ -140,19 +254,43 @@ impl SigVerifyStage {
         packet_receiver: PacketReceiver,
         verified_sender: CrossbeamSender<Vec<Packets>>,
         verifier: T,
-    ) -> Vec<JoinHandle<()>> {
-        let receiver = Arc::new(Mutex::new(packet_receiver));
-        (0..4)
-            .map(|id| {
-                Self::verifier_service(receiver.clone(), verified_sender.clone(), id, &verifier)
-            })
-            .collect()
+    ) -> JoinHandle<()> {
+        Self::verifier_service(packet_receiver, verified_sender, &verifier)
     }
 
     pub fn join(self) -> thread::Result<()> {
-        for thread_hdl in self.thread_hdls {
-            thread_hdl.join()?;
-        }
-        Ok(())
+        self.thread_hdl.join()
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use solana_perf::packet::Packet;
+
+    fn count_non_discard(packets: &[Packets]) -> usize {
+        packets
+            .iter()
+            .map(|pp| {
+                pp.packets
+                    .iter()
+                    .map(|p| if p.meta.discard { 0 } else { 1 })
+                    .sum::<usize>()
+            })
+            .sum::<usize>()
+    }
+
+    #[test]
+    fn test_packet_discard() {
+        solana_logger::setup();
+        let mut p = Packets::default();
+        p.packets.resize(10, Packet::default());
+        p.packets[3].meta.addr = [1u16; 8];
+        let mut packets = vec![p];
+        let max = 3;
+        SigVerifyStage::discard_excess_packets(&mut packets, max);
+        assert_eq!(count_non_discard(&packets), max);
+        assert!(!packets[0].packets[0].meta.discard);
+        assert!(!packets[0].packets[3].meta.discard);
     }
 }

core/src/tpu.rs

@@ -8,6 +8,8 @@ use crate::{
         ClusterInfoVoteListener, GossipDuplicateConfirmedSlotsSender, GossipVerifiedVoteHashSender,
         VerifiedVoteSender, VoteTracker,
     },
+    cost_model::CostModel,
+    cost_tracker::CostTracker,
     fetch_stage::FetchStage,
     sigverify::TransactionSigVerifier,
     sigverify_stage::SigVerifyStage,
@@ -71,6 +73,7 @@ impl Tpu {
         bank_notification_sender: Option<BankNotificationSender>,
         tpu_coalesce_ms: u64,
         cluster_confirmed_slot_sender: GossipDuplicateConfirmedSlotsSender,
+        cost_model: &Arc<RwLock<CostModel>>,
     ) -> Self {
         let (packet_sender, packet_receiver) = channel();
         let (vote_packet_sender, vote_packet_receiver) = channel();
@@ -120,6 +123,7 @@ impl Tpu {
             cluster_confirmed_slot_sender,
         );
 
+        let cost_tracker = Arc::new(RwLock::new(CostTracker::new(cost_model.clone())));
         let banking_stage = BankingStage::new(
             cluster_info,
             poh_recorder,
@@ -128,6 +132,7 @@ impl Tpu {
             verified_gossip_vote_packets_receiver,
             transaction_status_sender,
             replay_vote_sender,
+            cost_tracker,
         );
 
         let broadcast_stage = broadcast_type.new_broadcast_stage(

core/src/tvu.rs

@@ -12,6 +12,8 @@ use crate::{
     cluster_slots::ClusterSlots,
     completed_data_sets_service::CompletedDataSetsSender,
     consensus::Tower,
+    cost_model::CostModel,
+    cost_update_service::CostUpdateService,
     ledger_cleanup_service::LedgerCleanupService,
     replay_stage::{ReplayStage, ReplayStageConfig},
     retransmit_stage::RetransmitStage,
@@ -38,6 +40,7 @@ use solana_runtime::{
         AbsRequestHandler, AbsRequestSender, AccountsBackgroundService, SnapshotRequestHandler,
     },
     accounts_db::AccountShrinkThreshold,
+    bank::ExecuteTimings,
     bank_forks::{BankForks, SnapshotConfig},
     commitment::BlockCommitmentCache,
     vote_sender_types::ReplayVoteSender,
@@ -52,7 +55,7 @@ use std::{
     net::UdpSocket,
     sync::{
         atomic::AtomicBool,
-        mpsc::{channel, Receiver},
+        mpsc::{channel, Receiver, Sender},
         Arc, Mutex, RwLock,
     },
     thread,
@@ -67,6 +70,7 @@ pub struct Tvu {
     accounts_background_service: AccountsBackgroundService,
     accounts_hash_verifier: AccountsHashVerifier,
     voting_service: VotingService,
+    cost_update_service: CostUpdateService,
 }
 
 pub struct Sockets {
@@ -91,6 +95,7 @@ pub struct TvuConfig {
     pub rocksdb_max_compaction_jitter: Option<u64>,
     pub wait_for_vote_to_start_leader: bool,
     pub accounts_shrink_ratio: AccountShrinkThreshold,
+    pub disable_epoch_boundary_optimization: bool,
 }
 
 impl Tvu {
@@ -130,6 +135,7 @@ impl Tvu {
         gossip_confirmed_slots_receiver: GossipDuplicateConfirmedSlotsReceiver,
         tvu_config: TvuConfig,
         max_slots: &Arc<MaxSlots>,
+        cost_model: &Arc<RwLock<CostModel>>,
     ) -> Self {
         let keypair: Arc<Keypair> = cluster_info.keypair.clone();
 
@@ -273,6 +279,7 @@ impl Tvu {
             cache_block_meta_sender,
             bank_notification_sender,
             wait_for_vote_to_start_leader: tvu_config.wait_for_vote_to_start_leader,
+            disable_epoch_boundary_optimization: tvu_config.disable_epoch_boundary_optimization,
         };
 
         let (voting_sender, voting_receiver) = channel();
@@ -283,6 +290,17 @@ impl Tvu {
             bank_forks.clone(),
         );
 
+        let (cost_update_sender, cost_update_receiver): (
+            Sender<ExecuteTimings>,
+            Receiver<ExecuteTimings>,
+        ) = channel();
+        let cost_update_service = CostUpdateService::new(
+            exit.clone(),
+            blockstore.clone(),
+            cost_model.clone(),
+            cost_update_receiver,
+        );
+
         let replay_stage = ReplayStage::new(
             replay_stage_config,
             blockstore.clone(),
@@ -301,6 +319,7 @@ impl Tvu {
             gossip_verified_vote_hash_receiver,
             cluster_slots_update_sender,
             voting_sender,
+            cost_update_sender,
         );
 
         let ledger_cleanup_service = tvu_config.max_ledger_shreds.map(|max_ledger_shreds| {
@@ -332,6 +351,7 @@ impl Tvu {
             accounts_background_service,
             accounts_hash_verifier,
             voting_service,
+            cost_update_service,
         }
     }
 
@@ -346,6 +366,7 @@ impl Tvu {
         self.replay_stage.join()?;
         self.accounts_hash_verifier.join()?;
         self.voting_service.join()?;
+        self.cost_update_service.join()?;
         Ok(())
     }
 }
@@ -453,6 +474,7 @@ pub mod tests {
             gossip_confirmed_slots_receiver,
             TvuConfig::default(),
             &Arc::new(MaxSlots::default()),
+            &Arc::new(RwLock::new(CostModel::default())),
         );
         exit.store(true, Ordering::Relaxed);
         tvu.join().unwrap();

core/src/validator.rs

@@ -7,6 +7,7 @@ use {
         cluster_info_vote_listener::VoteTracker,
         completed_data_sets_service::CompletedDataSetsService,
         consensus::{reconcile_blockstore_roots_with_tower, Tower},
+        cost_model::CostModel,
         rewards_recorder_service::{RewardsRecorderSender, RewardsRecorderService},
         sample_performance_service::SamplePerformanceService,
         serve_repair::ServeRepair,
@@ -147,6 +148,7 @@ pub struct ValidatorConfig {
     pub validator_exit: Arc<RwLock<Exit>>,
     pub no_wait_for_vote_to_start_leader: bool,
     pub accounts_shrink_ratio: AccountShrinkThreshold,
+    pub disable_epoch_boundary_optimization: bool,
 }
 
 impl Default for ValidatorConfig {
@@ -204,6 +206,7 @@ impl Default for ValidatorConfig {
             validator_exit: Arc::new(RwLock::new(Exit::default())),
             no_wait_for_vote_to_start_leader: true,
             accounts_shrink_ratio: AccountShrinkThreshold::default(),
+            disable_epoch_boundary_optimization: false,
         }
     }
 }
@@ -679,6 +682,10 @@ impl Validator {
             bank_forks.read().unwrap().root_bank().deref(),
         ));
 
+        let mut cost_model = CostModel::default();
+        cost_model.initialize_cost_table(&blockstore.read_program_costs().unwrap());
+        let cost_model = Arc::new(RwLock::new(cost_model));
+
         let (retransmit_slots_sender, retransmit_slots_receiver) = unbounded();
         let (verified_vote_sender, verified_vote_receiver) = unbounded();
         let (gossip_verified_vote_hash_sender, gossip_verified_vote_hash_receiver) = unbounded();
@@ -753,8 +760,10 @@ impl Validator {
                 rocksdb_max_compaction_jitter: config.rocksdb_compaction_interval,
                 wait_for_vote_to_start_leader,
                 accounts_shrink_ratio: config.accounts_shrink_ratio,
+                disable_epoch_boundary_optimization: config.disable_epoch_boundary_optimization,
             },
             &max_slots,
+            &cost_model,
         );
 
         let tpu = Tpu::new(
@@ -781,6 +790,7 @@ impl Validator {
             bank_notification_sender,
             config.tpu_coalesce_ms,
             cluster_confirmed_slot_sender,
+            &cost_model,
         );
 
         datapoint_info!("validator-new", ("id", id.to_string(), String));

crate-features/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-crate-features"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Crate Features"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"

docs/src/cli/delegate-stake.md

@@ -1,6 +1,10 @@
 ---
-title: Delegate Stake
+title: Staking
 ---
+For an overview of staking, read first the
+[Staking and Inflation FAQ](https://solana.com/staking).
+
+------
 
 After you have [received SOL](transfer-tokens.md), you might consider putting
 it to use by delegating _stake_ to a validator. Stake is what we call tokens

docs/src/developing/clients/jsonrpc-api.md

@@ -3039,7 +3039,7 @@ curl http://localhost:8899 -X POST -H "Content-Type: application/json" -d '
 
 Result:
 ```json
-{"jsonrpc":"2.0","result":{"solana-core": "1.7.13"},"id":1}
+{"jsonrpc":"2.0","result":{"solana-core": "1.8.0"},"id":1}
 ```
 
 ### getVoteAccounts

docs/src/staking/stake-accounts.md

@@ -23,8 +23,8 @@ any control over the account. In fact, a keypair or private key may not even
 exist for a stake account's address.
 
 The only time a stake account's address has a keypair file is when [creating
-a stake account using the command line tools](../cli/delegate-stake.md#create-a-stake-account),
-a new keypair file is created first only to ensure that the stake account's
+a stake account using the command line tools](../cli/delegate-stake.md#create-a-stake-account).
+A new keypair file is created first only to ensure that the stake account's
 address is new and unique.
 
 #### Understanding Account Authorities
@@ -137,7 +137,5 @@ re-created for the address to be used again.
 
 #### Viewing Stake Accounts
 
-Stake account details can be viewed on the Solana Explorer by copying and pasting
-an account address into the search bar.
-
-- http://explorer.solana.com/accounts
+Stake account details can be viewed on the [Solana Explorer](http://explorer.solana.com/accounts)
+by copying and pasting an account address into the search bar.

docs/src/terminology.md

@@ -8,7 +8,7 @@ The following terms are used throughout the documentation.
 
 A record in the Solana ledger that either holds data or is an executable program.
 
-Like an account at a traditional bank, a Solana account may hold funds called [lamports](terminology.md#lamport). Like a file in Linux, it is addressable by a key, often referred to as a [public key](terminology.md#public-key-pubkey) or pubkey.
+Like an account at a traditional bank, a Solana account may hold funds called [lamports](#lamport). Like a file in Linux, it is addressable by a key, often referred to as a [public key](#public-key-pubkey) or pubkey.
 
 The key may be one of:
 
@@ -26,59 +26,55 @@ A front-end application that interacts with a Solana cluster.
 
 ## bank state
 
-The result of interpreting all programs on the ledger at a given [tick height](terminology.md#tick-height). It includes at least the set of all [accounts](terminology.md#account) holding nonzero [native tokens](terminology.md#native-token).
+The result of interpreting all programs on the ledger at a given [tick height](#tick-height). It includes at least the set of all [accounts](#account) holding nonzero [native tokens](#native-token).
 
 ## block
 
-A contiguous set of [entries](terminology.md#entry) on the ledger covered by a [vote](terminology.md#ledger-vote). A [leader](terminology.md#leader) produces at most one block per [slot](terminology.md#slot).
+A contiguous set of [entries](#entry) on the ledger covered by a [vote](#ledger-vote). A [leader](#leader) produces at most one block per [slot](#slot).
 
 ## blockhash
 
-A unique value ([hash](terminology.md#hash)) that identifies a record (block).  Solana computes a blockhash from the last [entry id](terminology.md#entry-id) of the block.
+A unique value ([hash](#hash)) that identifies a record (block).  Solana computes a blockhash from the last [entry id](#entry-id) of the block.
 
 ## block height
 
-The number of [blocks](terminology.md#block) beneath the current block. The first block after the [genesis block](terminology.md#genesis-block) has height one.
+The number of [blocks](#block) beneath the current block. The first block after the [genesis block](#genesis-block) has height one.
 
 ## bootstrap validator
 
-The [validator](terminology.md#validator) that produces the genesis (first) [block](terminology.md#block) of a block chain.
+The [validator](#validator) that produces the genesis (first) [block](#block) of a block chain.
 
 ## BPF loader
 
 The Solana program that owns and loads [BPF](developing/on-chain-programs/overview#berkeley-packet-filter-bpf) smart contract programs, allowing the program to interface with the runtime.
 
-## CBC block
-
-The smallest encrypted chunk of ledger, an encrypted ledger segment would be made of many CBC blocks. `ledger_segment_size / cbc_block_size` to be exact.
-
 ## client
 
-A computer program that accesses the Solana server network [cluster](terminology.md#cluster).
+A computer program that accesses the Solana server network [cluster](#cluster).
 
 ## cluster
 
-A set of [validators](terminology.md#validator) maintaining a single [ledger](terminology.md#ledger).
+A set of [validators](#validator) maintaining a single [ledger](#ledger).
 
 ## confirmation time
 
-The wallclock duration between a [leader](terminology.md#leader) creating a [tick entry](terminology.md#tick) and creating a [confirmed block](terminology.md#confirmed-block).
+The wallclock duration between a [leader](#leader) creating a [tick entry](#tick) and creating a [confirmed block](#confirmed-block).
 
 ## confirmed block
 
-A [block](terminology.md#block) that has received a [supermajority](terminology.md#supermajority) of [ledger votes](terminology.md#ledger-vote).
+A [block](#block) that has received a [supermajority](#supermajority) of [ledger votes](#ledger-vote).
 
 ## control plane
 
-A gossip network connecting all [nodes](terminology.md#node) of a [cluster](terminology.md#cluster).
+A gossip network connecting all [nodes](#node) of a [cluster](#cluster).
 
 ## cooldown period
 
-Some number of [epochs](terminology.md#epoch) after [stake](terminology.md#stake) has been deactivated while it progressively becomes available for withdrawal. During this period, the stake is considered to be "deactivating". More info about: [warmup and cooldown](implemented-proposals/staking-rewards.md#stake-warmup-cooldown-withdrawal)
+Some number of [epochs](#epoch) after [stake](#stake) has been deactivated while it progressively becomes available for withdrawal. During this period, the stake is considered to be "deactivating". More info about: [warmup and cooldown](implemented-proposals/staking-rewards.md#stake-warmup-cooldown-withdrawal)
 
 ## credit
 
-See [vote credit](terminology.md#vote-credit).
+See [vote credit](#vote-credit).
 
 ## cross-program invocation (CPI)
 
@@ -86,7 +82,7 @@ A call from one smart contract program to another. For more information, see [ca
 
 ## data plane
 
-A multicast network used to efficiently validate [entries](terminology.md#entry) and gain consensus.
+A multicast network used to efficiently validate [entries](#entry) and gain consensus.
 
 ## drone
 
@@ -94,21 +90,21 @@ An off-chain service that acts as a custodian for a user's private key. It typic
 
 ## entry
 
-An entry on the [ledger](terminology.md#ledger) either a [tick](terminology.md#tick) or a [transactions entry](terminology.md#transactions-entry).
+An entry on the [ledger](#ledger) either a [tick](#tick) or a [transactions entry](#transactions-entry).
 
 ## entry id
 
-A preimage resistant [hash](terminology.md#hash) over the final contents of an entry, which acts as the [entry's](terminology.md#entry) globally unique identifier. The hash serves as evidence of:
+A preimage resistant [hash](#hash) over the final contents of an entry, which acts as the [entry's](#entry) globally unique identifier. The hash serves as evidence of:
 
 - The entry being generated after a duration of time
-- The specified [transactions](terminology.md#transaction) are those included in the entry
-- The entry's position with respect to other entries in [ledger](terminology.md#ledger)
+- The specified [transactions](#transaction) are those included in the entry
+- The entry's position with respect to other entries in [ledger](#ledger)
 
-See [proof of history](terminology.md#proof-of-history-poh).
+See [proof of history](#proof-of-history-poh).
 
 ## epoch
 
-The time, i.e. number of [slots](terminology.md#slot), for which a [leader schedule](terminology.md#leader-schedule) is valid.
+The time, i.e. number of [slots](#slot), for which a [leader schedule](#leader-schedule) is valid.
 
 ## fee account
 
@@ -116,19 +112,19 @@ The fee account in the transaction is the account pays for the cost of including
 
 ## finality
 
-When nodes representing 2/3rd of the [stake](terminology.md#stake) have a common [root](terminology.md#root).
+When nodes representing 2/3rd of the [stake](#stake) have a common [root](#root).
 
 ## fork
 
-A [ledger](terminology.md#ledger) derived from common entries but then diverged.
+A [ledger](#ledger) derived from common entries but then diverged.
 
 ## genesis block
 
-The first [block](terminology.md#block) in the chain.
+The first [block](#block) in the chain.
 
 ## genesis config
 
-The configuration file that prepares the [ledger](terminology.md#ledger) for the [genesis block](terminology.md#genesis-block).
+The configuration file that prepares the [ledger](#ledger) for the [genesis block](#genesis-block).
 
 ## hash
 
@@ -140,76 +136,76 @@ An increase in token supply over time used to fund rewards for validation and to
 
 ## inner instruction
 
-See [cross-program invocation](terminology.md#cross-program-invocation-cpi).
+See [cross-program invocation](#cross-program-invocation-cpi).
 
 ## instruction
 
-The smallest contiguous unit of execution logic in a [program](terminology.md#program). An instruction specifies which program it is calling, which accounts it wants to read or modify, and additional data that serves as auxiliary input to the program. A [client](terminology.md#client) can include one or multiple instructions in a [transaction](terminology.md#transaction). An instruction may contain one or more [cross-program invocations](terminology.md#cross-program-invocation-cpi).
+The smallest contiguous unit of execution logic in a [program](#program). An instruction specifies which program it is calling, which accounts it wants to read or modify, and additional data that serves as auxiliary input to the program. A [client](#client) can include one or multiple instructions in a [transaction](#transaction). An instruction may contain one or more [cross-program invocations](#cross-program-invocation-cpi).
 
 ## keypair
 
-A [public key](terminology.md#public-key-pubkey) and corresponding [private key](terminology.md#private-key) for accessing an account.
+A [public key](#public-key-pubkey) and corresponding [private key](#private-key) for accessing an account.
 
 ## lamport
 
-A fractional [native token](terminology.md#native-token) with the value of 0.000000001 [sol](terminology.md#sol).
+A fractional [native token](#native-token) with the value of 0.000000001 [sol](#sol).
 
 ## leader
 
-The role of a [validator](terminology.md#validator) when it is appending [entries](terminology.md#entry) to the [ledger](terminology.md#ledger).
+The role of a [validator](#validator) when it is appending [entries](#entry) to the [ledger](#ledger).
 
 ## leader schedule
 
-A sequence of [validator](terminology.md#validator) [public keys](terminology.md#public-key-pubkey) mapped to [slots](terminology.md#slot). The cluster uses the leader schedule to determine which validator is the [leader](terminology.md#leader) at any moment in time.
+A sequence of [validator](#validator) [public keys](#public-key-pubkey) mapped to [slots](#slot). The cluster uses the leader schedule to determine which validator is the [leader](#leader) at any moment in time.
 
 ## ledger
 
-A list of [entries](terminology.md#entry) containing [transactions](terminology.md#transaction) signed by [clients](terminology.md#client).
-Conceptually, this can be traced back to the [genesis block](terminology.md#genesis-block), but an actual [validator](terminology.md#validator)'s ledger may have only newer [blocks](terminology.md#block) to reduce storage, as older ones are not needed for validation of future blocks by design.
+A list of [entries](#entry) containing [transactions](#transaction) signed by [clients](#client).
+Conceptually, this can be traced back to the [genesis block](#genesis-block), but an actual [validator](#validator)'s ledger may have only newer [blocks](#block) to reduce storage, as older ones are not needed for validation of future blocks by design.
 
 ## ledger vote
 
-A [hash](terminology.md#hash) of the [validator's state](terminology.md#bank-state) at a given [tick height](terminology.md#tick-height). It comprises a [validator's](terminology.md#validator) affirmation that a [block](terminology.md#block) it has received has been verified, as well as a promise not to vote for a conflicting [block](terminology.md#block) \(i.e. [fork](terminology.md#fork)\) for a specific amount of time, the [lockout](terminology.md#lockout) period.
+A [hash](#hash) of the [validator's state](#bank-state) at a given [tick height](#tick-height). It comprises a [validator's](#validator) affirmation that a [block](#block) it has received has been verified, as well as a promise not to vote for a conflicting [block](#block) \(i.e. [fork](#fork)\) for a specific amount of time, the [lockout](#lockout) period.
 
 ## light client
 
-A type of [client](terminology.md#client) that can verify it's pointing to a valid [cluster](terminology.md#cluster). It performs more ledger verification than a [thin client](terminology.md#thin-client) and less than a [validator](terminology.md#validator).
+A type of [client](#client) that can verify it's pointing to a valid [cluster](#cluster). It performs more ledger verification than a [thin client](#thin-client) and less than a [validator](#validator).
 
 ## loader
 
-A [program](terminology.md#program) with the ability to interpret the binary encoding of other on-chain programs.
+A [program](#program) with the ability to interpret the binary encoding of other on-chain programs.
 
 ## lockout
 
-The duration of time for which a [validator](terminology.md#validator) is unable to [vote](terminology.md#ledger-vote) on another [fork](terminology.md#fork).
+The duration of time for which a [validator](#validator) is unable to [vote](#ledger-vote) on another [fork](#fork).
 
 ## native token
 
-The [token](terminology.md#token) used to track work done by [nodes](terminology.md#node) in a cluster.
+The [token](#token) used to track work done by [nodes](#node) in a cluster.
 
 ## node
 
-A computer participating in a [cluster](terminology.md#cluster).
+A computer participating in a [cluster](#cluster).
 
 ## node count
 
-The number of [validators](terminology.md#validator) participating in a [cluster](terminology.md#cluster).
+The number of [validators](#validator) participating in a [cluster](#cluster).
 
 ## PoH
 
-See [Proof of History](terminology.md#proof-of-history-poh).
+See [Proof of History](#proof-of-history-poh).
 
 ## point
 
-A weighted [credit](terminology.md#credit) in a rewards regime. In the [validator](terminology.md#validator) [rewards regime](cluster/stake-delegation-and-rewards.md), the number of points owed to a [stake](terminology.md#stake) during redemption is the product of the [vote credits](terminology.md#vote-credit) earned and the number of lamports staked.
+A weighted [credit](#credit) in a rewards regime. In the [validator](#validator) [rewards regime](cluster/stake-delegation-and-rewards.md), the number of points owed to a [stake](#stake) during redemption is the product of the [vote credits](#vote-credit) earned and the number of lamports staked.
 
 ## private key
 
-The private key of a [keypair](terminology.md#keypair).
+The private key of a [keypair](#keypair).
 
 ## program
 
-The code that interprets [instructions](terminology.md#instruction).
+The code that interprets [instructions](#instruction).
 
 ## program derived account (PDA)
 
@@ -217,23 +213,23 @@ An account whose owner is a program and thus is not controlled by a private key
 
 ## program id
 
-The public key of the [account](terminology.md#account) containing a [program](terminology.md#program).
+The public key of the [account](#account) containing a [program](#program).
 
 ## proof of history (PoH)
 
-A stack of proofs, each which proves that some data existed before the proof was created and that a precise duration of time passed before the previous proof. Like a [VDF](terminology.md#verifiable-delay-function-vdf), a Proof of History can be verified in less time than it took to produce.
+A stack of proofs, each which proves that some data existed before the proof was created and that a precise duration of time passed before the previous proof. Like a [VDF](#verifiable-delay-function-vdf), a Proof of History can be verified in less time than it took to produce.
 
 ## public key (pubkey)
 
-The public key of a [keypair](terminology.md#keypair).
+The public key of a [keypair](#keypair).
 
 ## root
 
-A [block](terminology.md#block) or [slot](terminology.md#slot) that has reached maximum [lockout](terminology.md#lockout) on a [validator](terminology.md#validator). The root is the highest block that is an ancestor of all active forks on a validator. All ancestor blocks of a root are also transitively a root. Blocks that are not an ancestor and not a descendant of the root are excluded from consideration for consensus and can be discarded.
+A [block](#block) or [slot](#slot) that has reached maximum [lockout](#lockout) on a [validator](#validator). The root is the highest block that is an ancestor of all active forks on a validator. All ancestor blocks of a root are also transitively a root. Blocks that are not an ancestor and not a descendant of the root are excluded from consideration for consensus and can be discarded.
 
 ## runtime
 
-The component of a [validator](terminology.md#validator) responsible for [program](terminology.md#program) execution.
+The component of a [validator](#validator) responsible for [program](#program) execution.
 
 ## Sealevel
 
@@ -241,25 +237,25 @@ Solana's parallel smart contracts run-time.
 
 ## shred
 
-A fraction of a [block](terminology.md#block); the smallest unit sent between [validators](terminology.md#validator).
+A fraction of a [block](#block); the smallest unit sent between [validators](#validator).
 
 ## signature
 
 A 64-byte ed25519 signature of R (32-bytes) and S (32-bytes). With the requirement that R is a packed Edwards point not of small order and S is a scalar in the range of 0 <= S < L.
 This requirement ensures no signature malleability. Each transaction must have at least one signature for [fee account](terminology#fee-account).
-Thus, the first signature in transaction can be treated as [transacton id](terminology.md#transaction-id)
+Thus, the first signature in transaction can be treated as [transacton id](#transaction-id)
 
 ## skipped slot
 
-A past [slot](terminology.md#slot) that did not produce a [block](terminology.md#block), because the leader was offline or the [fork](terminology.md#fork) containing the slot was abandoned for a better alternative by cluster consensus. A skipped slot will not appear as an ancestor for blocks at subsequent slots, nor increment the [block height](terminology#block-height), nor expire the oldest `recent_blockhash`.
+A past [slot](#slot) that did not produce a [block](#block), because the leader was offline or the [fork](#fork) containing the slot was abandoned for a better alternative by cluster consensus. A skipped slot will not appear as an ancestor for blocks at subsequent slots, nor increment the [block height](terminology#block-height), nor expire the oldest `recent_blockhash`.
 
-Whether a slot has been skipped can only be determined when it becomes older than the latest [rooted](terminology.md#root) (thus not-skipped) slot.
+Whether a slot has been skipped can only be determined when it becomes older than the latest [rooted](#root) (thus not-skipped) slot.
 
 ## slot
 
-The period of time for which each [leader](terminology.md#leader) ingests transactions and produces a [block](terminology.md#block).
+The period of time for which each [leader](#leader) ingests transactions and produces a [block](#block).
 
-Collectively, slots create a logical clock. Slots are ordered sequentially and non-overlapping, comprising roughly equal real-world time as per [PoH](terminology.md#proof-of-history-poh).
+Collectively, slots create a logical clock. Slots are ordered sequentially and non-overlapping, comprising roughly equal real-world time as per [PoH](#proof-of-history-poh).
 
 ## smart contract
 
@@ -267,7 +263,7 @@ A program on a blockchain that can read and modify accounts over which it has co
 
 ## sol
 
-The [native token](terminology.md#native-token) of a Solana [cluster](terminology.md#cluster).
+The [native token](#native-token) of a Solana [cluster](#cluster).
 
 ## Solana Program Library (SPL)
 
@@ -275,27 +271,27 @@ A library of programs on Solana such as spl-token that facilitates tasks such as
 
 ## stake
 
-Tokens forfeit to the [cluster](terminology.md#cluster) if malicious [validator](terminology.md#validator) behavior can be proven.
+Tokens forfeit to the [cluster](#cluster) if malicious [validator](#validator) behavior can be proven.
 
 ## supermajority
 
-2/3 of a [cluster](terminology.md#cluster).
+2/3 of a [cluster](#cluster).
 
 ## sysvar
 
-A system [account](terminology.md#account).  [Sysvars](developing/runtime-facilities/sysvars.md) provide cluster state information such as current tick height, rewards [points](terminology.md#point) values, etc.  Programs can access Sysvars via a Sysvar account (pubkey) or by querying via a syscall.
+A system [account](#account).  [Sysvars](developing/runtime-facilities/sysvars.md) provide cluster state information such as current tick height, rewards [points](#point) values, etc.  Programs can access Sysvars via a Sysvar account (pubkey) or by querying via a syscall.
 
 ## thin client
 
-A type of [client](terminology.md#client) that trusts it is communicating with a valid [cluster](terminology.md#cluster).
+A type of [client](#client) that trusts it is communicating with a valid [cluster](#cluster).
 
 ## tick
 
-A ledger [entry](terminology.md#entry) that estimates wallclock duration.
+A ledger [entry](#entry) that estimates wallclock duration.
 
 ## tick height
 
-The Nth [tick](terminology.md#tick) in the [ledger](terminology.md#ledger).
+The Nth [tick](#tick) in the [ledger](#ledger).
 
 ## token
 
@@ -303,31 +299,31 @@ A digitally transferable asset.
 
 ## tps
 
-[Transactions](terminology.md#transaction) per second.
+[Transactions](#transaction) per second.
 
 ## transaction
 
-One or more [instructions](terminology.md#instruction) signed by a [client](terminology.md#client) using one or more [keypairs](terminology.md#keypair) and executed atomically with only two possible outcomes: success or failure.
+One or more [instructions](#instruction) signed by a [client](#client) using one or more [keypairs](#keypair) and executed atomically with only two possible outcomes: success or failure.
 
 ## transaction id
 
-The first [signature](terminology.md#signature) in a [transaction](terminology.md#transaction), which can be used to uniquely identify the transaction across the complete [ledger](terminology.md#ledger).
+The first [signature](#signature) in a [transaction](#transaction), which can be used to uniquely identify the transaction across the complete [ledger](#ledger).
 
 ## transaction confirmations
 
-The number of [confirmed blocks](terminology.md#confirmed-block) since the transaction was accepted onto the [ledger](terminology.md#ledger). A transaction is finalized when its block becomes a [root](terminology.md#root).
+The number of [confirmed blocks](#confirmed-block) since the transaction was accepted onto the [ledger](#ledger). A transaction is finalized when its block becomes a [root](#root).
 
 ## transactions entry
 
-A set of [transactions](terminology.md#transaction) that may be executed in parallel.
+A set of [transactions](#transaction) that may be executed in parallel.
 
 ## validator
 
-A full participant in a Solana network [cluster](terminology.md#cluster) that produces new [blocks](terminology.md#block).  A validator validates the transactions added to the [ledger](terminology.md#ledger)
+A full participant in a Solana network [cluster](#cluster) that produces new [blocks](#block).  A validator validates the transactions added to the [ledger](#ledger)
 
 ## VDF
 
-See [verifiable delay function](terminology.md#verifiable-delay-function-vdf).
+See [verifiable delay function](#verifiable-delay-function-vdf).
 
 ## verifiable delay function (VDF)
 
@@ -335,16 +331,16 @@ A function that takes a fixed amount of time to execute that produces a proof th
 
 ## vote
 
-See [ledger vote](terminology.md#ledger-vote).
+See [ledger vote](#ledger-vote).
 
 ## vote credit
 
-A reward tally for [validators](terminology.md#validator). A vote credit is awarded to a validator in its vote account when the validator reaches a [root](terminology.md#root).
+A reward tally for [validators](#validator). A vote credit is awarded to a validator in its vote account when the validator reaches a [root](#root).
 
 ## wallet
 
-A collection of [keypairs](terminology.md#keypair) that allows users to manage their funds.
+A collection of [keypairs](#keypair) that allows users to manage their funds.
 
 ## warmup period
 
-Some number of [epochs](terminology.md#epoch) after [stake](terminology.md#stake) has been delegated while it progressively becomes effective. During this period, the stake is considered to be "activating". More info about: [warmup and cooldown](cluster/stake-delegation-and-rewards.md#stake-warmup-cooldown-withdrawal)
+Some number of [epochs](#epoch) after [stake](#stake) has been delegated while it progressively becomes effective. During this period, the stake is considered to be "activating". More info about: [warmup and cooldown](cluster/stake-delegation-and-rewards.md#stake-warmup-cooldown-withdrawal)

dos/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-dos"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -14,17 +14,18 @@ clap = "2.33.1"
 log = "0.4.11"
 rand = "0.7.0"
 rayon = "1.5.0"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-core = { path = "../core", version = "=1.7.13" }
-solana-gossip = { path = "../gossip", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-gossip = { path = "../gossip", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-perf = { path = "../perf", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

dos/src/main.rs

@@ -12,6 +12,13 @@ use std::process::exit;
 use std::str::FromStr;
 use std::time::{Duration, Instant};
 
+fn get_repair_contact(nodes: &[ContactInfo]) -> ContactInfo {
+    let source = thread_rng().gen_range(0, nodes.len());
+    let mut contact = nodes[source].clone();
+    contact.id = solana_sdk::pubkey::new_rand();
+    contact
+}
+
 fn run_dos(
     nodes: &[ContactInfo],
     iterations: usize,
@@ -56,34 +63,35 @@ fn run_dos(
 
     let mut data = Vec::new();
 
-    if !nodes.is_empty() {
-        let source = thread_rng().gen_range(0, nodes.len());
-        let mut contact = nodes[source].clone();
-        contact.id = solana_sdk::pubkey::new_rand();
-        match data_type.as_str() {
-            "repair_highest" => {
-                let slot = 100;
-                let req = RepairProtocol::WindowIndexWithNonce(contact, slot, 0, 0);
-                data = bincode::serialize(&req).unwrap();
-            }
-            "repair_shred" => {
-                let slot = 100;
-                let req = RepairProtocol::HighestWindowIndexWithNonce(contact, slot, 0, 0);
-                data = bincode::serialize(&req).unwrap();
-            }
-            "repair_orphan" => {
-                let slot = 100;
-                let req = RepairProtocol::OrphanWithNonce(contact, slot, 0);
-                data = bincode::serialize(&req).unwrap();
-            }
-            "random" => {
-                data.resize(data_size, 0);
-            }
-            "get_account_info" => {}
-            "get_program_accounts" => {}
-            &_ => {
-                panic!("unknown data type");
-            }
+    match data_type.as_str() {
+        "repair_highest" => {
+            let slot = 100;
+            let req = RepairProtocol::WindowIndexWithNonce(get_repair_contact(nodes), slot, 0, 0);
+            data = bincode::serialize(&req).unwrap();
+        }
+        "repair_shred" => {
+            let slot = 100;
+            let req =
+                RepairProtocol::HighestWindowIndexWithNonce(get_repair_contact(nodes), slot, 0, 0);
+            data = bincode::serialize(&req).unwrap();
+        }
+        "repair_orphan" => {
+            let slot = 100;
+            let req = RepairProtocol::OrphanWithNonce(get_repair_contact(nodes), slot, 0);
+            data = bincode::serialize(&req).unwrap();
+        }
+        "random" => {
+            data.resize(data_size, 0);
+        }
+        "transaction" => {
+            let tx = solana_perf::test_tx::test_tx();
+            info!("{:?}", tx);
+            data = bincode::serialize(&tx).unwrap();
+        }
+        "get_account_info" => {}
+        "get_program_accounts" => {}
+        &_ => {
+            panic!("unknown data type");
         }
     }
 
@@ -183,6 +191,7 @@ fn main() {
                     "random",
                     "get_account_info",
                     "get_program_accounts",
+                    "transaction",
                 ])
                 .help("Type of data to send"),
         )

download-utils/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-download-utils"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Download Utils"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -15,8 +15,8 @@ console = "0.14.1"
 indicatif = "0.15.0"
 log = "0.4.11"
 reqwest = { version = "0.11.2", default-features = false, features = ["blocking", "rustls-tls", "json"] }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
 tar = "0.4.37"
 
 [lib]

faucet/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-faucet"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Faucet"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -16,12 +16,12 @@ clap = "2.33"
 log = "0.4.11"
 serde = "1.0.122"
 serde_derive = "1.0.103"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-cli-config = { path = "../cli-config", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-cli-config = { path = "../cli-config", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 spl-memo = { version = "=3.0.1", features = ["no-entrypoint"] }
 thiserror = "1.0"
 tokio = { version = "1", features = ["full"] }

frozen-abi/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-frozen-abi"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Frozen ABI"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -16,11 +16,11 @@ log = "0.4.11"
 serde = "1.0.122"
 serde_derive = "1.0.103"
 sha2 = "0.9.2"
-solana-frozen-abi-macro = { path = "macro", version = "=1.7.13" }
+solana-frozen-abi-macro = { path = "macro", version = "=1.8.0" }
 thiserror = "1.0"
 
 [target.'cfg(not(target_arch = "bpf"))'.dependencies]
-solana-logger = { path = "../logger", version = "=1.7.13" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
 generic-array = { version = "0.14.3", default-features = false, features = ["serde", "more_lengths"]}
 memmap2 = "0.1.0"
 

frozen-abi/macro/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-frozen-abi-macro"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Frozen ABI Macro"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"

genesis-utils/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-genesis-utils"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Genesis Utils"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,9 +10,9 @@ documentation = "https://docs.rs/solana-download-utils"
 edition = "2018"
 
 [dependencies]
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-download-utils = { path = "../download-utils", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-download-utils = { path = "../download-utils", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
 
 [lib]
 crate-type = ["lib"]

genesis/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-genesis"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -16,16 +16,16 @@ chrono = "0.4"
 serde = "1.0.122"
 serde_json = "1.0.56"
 serde_yaml = "0.8.13"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-cli-config = { path = "../cli-config", version = "=1.7.13" }
-solana-exchange-program = { path = "../programs/exchange", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-stake-program = { path = "../programs/stake", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-cli-config = { path = "../cli-config", version = "=1.8.0" }
+solana-exchange-program = { path = "../programs/exchange", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-stake-program = { path = "../programs/stake", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 tempfile = "3.1.0"
 
 [[bin]]

gossip/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-gossip"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -26,22 +26,22 @@ rayon = "1.5.0"
 serde = "1.0.122"
 serde_bytes = "0.11"
 serde_derive = "1.0.103"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-frozen-abi = { path = "../frozen-abi", version = "=1.7.13" }
-solana-frozen-abi-macro = { path = "../frozen-abi/macro", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-net-utils = { path = "../net-utils", version = "=1.7.13" }
-solana-perf = { path = "../perf", version = "=1.7.13" }
-solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-frozen-abi = { path = "../frozen-abi", version = "=1.8.0" }
+solana-frozen-abi-macro = { path = "../frozen-abi/macro", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-net-utils = { path = "../net-utils", version = "=1.8.0" }
+solana-perf = { path = "../perf", version = "=1.8.0" }
+solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 thiserror = "1.0"
 
 [dev-dependencies]

install/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-install"
 description = "The solana cluster software installer"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -25,12 +25,12 @@ reqwest = { version = "0.11.2", default-features = false, features = ["blocking"
 serde = { version = "1.0.122", features = ["derive"] }
 serde_json = "1.0.62"
 serde_yaml = "0.8.13"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-config-program = { path = "../programs/config", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-config-program = { path = "../programs/config", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 semver = "0.9.0"
 tar = "0.4.37"
 tempfile = "3.1.0"

install/src/command.rs

@@ -436,7 +436,7 @@ fn add_to_path(new_path: &str) -> bool {
 
 #[cfg(unix)]
 fn add_to_path(new_path: &str) -> bool {
-    let shell_export_string = format!(r#"export PATH="{}:$PATH""#, new_path);
+    let shell_export_string = format!("\nexport PATH=\"{}:$PATH\"", new_path);
     let mut modified_rcfiles = false;
 
     // Look for sh, bash, and zsh rc files

keygen/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-keygen"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana key generation utility"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -14,11 +14,11 @@ bs58 = "0.3.1"
 clap = "2.33"
 dirs-next = "2.0.0"
 num_cpus = "1.13.0"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-cli-config = { path = "../cli-config", version = "=1.7.13" }
-solana-remote-wallet = { path = "../remote-wallet", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-cli-config = { path = "../cli-config", version = "=1.8.0" }
+solana-remote-wallet = { path = "../remote-wallet", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 tiny-bip39 = "0.8.1"
 
 [[bin]]

ledger-tool/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-ledger-tool"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -14,6 +14,7 @@ bs58 = "0.3.1"
 bytecount = "0.6.0"
 clap = "2.33.1"
 csv = "1.1.3"
+dashmap = "4.0.2"
 futures = "0.3.8"
 futures-util = "0.3.5"
 histogram = "*"
@@ -23,18 +24,19 @@ regex = "1"
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0.56"
 serde_yaml = "0.8.13"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-cli-output = { path = "../cli-output", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-stake-program = { path = "../programs/stake", version = "=1.7.13" }
-solana-storage-bigtable = { path = "../storage-bigtable", version = "=1.7.13" }
-solana-transaction-status = { path = "../transaction-status", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-cli-output = { path = "../cli-output", version = "=1.8.0" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-stake-program = { path = "../programs/stake", version = "=1.8.0" }
+solana-storage-bigtable = { path = "../storage-bigtable", version = "=1.8.0" }
+solana-transaction-status = { path = "../transaction-status", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 tempfile = "3.1.0"
 tokio = { version = "1", features = ["full"] }
 

ledger-tool/src/main.rs

@@ -3,6 +3,7 @@ use clap::{
     crate_description, crate_name, value_t, value_t_or_exit, values_t_or_exit, App, AppSettings,
     Arg, ArgMatches, SubCommand,
 };
+use dashmap::DashMap;
 use itertools::Itertools;
 use log::*;
 use regex::Regex;
@@ -14,6 +15,9 @@ use solana_clap_utils::{
         is_parsable, is_pubkey, is_pubkey_or_keypair, is_slot, is_valid_percentage,
     },
 };
+use solana_core::cost_model::CostModel;
+use solana_core::cost_tracker::CostTracker;
+use solana_core::cost_tracker_stats::CostTrackerStats;
 use solana_ledger::entry::Entry;
 use solana_ledger::{
     ancestor_iterator::AncestorIterator,
@@ -57,7 +61,7 @@ use std::{
     path::{Path, PathBuf},
     process::{exit, Command, Stdio},
     str::FromStr,
-    sync::Arc,
+    sync::{Arc, RwLock},
 };
 
 mod bigtable;
@@ -726,6 +730,62 @@ fn load_bank_forks(
     )
 }
 
+fn compute_slot_cost(blockstore: &Blockstore, slot: Slot) -> Result<(), String> {
+    if blockstore.is_dead(slot) {
+        return Err("Dead slot".to_string());
+    }
+
+    let (entries, _num_shreds, _is_full) = blockstore
+        .get_slot_entries_with_shred_info(slot, 0, false)
+        .map_err(|err| format!(" Slot: {}, Failed to load entries, err {:?}", slot, err))?;
+
+    let mut transactions = 0;
+    let mut programs = 0;
+    let mut program_ids = HashMap::new();
+    let mut cost_model = CostModel::default();
+    cost_model.initialize_cost_table(&blockstore.read_program_costs().unwrap());
+    let cost_model = Arc::new(RwLock::new(cost_model));
+    let mut cost_tracker = CostTracker::new(cost_model.clone());
+    let mut cost_tracker_stats = CostTrackerStats::default();
+
+    for entry in &entries {
+        transactions += entry.transactions.len();
+        let mut cost_model = cost_model.write().unwrap();
+        for transaction in &entry.transactions {
+            programs += transaction.message().instructions.len();
+            let tx_cost = cost_model.calculate_cost(transaction, true);
+            if cost_tracker
+                .try_add(tx_cost, &mut cost_tracker_stats)
+                .is_err()
+            {
+                println!(
+                    "Slot: {}, CostModel rejected transaction {:?}, stats {:?}!",
+                    slot,
+                    transaction,
+                    cost_tracker.get_stats()
+                );
+            }
+            for instruction in &transaction.message().instructions {
+                let program_id =
+                    transaction.message().account_keys[instruction.program_id_index as usize];
+                *program_ids.entry(program_id).or_insert(0) += 1;
+            }
+        }
+    }
+
+    println!(
+        "Slot: {}, Entries: {}, Transactions: {}, Programs {}, {:?}",
+        slot,
+        entries.len(),
+        transactions,
+        programs,
+        cost_tracker.get_stats()
+    );
+    println!("  Programs: {:?}", program_ids);
+
+    Ok(())
+}
+
 fn open_genesis_config_by(ledger_path: &Path, matches: &ArgMatches<'_>) -> GenesisConfig {
     let max_genesis_archive_unpacked_size =
         value_t_or_exit!(matches, "max_genesis_archive_unpacked_size", u64);
@@ -1413,6 +1473,20 @@ fn main() {
                 .about("Output statistics in JSON format about \
                         all column families in the ledger rocksdb")
         )
+        .subcommand(
+            SubCommand::with_name("compute-slot-cost")
+            .about("runs cost_model over the block at the given slots, \
+                   computes how expensive a block was based on cost_model")
+            .arg(
+                Arg::with_name("slots")
+                    .index(1)
+                    .value_name("SLOTS")
+                    .validator(is_slot)
+                    .multiple(true)
+                    .takes_value(true)
+                    .help("Slots that their blocks are computed for cost, default to all slots in ledger"),
+            )
+        )
         .get_matches();
 
     info!("{} {}", crate_name!(), solana_version::version!());
@@ -2345,15 +2419,16 @@ fn main() {
                             skipped_reasons: String,
                         }
                         use solana_stake_program::stake_state::InflationPointCalculationEvent;
-                        let mut stake_calcuration_details: HashMap<Pubkey, CalculationDetail> =
-                            HashMap::new();
-                        let mut last_point_value = None;
+                        let stake_calculation_details: DashMap<Pubkey, CalculationDetail> =
+                            DashMap::new();
+                        let last_point_value = Arc::new(RwLock::new(None));
                         let tracer = |event: &RewardCalculationEvent| {
                             // Currently RewardCalculationEvent enum has only Staking variant
                             // because only staking tracing is supported!
                             #[allow(irrefutable_let_patterns)]
                             if let RewardCalculationEvent::Staking(pubkey, event) = event {
-                                let detail = stake_calcuration_details.entry(**pubkey).or_default();
+                                let mut detail =
+                                    stake_calculation_details.entry(**pubkey).or_default();
                                 match event {
                                     InflationPointCalculationEvent::CalculatedPoints(
                                         epoch,
@@ -2378,12 +2453,11 @@ fn main() {
                                         detail.point_value = Some(point_value.clone());
                                         // we have duplicate copies of `PointValue`s for possible
                                         // miscalculation; do some minimum sanity check
-                                        let point_value = detail.point_value.clone();
-                                        if point_value.is_some() {
-                                            if last_point_value.is_some() {
-                                                assert_eq!(last_point_value, point_value,);
-                                            }
-                                            last_point_value = point_value;
+                                        let mut last_point_value = last_point_value.write().unwrap();
+                                        if let Some(last_point_value) = last_point_value.as_ref() {
+                                            assert_eq!(last_point_value, point_value);
+                                        } else {
+                                            *last_point_value = Some(point_value.clone());
                                         }
                                     }
                                     InflationPointCalculationEvent::EffectiveStakeAtRewardedEpoch(stake) => {
@@ -2488,16 +2562,17 @@ fn main() {
                             },
                         );
 
-                        let mut unchanged_accounts = stake_calcuration_details
-                            .keys()
+                        let mut unchanged_accounts = stake_calculation_details
+                            .iter()
+                            .map(|entry| *entry.key())
                             .collect::<HashSet<_>>()
                             .difference(
                                 &rewarded_accounts
                                     .iter()
-                                    .map(|(pubkey, ..)| *pubkey)
+                                    .map(|(pubkey, ..)| **pubkey)
                                     .collect(),
                             )
-                            .map(|pubkey| (**pubkey, warped_bank.get_account(pubkey).unwrap()))
+                            .map(|pubkey| (*pubkey, warped_bank.get_account(pubkey).unwrap()))
                             .collect::<Vec<_>>();
                         unchanged_accounts.sort_unstable_by_key(|(pubkey, account)| {
                             (*account.owner(), account.lamports(), *pubkey)
@@ -2518,7 +2593,9 @@ fn main() {
 
                             if let Some(base_account) = base_bank.get_account(&pubkey) {
                                 let delta = warped_account.lamports() - base_account.lamports();
-                                let detail = stake_calcuration_details.get(&pubkey);
+                                let detail_ref = stake_calculation_details.get(&pubkey);
+                                let detail: Option<&CalculationDetail> =
+                                    detail_ref.as_ref().map(|detail_ref| detail_ref.value());
                                 println!(
                                     "{:<45}({}): {} => {} (+{} {:>4.9}%) {:?}",
                                     format!("{}", pubkey), // format! is needed to pad/justify correctly.
@@ -2641,10 +2718,18 @@ fn main() {
                                             ),
                                             commission: format_or_na(detail.map(|d| d.commission)),
                                             cluster_rewards: format_or_na(
-                                                last_point_value.as_ref().map(|pv| pv.rewards),
+                                                last_point_value
+                                                    .read()
+                                                    .unwrap()
+                                                    .clone()
+                                                    .map(|pv| pv.rewards),
                                             ),
                                             cluster_points: format_or_na(
-                                                last_point_value.as_ref().map(|pv| pv.points),
+                                                last_point_value
+                                                    .read()
+                                                    .unwrap()
+                                                    .clone()
+                                                    .map(|pv| pv.points),
                                             ),
                                             old_capitalization: base_bank.capitalization(),
                                             new_capitalization: warped_bank.capitalization(),
@@ -2952,6 +3037,28 @@ fn main() {
             ));
             println!("Ok.");
         }
+        ("compute-slot-cost", Some(arg_matches)) => {
+            let blockstore = open_blockstore(
+                &ledger_path,
+                AccessType::TryPrimaryThenSecondary,
+                wal_recovery_mode,
+            );
+
+            let mut slots: Vec<u64> = vec![];
+            if !arg_matches.is_present("slots") {
+                if let Ok(metas) = blockstore.slot_meta_iterator(0) {
+                    slots = metas.map(|(slot, _)| slot).collect();
+                }
+            } else {
+                slots = values_t_or_exit!(arg_matches, "slots", Slot);
+            }
+
+            for slot in slots {
+                if let Err(err) = compute_slot_cost(&blockstore, slot) {
+                    eprintln!("{}", err);
+                }
+            }
+        }
         ("", _) => {
             eprintln!("{}", matches.usage());
             exit(1);

ledger/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-ledger"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana ledger"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -33,21 +33,21 @@ rayon = "1.5.0"
 serde = "1.0.122"
 serde_bytes = "0.11.5"
 sha2 = "0.9.2"
-solana-bpf-loader-program = { path = "../programs/bpf_loader", version = "=1.7.13" }
-solana-frozen-abi = { path = "../frozen-abi", version = "=1.7.13" }
-solana-frozen-abi-macro = { path = "../frozen-abi/macro", version = "=1.7.13" }
-solana-transaction-status = { path = "../transaction-status", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-merkle-tree = { path = "../merkle-tree", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-perf = { path = "../perf", version = "=1.7.13" }
-solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-storage-bigtable = { path = "../storage-bigtable", version = "=1.7.13" }
-solana-storage-proto = { path = "../storage-proto", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-bpf-loader-program = { path = "../programs/bpf_loader", version = "=1.8.0" }
+solana-frozen-abi = { path = "../frozen-abi", version = "=1.8.0" }
+solana-frozen-abi-macro = { path = "../frozen-abi/macro", version = "=1.8.0" }
+solana-transaction-status = { path = "../transaction-status", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-merkle-tree = { path = "../merkle-tree", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-perf = { path = "../perf", version = "=1.8.0" }
+solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-storage-bigtable = { path = "../storage-bigtable", version = "=1.8.0" }
+solana-storage-proto = { path = "../storage-proto", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 tempfile = "3.1.0"
 thiserror = "1.0"
 tokio = { version = "1", features = ["full"] }
@@ -72,7 +72,7 @@ features = ["lz4"]
 [dev-dependencies]
 assert_matches = "1.3.0"
 matches = "0.1.6"
-solana-account-decoder = { path = "../account-decoder", version = "=1.7.13" }
+solana-account-decoder = { path = "../account-decoder", version = "=1.8.0" }
 
 [build-dependencies]
 rustc_version = "0.2"

ledger/src/block_cost_limits.rs

@@ -0,0 +1,56 @@
+//! defines block cost related limits
+//!
+use lazy_static::lazy_static;
+use solana_sdk::{
+    feature, incinerator, native_loader, pubkey::Pubkey, secp256k1_program, system_program,
+};
+use std::collections::HashMap;
+
+/// Static configurations:
+///
+/// Number of microseconds replaying a block should take, 400 millisecond block times
+/// is curerntly publicly communicated on solana.com
+pub const MAX_BLOCK_REPLAY_TIME_US: u64 = 400_000;
+/// number of concurrent processes,
+pub const MAX_CONCURRENCY: u64 = 10;
+
+/// Cluster data, method of collecting at https://github.com/solana-labs/solana/issues/19627
+///
+/// cluster avergaed compute unit to microsec conversion rate
+pub const COMPUTE_UNIT_TO_US_RATIO: u64 = 40;
+/// Number of compute units for one signature verification.
+pub const SIGNATURE_COST: u64 = COMPUTE_UNIT_TO_US_RATIO * 175;
+/// Number of compute units for one write lock
+pub const WRITE_LOCK_UNITS: u64 = COMPUTE_UNIT_TO_US_RATIO * 20;
+/// Number of data bytes per compute units
+pub const DATA_BYTES_UNITS: u64 = 220 /*bytes per us*/ / COMPUTE_UNIT_TO_US_RATIO;
+// Number of compute units for each built-in programs
+lazy_static! {
+    /// Number of compute units for each built-in programs
+    pub static ref BUILT_IN_INSTRUCTION_COSTS: HashMap<Pubkey, u64> = [
+        (feature::id(), COMPUTE_UNIT_TO_US_RATIO * 2),
+        (incinerator::id(), COMPUTE_UNIT_TO_US_RATIO * 2),
+        (native_loader::id(), COMPUTE_UNIT_TO_US_RATIO * 2),
+        (solana_sdk::stake::config::id(), COMPUTE_UNIT_TO_US_RATIO * 2),
+        (solana_sdk::stake::program::id(), COMPUTE_UNIT_TO_US_RATIO * 50),
+        (solana_vote_program::id(), COMPUTE_UNIT_TO_US_RATIO * 200),
+        (secp256k1_program::id(), COMPUTE_UNIT_TO_US_RATIO * 4),
+        (system_program::id(), COMPUTE_UNIT_TO_US_RATIO * 15),
+    ]
+    .iter()
+    .cloned()
+    .collect();
+}
+
+/// Statically computed data:
+///
+/// Number of compute units that a block is allowed. A block's compute units are
+/// accumualted by Transactions added to it; A transaction's compute units are
+/// calculated by cost_model, based on transaction's signarures, write locks,
+/// data size and built-in and BPF instructinos.
+pub const MAX_BLOCK_UNITS: u64 =
+    MAX_BLOCK_REPLAY_TIME_US * COMPUTE_UNIT_TO_US_RATIO * MAX_CONCURRENCY;
+/// Number of compute units that a writable account in a block is allowed. The
+/// limit is to prevent too many transactions write to same account, threrefore
+/// reduce block's paralellism.
+pub const MAX_WRITABLE_ACCOUNT_UNITS: u64 = MAX_BLOCK_REPLAY_TIME_US * COMPUTE_UNIT_TO_US_RATIO;

ledger/src/blockstore.rs

@@ -2701,6 +2701,26 @@ impl Blockstore {
         self.perf_samples_cf.put(index, perf_sample)
     }
 
+    pub fn read_program_costs(&self) -> Result<Vec<(Pubkey, u64)>> {
+        Ok(self
+            .db
+            .iter::<cf::ProgramCosts>(IteratorMode::End)?
+            .map(|(pubkey, data)| {
+                let program_cost: ProgramCost = deserialize(&data).unwrap();
+                (pubkey, program_cost.cost)
+            })
+            .collect())
+    }
+
+    pub fn write_program_cost(&self, key: &Pubkey, value: &u64) -> Result<()> {
+        self.program_costs_cf
+            .put(*key, &ProgramCost { cost: *value })
+    }
+
+    pub fn delete_program_cost(&self, key: &Pubkey) -> Result<()> {
+        self.program_costs_cf.delete(*key)
+    }
+
     /// Returns the entry vector for the slot starting with `shred_start_index`
     pub fn get_slot_entries(&self, slot: Slot, shred_start_index: u64) -> Result<Vec<Entry>> {
         self.get_slot_entries_with_shred_info(slot, shred_start_index, false)
@@ -8871,4 +8891,126 @@ pub mod tests {
 
         Blockstore::destroy(&blockstore_path).expect("Expected successful database destruction");
     }
+
+    #[test]
+    fn test_read_write_cost_table() {
+        let blockstore_path = get_tmp_ledger_path!();
+        {
+            let blockstore = Blockstore::open(&blockstore_path).unwrap();
+            let num_entries: usize = 10;
+            let mut cost_table: HashMap<Pubkey, u64> = HashMap::new();
+            for x in 1..num_entries + 1 {
+                cost_table.insert(Pubkey::new_unique(), (x + 100) as u64);
+            }
+
+            // write to db
+            for (key, cost) in cost_table.iter() {
+                blockstore
+                    .write_program_cost(key, cost)
+                    .expect("write a program");
+            }
+
+            // read back from db
+            let read_back = blockstore.read_program_costs().expect("read programs");
+            // verify
+            assert_eq!(read_back.len(), cost_table.len());
+            for (read_key, read_cost) in read_back {
+                assert_eq!(read_cost, *cost_table.get(&read_key).unwrap());
+            }
+
+            // update value, write to db
+            for val in cost_table.values_mut() {
+                *val += 100;
+            }
+            for (key, cost) in cost_table.iter() {
+                blockstore
+                    .write_program_cost(key, cost)
+                    .expect("write a program");
+            }
+            // add a new record
+            let new_program_key = Pubkey::new_unique();
+            let new_program_cost = 999;
+            blockstore
+                .write_program_cost(&new_program_key, &new_program_cost)
+                .unwrap();
+
+            // confirm value updated
+            let read_back = blockstore.read_program_costs().expect("read programs");
+            // verify
+            assert_eq!(read_back.len(), cost_table.len() + 1);
+            for (key, cost) in cost_table.iter() {
+                assert_eq!(*cost, read_back.iter().find(|(k, _v)| k == key).unwrap().1);
+            }
+            assert_eq!(
+                new_program_cost,
+                read_back
+                    .iter()
+                    .find(|(k, _v)| *k == new_program_key)
+                    .unwrap()
+                    .1
+            );
+
+            // test delete
+            blockstore
+                .delete_program_cost(&new_program_key)
+                .expect("delete a progrma");
+            let read_back = blockstore.read_program_costs().expect("read programs");
+            // verify
+            assert_eq!(read_back.len(), cost_table.len());
+            for (read_key, read_cost) in read_back {
+                assert_eq!(read_cost, *cost_table.get(&read_key).unwrap());
+            }
+        }
+        Blockstore::destroy(&blockstore_path).expect("Expected successful database destruction");
+    }
+
+    #[test]
+    fn test_delete_old_records_from_cost_table() {
+        let blockstore_path = get_tmp_ledger_path!();
+        {
+            let blockstore = Blockstore::open(&blockstore_path).unwrap();
+            let num_entries: usize = 10;
+            let mut cost_table: HashMap<Pubkey, u64> = HashMap::new();
+            for x in 1..num_entries + 1 {
+                cost_table.insert(Pubkey::new_unique(), (x + 100) as u64);
+            }
+
+            // write to db
+            for (key, cost) in cost_table.iter() {
+                blockstore
+                    .write_program_cost(key, cost)
+                    .expect("write a program");
+            }
+
+            // remove a record
+            let mut removed_key = Pubkey::new_unique();
+            for (key, cost) in cost_table.iter() {
+                if *cost == 101_u64 {
+                    removed_key = *key;
+                    break;
+                }
+            }
+            cost_table.remove(&removed_key);
+
+            // delete records from blockstore if they are no longer in cost_table
+            let db_records = blockstore.read_program_costs().expect("read programs");
+            db_records.iter().for_each(|(pubkey, _)| {
+                if !cost_table.iter().any(|(key, _)| key == pubkey) {
+                    assert_eq!(*pubkey, removed_key);
+                    blockstore
+                        .delete_program_cost(pubkey)
+                        .expect("delete old program");
+                }
+            });
+
+            // read back from db
+            let read_back = blockstore.read_program_costs().expect("read programs");
+            // verify
+            assert_eq!(read_back.len(), cost_table.len());
+            for (read_key, read_cost) in read_back {
+                assert_eq!(read_cost, *cost_table.get(&read_key).unwrap());
+            }
+        }
+        Blockstore::destroy(&blockstore_path).expect("Expected successful database destruction");
+    }
 }

ledger/src/blockstore_db.rs

@@ -537,6 +537,11 @@ impl Rocks {
         Ok(())
     }
 
+    fn delete_cf(&self, cf: &ColumnFamily, key: &[u8]) -> Result<()> {
+        self.0.delete_cf(cf, key)?;
+        Ok(())
+    }
+
     fn iterator_cf<C>(&self, cf: &ColumnFamily, iterator_mode: IteratorMode<C::Index>) -> DBIterator
     where
         C: Column,
@@ -1217,6 +1222,10 @@ where
         self.backend
             .put_cf(self.handle(), &C::key(key), &serialized_value)
     }
+
+    pub fn delete(&self, key: C::Index) -> Result<()> {
+        self.backend.delete_cf(self.handle(), &C::key(key))
+    }
 }
 
 impl<C> LedgerColumn<C>
@@ -1364,7 +1373,7 @@ fn get_cf_options<C: 'static + Column + ColumnName>(
     options.set_max_bytes_for_level_base(total_size_base);
     options.set_target_file_size_base(file_size_base);
 
-    // TransactionStatusIndex must be excluded from LedgerCleanupService's rocksdb
+    // TransactionStatusIndex and ProgramCosts must be excluded from LedgerCleanupService's rocksdb
     // compactions....
     if matches!(access_type, AccessType::PrimaryOnly) && !excludes_from_compaction(C::NAME) {
         options.set_compaction_filter_factory(PurgedSlotFilterFactory::<C> {

ledger/src/blockstore_processor.rs

@@ -1,4 +1,5 @@
 use crate::{
+    block_cost_limits::*,
     block_error::BlockError,
     blockstore::Blockstore,
     blockstore_db::BlockstoreError,
@@ -32,6 +33,7 @@ use solana_runtime::{
 };
 use solana_sdk::{
     clock::{Slot, MAX_PROCESSING_AGE},
+    feature_set,
     genesis_config::GenesisConfig,
     hash::Hash,
     pubkey::Pubkey,
@@ -48,11 +50,40 @@ use std::{
     collections::{HashMap, HashSet},
     path::PathBuf,
     result,
-    sync::Arc,
+    sync::{Arc, RwLock},
     time::{Duration, Instant},
 };
 use thiserror::Error;
 
+// it tracks the block cost available capacity - number of compute-units allowed
+// by max blockl cost limit
+#[derive(Debug)]
+pub struct BlockCostCapacityMeter {
+    pub capacity: u64,
+    pub accumulated_cost: u64,
+}
+
+impl Default for BlockCostCapacityMeter {
+    fn default() -> Self {
+        BlockCostCapacityMeter::new(MAX_BLOCK_UNITS)
+    }
+}
+
+impl BlockCostCapacityMeter {
+    pub fn new(capacity_limit: u64) -> Self {
+        Self {
+            capacity: capacity_limit,
+            accumulated_cost: 0_u64,
+        }
+    }
+
+    // return the remaining capacity
+    pub fn accumulate(&mut self, cost: u64) -> u64 {
+        self.accumulated_cost += cost;
+        self.capacity.saturating_sub(self.accumulated_cost)
+    }
+}
+
 pub type BlockstoreProcessorResult =
     result::Result<(BankForks, LeaderScheduleCache), BlockstoreProcessorError>;
 
@@ -100,12 +131,26 @@ fn get_first_error(
     first_err
 }
 
+fn aggregate_total_execution_units(execute_timings: &ExecuteTimings) -> u64 {
+    let mut execute_cost_units: u64 = 0;
+    for (program_id, timing) in &execute_timings.details.per_program_timings {
+        if timing.count < 1 {
+            continue;
+        }
+        execute_cost_units =
+            execute_cost_units.saturating_add(timing.accumulated_units / timing.count as u64);
+        trace!("aggregated execution cost of {:?} {:?}", program_id, timing);
+    }
+    execute_cost_units
+}
+
 fn execute_batch(
     batch: &TransactionBatch,
     bank: &Arc<Bank>,
     transaction_status_sender: Option<&TransactionStatusSender>,
     replay_vote_sender: Option<&ReplayVoteSender>,
     timings: &mut ExecuteTimings,
+    cost_capacity_meter: Arc<RwLock<BlockCostCapacityMeter>>,
 ) -> Result<()> {
     let record_token_balances = transaction_status_sender.is_some();
 
@@ -117,6 +162,8 @@ fn execute_batch(
         vec![]
     };
 
+    let pre_process_units: u64 = aggregate_total_execution_units(timings);
+
     let (tx_results, balances, inner_instructions, transaction_logs) =
         batch.bank().load_execute_and_commit_transactions(
             batch,
@@ -127,6 +174,29 @@ fn execute_batch(
             timings,
         );
 
+    if bank
+        .feature_set
+        .is_active(&feature_set::gate_large_block::id())
+    {
+        let execution_cost_units = aggregate_total_execution_units(timings) - pre_process_units;
+        let remaining_block_cost_cap = cost_capacity_meter
+            .write()
+            .unwrap()
+            .accumulate(execution_cost_units);
+
+        debug!(
+            "bank {} executed a batch, number of transactions {}, total execute cu {}, remaining block cost cap {}",
+            bank.slot(),
+            batch.hashed_transactions().len(),
+            execution_cost_units,
+            remaining_block_cost_cap,
+        );
+
+        if remaining_block_cost_cap == 0_u64 {
+            return Err(TransactionError::WouldExceedMaxBlockCostLimit);
+        }
+    }
+
     bank_utils::find_and_send_votes(batch.hashed_transactions(), &tx_results, replay_vote_sender);
 
     let TransactionResults {
@@ -170,6 +240,7 @@ fn execute_batches(
     transaction_status_sender: Option<&TransactionStatusSender>,
     replay_vote_sender: Option<&ReplayVoteSender>,
     timings: &mut ExecuteTimings,
+    cost_capacity_meter: Arc<RwLock<BlockCostCapacityMeter>>,
 ) -> Result<()> {
     inc_new_counter_debug!("bank-par_execute_entries-count", batches.len());
     let (results, new_timings): (Vec<Result<()>>, Vec<ExecuteTimings>) =
@@ -185,6 +256,7 @@ fn execute_batches(
                             transaction_status_sender,
                             replay_vote_sender,
                             &mut timings,
+                            cost_capacity_meter.clone(),
                         );
                         if let Some(entry_callback) = entry_callback {
                             entry_callback(bank);
@@ -226,6 +298,7 @@ pub fn process_entries(
         transaction_status_sender,
         replay_vote_sender,
         &mut timings,
+        Arc::new(RwLock::new(BlockCostCapacityMeter::default())),
     );
 
     debug!("process_entries: {:?}", timings);
@@ -241,6 +314,7 @@ fn process_entries_with_callback(
     transaction_status_sender: Option<&TransactionStatusSender>,
     replay_vote_sender: Option<&ReplayVoteSender>,
     timings: &mut ExecuteTimings,
+    cost_capacity_meter: Arc<RwLock<BlockCostCapacityMeter>>,
 ) -> Result<()> {
     // accumulator for entries that can be processed in parallel
     let mut batches = vec![];
@@ -262,6 +336,7 @@ fn process_entries_with_callback(
                         transaction_status_sender,
                         replay_vote_sender,
                         timings,
+                        cost_capacity_meter.clone(),
                     )?;
                     batches.clear();
                     for hash in &tick_hashes {
@@ -313,6 +388,7 @@ fn process_entries_with_callback(
                             transaction_status_sender,
                             replay_vote_sender,
                             timings,
+                            cost_capacity_meter.clone(),
                         )?;
                         batches.clear();
                     }
@@ -327,6 +403,7 @@ fn process_entries_with_callback(
         transaction_status_sender,
         replay_vote_sender,
         timings,
+        cost_capacity_meter,
     )?;
     for hash in tick_hashes {
         bank.register_tick(hash);
@@ -806,6 +883,7 @@ pub fn confirm_slot(
     let mut entries = check_result.unwrap();
     let mut replay_elapsed = Measure::start("replay_elapsed");
     let mut execute_timings = ExecuteTimings::default();
+    let cost_capacity_meter = Arc::new(RwLock::new(BlockCostCapacityMeter::default()));
     // Note: This will shuffle entries' transactions in-place.
     let process_result = process_entries_with_callback(
         bank,
@@ -815,6 +893,7 @@ pub fn confirm_slot(
         transaction_status_sender,
         replay_vote_sender,
         &mut execute_timings,
+        cost_capacity_meter,
     )
     .map_err(BlockstoreProcessorError::from);
     replay_elapsed.stop();

ledger/src/lib.rs

@@ -11,6 +11,7 @@ pub mod block_error;
 #[macro_use]
 pub mod blockstore;
 pub mod ancestor_iterator;
+pub mod block_cost_limits;
 pub mod blockstore_db;
 pub mod blockstore_meta;
 pub mod blockstore_processor;

local-cluster/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-local-cluster"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -17,22 +17,22 @@ fs_extra = "1.2.0"
 log = "0.4.11"
 rand = "0.7.0"
 rayon = "1.5.0"
-solana-config-program = { path = "../programs/config", version = "=1.7.13" }
-solana-core = { path = "../core", version = "=1.7.13" }
-solana-client = { path = "../client", version = "=1.7.13" }
-solana-download-utils = { path = "../download-utils", version = "=1.7.13" }
-solana-exchange-program = { path = "../programs/exchange", version = "=1.7.13" }
-solana-faucet = { path = "../faucet", version = "=1.7.13" }
-solana-gossip = { path = "../gossip", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.7.13" }
-solana-rpc = { path = "../rpc", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-stake-program = { path = "../programs/stake", version = "=1.7.13" }
-solana-streamer = { path = "../streamer", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-config-program = { path = "../programs/config", version = "=1.8.0" }
+solana-core = { path = "../core", version = "=1.8.0" }
+solana-client = { path = "../client", version = "=1.8.0" }
+solana-download-utils = { path = "../download-utils", version = "=1.8.0" }
+solana-exchange-program = { path = "../programs/exchange", version = "=1.8.0" }
+solana-faucet = { path = "../faucet", version = "=1.8.0" }
+solana-gossip = { path = "../gossip", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.8.0" }
+solana-rpc = { path = "../rpc", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-stake-program = { path = "../programs/stake", version = "=1.8.0" }
+solana-streamer = { path = "../streamer", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 tempfile = "3.1.0"
 
 [dev-dependencies]

local-cluster/src/validator_configs.rs

@@ -57,6 +57,7 @@ pub fn safe_clone_config(config: &ValidatorConfig) -> ValidatorConfig {
         poh_hashes_per_batch: config.poh_hashes_per_batch,
         no_wait_for_vote_to_start_leader: config.no_wait_for_vote_to_start_leader,
         accounts_shrink_ratio: config.accounts_shrink_ratio,
+        disable_epoch_boundary_optimization: config.disable_epoch_boundary_optimization,
     }
 }
 

log-analyzer/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.com>"]
 edition = "2018"
 name = "solana-log-analyzer"
 description = "The solana cluster network analysis tool"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -14,9 +14,9 @@ byte-unit = "4.0.9"
 clap = "2.33.1"
 serde = "1.0.122"
 serde_json = "1.0.56"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 
 [[bin]]
 name = "solana-log-analyzer"

logger/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-logger"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Logger"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"

measure/Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "solana-measure"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 homepage = "https://solana.com/"
 documentation = "https://docs.rs/solana-measure"
 readme = "../README.md"
@@ -12,8 +12,8 @@ edition = "2018"
 
 [dependencies]
 log = "0.4.11"
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

merkle-root-bench/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-merkle-root-bench"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -10,11 +10,11 @@ publish = false
 
 [dependencies]
 log = "0.4.11"
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 clap = "2.33.1"
 
 [package.metadata.docs.rs]

merkle-tree/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-merkle-tree"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Merkle Tree"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-merkle-tree"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../sdk/program", version = "=1.8.0" }
 fast-math = "0.1"
 
 # This can go once the BPF toolchain target Rust 1.42.0+

metrics/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-metrics"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Metrics"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -15,7 +15,7 @@ gethostname = "0.2.1"
 lazy_static = "1.4.0"
 log = "0.4.11"
 reqwest = { version = "0.11.2", default-features = false, features = ["blocking", "rustls-tls", "json"] }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
 
 [dev-dependencies]
 rand = "0.7.0"

net-shaper/Cargo.toml

@@ -3,7 +3,7 @@ authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-net-shaper"
 description = "The solana cluster network shaping tool"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -13,8 +13,8 @@ publish = false
 clap = "2.33.1"
 serde = "1.0.122"
 serde_json = "1.0.56"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
 rand = "0.7.0"
 
 [[bin]]

net-utils/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-net-utils"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Network Utilities"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -18,10 +18,10 @@ rand = "0.7.0"
 serde = "1.0.122"
 serde_derive = "1.0.103"
 socket2 = "0.3.17"
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
 tokio = { version = "1", features = ["full"] }
 url = "2.1.1"
 

net/gce.sh

@@ -805,7 +805,6 @@ $(
     install-earlyoom.sh \
     install-iftop.sh \
     install-libssl-compatability.sh \
-    install-nodejs.sh \
     install-redis.sh \
     install-rsync.sh \
     localtime.sh \

notifier/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-notifier"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Notifier"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"

perf/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-perf"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana Performance APIs"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -19,11 +19,11 @@ log = "0.4.11"
 rand = "0.7.0"
 rayon = "1.5.0"
 serde = "1.0.126"
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-rayon-threadlimit = { path = "../rayon-threadlimit", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 
 [lib]
 name = "solana_perf"

poh-bench/Cargo.toml

@@ -2,7 +2,7 @@
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 edition = "2018"
 name = "solana-poh-bench"
-version = "1.7.13"
+version = "1.8.0"
 repository = "https://github.com/solana-labs/solana"
 license = "Apache-2.0"
 homepage = "https://solana.com/"
@@ -13,13 +13,13 @@ clap = "2.33.1"
 log = "0.4.11"
 rand = "0.7.0"
 rayon = "1.5.0"
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-clap-utils = { path = "../clap-utils", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-version = { path = "../version", version = "=1.7.13" }
-solana-perf = { path = "../perf", version = "=1.7.13" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-clap-utils = { path = "../clap-utils", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-version = { path = "../version", version = "=1.8.0" }
+solana-perf = { path = "../perf", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

poh/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-poh"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana PoH"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -13,20 +13,20 @@ edition = "2018"
 core_affinity = "0.5.10"
 crossbeam-channel = "0.4"
 log = "0.4.11"
-solana-ledger = { path = "../ledger", version = "=1.7.13" }
-solana-measure = { path = "../measure", version = "=1.7.13" }
-solana-metrics = { path = "../metrics", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-sys-tuner = { path = "../sys-tuner", version = "=1.7.13" }
+solana-ledger = { path = "../ledger", version = "=1.8.0" }
+solana-measure = { path = "../measure", version = "=1.8.0" }
+solana-metrics = { path = "../metrics", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-sys-tuner = { path = "../sys-tuner", version = "=1.8.0" }
 thiserror = "1.0"
 
 [dev-dependencies]
 bincode = "1.3.1"
 matches = "0.1.6"
 rand = "0.7.0"
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-perf = { path = "../perf", version = "=1.7.13" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-perf = { path = "../perf", version = "=1.8.0" }
 
 [lib]
 crate-type = ["lib"]

program-test/Cargo.toml

@@ -5,7 +5,7 @@ edition = "2018"
 license = "Apache-2.0"
 name = "solana-program-test"
 repository = "https://github.com/solana-labs/solana"
-version = "1.7.13"
+version = "1.8.0"
 
 [dependencies]
 async-trait = "0.1.42"
@@ -17,13 +17,13 @@ log = "0.4.11"
 mio = "0.7.6"
 serde = "1.0.112"
 serde_derive = "1.0.103"
-solana-banks-client = { path = "../banks-client", version = "=1.7.13" }
-solana-banks-server = { path = "../banks-server", version = "=1.7.13" }
-solana-bpf-loader-program = { path = "../programs/bpf_loader", version = "=1.7.13" }
-solana-logger = { path = "../logger", version = "=1.7.13" }
-solana-runtime = { path = "../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../sdk", version = "=1.7.13" }
-solana-vote-program = { path = "../programs/vote", version = "=1.7.13" }
+solana-banks-client = { path = "../banks-client", version = "=1.8.0" }
+solana-banks-server = { path = "../banks-server", version = "=1.8.0" }
+solana-bpf-loader-program = { path = "../programs/bpf_loader", version = "=1.8.0" }
+solana-logger = { path = "../logger", version = "=1.8.0" }
+solana-runtime = { path = "../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../sdk", version = "=1.8.0" }
+solana-vote-program = { path = "../programs/vote", version = "=1.8.0" }
 thiserror = "1.0"
 tokio = { version = "1", features = ["full"] }
 

program-test/tests/warp.rs

@@ -1,6 +1,5 @@
 #![allow(clippy::integer_arithmetic)]
 use {
-    assert_matches::assert_matches,
     bincode::deserialize,
     solana_banks_client::BanksClient,
     solana_program_test::{processor, ProgramTest, ProgramTestContext, ProgramTestError},
@@ -15,7 +14,7 @@ use {
         signature::{Keypair, Signer},
         stake::{
             instruction as stake_instruction,
-            state::{Authorized, Lockup, StakeState},
+            state::{Authorized, Lockup, StakeActivationStatus, StakeState},
         },
         system_instruction, system_program,
         sysvar::{
@@ -298,11 +297,11 @@ async fn stake_rewards_from_warp() {
     let stake_history: StakeHistory = deserialize(&stake_history_account.data).unwrap();
     let clock: Clock = deserialize(&clock_account.data).unwrap();
     let stake = stake_state.stake().unwrap();
-    assert_matches!(
+    assert_eq!(
         stake
             .delegation
             .stake_activating_and_deactivating(clock.epoch, Some(&stake_history)),
-        (_, 0, 0)
+        StakeActivationStatus::with_effective(stake.delegation.stake),
     );
 }
 

programs/bpf/Cargo.lock

@@ -83,9 +83,9 @@ checksum = "eab1c04a571841102f5345a8fc0f6bb3d31c315dec879b5c6e42e40ce7ffa34e"
 
 [[package]]
 name = "assert_matches"
-version = "1.4.0"
+version = "1.5.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "695579f0f2520f3774bb40461e5adb066459d4e0af4d59d20175484fb8e9edf1"
+checksum = "9b34d609dfbaf33d6889b2b7106d3ca345eacad44200913df5ba02bfd31d2ba9"
 
 [[package]]
 name = "async-trait"
@@ -155,11 +155,10 @@ checksum = "904dfeac50f3cdaba28fc6f57fdcddb75f49ed61346676a78c4ffe55877802fd"
 
 [[package]]
 name = "bincode"
-version = "1.3.1"
+version = "1.3.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f30d3a39baa26f9651f17b375061f3233dde33424a8b72b0dbe93a68a0bc896d"
+checksum = "b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad"
 dependencies = [
- "byteorder 1.3.4",
  "serde",
 ]
 
@@ -272,6 +271,12 @@ version = "0.3.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "476e9cd489f9e121e02ffa6014a8ef220ecb15c05ed23fc34cca13925dc283fb"
 
+[[package]]
+name = "bs58"
+version = "0.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "771fe0050b883fcc3ea2359b1a96bcfbc090b7116eae7c3c512c7a083fdf23d3"
+
 [[package]]
 name = "bumpalo"
 version = "3.3.0"
@@ -2592,12 +2597,12 @@ dependencies = [
 
 [[package]]
 name = "solana-account-decoder"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "Inflector",
  "base64 0.12.3",
  "bincode",
- "bs58",
+ "bs58 0.3.1",
  "bv",
  "lazy_static",
  "serde",
@@ -2613,7 +2618,7 @@ dependencies = [
 
 [[package]]
 name = "solana-banks-client"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "borsh",
@@ -2621,7 +2626,7 @@ dependencies = [
  "futures",
  "mio",
  "solana-banks-interface",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
  "solana-sdk",
  "tarpc",
  "tokio",
@@ -2630,7 +2635,7 @@ dependencies = [
 
 [[package]]
 name = "solana-banks-interface"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "mio",
  "serde",
@@ -2640,7 +2645,7 @@ dependencies = [
 
 [[package]]
 name = "solana-banks-server"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "futures",
@@ -2658,7 +2663,7 @@ dependencies = [
 
 [[package]]
 name = "solana-bpf-loader-program"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "byteorder 1.3.4",
@@ -2677,7 +2682,7 @@ dependencies = [
 
 [[package]]
 name = "solana-bpf-programs"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "byteorder 1.3.4",
@@ -2689,7 +2694,7 @@ dependencies = [
  "solana-account-decoder",
  "solana-bpf-loader-program",
  "solana-cli-output",
- "solana-logger 1.7.13",
+ "solana-logger 1.8.0",
  "solana-measure",
  "solana-runtime",
  "solana-sdk",
@@ -2700,288 +2705,288 @@ dependencies = [
 
 [[package]]
 name = "solana-bpf-rust-128bit"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "solana-bpf-rust-128bit-dep",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-128bit-dep"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-alloc"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-call-depth"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-caller-access"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-custom-heap"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-dep-crate"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "byteorder 1.3.4",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-deprecated-loader"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-dup-accounts"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-error-handling"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "num-derive 0.2.5",
  "num-traits",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
  "thiserror",
 ]
 
 [[package]]
 name = "solana-bpf-rust-external-spend"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-finalize"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-instruction-introspection"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-invoke"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "solana-bpf-rust-invoked",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-invoke-and-error"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-invoke-and-ok"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-invoke-and-return"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-invoked"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-iter"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-many-args"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "solana-bpf-rust-many-args-dep",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-many-args-dep"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-mem"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
  "solana-program-test",
  "solana-sdk",
 ]
 
 [[package]]
 name = "solana-bpf-rust-membuiltins"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "solana-bpf-rust-mem",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-noop"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-panic"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-param-passing"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "solana-bpf-rust-param-passing-dep",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-param-passing-dep"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-rand"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "getrandom 0.1.14",
  "rand 0.7.3",
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-ro-account_modify"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-ro-modify"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-sanity"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-secp256k1-recover"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-sha"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-spoof1"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-spoof1-system"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-sysvar"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
  "solana-program-test",
  "solana-sdk",
 ]
 
 [[package]]
 name = "solana-bpf-rust-upgradeable"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-bpf-rust-upgraded"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "solana-program 1.7.13",
+ "solana-program 1.8.0",
 ]
 
 [[package]]
 name = "solana-clap-utils"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "chrono",
  "clap",
@@ -2996,7 +3001,7 @@ dependencies = [
 
 [[package]]
 name = "solana-cli-config"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "dirs-next",
  "lazy_static",
@@ -3008,7 +3013,7 @@ dependencies = [
 
 [[package]]
 name = "solana-cli-output"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "Inflector",
  "base64 0.13.0",
@@ -3031,11 +3036,11 @@ dependencies = [
 
 [[package]]
 name = "solana-client"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "base64 0.13.0",
  "bincode",
- "bs58",
+ "bs58 0.3.1",
  "clap",
  "indicatif",
  "jsonrpc-core",
@@ -3061,9 +3066,16 @@ dependencies = [
  "url",
 ]
 
+[[package]]
+name = "solana-compute-budget-program"
+version = "1.8.0"
+dependencies = [
+ "solana-sdk",
+]
+
 [[package]]
 name = "solana-config-program"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "chrono",
@@ -3076,7 +3088,7 @@ dependencies = [
 
 [[package]]
 name = "solana-crate-features"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "backtrace",
  "bytes 0.4.12",
@@ -3098,7 +3110,7 @@ dependencies = [
 
 [[package]]
 name = "solana-faucet"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "byteorder 1.3.4",
@@ -3108,7 +3120,7 @@ dependencies = [
  "serde_derive",
  "solana-clap-utils",
  "solana-cli-config",
- "solana-logger 1.7.13",
+ "solana-logger 1.8.0",
  "solana-metrics",
  "solana-sdk",
  "solana-version",
@@ -3123,7 +3135,7 @@ version = "1.7.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "b0b98d31e0662fedf3a1ee30919c655713874d578e19e65affe46109b1b927f9"
 dependencies = [
- "bs58",
+ "bs58 0.3.1",
  "bv",
  "generic-array 0.14.3",
  "log",
@@ -3139,9 +3151,9 @@ dependencies = [
 
 [[package]]
 name = "solana-frozen-abi"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "bs58",
+ "bs58 0.3.1",
  "bv",
  "generic-array 0.14.3",
  "log",
@@ -3150,8 +3162,8 @@ dependencies = [
  "serde",
  "serde_derive",
  "sha2",
- "solana-frozen-abi-macro 1.7.13",
- "solana-logger 1.7.13",
+ "solana-frozen-abi-macro 1.8.0",
+ "solana-logger 1.8.0",
  "thiserror",
 ]
 
@@ -3169,7 +3181,7 @@ dependencies = [
 
 [[package]]
 name = "solana-frozen-abi-macro"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "proc-macro2 1.0.24",
  "quote 1.0.6",
@@ -3190,7 +3202,7 @@ dependencies = [
 
 [[package]]
 name = "solana-logger"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "env_logger",
  "lazy_static",
@@ -3199,7 +3211,7 @@ dependencies = [
 
 [[package]]
 name = "solana-measure"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "log",
  "solana-metrics",
@@ -3208,7 +3220,7 @@ dependencies = [
 
 [[package]]
 name = "solana-metrics"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "env_logger",
  "gethostname",
@@ -3220,7 +3232,7 @@ dependencies = [
 
 [[package]]
 name = "solana-net-utils"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "clap",
@@ -3231,7 +3243,7 @@ dependencies = [
  "serde_derive",
  "socket2 0.3.17",
  "solana-clap-utils",
- "solana-logger 1.7.13",
+ "solana-logger 1.8.0",
  "solana-sdk",
  "solana-version",
  "tokio",
@@ -3248,7 +3260,7 @@ dependencies = [
  "blake3",
  "borsh",
  "borsh-derive",
- "bs58",
+ "bs58 0.3.1",
  "bv",
  "curve25519-dalek 2.1.0",
  "hex",
@@ -3275,13 +3287,13 @@ dependencies = [
 
 [[package]]
 name = "solana-program"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "blake3",
  "borsh",
  "borsh-derive",
- "bs58",
+ "bs58 0.3.1",
  "bv",
  "curve25519-dalek 2.1.0",
  "hex",
@@ -3299,16 +3311,16 @@ dependencies = [
  "serde_derive",
  "sha2",
  "sha3",
- "solana-frozen-abi 1.7.13",
- "solana-frozen-abi-macro 1.7.13",
- "solana-logger 1.7.13",
- "solana-sdk-macro 1.7.13",
+ "solana-frozen-abi 1.8.0",
+ "solana-frozen-abi-macro 1.8.0",
+ "solana-logger 1.8.0",
+ "solana-sdk-macro 1.8.0",
  "thiserror",
 ]
 
 [[package]]
 name = "solana-program-test"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "async-trait",
  "base64 0.12.3",
@@ -3322,7 +3334,7 @@ dependencies = [
  "solana-banks-client",
  "solana-banks-server",
  "solana-bpf-loader-program",
- "solana-logger 1.7.13",
+ "solana-logger 1.8.0",
  "solana-runtime",
  "solana-sdk",
  "solana-vote-program",
@@ -3332,7 +3344,7 @@ dependencies = [
 
 [[package]]
 name = "solana-rayon-threadlimit"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "lazy_static",
  "num_cpus",
@@ -3340,7 +3352,7 @@ dependencies = [
 
 [[package]]
 name = "solana-remote-wallet"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "base32",
  "console 0.14.1",
@@ -3359,7 +3371,7 @@ dependencies = [
 
 [[package]]
 name = "solana-runtime"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "arrayref",
  "bincode",
@@ -3388,10 +3400,11 @@ dependencies = [
  "rustc_version",
  "serde",
  "serde_derive",
+ "solana-compute-budget-program",
  "solana-config-program",
- "solana-frozen-abi 1.7.13",
- "solana-frozen-abi-macro 1.7.13",
- "solana-logger 1.7.13",
+ "solana-frozen-abi 1.8.0",
+ "solana-frozen-abi-macro 1.8.0",
+ "solana-logger 1.8.0",
  "solana-measure",
  "solana-metrics",
  "solana-rayon-threadlimit",
@@ -3408,11 +3421,13 @@ dependencies = [
 
 [[package]]
 name = "solana-sdk"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "assert_matches",
  "bincode",
- "bs58",
+ "borsh",
+ "borsh-derive",
+ "bs58 0.4.0",
  "bv",
  "byteorder 1.3.4",
  "chrono",
@@ -3444,11 +3459,11 @@ dependencies = [
  "sha2",
  "sha3",
  "solana-crate-features",
- "solana-frozen-abi 1.7.13",
- "solana-frozen-abi-macro 1.7.13",
- "solana-logger 1.7.13",
- "solana-program 1.7.13",
- "solana-sdk-macro 1.7.13",
+ "solana-frozen-abi 1.8.0",
+ "solana-frozen-abi-macro 1.8.0",
+ "solana-logger 1.8.0",
+ "solana-program 1.8.0",
+ "solana-sdk-macro 1.8.0",
  "thiserror",
  "uriparse",
 ]
@@ -3459,7 +3474,7 @@ version = "1.7.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "84710ce45a21cccd9f2b09d8e9aad529080bb2540f27b1253874b6e732b465b9"
 dependencies = [
- "bs58",
+ "bs58 0.3.1",
  "proc-macro2 1.0.24",
  "quote 1.0.6",
  "rustversion",
@@ -3468,9 +3483,9 @@ dependencies = [
 
 [[package]]
 name = "solana-sdk-macro"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
- "bs58",
+ "bs58 0.3.1",
  "proc-macro2 1.0.24",
  "quote 1.0.6",
  "rustversion",
@@ -3479,14 +3494,14 @@ dependencies = [
 
 [[package]]
 name = "solana-secp256k1-program"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "solana-sdk",
 ]
 
 [[package]]
 name = "solana-stake-program"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "log",
@@ -3496,8 +3511,8 @@ dependencies = [
  "serde",
  "serde_derive",
  "solana-config-program",
- "solana-frozen-abi 1.7.13",
- "solana-frozen-abi-macro 1.7.13",
+ "solana-frozen-abi 1.8.0",
+ "solana-frozen-abi-macro 1.8.0",
  "solana-metrics",
  "solana-sdk",
  "solana-vote-program",
@@ -3506,12 +3521,12 @@ dependencies = [
 
 [[package]]
 name = "solana-transaction-status"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "Inflector",
  "base64 0.12.3",
  "bincode",
- "bs58",
+ "bs58 0.3.1",
  "lazy_static",
  "serde",
  "serde_derive",
@@ -3528,21 +3543,21 @@ dependencies = [
 
 [[package]]
 name = "solana-version"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "log",
  "rustc_version",
  "serde",
  "serde_derive",
- "solana-frozen-abi 1.7.13",
- "solana-frozen-abi-macro 1.7.13",
- "solana-logger 1.7.13",
+ "solana-frozen-abi 1.8.0",
+ "solana-frozen-abi-macro 1.8.0",
+ "solana-logger 1.8.0",
  "solana-sdk",
 ]
 
 [[package]]
 name = "solana-vote-program"
-version = "1.7.13"
+version = "1.8.0"
 dependencies = [
  "bincode",
  "log",
@@ -3551,9 +3566,9 @@ dependencies = [
  "rustc_version",
  "serde",
  "serde_derive",
- "solana-frozen-abi 1.7.13",
- "solana-frozen-abi-macro 1.7.13",
- "solana-logger 1.7.13",
+ "solana-frozen-abi 1.8.0",
+ "solana-frozen-abi-macro 1.8.0",
+ "solana-logger 1.8.0",
  "solana-metrics",
  "solana-sdk",
  "thiserror",

programs/bpf/Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "solana-bpf-programs"
 description = "Blockchain, Rebuilt for Scale"
-version = "1.7.13"
+version = "1.8.0"
 documentation = "https://docs.rs/solana"
 homepage = "https://solana.com/"
 readme = "README.md"
@@ -26,15 +26,15 @@ itertools = "0.10.0"
 log = "0.4.11"
 miow = "0.2.2"
 net2 = "0.2.37"
-solana-bpf-loader-program = { path = "../bpf_loader", version = "=1.7.13" }
-solana-cli-output = { path = "../../cli-output", version = "=1.7.13" }
-solana-logger = { path = "../../logger", version = "=1.7.13" }
-solana-measure = { path = "../../measure", version = "=1.7.13" }
+solana-bpf-loader-program = { path = "../bpf_loader", version = "=1.8.0" }
+solana-cli-output = { path = "../../cli-output", version = "=1.8.0" }
+solana-logger = { path = "../../logger", version = "=1.8.0" }
+solana-measure = { path = "../../measure", version = "=1.8.0" }
 solana_rbpf = "=0.2.11"
-solana-runtime = { path = "../../runtime", version = "=1.7.13" }
-solana-sdk = { path = "../../sdk", version = "=1.7.13" }
-solana-transaction-status = { path = "../../transaction-status", version = "=1.7.13" }
-solana-account-decoder = { path = "../../account-decoder", version = "=1.7.13" }
+solana-runtime = { path = "../../runtime", version = "=1.8.0" }
+solana-sdk = { path = "../../sdk", version = "=1.8.0" }
+solana-transaction-status = { path = "../../transaction-status", version = "=1.8.0" }
+solana-account-decoder = { path = "../../account-decoder", version = "=1.8.0" }
 
 
 [[bench]]

programs/bpf/rust/128bit/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-128bit"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,8 +10,8 @@ documentation = "https://docs.rs/solana-bpf-rust-128bit"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
-solana-bpf-rust-128bit-dep = { path = "../128bit_dep", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
+solana-bpf-rust-128bit-dep = { path = "../128bit_dep", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/128bit_dep/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-128bit-dep"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-128bit-dep"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [package.metadata.docs.rs]
 targets = ["x86_64-unknown-linux-gnu"]

programs/bpf/rust/alloc/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-alloc"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-alloc"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/call_depth/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-call-depth"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-call-depth"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/caller_access/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-caller-access"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-caller-access"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/custom_heap/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-custom-heap"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-custom-heap"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [features]
 default = ["custom-heap"]

programs/bpf/rust/dep_crate/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-dep-crate"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -11,7 +11,7 @@ edition = "2018"
 
 [dependencies]
 byteorder = { version = "1", default-features = false }
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/deprecated_loader/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-deprecated-loader"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-deprecated-loader"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/dup_accounts/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-dup-accounts"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-dup-accounts"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/error_handling/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-error-handling"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -12,7 +12,7 @@ edition = "2018"
 [dependencies]
 num-derive = "0.2"
 num-traits = "0.2"
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 thiserror = "1.0"
 
 [lib]

programs/bpf/rust/external_spend/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-external-spend"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-external-spend"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/finalize/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-finalize"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-finalize"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/instruction_introspection/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-instruction-introspection"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-instruction-introspection"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/invoke/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-invoke"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -11,7 +11,7 @@ edition = "2018"
 
 [dependencies]
 solana-bpf-rust-invoked = { path = "../invoked", default-features = false }
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]

programs/bpf/rust/invoke_and_error/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "solana-bpf-rust-invoke-and-error"
-version = "1.7.13"
+version = "1.8.0"
 description = "Solana BPF test program written in Rust"
 authors = ["Solana Maintainers <maintainers@solana.foundation>"]
 repository = "https://github.com/solana-labs/solana"
@@ -10,7 +10,7 @@ documentation = "https://docs.rs/solana-bpf-rust-invoke-and-error"
 edition = "2018"
 
 [dependencies]
-solana-program = { path = "../../../../sdk/program", version = "=1.7.13" }
+solana-program = { path = "../../../../sdk/program", version = "=1.8.0" }
 
 [lib]
 crate-type = ["cdylib"]