2021-05-19 09:31:47 -05:00
|
|
|
#![cfg_attr(RUSTC_WITH_SPECIALIZATION, feature(min_specialization))]
|
2021-02-16 14:48:20 -07:00
|
|
|
#![allow(clippy::integer_arithmetic)]
|
2018-06-06 09:49:22 -06:00
|
|
|
//! The `solana` library implements the Solana high-performance blockchain architecture.
|
2018-06-07 14:51:29 -06:00
|
|
|
//! It includes a full Rust implementation of the architecture (see
|
2019-05-23 22:05:16 -07:00
|
|
|
//! [Validator](server/struct.Validator.html)) as well as hooks to GPU implementations of its most
|
2018-06-07 14:51:29 -06:00
|
|
|
//! paralellizable components (i.e. [SigVerify](sigverify/index.html)). It also includes
|
2019-10-11 13:30:52 -06:00
|
|
|
//! command-line tools to spin up validators and a Rust library
|
2018-06-06 09:49:22 -06:00
|
|
|
//!
|
|
|
|
|
|
2020-03-16 08:37:31 -07:00
|
|
|
pub mod accounts_hash_verifier;
|
2018-05-14 17:31:13 -06:00
|
|
|
pub mod banking_stage;
|
2019-03-01 20:43:30 -07:00
|
|
|
pub mod broadcast_stage;
|
2021-05-26 22:16:16 -06:00
|
|
|
pub mod cache_block_meta_service;
|
2019-02-01 05:21:29 +05:30
|
|
|
pub mod cluster_info_vote_listener;
|
2021-07-07 14:30:55 +00:00
|
|
|
pub mod cluster_nodes;
|
2021-03-24 23:41:52 -07:00
|
|
|
pub mod cluster_slot_state_verifier;
|
2020-03-11 21:31:50 -07:00
|
|
|
pub mod cluster_slots;
|
2020-08-11 12:48:13 -07:00
|
|
|
pub mod cluster_slots_service;
|
2021-05-26 09:15:46 -06:00
|
|
|
pub mod commitment_service;
|
|
|
|
|
pub mod completed_data_sets_service;
|
2019-06-24 13:41:23 -07:00
|
|
|
pub mod consensus;
|
Cost model 1.7 (#20188)
* Cost Model to limit transactions which are not parallelizeable (#16694)
* * Add following to banking_stage:
1. CostModel as immutable ref shared between threads, to provide estimated cost for transactions.
2. CostTracker which is shared between threads, tracks transaction costs for each block.
* replace hard coded program ID with id() calls
* Add Account Access Cost as part of TransactionCost. Account Access cost are weighted differently between read and write, signed and non-signed.
* Establish instruction_execution_cost_table, add function to update or insert instruction cost, unit tested. It is read-only for now; it allows Replay to insert realtime instruction execution costs to the table.
* add test for cost_tracker atomically try_add operation, serves as safety guard for future changes
* check cost against local copy of cost_tracker, return transactions that would exceed limit as unprocessed transaction to be buffered; only apply bank processed transactions cost to tracker;
* bencher to new banking_stage with max cost limit to allow cost model being hit consistently during bench iterations
* replay stage feed back program cost (#17731)
* replay stage feeds back realtime per-program execution cost to cost model;
* program cost execution table is initialized into empty table, no longer populated with hardcoded numbers;
* changed cost unit to microsecond, using value collected from mainnet;
* add ExecuteCostTable with fixed capacity for security concern, when its limit is reached, programs with old age AND less occurrence will be pushed out to make room for new programs.
* investigate system performance test degradation (#17919)
* Add stats and counter around cost model ops, mainly:
- calculate transaction cost
- check transaction can fit in a block
- update block cost tracker after transactions are added to block
- replay_stage to update/insert execution cost to table
* Change mutex on cost_tracker to RwLock
* removed cloning cost_tracker for local use, as the metrics show clone is very expensive.
* acquire and hold locks for block of TXs, instead of acquire and release per transaction;
* remove redundant would_fit check from cost_tracker update execution path
* refactor cost checking with less frequent lock acquiring
* avoid many Transaction_cost heap allocation when calculate cost, which
is in the hot path - executed per transaction.
* create hashmap with new_capacity to reduce runtime heap realloc.
* code review changes: categorize stats, replace explicit drop calls, concisely initiate to default
* address potential deadlock by acquiring locks one at time
* Persist cost table to blockstore (#18123)
* Add `ProgramCosts` Column Family to blockstore, implement LedgerColumn; add `delete_cf` to Rocks
* Add ProgramCosts to compaction excluding list alone side with TransactionStatusIndex in one place: `excludes_from_compaction()`
* Write cost table to blockstore after `replay_stage` replayed active banks; add stats to measure persist time
* Deletes program from `ProgramCosts` in blockstore when they are removed from cost_table in memory
* Only try to persist to blockstore when cost_table is changed.
* Restore cost table during validator startup
* Offload `cost_model` related operations from replay main thread to dedicated service thread, add channel to send execute_timings between these threads;
* Move `cost_update_service` to its own module; replay_stage is now decoupled from cost_model.
* log warning when channel send fails (#18391)
* Aggregate cost_model into cost_tracker (#18374)
* * aggregate cost_model into cost_tracker, decouple it from banking_stage to prevent accidental deadlock. * Simplified code, removed unused functions
* review fixes
* update ledger tool to restore cost table from blockstore (#18489)
* update ledger tool to restore cost model from blockstore when compute-slot-cost
* Move initialize_cost_table into cost_model, so the function can be tested and shared between validator and ledger-tool
* refactor and simplify a test
* manually fix merge conflicts
* Per-program id timings (#17554)
* more manual fixing
* solve a merge conflict
* featurize cost model
* more merge fix
* cost model uses compute_unit to replace microsecond as cost unit
(#18934)
* Reject blocks for costs above the max block cost (#18994)
* Update block max cost limit to fix performance regession (#19276)
* replace function with const var for better readability (#19285)
* Add few more metrics data points (#19624)
* periodically report sigverify_stage stats (#19674)
* manual merge
* cost model nits (#18528)
* Accumulate consumed units (#18714)
* tx wide compute budget (#18631)
* more manual merge
* ignore zerorize drop security
* - update const cost values with data collected by #19627
- update cost calculation to closely proposed fee schedule #16984
* add transaction cost histogram metrics (#20350)
* rebase to 1.7.15
* add tx count and thread id to stats (#20451)
each stat reports and resets when slot changes
* remove cost_model feature_set
* ignore vote transactions from cost model
Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Jeff Biseda <jbiseda@gmail.com>
Co-authored-by: Jack May <jack@solana.com>
2021-10-06 15:11:41 -05:00
|
|
|
pub mod cost_model;
|
|
|
|
|
pub mod cost_tracker;
|
|
|
|
|
pub mod cost_tracker_stats;
|
|
|
|
|
pub mod cost_update_service;
|
|
|
|
|
pub mod execute_cost_table;
|
2018-05-29 11:18:12 -06:00
|
|
|
pub mod fetch_stage;
|
2020-06-11 12:16:04 -07:00
|
|
|
pub mod fork_choice;
|
2019-01-25 22:58:35 -07:00
|
|
|
pub mod gen_keys;
|
2020-06-11 12:16:04 -07:00
|
|
|
pub mod heaviest_subtree_fork_choice;
|
2021-04-21 14:40:35 -07:00
|
|
|
pub mod latest_validator_votes_for_frozen_banks;
|
2019-07-20 13:13:55 -07:00
|
|
|
pub mod ledger_cleanup_service;
|
2020-07-28 02:33:27 -07:00
|
|
|
pub mod optimistic_confirmation_verifier;
|
2021-04-20 09:37:33 -07:00
|
|
|
pub mod outstanding_requests;
|
2020-12-15 16:50:40 -08:00
|
|
|
pub mod packet_hasher;
|
2020-03-26 19:57:27 -07:00
|
|
|
pub mod progress_map;
|
2020-05-19 12:38:18 -07:00
|
|
|
pub mod repair_response;
|
2019-02-07 15:10:54 -08:00
|
|
|
pub mod repair_service;
|
2020-07-06 22:49:40 -07:00
|
|
|
pub mod repair_weight;
|
2020-07-02 14:33:04 -07:00
|
|
|
pub mod repair_weighted_traversal;
|
2018-12-07 15:09:29 -07:00
|
|
|
pub mod replay_stage;
|
2021-04-20 09:37:33 -07:00
|
|
|
pub mod request_response;
|
2020-01-02 20:50:43 -07:00
|
|
|
mod result;
|
2018-08-09 13:03:34 -06:00
|
|
|
pub mod retransmit_stage;
|
2020-02-04 19:50:24 -07:00
|
|
|
pub mod rewards_recorder_service;
|
2021-05-26 09:15:46 -06:00
|
|
|
pub mod sample_performance_service;
|
2020-01-31 14:23:51 -08:00
|
|
|
pub mod serve_repair;
|
|
|
|
|
pub mod serve_repair_service;
|
2021-05-26 09:15:46 -06:00
|
|
|
pub mod shred_fetch_stage;
|
2018-05-25 16:52:17 -06:00
|
|
|
pub mod sigverify;
|
2019-10-28 10:29:38 -07:00
|
|
|
pub mod sigverify_shreds;
|
2018-05-25 16:52:17 -06:00
|
|
|
pub mod sigverify_stage;
|
2019-10-18 15:58:16 -06:00
|
|
|
pub mod snapshot_packager_service;
|
2020-09-18 22:21:44 -07:00
|
|
|
pub mod test_validator;
|
2018-05-14 17:36:19 -06:00
|
|
|
pub mod tpu;
|
2020-07-14 00:38:48 -07:00
|
|
|
pub mod tree_diff;
|
2018-05-12 00:31:32 -06:00
|
|
|
pub mod tvu;
|
2021-04-21 14:40:35 -07:00
|
|
|
pub mod unfrozen_gossip_verified_vote_hashes;
|
2019-05-23 22:05:16 -07:00
|
|
|
pub mod validator;
|
2020-03-15 20:31:05 -07:00
|
|
|
pub mod verified_vote_packets;
|
2020-07-28 02:33:27 -07:00
|
|
|
pub mod vote_stake_tracker;
|
2021-07-16 22:12:04 +02:00
|
|
|
pub mod voting_service;
|
2018-09-07 16:00:26 -06:00
|
|
|
pub mod window_service;
|
2018-12-08 22:44:20 -07:00
|
|
|
|
2018-03-07 13:47:13 -08:00
|
|
|
#[macro_use]
|
2018-03-19 10:18:48 -06:00
|
|
|
extern crate log;
|
2018-12-08 22:44:20 -07:00
|
|
|
|
2018-02-20 16:26:11 -07:00
|
|
|
#[macro_use]
|
|
|
|
|
extern crate serde_derive;
|
2019-05-16 08:23:31 -07:00
|
|
|
|
2019-02-18 23:26:22 -07:00
|
|
|
#[macro_use]
|
|
|
|
|
extern crate solana_metrics;
|
|
|
|
|
|
2020-07-06 20:22:23 +09:00
|
|
|
#[macro_use]
|
2020-10-19 21:07:46 -07:00
|
|
|
extern crate solana_frozen_abi_macro;
|
2020-07-06 20:22:23 +09:00
|
|
|
|
2018-03-07 13:47:13 -08:00
|
|
|
#[cfg(test)]
|
|
|
|
|
#[macro_use]
|
|
|
|
|
extern crate matches;
|