151 Commits

Author SHA1 Message Date
Michael Vines
140954a53c Remove Tpu::is_leader(), fullnode doesn't need it anymore 2019-02-27 11:55:21 -08:00
Rob Walker
9adbc1dd60 nit: always pass &Arc<Bank>, clone() only where consumed 2019-02-27 10:55:43 -08:00
anatoly yakovenko
6dcb97af9e
Move PohService and PohRecorder out of banking_stage and into fullnode (#2852)
* Move PohService out of banking_stage and into fullnode.

* 10 second slots
2019-02-26 10:48:18 -08:00
Carl
3e893ffddc Remove max_tick_height, leader_scheduler from broadcast_service 2019-02-25 18:17:36 -08:00
Greg Fitzgerald
e19dbdc527 Use Bank for ticks_per_slot 2019-02-22 22:02:23 -07:00
Michael Vines
b501090443 Route BankForks into the ReplayStage 2019-02-21 19:25:17 -08:00
Michael Vines
d25fc7a649 Stop passing blob_index unnecessarily into ReplayStage 2019-02-21 15:33:01 -07:00
Michael Vines
dcf1200d2a Make Fullnode do less work on rotation, ReplayStage can just pass along more details 2019-02-21 11:13:06 -08:00
Michael Vines
28a53959e0 Remove dead types 2019-02-20 18:39:32 -08:00
Greg Fitzgerald
dde886f058 Move Bank to its own crate
Also:
* counters.rs to solana_metrics
* genesis_block.rs to solana_sdk
2019-02-19 07:17:04 -07:00
Carl
f33c6eb95f delete leader rotation signal from banking stage 2019-02-17 19:30:45 -07:00
Carl
4e3d71c2c9 Batch joins on entire tpumode struct instead of individual services 2019-02-17 19:30:23 -07:00
Carl
a074cb78cd Ensure leader services are closed before starting new ones 2019-02-17 19:30:23 -07:00
Greg Fitzgerald
97c93629a5 Don't use the Bank's LeaderScheduler 2019-02-16 10:55:58 -07:00
Michael Vines
626a381ddc Collect and re-forward packets received while TpuForwarder is shutting down 2019-02-14 18:40:31 -08:00
Michael Vines
cceeb8e52d On leader rotation forward any unprocessed transaction packets to the new leader 2019-02-14 14:49:48 -08:00
Michael Vines
0f8ff07b51 tpu now hangs on to its cluster_info 2019-02-13 16:16:18 -08:00
Pankaj Garg
0002b5dd02 Write to ledger in BroadcastService
- Also disconnect the channel between TPU and TVU
2019-02-12 08:41:02 -08:00
Michael Vines
095afdfe47 Merge leader_to_validator/validator_to_leader 2019-02-11 08:57:44 -08:00
Michael Vines
a6aaca814c Rename enum Config to enum PohServiceConfig 2019-02-09 10:12:32 -08:00
Michael Vines
5200435bab Strip unused return type 2019-02-08 08:43:11 -08:00
Michael Vines
56734dca3b Align Tpu::new() and Tpu::switch_to_leader() arguments 2019-02-07 21:33:49 -08:00
Michael Vines
dbaf8e66ab Remove code duplication 2019-02-07 21:33:49 -08:00
carllin
fd7db7a954
Support multiple forks in the ledger (#2277)
* Modify db_ledger to support per_slot metadata, add signal for updates, and add chaining to slots in db_ledger

* Modify replay stage to ask db_ledger for updates based on slots

* Add repair send/receive metrics

* Add repair service, remove old repair code

* Fix tmp_copy_ledger and setup for tests to account for multiple slots and tick limits within slots
2019-02-07 15:10:54 -08:00
Michael Vines
95ac6305bc Remove unnecessary dependencies on fullnode mod 2019-02-06 21:31:48 -08:00
Sagar Dhawan
acc6bf1564 Don't over complicate the solution 2019-02-06 19:55:12 -08:00
Sagar Dhawan
db688207a5 Add abort signals to tvu/tpu receivers 2019-02-06 19:55:12 -08:00
Michael Vines
c5a74ada05 leader_scheduler: remove bootstrap_height 2019-02-06 14:23:10 -08:00
carllin
37003da854
Fix potential of checking tvu bank for truth when its behind (#2614)
* Fix race between tpu and tvu, where tvu bank is not caught up to tpu bank

* Add test

* Cleanup Fullnode tests
2019-01-31 19:21:02 -08:00
Sagar Dhawan
ed478675ba
Push and query the ClusterInfo for votes. (#2622) 2019-02-01 05:21:29 +05:30
Pankaj Garg
32162ef0f1
Connect TPU's broadcast service with TVU's blob fetch stage (#2587)
* Connect TPU's broadcast service with TVU's blob fetch stage

- This is needed since ledger is being written only in TVU now

* fix clippy warnings

* fix failing test

* fix broken tests

* fixed failing tests
2019-01-31 13:43:22 -08:00
Sagar Dhawan
d65e7b9fcc
Speedup rotation (#2468)
Speedup leader to validator transitions
2019-01-26 13:58:08 +05:30
Greg Fitzgerald
d2431128c7 Remove WriteStage from TPU/TVU diagrams
Fixes #2312
2019-01-08 08:42:06 -08:00
Michael Vines
1f6346d880 De-dup ledgers - db_ledger is now the only ledger written to disk 2019-01-04 16:37:00 -08:00
Greg Fitzgerald
ec5a8141eb cargo fix --edition 2018-12-08 23:19:55 -07:00
Sathish
2de45a4da5
Update airdrop tokens to 3 for fullnode (#2051)
Filter out leader while computing the super majority stake
2018-12-08 16:54:42 -08:00
Greg Fitzgerald
0a83b17cdd
Upgrade to Rust 1.31.0 (#2052)
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Michael Vines
d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
Greg Fitzgerald
2a6046de8e
Cleanup TVU code to look like its block diagram (#1737)
* Reorg TVU code to look like TVU diagram

And move channel creation into LedgerWriteStage so that it can
be used in the same was as all the other stages.

* Delete commented out code
2018-11-07 19:25:36 -07:00
Greg Fitzgerald
8be7c13d2d
Stub out architecture book (#1674)
* Stub out architecture documentation

* Add book HTML generation and book tests to CI

* Add heading

* Better table of contents

* Reference existing documentation

Move ASCII art from code comments into rendered SVG

* Attempt to fix CI

* Add lamport docs

And truncate lines to 80 characters

* Fix links

And reference shorter, newer description of PoH.

* Replace ASCII art with SVG

* Streamline for Pillbox

* Update path before optional install

* Use $CARGO_HOME instead of $HOME

* Delete code

Attempt to describe all data structures without code.

* Boot RPU from docs, add JsonRpcService

Also, use Rust naming conventions in the block diagrams to
minimize the jump from docs to code.

* Latest code uses tick_height

* Rename bob/ folder to art/

A home for any ASCII art

* Import JSON RPC API

* More mdbook docs

* Add Ncp

* Cleanup links

* Move pipelining description into fullnode description

* Move high-level transaction docs into top-level doc

* Delete unused files
2018-11-06 18:00:58 -07:00
Rob Walker
9b43b00d5c
remove tick_count, leader_scheduler, from broadcast code (#1725) 2018-11-06 13:17:41 -08:00
carllin
0636399b7a
Compute finality computation in new ComputeLeaderFinalityService (#1652)
* Move finality computation into a service run from the banking stage, ComputeLeaderFinalityService

* Change last ids nth to tick height, remove separate tick height from bank
2018-11-02 15:49:14 -07:00
Rob Walker
13bfdde228
remove ledger tail code, WINDOW_SIZE begone (#1617)
* remove WINDOW_SIZE, use window.window_size()
* move ledger tail, redundant with ledger-based repair
2018-10-30 10:05:18 -07:00
carllin
f6c8e1a4bf
Vote contract (#1552)
* Add Vote Contract

* Move ownership of LeaderScheduler from Fullnode to the bank

* Modified ReplicateStage to consume leader information from bank

* Restart RPC Services in Leader To Validator Transition

* Make VoteContract Context Free

* Remove voting from ClusterInfo and Tpu

* Remove dependency on ActiveValidators in LeaderScheduler

* Switch VoteContract to have two steps 1) Register 2) Vote. Change thin client to create + register a voting account on fullnode startup

* Remove check in leader_to_validator transition for unique references to bank, b/c jsonrpc service and rpcpubsub hold references through jsonhttpserver
2018-10-25 16:58:40 -07:00
carllin
55833e20b1
Create Poh Service (#1604)
* Create new Poh Service, replace tick generation in BankingStage
2018-10-25 14:56:21 -07:00
Pankaj Garg
8d9912b4e2
Move ledger write to its own stage (#1577)
* Move ledger write to its own stage

- Also, rename write_stage to leader_vote_stage, as write functionality
  is moved to a different stage

* Address review comments

* Fix leader rotation test failure

* address review comments
2018-10-23 14:42:48 -07:00
carllin
0bd1412562
Switch leader scheduler to use PoH ticks instead of Entry height (#1519)
* Add PoH height to process_ledger()

* Moved broadcast_stage Leader Scheduling logic to use Poh height instead of entry_height

* Moved LeaderScheduler logic to PoH in ReplicateStage

* Fix Leader scheduling tests to use PoH instead of entry height

* Change is_leader detection in repair() to use PoH instead of entry height

* Add tests to LeaderScheduler for new functionality

* fix Entry::new and genesis block PoH counts

* Moved LeaderScheduler to PoH ticks

* Cleanup to resolve PR comments
2018-10-18 22:57:48 -07:00
carllin
47f69f2d24
1) Switch broken tests to generate an empty tick in their ledgers to use as last_id, 2) Fix bug where PoH generator in BankingStage did not referenced the last tick instead of the last entry on startup, causing ledger verification to fail on the new tick added by the PoH generator (#1479) 2018-10-12 00:39:10 -07:00
carllin
9931ac9780
Leader scheduler plumbing (#1440)
* Added LeaderScheduler module and tests

* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage

* Add LeaderScheduler plumbing for Tvu, window, and tests

* Fix bank and switch tests to use new LeaderScheduler

* move leader rotation check from window service to replicate stage

* Add replicate_stage leader rotation exit test

* removed leader scheduler from the window service and associated modules/tests

* Corrected is_leader calculation in repair() function in window.rs

* Integrate LeaderScheduler with write_stage for leader to validator transitions

* Integrated LeaderScheduler with BroadcastStage

* Removed gossip leader rotation from crdt

* Add multi validator, leader test

* Comments and cleanup

* Remove unneeded checks from broadcast stage

* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role

* Set new leader in validator -> validator transitions

* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail

* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops

* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
2018-10-10 16:49:41 -07:00
Greg Fitzgerald
95701114e3 Crdt -> ClusterInfo 2018-10-09 03:49:39 -06:00