Commit Graph

60 Commits

Author SHA1 Message Date
Rob Walker
9b43b00d5c remove tick_count, leader_scheduler, from broadcast code (#1725) 2018-11-06 13:17:41 -08:00
Rob Walker
13bfdde228 remove ledger tail code, WINDOW_SIZE begone (#1617)
* remove WINDOW_SIZE, use window.window_size()
* move ledger tail, redundant with ledger-based repair
2018-10-30 10:05:18 -07:00
carllin
f6c8e1a4bf Vote contract (#1552)
* Add Vote Contract

* Move ownership of LeaderScheduler from Fullnode to the bank

* Modified ReplicateStage to consume leader information from bank

* Restart RPC Services in Leader To Validator Transition

* Make VoteContract Context Free

* Remove voting from ClusterInfo and Tpu

* Remove dependency on ActiveValidators in LeaderScheduler

* Switch VoteContract to have two steps 1) Register 2) Vote. Change thin client to create + register a voting account on fullnode startup

* Remove check in leader_to_validator transition for unique references to bank, b/c jsonrpc service and rpcpubsub hold references through jsonhttpserver
2018-10-25 16:58:40 -07:00
Michael Vines
e47fcb196b s/solana_program_interface/solana[_-]sdk/g 2018-10-25 12:31:45 -07:00
Pankaj Garg
bbb38ac106 Increase window size (#1578)
Addresses the following problem
- Validators are not able to keep up with the leader
- The future blobs (outside of window) get dropped
- The validators won't process repair requests for these future blobs
2018-10-23 10:25:01 -07:00
carllin
0bd1412562 Switch leader scheduler to use PoH ticks instead of Entry height (#1519)
* Add PoH height to process_ledger()

* Moved broadcast_stage Leader Scheduling logic to use Poh height instead of entry_height

* Moved LeaderScheduler logic to PoH in ReplicateStage

* Fix Leader scheduling tests to use PoH instead of entry height

* Change is_leader detection in repair() to use PoH instead of entry height

* Add tests to LeaderScheduler for new functionality

* fix Entry::new and genesis block PoH counts

* Moved LeaderScheduler to PoH ticks

* Cleanup to resolve PR comments
2018-10-18 22:57:48 -07:00
carllin
9931ac9780 Leader scheduler plumbing (#1440)
* Added LeaderScheduler module and tests

* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage

* Add LeaderScheduler plumbing for Tvu, window, and tests

* Fix bank and switch tests to use new LeaderScheduler

* move leader rotation check from window service to replicate stage

* Add replicate_stage leader rotation exit test

* removed leader scheduler from the window service and associated modules/tests

* Corrected is_leader calculation in repair() function in window.rs

* Integrate LeaderScheduler with write_stage for leader to validator transitions

* Integrated LeaderScheduler with BroadcastStage

* Removed gossip leader rotation from crdt

* Add multi validator, leader test

* Comments and cleanup

* Remove unneeded checks from broadcast stage

* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role

* Set new leader in validator -> validator transitions

* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail

* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops

* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
2018-10-10 16:49:41 -07:00
Greg Fitzgerald
95701114e3 Crdt -> ClusterInfo 2018-10-09 03:49:39 -06:00
Greg Fitzgerald
423e7ebc3f Pacify clippy 2018-09-27 16:21:12 -06:00
Pankaj Garg
e10574c64d Remove recycler and it's usage
- The memory usage due to recycler was high, and incrementing with
  time.
2018-09-27 10:42:37 -06:00
jackcmay
9c47e022dc break dependency of programs on solana core (#1371)
* break dependency of programs on Solana core
2018-09-27 07:49:26 -07:00
Greg Fitzgerald
b7ae5b712a Move Pubkey into its own module 2018-09-26 20:40:40 -06:00
Stephen Akridge
a5b28349ed Add max entry height to download for replicator 2018-09-26 09:57:22 -07:00
carllin
e7383a7e66 Validator to leader (#1303)
* Add check in window_service to exit in checks for leader rotation, and propagate that service exit up to fullnode

* Added logic to shutdown Tvu once ReplicateStage finishes

* Added test for successfully shutting down validator and starting up leader

* Add test for leader validator interaction

* fix streamer to check for exit signal before checking socket again to prevent busy leaders from never returning

* PR comments - Rewrite make_consecutive_blobs() function, revert genesis function change
2018-09-25 15:41:29 -07:00
Pankaj Garg
0f4fd8367d Add counters for channel pressure and time spent in TPU pipeline (#1324)
* Add counters for channel pressure and time spent in TPU pipeline

* Fixed failing tests

* Fix rust format issue
2018-09-24 17:13:49 -07:00
Rob Walker
be31da3dce lastidnotfound step 2: (#1300)
lastidnotfound step 2:
  * move "record stage", aka poh_service into banking stage
  * remove Entry.has_more, is incompatible with leader rotation
  * rewrite entry_next_hash in terms of Poh
  * simplify and unify transaction hashing (no embedded nulls)
  * register_last_entry from banking stage, fixes #1171 (w00t!)
  * new PoH doesn't generate empty ledger entries, so some fixes necessary in 
         multinode tests that rely on that (e.g. giving validators airdrops)
  * make window repair less patient, if we've been waiting for an answer, 
          don't be shy about most recent blobs
   * delete recorder and record stage
   * make more verbost  thin_client error reporting
   * more tracing in window (sigh)
2018-09-21 21:01:13 -07:00
carllin
c50ac96f75 Moved deserialization of blobs to entries from replicate_stage to window_service (#1287) 2018-09-21 16:01:24 -07:00
Michael Vines
beeb09646a suppress warning: unused variable: recycler 2018-09-20 19:46:20 -07:00
Greg Fitzgerald
6073cd57fa Boot Recycler::recycle() 2018-09-20 17:08:51 -06:00
Anatoly Yakovenko
431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
Jay Kickliter
fd51599fa8 Replace replace(..., None) with take()
This is strictly for simplicity, since Option::take() is imlemented with replace().
2018-09-15 05:12:09 -06:00
Michael Vines
4196cf43e8 cargo fmt 2018-09-14 16:37:49 -07:00
Greg Fitzgerald
a093d5c809 Fix erasure build 2018-09-10 11:40:26 -06:00
Greg Fitzgerald
fc64e1853c Initialize Window, not SharedWindow
Wrap with Arc<RwLock>> when/if needed, no earlier.
2018-09-10 11:40:26 -06:00
Greg Fitzgerald
7f669094de Split window into two modules 2018-09-10 11:40:26 -06:00
Greg Fitzgerald
5025d89c88 Inline window method implementations 2018-09-10 11:40:26 -06:00
Greg Fitzgerald
2b44c4504a Use WindowUtil for more idiomatic code 2018-09-10 11:40:26 -06:00
Greg Fitzgerald
d2c9beb843 Add a trait to pretend Window is an object 2018-09-10 11:40:26 -06:00
Greg Fitzgerald
71a617b4dc Fix erasure build 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
e9f44b6661 window -> window_service 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
9693de1867 Reposition parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
f7ea95aed1 Hoist lock, reposition parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
f07ce59be8 Toggle parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
da423b6cf0 Hoist read lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
d5f60b68e4 Hoist window write lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
78b3a8f7f9 Hoist repair_window() branches
This probably would have been done if repair_window() was unit-tested.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald
d77699c126 Do the easy check first
All functions above operate on immutable values, so this shouldn't
change functionality, but no repair_window() tests to be certain.hI
2018-09-07 13:18:19 -06:00
Greg Fitzgerald
09ba0dae15 Remove redundant clone() 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
a5c7575207 Rewrite find_next_missing, call it clear_slots 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
50f040530b Remove redundant cast 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
7f99c90539 Simplify using early return and Result::ok() 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
d8564b725c Don't reference window to get each slot 2018-09-07 13:18:19 -06:00
Greg Fitzgerald
e4de25442a Hoist write lock
It needed to be passed the lock before, because it contained a
branch where one side didn't require locking. Now that that
defensive programming was hoisted, we can hoist the write lock
as well, leaving a simpler function for unit testing.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald
3b2ea8fd40 Hoist untested branch in window
If there were unit tests for this function, the author would have
written it this way to make their own life easier.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald
0fa27f65bb Use the default Pubkey formatter instead of debug_id() 2018-09-06 16:31:47 -06:00
Greg Fitzgerald
68c72d6f34 Fix nightly build 2018-09-04 20:56:40 -06:00
Rob Walker
226d3b9471 Trace recycle() calls (#968)
* trace recycle() calls fixes #810
2018-09-05 05:07:02 +09:00
Greg Fitzgerald
2aa72cc72e Return a Vec from to_blobs() instead of using a mut parameter 2018-09-04 07:50:23 -10:00
Greg Fitzgerald
8cc030ef84 Use Vec instead of VecDeque for SharedBlobs 2018-09-04 07:50:23 -10:00
Greg Fitzgerald
4e2deaa33b Less mut 2018-09-04 06:01:32 -10:00