102 Commits

Author SHA1 Message Date
Greg Fitzgerald
95701114e3 Crdt -> ClusterInfo 2018-10-09 03:49:39 -06:00
Greg Fitzgerald
423e7ebc3f Pacify clippy 2018-09-27 16:21:12 -06:00
Anatoly Yakovenko
93c4f6c9b8 Synchronize PoH, bank last_id queue and ledger entry channel.
PoH, bank's last_id queue and the Entry channel need to have a synchronized order of ids.
2018-09-26 16:19:03 -07:00
Rob Walker
be31da3dce
lastidnotfound step 2: (#1300)
lastidnotfound step 2:
  * move "record stage", aka poh_service into banking stage
  * remove Entry.has_more, is incompatible with leader rotation
  * rewrite entry_next_hash in terms of Poh
  * simplify and unify transaction hashing (no embedded nulls)
  * register_last_entry from banking stage, fixes #1171 (w00t!)
  * new PoH doesn't generate empty ledger entries, so some fixes necessary in 
         multinode tests that rely on that (e.g. giving validators airdrops)
  * make window repair less patient, if we've been waiting for an answer, 
          don't be shy about most recent blobs
   * delete recorder and record stage
   * make more verbost  thin_client error reporting
   * more tracing in window (sigh)
2018-09-21 21:01:13 -07:00
Anatoly Yakovenko
431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
Carl
bfe64f5f6e Added integration test for transitioning leader to validator to see that tpu pipeline can exit and restart a tvu. Fixed Tpu and broadcast stage so that exiting later stages in the pipeline also causes earlier stages to exit. 2018-09-19 10:48:05 -06:00
Carl
6d27751365 give fullnode ownership of state needed to dynamically start up a tpu or tvu for role transition 2018-09-19 10:48:05 -06:00
Carl
1fb1c0a681 added jointypes to the stages in the tpu involved in leader rotation 2018-09-19 10:48:05 -06:00
Carl
f28ba3937b Added check in write stage to exit when scheduled entry_height for leader rotation is detected 2018-09-19 10:48:05 -06:00
Pankaj Garg
5310b6e5a2
Move entry->blob creation out of write stage (#1257)
- The write stage will output vector of entries
- Broadcast stage will create blobs out of the entries
- Helps reduce MIPS requirements for write stage
2018-09-18 13:49:10 -07:00
Pankaj Garg
7d14f44a7c
Move register_entry_id() call out of write stage (#1253)
* Move register_entry_id() call out of write stage

- Write stage is MIPS intensive and has become a bottleneck for
  TPU pipeline
- This will reduce the MIPS requirements for the stage

* Fix rust format issues
2018-09-18 11:42:25 -07:00
Anatoly Yakovenko
fd36954477 clippy 2018-09-15 05:12:53 -06:00
sakridge
4dc30ea104
Add recycler stats (#1187) 2018-09-13 14:49:48 -07:00
carllin
8706774ea7
Rewrote service trait join() method to allow thread join handles to return values other than () (#1213) 2018-09-13 14:00:17 -07:00
Rob Walker
a8fdb8a5a7 use a single BlobRecycler per fullnode 2018-09-11 16:56:54 -07:00
Pankaj Garg
05460eec0d
Open multiple sockets for transaction UDP port (#1128)
* Reuse UDP port and open multiple sockets for transaction address

* Fixed failing crdt tests

* Add tests for reusing UDP ports

* Address review comments

* Updated bench-streamer to use multiple receive sockets

* Fix minimum number of recv sockets for bench-streamer

* Address review comments

Fixes #1132

* Moved bind_to function to nat.rs
2018-09-06 14:13:40 -07:00
Rob Walker
1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
Greg Fitzgerald
c2bbe4344e Rename KeyPair to Keypair 2018-08-09 13:41:37 -06:00
Rob Walker
5193819d8e Revert "Revert "plug in new ledger""
This reverts commit 57e928d1d02a1c841d9bfd2c7717312084a6a24e.
2018-08-06 08:51:41 -07:00
Rob Walker
57e928d1d0 Revert "plug in new ledger"
This reverts commit 46d9ba5ca0ff1429dfd1aab1c824af33f454325c.
2018-08-03 10:24:51 -07:00
Rob Walker
46d9ba5ca0 plug in new ledger 2018-08-02 21:50:47 -07:00
sakridge
2ea6f86199 Submit leader's vote after observing 2/3 validator votes (#780)
* fixup!

* fixups!

* send the vote and count it

* actually vote

* test

* Spelling fixes

* Process the voting transaction in the leader's bank

* Send tokens to the leader

* Give leader tokens in more cases

* Test for write_stage::leader_vote

* Request airdrop inside fullnode and not the script

* Change readme to indicate that drone should be up before leader

And start drone before leader in snap scripts

* Rename _kp => _keypair for keypairs and other review fixups

* Remove empty else
* tweak test_leader_vote numbers to be closer to testing 2/3 boundary
* combine creating blob and transaction for leader/validator
2018-07-31 22:07:53 -07:00
pgarg66
7c5172a65e
Converted sigverify disable flag to runtime check instead of "cfg" (#799) 2018-07-31 16:54:24 -07:00
Greg Fitzgerald
28af9a39b4 Don't clone before borrowing
Clippy told us to change function parameters to references, but
wasn't able to then tell us that the clone() before borrowing
was superfluous. This patch removes those by hand.

No expectation of a performance improvement here, since we were
just cloning reference counts. Just removes a bunch of noise.
2018-07-18 08:04:31 -04:00
Greg Fitzgerald
30f0c25b65 Fix all remaining clippy warnings
Fixes #586
2018-07-12 09:40:40 -06:00
Greg Fitzgerald
73ae3c3301 Apply most of clippy's feedback 2018-07-12 09:40:40 -06:00
Anatoly Yakovenko
be2bf69c93 initial vote stage
wip

voting

wip

move voting into the replicate stage

update

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

update

fixup!

fixup!

fixup!

tpu processing votes in entries before write stage

fixup!

fixup!

txs

make sure validators have an account

fixup!

fixup!

fixup!

exit fullnode correctly

exit on exit not err

try 50

add delay for voting

300

300

startup logs

par start

100

no rayon

retry longer

log leader drop

fix distance

50 nodes

100

handle deserialize error

update

fix broadcast

new table every time

tweaks

table

update

try shuffle table

skip kill

skip add

purge test

fixed tests

rebase 2

fixed tests

fixed rebase

cleanup

ok for blobs to be longer then window

fix init window

60 nodes
2018-07-10 13:32:31 -06:00
Greg Fitzgerald
c65c0d9b23 Expose fewer exit variables 2018-07-10 11:11:36 -06:00
Greg Fitzgerald
81477246be Remove exit variable from VerifyStage 2018-07-05 17:32:41 -06:00
Greg Fitzgerald
cbd664ba4b Remove exit variable from BankingStage 2018-07-05 17:32:41 -06:00
Greg Fitzgerald
4bb7cefa15 Remove exit variable from WriteStage 2018-07-05 17:32:41 -06:00
Greg Fitzgerald
77bf17064a Add Service trait
Added a consistent interface to all the microservices.
2018-07-04 16:40:34 -06:00
Greg Fitzgerald
5d17c2b58f Return output receivers from each stage
Reaching into the stages' structs for their receivers is, in hindsight,
more awkward than returning multiple values from constructors. By
returning the receiver, the caller can name the receiver whatever it
wants (as you would with any return value), and doesn't need to
reach into the struct for the field (which is super awkward in
combination with move semantics).
2018-07-02 16:18:32 -06:00
Greg Fitzgerald
56e37ad2f4 Limit sticky mutex to WriteStage 2018-07-01 17:29:24 -06:00
anatoly yakovenko
c24b0a1a3f
TVU rework (#352)
Refactored TVU, into stages
* blob fetch stage for blobs
* window stage for maintaining the blob window
* pulled out NCP out of the TVU so they can be separate units
TVU is now just the fetch -> window -> request and bank processing
2018-06-13 21:52:23 -07:00
Greg Fitzgerald
928e6165bc Add TPU & RPU diagrams 2018-06-07 15:24:44 -06:00
Greg Fitzgerald
60609a44ba Initialize recorder from bank's last_id 2018-05-31 12:13:18 -06:00
Greg Fitzgerald
7aab7d2f82 Sleep between events if PoH is disabled 2018-05-30 15:55:10 -06:00
Anatoly Yakovenko
cef1c208a5 Crdt pipeline, coalesce window repair requests in the listener by examining all of them at once, and ublock those threads from doing io. 2018-05-30 14:04:48 -06:00
Greg Fitzgerald
f2ccc133a2 Finally made fetch happen 2018-05-29 14:04:27 -06:00
Greg Fitzgerald
5e824b39dd Move multinode communication outside TPU 2018-05-29 14:04:27 -06:00
Greg Fitzgerald
46c19a5783 Rename sigverify modules 2018-05-25 17:31:07 -06:00
Greg Fitzgerald
9f5a3d6064 events -> transactions 2018-05-25 16:47:21 -06:00
Greg Fitzgerald
5a45eef1dc
Exit cleanup (#252)
* Ignore record_stage exit reason. We only really care about panic exit versus graceful exit.
* Ignore coverage build in CI
2018-05-24 10:03:17 -06:00
Greg Fitzgerald
b826f837f8 First attempt to pull TPU into the server 2018-05-15 11:25:55 -06:00
Greg Fitzgerald
5855e18a4e Let server own the bank, not TPU/RPU 2018-05-15 11:21:48 -06:00
Greg Fitzgerald
e9ee020b5f Rename constructors 2018-05-15 10:45:36 -06:00
Greg Fitzgerald
1bcf3891b4 New TPU/RPU constructors 2018-05-15 10:44:47 -06:00
Greg Fitzgerald
5456de63e9 Less state 2018-05-15 10:38:17 -06:00
Greg Fitzgerald
9026c70952 Inline Rpu::new 2018-05-15 10:33:16 -06:00