95 Commits

Author SHA1 Message Date
Carl
1fb1c0a681 added jointypes to the stages in the tpu involved in leader rotation 2018-09-19 10:48:05 -06:00
Carl
f28ba3937b Added check in write stage to exit when scheduled entry_height for leader rotation is detected 2018-09-19 10:48:05 -06:00
Pankaj Garg
5310b6e5a2
Move entry->blob creation out of write stage (#1257)
- The write stage will output vector of entries
- Broadcast stage will create blobs out of the entries
- Helps reduce MIPS requirements for write stage
2018-09-18 13:49:10 -07:00
Pankaj Garg
7d14f44a7c
Move register_entry_id() call out of write stage (#1253)
* Move register_entry_id() call out of write stage

- Write stage is MIPS intensive and has become a bottleneck for
  TPU pipeline
- This will reduce the MIPS requirements for the stage

* Fix rust format issues
2018-09-18 11:42:25 -07:00
Anatoly Yakovenko
fd36954477 clippy 2018-09-15 05:12:53 -06:00
sakridge
4dc30ea104
Add recycler stats (#1187) 2018-09-13 14:49:48 -07:00
carllin
8706774ea7
Rewrote service trait join() method to allow thread join handles to return values other than () (#1213) 2018-09-13 14:00:17 -07:00
Rob Walker
a8fdb8a5a7 use a single BlobRecycler per fullnode 2018-09-11 16:56:54 -07:00
Pankaj Garg
05460eec0d
Open multiple sockets for transaction UDP port (#1128)
* Reuse UDP port and open multiple sockets for transaction address

* Fixed failing crdt tests

* Add tests for reusing UDP ports

* Address review comments

* Updated bench-streamer to use multiple receive sockets

* Fix minimum number of recv sockets for bench-streamer

* Address review comments

Fixes #1132

* Moved bind_to function to nat.rs
2018-09-06 14:13:40 -07:00
Rob Walker
1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
Greg Fitzgerald
c2bbe4344e Rename KeyPair to Keypair 2018-08-09 13:41:37 -06:00
Rob Walker
5193819d8e Revert "Revert "plug in new ledger""
This reverts commit 57e928d1d02a1c841d9bfd2c7717312084a6a24e.
2018-08-06 08:51:41 -07:00
Rob Walker
57e928d1d0 Revert "plug in new ledger"
This reverts commit 46d9ba5ca0ff1429dfd1aab1c824af33f454325c.
2018-08-03 10:24:51 -07:00
Rob Walker
46d9ba5ca0 plug in new ledger 2018-08-02 21:50:47 -07:00
sakridge
2ea6f86199 Submit leader's vote after observing 2/3 validator votes (#780)
* fixup!

* fixups!

* send the vote and count it

* actually vote

* test

* Spelling fixes

* Process the voting transaction in the leader's bank

* Send tokens to the leader

* Give leader tokens in more cases

* Test for write_stage::leader_vote

* Request airdrop inside fullnode and not the script

* Change readme to indicate that drone should be up before leader

And start drone before leader in snap scripts

* Rename _kp => _keypair for keypairs and other review fixups

* Remove empty else
* tweak test_leader_vote numbers to be closer to testing 2/3 boundary
* combine creating blob and transaction for leader/validator
2018-07-31 22:07:53 -07:00
pgarg66
7c5172a65e
Converted sigverify disable flag to runtime check instead of "cfg" (#799) 2018-07-31 16:54:24 -07:00
Greg Fitzgerald
28af9a39b4 Don't clone before borrowing
Clippy told us to change function parameters to references, but
wasn't able to then tell us that the clone() before borrowing
was superfluous. This patch removes those by hand.

No expectation of a performance improvement here, since we were
just cloning reference counts. Just removes a bunch of noise.
2018-07-18 08:04:31 -04:00
Greg Fitzgerald
30f0c25b65 Fix all remaining clippy warnings
Fixes #586
2018-07-12 09:40:40 -06:00
Greg Fitzgerald
73ae3c3301 Apply most of clippy's feedback 2018-07-12 09:40:40 -06:00
Anatoly Yakovenko
be2bf69c93 initial vote stage
wip

voting

wip

move voting into the replicate stage

update

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

update

fixup!

fixup!

fixup!

tpu processing votes in entries before write stage

fixup!

fixup!

txs

make sure validators have an account

fixup!

fixup!

fixup!

exit fullnode correctly

exit on exit not err

try 50

add delay for voting

300

300

startup logs

par start

100

no rayon

retry longer

log leader drop

fix distance

50 nodes

100

handle deserialize error

update

fix broadcast

new table every time

tweaks

table

update

try shuffle table

skip kill

skip add

purge test

fixed tests

rebase 2

fixed tests

fixed rebase

cleanup

ok for blobs to be longer then window

fix init window

60 nodes
2018-07-10 13:32:31 -06:00
Greg Fitzgerald
c65c0d9b23 Expose fewer exit variables 2018-07-10 11:11:36 -06:00
Greg Fitzgerald
81477246be Remove exit variable from VerifyStage 2018-07-05 17:32:41 -06:00
Greg Fitzgerald
cbd664ba4b Remove exit variable from BankingStage 2018-07-05 17:32:41 -06:00
Greg Fitzgerald
4bb7cefa15 Remove exit variable from WriteStage 2018-07-05 17:32:41 -06:00
Greg Fitzgerald
77bf17064a Add Service trait
Added a consistent interface to all the microservices.
2018-07-04 16:40:34 -06:00
Greg Fitzgerald
5d17c2b58f Return output receivers from each stage
Reaching into the stages' structs for their receivers is, in hindsight,
more awkward than returning multiple values from constructors. By
returning the receiver, the caller can name the receiver whatever it
wants (as you would with any return value), and doesn't need to
reach into the struct for the field (which is super awkward in
combination with move semantics).
2018-07-02 16:18:32 -06:00
Greg Fitzgerald
56e37ad2f4 Limit sticky mutex to WriteStage 2018-07-01 17:29:24 -06:00
anatoly yakovenko
c24b0a1a3f
TVU rework (#352)
Refactored TVU, into stages
* blob fetch stage for blobs
* window stage for maintaining the blob window
* pulled out NCP out of the TVU so they can be separate units
TVU is now just the fetch -> window -> request and bank processing
2018-06-13 21:52:23 -07:00
Greg Fitzgerald
928e6165bc Add TPU & RPU diagrams 2018-06-07 15:24:44 -06:00
Greg Fitzgerald
60609a44ba Initialize recorder from bank's last_id 2018-05-31 12:13:18 -06:00
Greg Fitzgerald
7aab7d2f82 Sleep between events if PoH is disabled 2018-05-30 15:55:10 -06:00
Anatoly Yakovenko
cef1c208a5 Crdt pipeline, coalesce window repair requests in the listener by examining all of them at once, and ublock those threads from doing io. 2018-05-30 14:04:48 -06:00
Greg Fitzgerald
f2ccc133a2 Finally made fetch happen 2018-05-29 14:04:27 -06:00
Greg Fitzgerald
5e824b39dd Move multinode communication outside TPU 2018-05-29 14:04:27 -06:00
Greg Fitzgerald
46c19a5783 Rename sigverify modules 2018-05-25 17:31:07 -06:00
Greg Fitzgerald
9f5a3d6064 events -> transactions 2018-05-25 16:47:21 -06:00
Greg Fitzgerald
5a45eef1dc
Exit cleanup (#252)
* Ignore record_stage exit reason. We only really care about panic exit versus graceful exit.
* Ignore coverage build in CI
2018-05-24 10:03:17 -06:00
Greg Fitzgerald
b826f837f8 First attempt to pull TPU into the server 2018-05-15 11:25:55 -06:00
Greg Fitzgerald
5855e18a4e Let server own the bank, not TPU/RPU 2018-05-15 11:21:48 -06:00
Greg Fitzgerald
e9ee020b5f Rename constructors 2018-05-15 10:45:36 -06:00
Greg Fitzgerald
1bcf3891b4 New TPU/RPU constructors 2018-05-15 10:44:47 -06:00
Greg Fitzgerald
5456de63e9 Less state 2018-05-15 10:38:17 -06:00
Greg Fitzgerald
9026c70952 Inline Rpu::new 2018-05-15 10:33:16 -06:00
Greg Fitzgerald
99dc4ea4a9 Spin up threads from Rpu/Tpu constructors 2018-05-15 10:30:52 -06:00
Greg Fitzgerald
0aaa500f7c Rpu/Tpu serve() functions now only spin up threads 2018-05-15 10:10:45 -06:00
Greg Fitzgerald
7e44005a0f Don't do error-prone things in functions that spawn threads 2018-05-15 09:53:51 -06:00
Greg Fitzgerald
2a268aa528 Reorder to reflect dependencies 2018-05-15 09:17:48 -06:00
Greg Fitzgerald
ef6bd7e3b8 Add TPU 2018-05-14 17:36:19 -06:00
Greg Fitzgerald
421d9aa501 Free up the name 'tpu' 2018-05-12 10:53:25 -06:00
Greg Fitzgerald
898f4971a2 Free up name 'thin_client_service' 2018-05-12 10:50:22 -06:00