188 Commits

Author SHA1 Message Date
Rob Walker
e9e01557b7 fix leaked threads from unclosed fullnode 2019-01-25 03:02:49 -08:00
Tyera Eulberg
b7be5b9a7a Add no-signer argument 2019-01-24 10:47:37 -08:00
Michael Vines
6e8b69fc88 Cleanup leader_addr, it's really entrypoint_addr 2019-01-21 13:06:30 -08:00
Pankaj Garg
6611188edf
Move subscriptions to rpc_pubsub (#2490)
* Move subscriptions to rpc_pubsub

- this helps avoid recreating pubsub_service on node's role change

* fixed tests and addressed review comments

* fix clippy errors

* address review comments
2019-01-21 09:59:09 -08:00
Michael Vines
132d59ca6a new_bank_from_db_ledger need not be public 2019-01-21 08:11:13 -08:00
Michael Vines
5b73a8eceb Rework fullnode boot sequence 2019-01-19 21:35:46 -08:00
Stephen Akridge
1fd7bd7ede Storage fixes
* replicators generate their sample values
* fixes to replicator block height logic
2019-01-18 13:05:35 -08:00
Michael Vines
f8bd19f5db Log the time it look to process the ledger for easier log inspection 2019-01-16 10:45:47 -08:00
Michael Vines
4369c1a113 RPC port is no longer reset on leader-to-validator transition 2019-01-15 16:06:56 -08:00
Rob Walker
022a97da99
revert revert of kill window (#2427)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-15 10:51:53 -08:00
Rob Walker
447243f994
Revert "remove window code from most places" (#2417)
* Revert "Fix link to book in Local Testnet section (#2416)"

This reverts commit 710c0c9980d090c531b5bc1fe4c2e2aaa17c856f.

* Revert "Add current leader information to dashboard (#2413)"

This reverts commit f0300c1711e9631cdcf65b67f252783f12ad0193.

* Revert "remove window code from most places (#2389)"

This reverts commit e3c0bd5a3fc732affb4bb36102a15063a09d90fd.
2019-01-14 15:11:18 -08:00
Rob Walker
e3c0bd5a3f
remove window code from most places (#2389)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-14 12:11:55 -08:00
Pankaj Garg
d7594b19fc
Implemented a trait for vote signer service (#2386)
* Implemented a trait for vote signer service

* removes need for RPC in unit tests for vote signing

* fix build errors

* address some review comments
2019-01-11 12:58:31 -08:00
Pankaj Garg
2dbe8fc1a9
Refactor vote signer code (#2368)
* Refactor vote signer code

* fixed test compilation errors

* address clippy errors

* fix missing macro_use

* move macro use

* review comments
2019-01-10 09:21:38 -08:00
Stephen Akridge
491bca5e4b Remove ledger.rs
Split into entry.rs for entry-constructing functions and EntrySlice
trait and db_ledger.rs for ledger helper test functions.
2019-01-09 15:15:47 -08:00
Greg Fitzgerald
f53810fcd2 Remove unused exit variable
The exit variable was only used by a test.
2019-01-08 20:22:31 -08:00
Michael Vines
a8b9899dee Add retry, restore ignored tests 2019-01-07 19:30:08 -08:00
Stephen Akridge
d2cb4e003c Re-enable the --lib tests 2019-01-07 15:28:20 -08:00
Pankaj Garg
91bd38504e
Use vote signer service in fullnode (#2009)
* Use vote signer service in fullnode

* Use native types for signature and pubkey, and address other review comments

* Start local vote signer if a remote service address is not provided

* Rebased to master

* Fixes after rebase
2019-01-05 12:57:52 -08:00
Michael Vines
b7dc9dbc76 RPC API now assumes a drone running on the bootstrap leader 2019-01-04 18:45:55 -08:00
Michael Vines
1f6346d880 De-dup ledgers - db_ledger is now the only ledger written to disk 2019-01-04 16:37:00 -08:00
Sagar Dhawan
0bea870b22
Dynamic N layer 'avalanche' broadcast and retransmit (#2058)
* Dynamic N layer avalanche broadcast and retransmit
2019-01-02 14:16:15 +05:30
Michael Vines
37d7ad819b Purge DB::destroy() usage 2018-12-20 10:38:03 -08:00
Pankaj Garg
974249f2a5
Parallelize entry processing in replay stage in validators (#2212)
* Parallelize entry processing in replay stage in validators

- single threaded entry processing is not utlizing CPU cores to the fullest

* fix tests and address review comments
2018-12-18 16:06:05 -08:00
Rob Walker
a65022aed7
DbLedger doesn't need to be mut, doesn't need an RwLock (#2215)
* DbLedger doesn't need to be mut, doesn't need an RwLock

* fix erasure cases
2018-12-18 15:18:57 -08:00
Michael Vines
07202205c4 Revert "ignore unstable tests"
This reverts commit bd7ef5d445071329a3b49b1f8be71b602226bbec.
2018-12-17 10:47:32 -08:00
anatoly yakovenko
4be6d01dfb
Move last ids (#2187)
* Break out last_ids into its own module
* Boot SignatureNotFound from BankError
* No longer return BankError from LastIds methods
* No longer piggypack on BankError for a LastIds signature status
* Drop all dependencies on the bank
* SignatureStatus -> Status and LastIds -> StatusDeque
* Unstable tests, issue 2193
2018-12-17 07:55:56 -08:00
Michael Vines
5bf9a20d42 fullnode-config no longer depends on src/ 2018-12-14 20:13:34 -08:00
Stephen Akridge
7cdbbfa88e Storage stage updates
* Remove logging init from storage program: saw a crash in a test
  indicating the logger being init'ed twice.
* Add entry_height mining proof to indicate which segment the result is
  for
* Add an interface to get storage miner pubkeys for a given entry_height
* Add an interface to get the current storage mining entry_height
* Set the tvu socket to 0.0.0.0:0 in replicator to stop getting entries
  after the desired ledger segment is downloaded.
* Use signature of PoH height to determine which block to download for
  replicator.
2018-12-13 11:30:12 -08:00
Rob Walker
4f48f1a850
add db_ledger genesis, rework to_blob(), to_blobs() (#2135) 2018-12-12 20:42:12 -08:00
Carl
245362db96 Make a dummy version of serving repairs from db_ledger 2018-12-12 19:12:51 -08:00
Mark
b1b190b80d Fix too many args in Tvu::new (#2114)
* Reduce args in Tvu::new under to 8

Now pass in sockets through a the crate::tvu::Sockets struct

Move ClusterInfo.keypair to pub(crate) in order to remove redundant
signing keypair parameter

* remove commented code
2018-12-12 18:57:48 -08:00
carllin
ae903f190e
Broadcast for slots (#2081)
* Insert blobs into db_ledger in broadcast stage to support leader to validator transitions

* Add transmitting real slots to broadcast stage

* Handle real slots instead of default slots in window

* Switch to dummy repair on slots and modify erasure to support leader rotation

* Shorten length of holding locks

* Remove logger from replicator test
2018-12-12 15:58:29 -08:00
Pankaj Garg
3413ecc2bd
Change query used to find list of nodes in the network (#2124)
* Change query used to find list of nodes in the network

* include "All" option for host selection
2018-12-12 12:38:00 -08:00
Greg Fitzgerald
5e703dc70a Free up the term 'replicate' for exclusive use in replicator
Also, align Sockets field names with ContactInfo.
2018-12-10 15:26:43 -07:00
Greg Fitzgerald
ec5a8141eb cargo fix --edition 2018-12-08 23:19:55 -07:00
Sathish
2de45a4da5
Update airdrop tokens to 3 for fullnode (#2051)
Filter out leader while computing the super majority stake
2018-12-08 16:54:42 -08:00
Greg Fitzgerald
0a83b17cdd
Upgrade to Rust 1.31.0 (#2052)
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Greg Fitzgerald
97b1156a7a Rename Ncp to GossipService
And BroadcastStage to BroadcastService since it's not included in the
TPU pipeline.
2018-12-06 15:48:19 -07:00
Michael Vines
c99f93e40a Remove signature.rs indirection 2018-12-03 13:50:00 -08:00
carllin
4ae58cc854
Change range of leader scheduler to match current broadcasts (#1920) 2018-12-03 00:10:43 -08:00
Sagar Dhawan
34c3a0cc1f
Add signature verification to gossip (#1937) 2018-12-01 12:00:30 -08:00
carllin
57a384d6a0
Rocks db window service (#1888)
* Add db_window module for windowing functions from RocksDb

* Replace window with db_window functions in window_service

* Fix tests

* Make note of change in db_window

* Create RocksDb ledger in bin/fullnode

* Make db_ledger functions generic

* Add db_ledger to bin/replicator
2018-11-24 19:32:33 -08:00
Rob Walker
6b910d1bd4 add tpu_forwarding, simplify ClusterInfo::new() from Result<Self> to Self 2018-11-19 20:45:49 -08:00
Rob Walker
3d113611cc
remove Result<> return from ClusterInfo::new() (#1869)
strip Result<> for ClusterInfo::new()
2018-11-19 11:25:14 -08:00
Michael Vines
d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
Michael Vines
6ac5700f2e Move metrics into its own crate 2018-11-16 15:10:07 -08:00
Michael Vines
a0dd8617be Remove airdrop from fullnode 2018-11-16 13:25:55 -08:00
anatoly yakovenko
a41254e18c
Add scalable gossip library (#1546)
* Cluster Replicated Data Store

Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.

Simulation shows fast convergence with 20k nodes.
2018-11-15 13:23:26 -08:00
Sagar Dhawan
729d28d910 Add poh verification before processing entries
- Replicate stage now verifies entries delivered
  by the window
- Minor refactor of entries_from_blobs
2018-11-13 14:17:00 -08:00