Commit Graph

94 Commits

Author SHA1 Message Date
176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
f7c0d30167 Disallow localhost in deployment (#1064)
* disallow localhost in deployment

* tests

* fmt

* integration tests do not have a flag to check

* fmt
2018-08-25 21:09:18 -07:00
be58fdf1bb Less constructors 2018-08-23 13:42:54 -07:00
9c5e69bf3d Don't offer two ways to specify a leader 2018-08-23 13:42:54 -07:00
24b0031925 Reduce number of nodes in multinode test (#1003) 2018-08-20 13:40:42 -07:00
f73f3941cd Revert ill-advised jsonrpc marker, and handle jsonrpc server close 2018-08-18 12:29:49 -06:00
1eb8724a89 Disable Rpc module for other tests to prevent port conflicts 2018-08-18 12:29:49 -06:00
3ae867bdd6 fixups 2018-08-18 02:22:52 -07:00
f6f0900506 Large network test to not poll validator for sigs (#998)
- The finality is already reached. The test will check the signature
  in validators once, instead of polling. This will help speed up the test.
2018-08-17 14:38:19 -07:00
a284030ecc Account type with state
comments

fixups!

fixups!

fixups for a real Result<> from get_balance()

on 2nd thought, be more rigorous

Merge branch 'rob-solana-accounts_with_state' into accounts_with_state

update

review comments

comments

get rid of option
2018-08-16 14:44:51 -07:00
c2983f824e Refactored large network test to use finality to assert success (#978) 2018-08-15 20:05:43 -07:00
88d6fea999 Revert "Accounts with state (#954)"
This reverts commit c23fa289c3.
2018-08-15 19:44:39 -07:00
c23fa289c3 Accounts with state (#954)
* Account type with state

* fixed test according to @rob-solana
2018-08-15 14:32:11 -07:00
db35f220f7 Run multinode test for enough iterations for a small node count test (#971) 2018-08-15 10:44:14 -07:00
982afa87a6 Retransmit blobs from leader from window (#975)
- Some nodes don't have leader information while leader is broadcasting
  blobs to those nodes. Such blobs are not retransmitted. This change
  rertansmits the blobs once the leader's identity is know.
2018-08-14 21:51:37 -07:00
b801ca477d Declare fullnode a word 2018-08-13 20:17:16 -06:00
2727067b94 Move winow into its own module 2018-08-13 20:17:16 -06:00
bf15cad36b Add get_finality request and use it from multinode test (#941) 2018-08-13 08:55:13 -07:00
f07c038266 Fix bank coalescing (#949)
* fix bank coalescing

* comments

* fix bench

* fix bench

* backout banking stage coalescing

* 120 nodes

* 100
2018-08-12 10:04:21 -07:00
96d6985895 rework read_ledger, LedgerWriter, and LedgerWindow for recover()
fixes #910
2018-08-10 18:07:23 -07:00
ad331e6d56 Rename PublicKey type to Pubkey
Recognize pubkey as a noun meaning the public key of a keypair.
2018-08-09 13:41:37 -06:00
c2bbe4344e Rename KeyPair to Keypair 2018-08-09 13:41:37 -06:00
8567253833 Ignore flaky test 2018-08-09 10:15:10 -06:00
a6857dbaaa Updated node count to 230. Increased wmem on CI large 2018-08-08 13:13:18 -07:00
75854cc234 Update dynamic network test with more nodes (#904)
- Check for correct OS params in test-large-network.sh
2018-08-08 06:52:57 -07:00
9783d47fd1 write a "unit" test for WindowLedger (it was working ;)
clear flags on fresh blobs, lest they sometimes impersonate coding blobs...

fix bug: advance *received whether the blob_index is in the window or not,
  failure to do so results in a stalled repair request pipeline
2018-08-08 04:28:09 -07:00
38be61bd22 Check for log level before doing perf counter work
Perf counters, especially when running the dynamic test can cause
functions like crdt::apply_updates to be really slow (>500ms).
2018-08-08 00:16:53 -07:00
fbc754ea25 plug in LedgerWindow
fixes #872
2018-08-07 17:27:53 -07:00
16772d3d51 Coalesce multiple blobs if received close together (#869)
- This helps reduce unnecessary growth of window if small blobs
  are received in small space of time
2018-08-07 10:29:57 -07:00
9c1b6288a4 Use ? instead of unwrap()
This change addresses #833, while there are still some unwrap() though.
2018-08-07 08:10:22 -07:00
3beb38ac8a /tmp/farf no good on multi-user machine 2018-08-06 14:53:40 -07:00
039ed01abf on 2nd thought: do not copy_ledger() for this test 2018-08-06 08:51:41 -07:00
ead0eb2754 move copy_ledger() back into ledger.rs
Don't recover() for copy(), as copy() is already tolerant of things
    recover() guards against.  Note: recover() is problematic if the ledger is
    "live", i.e. is currently being written to.
2018-08-06 08:51:41 -07:00
c3db2df7eb tweak random access ledger
* add recover_ledger() to deal with expected common ledger corruptions
  * add verify_ledger() for future use cases (ledger-tool)
  * increase ledger testing
  * allow replicate stage to run without a ledger
  * ledger-tool to output valid json
2018-08-06 08:51:41 -07:00
5193819d8e Revert "Revert "plug in new ledger""
This reverts commit 57e928d1d0.
2018-08-06 08:51:41 -07:00
f2b1a04bca cargo fmt fixups 2018-08-03 11:59:25 -07:00
57e928d1d0 Revert "plug in new ledger"
This reverts commit 46d9ba5ca0.
2018-08-03 10:24:51 -07:00
46d9ba5ca0 plug in new ledger 2018-08-02 21:50:47 -07:00
fda3b9bbd4 Use new PublicKey format instead of hex 2018-08-01 16:15:14 -07:00
d622742b84 Mark test-multinode-basic as ignore 2018-08-01 10:13:05 -06:00
2ea6f86199 Submit leader's vote after observing 2/3 validator votes (#780)
* fixup!

* fixups!

* send the vote and count it

* actually vote

* test

* Spelling fixes

* Process the voting transaction in the leader's bank

* Send tokens to the leader

* Give leader tokens in more cases

* Test for write_stage::leader_vote

* Request airdrop inside fullnode and not the script

* Change readme to indicate that drone should be up before leader

And start drone before leader in snap scripts

* Rename _kp => _keypair for keypairs and other review fixups

* Remove empty else
* tweak test_leader_vote numbers to be closer to testing 2/3 boundary
* combine creating blob and transaction for leader/validator
2018-07-31 22:07:53 -07:00
7c5172a65e Converted sigverify disable flag to runtime check instead of "cfg" (#799) 2018-07-31 16:54:24 -07:00
0c33c9e0d7 Dynamic network test changes (#795)
- No sigverify if feature sigverify_cpu_disable is used
- Purge validators in the test if lag count increases beyond
  SOLANA_DYNAMIC_NODES_PURGE_LAG environment variable
- Other useful log messages in the test
2018-07-30 13:57:10 -07:00
0b47404ba6 Check for default leader and use cmp::max for a bit nicer code (#779) 2018-07-27 15:53:31 -07:00
7f4844f426 More stats in dynamic multinode test 2018-07-27 11:55:09 -07:00
c52d4eca0b Stop validator first to stop voting before the leader stops 2018-07-26 17:29:32 -07:00
7672506b45 Validators now vote once a second regardless 2018-07-26 17:07:42 -07:00
297896bc49 honor environment variable SOLANA_DYNAMIC_NODES, obviating need to edit+compile for re-test 2018-07-25 20:45:16 -07:00