Commit Graph

98 Commits

Author SHA1 Message Date
57e90948a8 Remove dead code from cluster_info (#6051) 2019-09-24 10:27:59 -07:00
d379786c90 Fix bind errors (#5986)
* Add ability to bind to a common tcp/udp port

* Extend port range for local-net sanity and fix validator executable
2019-09-19 17:16:22 -07:00
783e8672e7 Removed Shred enum (#5963)
* Remove shred enum and it's references

* rename ShredInfo to Shred

* clippy
2019-09-18 16:24:30 -07:00
6c4e656795 Remove obsoleted code from shred (#5954)
* Remove obsoleted code from shred

* fix broken test
2019-09-18 13:56:44 -07:00
ff608992ee Replace Shred usage with ShredInfo (#5939)
* Replace Shred usage with ShredInfo

* Fix tests

* fix clippy
2019-09-17 18:22:46 -07:00
c44e7ce184 Leaders should not broadcast to replicators (#5917) 2019-09-16 17:56:34 -07:00
5f54573613 More shred related cleanup (#5909)
* More shred related cleanup

* fix uncle
2019-09-16 10:28:28 -07:00
e8d88f3237 Split SDK's timing.rs (#5823) 2019-09-06 14:30:56 -07:00
3d3b03a123 Verify signature of recovered shred before adding them to blocktree (#5811)
* Verify signature of recovered shred before adding them to blocktree

* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
f78b865cba Cleanup shreds to remove FirstShred data structure (#5789)
* Cleanup shreds to remove FirstShred data structure

* Also reduce size used by parent slot information in shred header

* clippy

* fixes

* fix chacha test
2019-09-04 21:06:47 -07:00
02ee2a601c Further cleanup of blocktree after Blob deprecation (#5780) 2019-09-04 12:47:09 -07:00
3b0d48e3b8 Remove blocktree blob references (#5691)
* Remove blocktree blob references

* fixes and cleanup

* replace uninitialized() call with MaybeUninit

* fix bench
2019-09-03 21:32:51 -07:00
a29f0484dc Add newline before cluster info log (#5671) 2019-08-27 08:33:48 -07:00
db768b4c3a Log contact info every 10 seconds (#5663) 2019-08-26 18:31:14 -07:00
4ac1213c9c Integrate coding shreds and recovery (#5625)
* Integrate coding shreds and recovery

* More tests for shreds and some fixes

* address review comments

* fixes to code shred generation

* unignore tests

* fixes to recovery
2019-08-26 18:27:45 -07:00
4798e7fa73 Integrate data shreds (#5541)
* Insert data shreds in blocktree and database

* Integrate data shreds with rest of the code base

* address review comments, and some clippy fixes

* Fixes to some tests

* more test fixes

* ignore some local cluster tests

* ignore replicator local cluster tests
2019-08-20 17:16:06 -07:00
bd193535c9 Cap CrdsFilter sizes such that PullRequest no longer exceeds MTU (#5561) 2019-08-19 18:14:10 -07:00
e73cbdda61 Reduce log level for known issue (#5536)
automerge
2019-08-15 19:42:27 -07:00
4ee212ae4c Coalesce gossip pull requests and serve them in batches (#5501)
* Coalesce gossip pull requests and serve them in batches

* batch all filters and immediately respond to messages in gossip

* Fix tests

* make download_from_replicator perform a greedy recv
2019-08-15 17:04:45 -07:00
1d0608200c Restore blob size fix (#5516)
* Revert "Revert "Fix gossip messages growing beyond blob size  (#5460)" (#5512)"

This reverts commit 97d57d168b.

* Fix Crds filters
2019-08-13 18:04:14 -07:00
97d57d168b Revert "Fix gossip messages growing beyond blob size (#5460)" (#5512)
This reverts commit a8eb0409b7.
2019-08-13 10:29:26 -07:00
0fde19239b Rate limit counter metrics points to one per second (#5496)
* Rate limit counter metrics points to one per second

* Remove old env var

* Test that metrics counter is incrementing

* Fix typo
2019-08-12 18:15:34 -04:00
a8eb0409b7 Fix gossip messages growing beyond blob size (#5460)
* fixed bloom filter math

* Add split each pull request into multiple pulls with different filters

* Rework CrdsFilter to generate all possible masks to cover the keyspace

* Limit the bloom sizes such that each pull request is no larger than mtu
2019-08-12 13:51:29 -07:00
4f86c0b74a Rate limit transaction counters (#5447)
* Rate limit transaction counters

* @sakridge feedback

* Set default high metrics rate for multinode demo

* Fix tests

* Swap defaults and fix env var tests

* Only set metrics rate if not already set
2019-08-08 17:05:06 -04:00
d7a2b790dc Limit the size of gossip push and gossip pull response (#5348)
* Limit the size of gossip push and gossip pull response

* Remove Default::default

* Rename var
2019-07-30 15:43:17 -07:00
a7a10e12c7 Forward transactions as packets instead of blobs (#5334)
* Forward transactions as packets instead of blobs

* clippy
2019-07-30 14:50:02 -07:00
84368697af Fix metrics when leader does not report metrics (#5291) 2019-07-30 16:18:33 -04:00
4c90898f0b Dynamic erasure set configuration (#5018)
* Use local erasure session to create/broadcast coding blobs

* Individual session for each recovery (as the config might be different)

* address review comments

* new constructors for session and coding generator

* unit test for dynamic erasure config
2019-07-11 13:58:33 -07:00
f537482c86 remove set_leader from cluster_info (#4998) 2019-07-09 22:06:47 -07:00
65adce65fa Always send pull responses to the origin addr (#4894) 2019-07-01 16:49:05 -07:00
861d6468ca Stake weighted pruning for the gossip network (#4769)
* Stake weighted pruning

* Fix compile error

* Fix clippy errors

* Add helper for creating a connected staked network

* Bug fixes and test groundwork

* Small refactor

* Anatoly's feedback and tests

* Doc updates

* @rob-solana's feedback

* Fix test bug and add log trace

* @rob-solana's feedback
2019-06-26 00:30:16 -07:00
46bb79df29 Support for custom BroadcastStage in local cluster tests (#4716)
* Refactor BroadcastStage to support custom implementations, add FailEntryVerificationBroadcastRun implementation

* Plumb switch on broadcast type through validator

* Add test for validator generating non-verifiable entries to local_cluster

* Fix bad initializers

* Refactor broadcast run code into utils
2019-06-19 00:13:19 -07:00
cc48773b03 Add "download from replicator" utility (#4709)
automerge
2019-06-17 18:12:13 -07:00
8c1b9a0b67 Data plane verification (#4639)
* Add signature to blob

* Change Signable trait to support returning references to signable data

* Add signing to broadcast

* Verify signatures in window_service

* Add testing for signatures to erasure

* Add RPC for getting current slot, consume RPC call in test_repairman_catchup for more deterministic results
2019-06-12 16:43:05 -07:00
82df267ec9 s/avalanche/turbine (#4561)
* s/avalanche/turbine/g
2019-06-06 12:48:40 -07:00
70a16e91a5 Randomize avalanche broadcast peer table for each blob (#4529)
* fix clippy warnings
2019-06-03 20:38:05 -07:00
3574469052 Add random distribution for avalanche peers (#4493)
* Add random distribution for avalanche peers

* fix clippy warnings

* bug fixes

* nits
2019-06-01 07:55:43 -07:00
182096dc1a Create bank snapshots (#4244)
* Revert "Revert "Create bank snapshots (#3671)" (#4243)"

This reverts commit 81fa69d347.

* keep saved and unsaved copies of status cache

* fix format check

* bench for status cache serialize

* misc cleanup

* remove appendvec storage on purge

* fix accounts restore

* cleanup

* Pass snapshot path as args

* Fix clippy
2019-05-30 21:31:35 -07:00
335dfdc4d5 Fix Gossip skipping push for some values (#4463)
* Make gossip skip over values from Pruned nodes

* Add test and init blooms to contain the origin
2019-05-28 18:39:40 -07:00
cfe5afd34c _id => _pubkey variable renaming (#4419)
* wallet: rename *_account_id to *_account_pubkey

* s/from_id/from_pubkey/g

* s/node_id/node_pubkey/g

* s/stake_id/stake_pubkey/g

* s/voter_id/voter_pubkey/g

* s/vote_id/vote_pubkey/g

* s/delegate_id/delegate_pubkey/g

* s/account_id/account_pubkey/g

* s/to_id/to_pubkey/g

* s/my_id/my_pubkey/g

* cargo fmt

* s/staker_id/staker_pubkey/g

* s/mining_pool_id/mining_pool_pubkey/g

* s/leader_id/leader_pubkey/g

* cargo fmt

* s/funding_id/funding_pubkey/g
2019-05-23 23:20:04 -07:00
6a9e0bc593 Change EpochSlots to use BtreeSet so that serialization/deserialization returns the same order (#4404)
automerge
2019-05-23 03:50:41 -07:00
591fd72e0b Implement listener for serving repairs through Repairman protocol (#4306)
* Make listener for serving repairs through Repairman protocol
2019-05-23 03:10:16 -07:00
f877fb8c8f Don't print leader update message unless leader actually updates (#4365) 2019-05-21 11:06:56 -07:00
9476fe5ce3 Use log levels for datapoint metrics (#4335)
* Use log levels for datapoint metrics

* address review comments

* fix cyclomatic complexity
2019-05-17 17:34:05 -07:00
b2ce5dc9f5 Adjust log level for counter metrics (#4323) 2019-05-17 07:00:06 -07:00
a0ffbf50a5 Correctly remove replicator from data plane after its done repairing (#4301)
* Correctly remove replicator from data plane after its done repairing

* Update discover to report nodes and replicators separately

* Fix print and condition to be spy
2019-05-16 07:14:58 -07:00
0c1191c3ee rework staking_utils (#4283) 2019-05-14 16:15:51 -07:00
f567877d1d Cleanup metrics (#4230) 2019-05-10 08:33:58 -07:00
81fa69d347 Revert "Create bank snapshots (#3671)" (#4243)
This reverts commit abf2b300da.
2019-05-09 19:27:27 -07:00
abf2b300da Create bank snapshots (#3671)
* Be able to create bank snapshots

* fix clippy

* load snapshot on start

* regenerate account index from the storage

* Remove rc feature dependency

* cleanup

* save snapshot for slot 0
2019-05-09 19:27:06 -07:00