Commit Graph

219 Commits

Author SHA1 Message Date
Sagar Dhawan
801337a422 Refactor Weighted Shuffle (#6614)
automerge
2019-10-29 21:02:11 -07:00
Michael Vines
397ea05aa7 spy nodes are now gossip entrypoints (#6532) 2019-10-24 15:35:33 -07:00
Greg Fitzgerald
9232057e95 Rename replicator to archiver (#6464)
* Rename replicator to archiver

* cargo fmt

* Fix grammar
2019-10-21 11:29:37 -06:00
Greg Fitzgerald
2636418659 Move blocktree_processor to solana_ledger (#6460)
* Drop core::result dependency in bank_forks

* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
Pankaj Garg
badeb4d31a Rework shred headers to fix position of signature (#6451)
* Rework shred headers to fix position of signature

* fix clippy
2019-10-18 22:55:59 -07:00
Greg Fitzgerald
5468be2ef9 Add solana-ledger crate (#6415)
automerge
2019-10-18 09:28:51 -07:00
Pankaj Garg
33052c1dd2 Cleanup shred header structures (#6378)
automerge
2019-10-15 20:48:45 -07:00
sakridge
1afc527919 Lower cluster_info-num_nodes datapoint (#6368) 2019-10-15 14:42:19 -07:00
Pankaj Garg
364781366a Use sendmmsg for broadcasting shreds (#6325)
* Replace packet with slice of data in sendmmsg

* fixes

* fix bench
2019-10-10 19:38:48 -07:00
Greg Fitzgerald
c6e4641781 Remove many uses of legacy term 'fullnode' (#6324) 2019-10-10 17:33:00 -06:00
Pankaj Garg
753bd77b41 Use multicast to send retransmit packets (#6319) 2019-10-10 15:02:36 -07:00
sakridge
1b775044f7 Use multiple retransmit stage threads/sockets (#6279) 2019-10-10 13:24:03 -07:00
carllin
dd66d16fdb Broadcast final shred for slots that are interrupted (#6269)
* Broadcast final shred for slots that are interrupted
2019-10-09 16:07:18 -07:00
Sagar Dhawan
32312f3c16 Do not retransmit Repair responses (#6284)
* Do not retransmit Repair responses

* Add a test

* Refactor neighboring functionality
2019-10-09 13:11:19 -07:00
Sagar Dhawan
723f9a9b81 Remove unnecessary locking in retransmit stage (#6276)
* Add more detailed metrics to retransmit

* Remove unnecessary locking and add more metrics
2019-10-08 14:41:16 -07:00
carllin
ac2374e9a1 Shred entries in parallel (#6180)
* Make shredding more parallel

* Fix erasure tests

* Fix replicator test

* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
sakridge
5ef012b2c1 Tweak debug to remove unreadable datapoints (#6060) 2019-10-04 16:25:22 -07:00
Michael Vines
7f53737000 Periodically pull from the entrypoint if it's no longer in Crdt (#6240) 2019-10-04 14:18:07 -07:00
Sagar Dhawan
23ea8ae56b Optimize retransmit stage (#6231)
* Optimize retransmit stage

* Remove comment

* Fix test

* Skip iteration to fixup 0 stakes
2019-10-04 11:52:02 -07:00
Michael Vines
f9f5bc2eb5 More clippy 2019-10-02 21:21:07 -07:00
sakridge
ae7700296d broadcast_shreds opt (#6175)
* Don't clone/copy/sort ContactInfo vec
2019-10-01 09:38:29 -07:00
Pankaj Garg
57e90948a8 Remove dead code from cluster_info (#6051) 2019-09-24 10:27:59 -07:00
Sagar Dhawan
d379786c90 Fix bind errors (#5986)
* Add ability to bind to a common tcp/udp port

* Extend port range for local-net sanity and fix validator executable
2019-09-19 17:16:22 -07:00
Pankaj Garg
783e8672e7 Removed Shred enum (#5963)
* Remove shred enum and it's references

* rename ShredInfo to Shred

* clippy
2019-09-18 16:24:30 -07:00
Pankaj Garg
6c4e656795 Remove obsoleted code from shred (#5954)
* Remove obsoleted code from shred

* fix broken test
2019-09-18 13:56:44 -07:00
Pankaj Garg
ff608992ee Replace Shred usage with ShredInfo (#5939)
* Replace Shred usage with ShredInfo

* Fix tests

* fix clippy
2019-09-17 18:22:46 -07:00
Sagar Dhawan
c44e7ce184 Leaders should not broadcast to replicators (#5917) 2019-09-16 17:56:34 -07:00
Pankaj Garg
5f54573613 More shred related cleanup (#5909)
* More shred related cleanup

* fix uncle
2019-09-16 10:28:28 -07:00
Jack May
e8d88f3237 Split SDK's timing.rs (#5823) 2019-09-06 14:30:56 -07:00
Pankaj Garg
3d3b03a123 Verify signature of recovered shred before adding them to blocktree (#5811)
* Verify signature of recovered shred before adding them to blocktree

* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
Pankaj Garg
f78b865cba Cleanup shreds to remove FirstShred data structure (#5789)
* Cleanup shreds to remove FirstShred data structure

* Also reduce size used by parent slot information in shred header

* clippy

* fixes

* fix chacha test
2019-09-04 21:06:47 -07:00
Pankaj Garg
02ee2a601c Further cleanup of blocktree after Blob deprecation (#5780) 2019-09-04 12:47:09 -07:00
Pankaj Garg
3b0d48e3b8 Remove blocktree blob references (#5691)
* Remove blocktree blob references

* fixes and cleanup

* replace uninitialized() call with MaybeUninit

* fix bench
2019-09-03 21:32:51 -07:00
Michael Vines
a29f0484dc Add newline before cluster info log (#5671) 2019-08-27 08:33:48 -07:00
Michael Vines
db768b4c3a Log contact info every 10 seconds (#5663) 2019-08-26 18:31:14 -07:00
Pankaj Garg
4ac1213c9c Integrate coding shreds and recovery (#5625)
* Integrate coding shreds and recovery

* More tests for shreds and some fixes

* address review comments

* fixes to code shred generation

* unignore tests

* fixes to recovery
2019-08-26 18:27:45 -07:00
Pankaj Garg
4798e7fa73 Integrate data shreds (#5541)
* Insert data shreds in blocktree and database

* Integrate data shreds with rest of the code base

* address review comments, and some clippy fixes

* Fixes to some tests

* more test fixes

* ignore some local cluster tests

* ignore replicator local cluster tests
2019-08-20 17:16:06 -07:00
Sagar Dhawan
bd193535c9 Cap CrdsFilter sizes such that PullRequest no longer exceeds MTU (#5561) 2019-08-19 18:14:10 -07:00
Sagar Dhawan
e73cbdda61 Reduce log level for known issue (#5536)
automerge
2019-08-15 19:42:27 -07:00
Sagar Dhawan
4ee212ae4c Coalesce gossip pull requests and serve them in batches (#5501)
* Coalesce gossip pull requests and serve them in batches

* batch all filters and immediately respond to messages in gossip

* Fix tests

* make download_from_replicator perform a greedy recv
2019-08-15 17:04:45 -07:00
Sagar Dhawan
1d0608200c Restore blob size fix (#5516)
* Revert "Revert "Fix gossip messages growing beyond blob size  (#5460)" (#5512)"

This reverts commit 97d57d168b.

* Fix Crds filters
2019-08-13 18:04:14 -07:00
Sagar Dhawan
97d57d168b Revert "Fix gossip messages growing beyond blob size (#5460)" (#5512)
This reverts commit a8eb0409b7.
2019-08-13 10:29:26 -07:00
Justin Starry
0fde19239b Rate limit counter metrics points to one per second (#5496)
* Rate limit counter metrics points to one per second

* Remove old env var

* Test that metrics counter is incrementing

* Fix typo
2019-08-12 18:15:34 -04:00
Sagar Dhawan
a8eb0409b7 Fix gossip messages growing beyond blob size (#5460)
* fixed bloom filter math

* Add split each pull request into multiple pulls with different filters

* Rework CrdsFilter to generate all possible masks to cover the keyspace

* Limit the bloom sizes such that each pull request is no larger than mtu
2019-08-12 13:51:29 -07:00
Justin Starry
4f86c0b74a Rate limit transaction counters (#5447)
* Rate limit transaction counters

* @sakridge feedback

* Set default high metrics rate for multinode demo

* Fix tests

* Swap defaults and fix env var tests

* Only set metrics rate if not already set
2019-08-08 17:05:06 -04:00
Sagar Dhawan
d7a2b790dc Limit the size of gossip push and gossip pull response (#5348)
* Limit the size of gossip push and gossip pull response

* Remove Default::default

* Rename var
2019-07-30 15:43:17 -07:00
Pankaj Garg
a7a10e12c7 Forward transactions as packets instead of blobs (#5334)
* Forward transactions as packets instead of blobs

* clippy
2019-07-30 14:50:02 -07:00
Justin Starry
84368697af Fix metrics when leader does not report metrics (#5291) 2019-07-30 16:18:33 -04:00
Pankaj Garg
4c90898f0b Dynamic erasure set configuration (#5018)
* Use local erasure session to create/broadcast coding blobs

* Individual session for each recovery (as the config might be different)

* address review comments

* new constructors for session and coding generator

* unit test for dynamic erasure config
2019-07-11 13:58:33 -07:00
Rob Walker
f537482c86 remove set_leader from cluster_info (#4998) 2019-07-09 22:06:47 -07:00