* Cluster Replicated Data Store
Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.
Simulation shows fast convergence with 20k nodes.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
* Use multiple sockets for receiving blobs on validators
- The blobs that are broadcasted by leader or retransmitted by peer
validators are received on replicate_port
- Using reuse_addr/reuse_port, multiple sockets can be opened for
the same port
- This allows the kernel to queue data to user space app on multiple
socket queues, preventing over-running one queue
- This helps with reducing packets dropped due to queue over-runs
Fixes#1224
* Fixed failing tests
* rename NodeInfo field of Node from "data" to "info"
(touches a lot of files)
* update client to use gossip to find leader, a la drone
* rework multinode scripts
* move more stuff into rust
* added usage to all
* no more rsync unless you're a validator (TODO: whack that, too)
* fullnode doesn't bail if drone isn't up yet, just keeps trying
* drone doesn't bail if network isn't up yet, just keeps trying
* remove trailing whitespace in ci/audit.sh
* code review fixups
* rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
* remove out-of-date TODO in localnet-sanity.sh
* remove features=test and code that was using it (localhost prohibitions in
crdt) added TODO in crdt.rs, maybe we should boot localhost in production
networks?
* boot tvu_window from NodeInfo: instead, send repair requests from the repair
socket (to gossip on peer) and answer repair requests via the sockaddr
from the repair request
* remove various unused pub functions
* banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
* move gossip/NCP off assuming anything about its address
* use a single socket to send and receive gossip
* remove --addr/-a from CLIs
* rearrange networking utility code
* use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
* rename TestNode to Node
TODO:
* re-enable 127.0.0.1 as a valid address in crdt
* change repair request/response to a similar, single socket
* pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
* update contact_info with network truthiness instead of what the node
says?
* log responder error to warn
* log responder error to warn
* fixup!
* fixed assert
* fixed bad ports issue
* comments
* test for dummy address in Crdt::new instaad of NodeInfo::new
* return error if ContactInfo supplied to Crdt::new cannot be used to connect to network
* comments
