gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Accountsdb replication installment 2 (#19325)

Browse Source 
This is the 2nd installment for the AccountsDb replication.

Summary of Changes

The basic google protocol buffer protocol for replicating updated slots and accounts. tonic/tokio is used for transporting the messages.

The basic framework of the client and server for replicating slots and accounts -- the persisting of accounts in the replica-side will be done at the next PR -- right now -- the accounts are streamed to the replica-node and dumped. Replication for information about Bank is also not done in this PR -- to be addressed in the next PR to limit the change size.

Functionality used by both the client and server side are encapsulated in the replica-lib crate.

There is no impact to the existing validator by default.

Tests:

Observe the confirmed slots replicated to the replica-node.
Observe the accounts for the confirmed slot are received at the replica-node side.
This commit is contained in:
Lijun Wang
2021-09-01 14:10:16 -07:00
committed by GitHub
parent 27c2180db9
commit 8378e8790f
28 changed files with 994 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
replica-node/src/main.rs
View File

@@ -67,12 +67,28 @@ pub fn main() {
.help("Use DIR as snapshot location [default: --ledger value]"),
)
.arg(
Arg::with_name("peer")
.long("peer")
.value_name("IP:PORT")
Arg::with_name("peer_address")
.long("peer-address")
.value_name("IP")
.takes_value(true)
.required(true)
.help("The the IP:PORT for the peer validator/replica to download from"),
.help("The the address for the peer validator/replica to download from"),
)
.arg(
Arg::with_name("peer_rpc_port")
.long("peer-rpc-port")
.value_name("PORT")
.takes_value(true)
.required(true)
.help("The the PORT for the peer validator/replica from which to download the snapshots"),
)
.arg(
Arg::with_name("peer_accountsdb_repl_port")
.long("peer-accountsdb-repl-port")
.value_name("PORT")
.takes_value(true)
.required(true)
.help("The the PORT for the peer validator/replica serving the AccountsDb replication"),
)
.arg(
Arg::with_name("peer_pubkey")
@@ -296,19 +312,30 @@ pub fn main() {
vec![ledger_path.join("accounts")]
};
let rpc_source_addr =
solana_net_utils::parse_host_port(matches.value_of("peer").unwrap_or_else(|| {
let peer_address = solana_net_utils::parse_host(matches.value_of("peer_address").unwrap())
.expect("invalid peer_address");
let peer_rpc_port = value_t!(matches, "peer_rpc_port", u16).unwrap_or_else(|_| {
clap::Error::with_description(
"The --peer-rpc-port <PORT> argument is required",
clap::ErrorKind::ArgumentNotFound,
)
.exit();
});
let rpc_peer_addr = SocketAddr::new(peer_address, peer_rpc_port);
let peer_accountsdb_repl_port = value_t!(matches, "peer_accountsdb_repl_port", u16)
.unwrap_or_else(|_| {
clap::Error::with_description(
"The --peer <IP:PORT> argument is required",
"The --peer-accountsdb-repl-port <PORT> argument is required",
clap::ErrorKind::ArgumentNotFound,
)
.exit();
}))
.unwrap_or_else(|e| {
eprintln!("failed to parse entrypoint address: {}", e);
exit(1);
});
let accountsdb_repl_peer_addr = SocketAddr::new(peer_address, peer_accountsdb_repl_port);
let rpc_port = value_t!(matches, "rpc_port", u16).unwrap_or_else(|_| {
clap::Error::with_description(
"The --rpc-port <PORT> argument is required",
@@ -358,7 +385,8 @@ pub fn main() {
);
let config = ReplicaNodeConfig {
rpc_source_addr,
rpc_peer_addr,
accountsdb_repl_peer_addr: Some(accountsdb_repl_peer_addr),
rpc_addr: rpc_addrs.0,
rpc_pubsub_addr: rpc_addrs.1,
ledger_path,
@@ -376,6 +404,6 @@ pub fn main() {
replica_exit: Arc::new(RwLock::new(Exit::default())),
};
let validator = ReplicaNode::new(config);
validator.join();
let replica = ReplicaNode::new(config);
replica.join();
}