gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add incremental snapshot utils (#18504)

Browse Source 
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
This commit is contained in:
Brooks Prumo
2021-07-22 14:40:37 -05:00
committed by GitHub
parent 7fc4cfebc8
commit d1debcd971
19 changed files with 1773 additions and 507 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
download-utils/src/lib.rs
View File

@@ -246,13 +246,13 @@ pub fn download_genesis_if_missing(
pub fn download_snapshot<'a, 'b>(
rpc_addr: &SocketAddr,
snapshot_output_dir: &Path,
snapshot_archives_dir: &Path,
desired_snapshot_hash: (Slot, Hash),
use_progress_bar: bool,
maximum_snapshots_to_retain: usize,
progress_notify_callback: &'a mut DownloadProgressCallbackOption<'b>,
) -> Result<(), String> {
snapshot_utils::purge_old_snapshot_archives(snapshot_output_dir, maximum_snapshots_to_retain);
snapshot_utils::purge_old_snapshot_archives(snapshot_archives_dir, maximum_snapshots_to_retain);
for compression in &[
ArchiveFormat::TarZstd,
@@ -260,8 +260,8 @@ pub fn download_snapshot<'a, 'b>(
ArchiveFormat::TarBzip2,
ArchiveFormat::Tar, // `solana-test-validator` creates uncompressed snapshots
] {
let desired_snapshot_package = snapshot_utils::build_snapshot_archive_path(
snapshot_output_dir.to_path_buf(),
let desired_snapshot_package = snapshot_utils::build_full_snapshot_archive_path(
snapshot_archives_dir.to_path_buf(),
desired_snapshot_hash.0,
&desired_snapshot_hash.1,
*compression,