Commit Graph

52 Commits

Author SHA1 Message Date
Brooks Prumo
45d54b1fc6 Add SnapshotArchiveInfo and refactor functions in snapshot_utils (#18232) 2021-07-01 12:20:56 -05:00
dependabot[bot]
05924423c2 chore: bump reqwest from 0.11.2 to 0.11.4 (#18362)
* chore: bump reqwest from 0.11.2 to 0.11.4

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.2 to 0.11.4.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.2...v0.11.4)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-07-01 09:59:30 -06:00
Brooks Prumo
89a3e4f91e Move SnapshotConfig into its own module (#18331)
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
2021-07-01 08:55:26 -05:00
dependabot[bot]
78968d132f chore: bump log from 0.4.11 to 0.4.14 (#18323)
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-30 18:32:01 +00:00
dependabot[bot]
8e1e57a181 chore: bump tar from 0.4.28 to 0.4.35 (#17666)
* chore: bump tar from 0.4.28 to 0.4.35

Bumps [tar](https://github.com/alexcrichton/tar-rs) from 0.4.28 to 0.4.35.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases)
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.28...0.4.35)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-06-02 14:17:58 -06:00
dependabot[bot]
a43c29e858 Bump indicatif from 0.15.0 to 0.16.2 (#17628)
* Bump indicatif from 0.15.0 to 0.16.2

Bumps [indicatif](https://github.com/mitsuhiko/indicatif) from 0.15.0 to 0.16.2.
- [Release notes](https://github.com/mitsuhiko/indicatif/releases)
- [Commits](https://github.com/mitsuhiko/indicatif/compare/0.15.0...0.16.2)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Fix message types

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-01 17:38:02 +00:00
dependabot[bot]
ec1a307a7c Bump console from 0.11.3 to 0.14.1 (#16301)
* Bump console from 0.11.3 to 0.14.1

Bumps [console](https://github.com/mitsuhiko/console) from 0.11.3 to 0.14.1.
- [Release notes](https://github.com/mitsuhiko/console/releases)
- [Changelog](https://github.com/mitsuhiko/console/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/console/compare/0.11.3...0.14.1)

Signed-off-by: dependabot[bot] <support@github.com>

* Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-05-27 18:55:27 +00:00
Michael Vines
8eab0e8602 Bump version to v1.8.0 (#17541) 2021-05-27 08:51:53 -07:00
Lijun Wang
54f0fc9f0f Use type alias for DownloadProgress callback (#17518)
Convert to use type alias for the callback and cascade the changes to callers. Thanks @jeffwashington for the help making it possible.
Changed the closure for the progress update in the validator main to FnMut and modify the abort count in the closure which is more reliable.
2021-05-26 13:26:07 -07:00
Lijun Wang
4c17243157 snapshot download enhancement (#17415)
1. Allow the validator bootstrap code to specify the minimal snapshot download speed. If the snapshot download speed is detected below that, a different RPC can be retried. The default is 10MB/sec.

2. To prevent spinning on a number of sub-optimal choices and not making progress, the abort/retry logic is implemented with the following safe guards:
2.1 at maximum we do this retry for 5 times -- this number is configurable with default 5.
2.2 if the download in one notification round (5 second) is more than 2%, do not do retry -- it is not as bad anyway.
2.3 if the remaining estimate time is less than 1 minutes, do not abort retry as it will be done quickly anyway.
2.4 We do this abort/retry logic only at the first notification to avoid wasting download efforts -- the reasoning is being opportunistic and too greedy may not achieve overall shorter download time.

3. The download_snapshot and download_file is modified with the option allowing caller to notified of download progress via a callback. This allows the business logic of retrying to the place it belongs.
2021-05-25 09:32:12 -07:00
sakridge
a8dca3976b Refactor genesis download/load/check functions (#17276)
* Refactor genesis ingest functions

* Consolidate genesis.bin/genesis.tar.bz2 references
2021-05-24 16:45:36 +02:00
Lijun Wang
9c42a89a43 Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Trent Nelson
e61b4b7d70 dl-utils: use wide_msg everywhere for truncation on narrow terminals 2021-04-14 23:50:42 +00:00
Michael Vines
02b81dd05d Update to reqwest 0.11.2 2021-03-18 11:59:41 -07:00
DimAn
0209d334bd Add option for separate snapshot location
(cherry picked from commit 6126878f509c69e23480a5ec22b3271e2b16e072)
2021-03-17 10:31:01 -07:00
Michael Vines
58b980f9cd Download snapshot files with a tmp- prefix so they'll automatically be cleaned up if interrupted 2021-03-17 09:10:10 -07:00
Michael Vines
a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Michael Vines
0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Trent Nelson
24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
Trent Nelson
7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Jeff Washington (jwash)
600ff0d915 calculate hash from store instead of index (#15034)
* calculate hash from store instead of index

* restore update hash in abs
2021-02-04 09:00:33 -06:00
Michael Vines
bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
Michael Vines
7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
Michael Vines
90deadb396 Avoid creating "..tmp" files 2020-12-28 08:58:09 -08:00
sakridge
7b49c85aa7 Fix download speed (#14291) 2020-12-27 16:25:41 -08:00
Michael Vines
93ae177503 Do not delete ALL other snapshots before downloading a new snapshot 2020-12-21 08:58:52 +00:00
Michael Vines
1e977ac025 Bump version to v1.6.0 2020-12-15 18:28:04 +00:00
dependabot[bot]
88d950e5a8 chore: bump log from 0.4.8 to 0.4.11 (#13691)
* chore: bump log from 0.4.8 to 0.4.11

Bumps [log](https://github.com/rust-lang/log) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
sakridge
7ee0c9e68a Add non-progress bar download status (#13370) 2020-11-04 10:32:27 -08:00
Michael Vines
8a119c1483 Bump reqwest/rayon to get past cargo audit 2020-10-11 18:51:09 +00:00
Michael Vines
11df2e2236 Bump version to v1.5.0 2020-10-08 04:51:36 +00:00
Trent Nelson
e12ab9d0dd Bump version to 1.4.0 2020-08-05 12:04:15 -06:00
Greg Fitzgerald
6ee222363e Move BankForks to solana_runtime (#10637)
* Move BankForks to solana_runtime

* Update imports
2020-06-17 15:27:03 +00:00
dependabot-preview[bot]
943ae9793d Bump indicatif from 0.14.0 to 0.15.0
Bumps [indicatif](https://github.com/mitsuhiko/indicatif) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/mitsuhiko/indicatif/releases)
- [Commits](https://github.com/mitsuhiko/indicatif/compare/0.14.0...0.15.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-14 23:58:57 -07:00
R. M. Shea
48febf153b Change documentation introduction to indicate Foundation IP transfer (#10440) 2020-06-13 14:41:05 -07:00
dependabot-preview[bot]
6cf30c38a4 Bump reqwest from 0.10.5 to 0.10.6
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.10.5 to 0.10.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-03 09:01:36 -07:00
dependabot-preview[bot]
b787254bd6 Bump console from 0.10.3 to 0.11.3 (#10314)
Bumps [console](https://github.com/mitsuhiko/console) from 0.10.3 to 0.11.3.
- [Release notes](https://github.com/mitsuhiko/console/releases)
- [Commits](https://github.com/mitsuhiko/console/commits/v0.11.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-30 07:47:41 -06:00
Michael Vines
9eefb2c929 Bump version to 1.3.0 2020-05-28 15:01:13 -07:00
dependabot-preview[bot]
2d3a906d55 Bump tar from 0.4.27 to 0.4.28 (#10185)
Bumps [tar](https://github.com/alexcrichton/tar-rs) from 0.4.27 to 0.4.28.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases)
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.27...0.4.28)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-22 15:55:40 -07:00
dependabot-preview[bot]
8da2e1b2f7 Bump tar from 0.4.26 to 0.4.27 (#10150)
Bumps [tar](https://github.com/alexcrichton/tar-rs) from 0.4.26 to 0.4.27.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases)
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.26...0.4.27)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-20 20:04:22 -07:00
Michael Vines
d67ad70443 Revert "Bump console from 0.10.1 to 0.11.2 (#9763)"
This reverts commit 2de999fb61.
2020-04-28 10:43:18 -07:00
dependabot-preview[bot]
2de999fb61 Bump console from 0.10.1 to 0.11.2 (#9763)
Bumps [console](https://github.com/mitsuhiko/console) from 0.10.1 to 0.11.2.
- [Release notes](https://github.com/mitsuhiko/console/releases)
- [Commits](https://github.com/mitsuhiko/console/compare/0.10.1...0.11.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-04-28 09:36:07 -07:00
dependabot-preview[bot]
c5a7df9221 Bump console from 0.10.0 to 0.10.1 (#9726)
Bumps [console](https://github.com/mitsuhiko/console) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/mitsuhiko/console/releases)
- [Commits](https://github.com/mitsuhiko/console/compare/0.10.0...0.10.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-04-27 09:16:24 -06:00
Michael Vines
47ae57610a Only build x86_64-unknown-linux-gnu on docs.rs 2020-04-16 19:06:17 -07:00
sakridge
9493de4443 Add snapshot compression option (#9276) 2020-04-03 13:13:49 -07:00
Michael Vines
d7fa40087c Bump version to 1.2.0 2020-03-28 09:44:13 -07:00
Ryo Onodera
f987c18a7e Strictly validate the contents of snapshot/genesis (#8959)
automerge
2020-03-25 02:46:41 -07:00
Michael Vines
bf60345b7a Remove all snapshots not matching the desired hash 2020-03-17 22:58:01 -07:00
Michael Vines
3a501ad69e Remove all snapshot not matching the desired hash 2020-03-17 22:18:25 -07:00
dependabot-preview[bot]
215650f6e7 Bump console from 0.9.2 to 0.10.0 (#8786)
Bumps [console](https://github.com/mitsuhiko/console) from 0.9.2 to 0.10.0.
- [Release notes](https://github.com/mitsuhiko/console/releases)
- [Commits](https://github.com/mitsuhiko/console/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-11 12:30:15 -06:00