155 Commits

Author SHA1 Message Date
587d45769d Thin client quic (#23973) (#24266)
Change thin-client to use connection-cache

(cherry picked from commit 8b72200afb)

Co-authored-by: ryleung-solana <91908731+ryleung-solana@users.noreply.github.com>
2022-04-11 23:54:39 -06:00
0c684721d8 Separate remotely downloaded snapshot archives (#23510)
* seperate remotely downloaded snapshot archives

* add str const for snapshot download dir

* only walk remote sub directory

* move directory creation outside of loop

* move is_remote to traits

* clippy simplify

* clippy

* clippy

* add unittest

* fix local cluster tests

* look for remote snapshot archive in remote foler

* create remote dir in tests

* use snapshot download dir constant

* extract build_remote_dir fn

* fix build

* code review - walking snapshot archived dirs explicitly

* fix build

* fix build

* fix comments

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* clippy

* borrow to avoid copy

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-14 14:03:59 -05:00
17b00ad3a4 Add quic-client module (#23166)
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
9bbccbe27c Use AsRef<Path> instead of PathBuf for parameters (#23560) 2022-03-09 16:08:33 -06:00
bca1d51735 Fix flaky optimistic confirmation tests (#23178) 2022-02-16 16:13:58 -05:00
ab92578b02 Fix the flaky test test_restart_tower_rollback (#23129)
* Add flag to disable voting until a slot to avoid duplicate voting

* Fix the tower rollback test and remove it from flaky.
2022-02-15 13:19:34 -07:00
5acf0f6331 Add feature gate for new vote instruction and plumb through replay (#21683)
* Add feature gate for new vote instruction and plumb through replay

Add tower versions

* Add check for slot hashes history

* Update is_recent check to exclude voting on hard fork root slot

* Move tower rollback test to flaky and ignore it until #22551 lands
2022-02-07 14:06:19 -08:00
eac4a6df68 rpc: use minimal mode by default 2022-02-01 19:00:06 -07:00
d9c259a231 Set the correct root in block commitment cache initialization (#22750)
* Set the correct root in block commitment cache initialization

* clean up test

* bump
2022-01-27 00:48:00 +08:00
6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
4ab7d6c23e Filter out outdated slots (#22450)
* Filter out outdated slots

* Fixup error
2022-01-13 19:51:00 -05:00
eeec1ce2ad Add local cluster test to repro slot hash expiry bug (#21873) 2022-01-10 00:58:21 -05:00
0e1afcbb26 Split up local cluster tests into separate CI steps (#22295)
* Split up local cluster tests into separate CI steps

* Update buildkite-pipeline.sh
2022-01-05 14:44:15 +00:00
e8b7f96a89 Add struct BlockstoreOptions (#22121) 2022-01-03 18:30:45 -10:00
8d980f07ba uses Option<Slot> for SlotMeta.parent_slot (#21808)
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
f0acf7681e Add vote instructions that directly update on chain vote state (#21531)
* Add vote state instructions

UpdateVoteState and UpdateVoteStateSwitch

* cargo tree

* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
f493a88258 Fixup flaky tests (#21617)
* Fixup flaky tests

* Fixup listeners
2021-12-06 17:14:38 -05:00
b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
b30c94ce55 ClusterInfoVoteListener send only missing votes to BankingStage (#20873) 2021-11-18 15:20:41 -08:00
b0ca335463 Rename "trusted" to "known" in validators/ (#21197)
* Replaced trusted with known validator

* Format Convention
2021-11-12 11:57:55 -07:00
7bb347faac Tone down the optimistic confirmation printing (#21045) 2021-10-28 15:14:36 +02:00
44ff30b65b Retry SampleNotDuplicateConfirmed decisions in AncestorHashesService (#20240) 2021-10-15 11:40:03 -07:00
7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
258c3bca65 Fix copy fn in local-cluster test (#20424) 2021-10-04 19:04:12 -05:00
2b967202f3 Add tower rollback test (#20158) 2021-10-04 15:21:19 -07:00
4895c69fea Add test for startup processing new roots past full snapshot interval (#19876) 2021-09-20 18:50:29 -05:00
79ade5ec68 Add test_incremental_snapshot_download() to local-cluster (#19746) 2021-09-13 21:44:48 -05:00
62c8bcf565 Add default() to SnapshotConfig (#19776) 2021-09-12 13:44:27 -05:00
9d9482b9d8 Plumb maximum_incremental_snapshot_archives_to_retain (#19640) 2021-09-06 18:01:56 -05:00
5e25ee5ebe Add maximum_incremental_snapshot_archives_to_retain to SnapshotConfig (#19612) 2021-09-03 20:21:32 +00:00
7ab0aec61f Rename maximum_full_snapshot_archives_to_retain (#19610)
To prepare for adding maximum_incremental_snapshot_archives_to_retain,
rename the current field in SnapshotConfig.
2021-09-03 11:28:10 -05:00
1d5a8ebc6a Revert "Add LastFullSnapshotSlot to SnapshotConfig (#19341)" (#19529)
This reverts commit 4d361af976.
2021-08-31 22:03:19 -05:00
aab2df208a fixup! Fix fork race condition in optimistic violation tower tests (#19192) (#19459) 2021-08-27 00:21:33 -05:00
73c1ea7de6 Fix fork race condition in optimistic violation tower tests (#19192)
* Fix fork race condition in optimistic violation tower tests

* clippy

* pr comments
2021-08-26 21:41:05 -07:00
6d939811e9 Name snapshots consistently (#19346)
#### Problem

Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:

```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```

#### Summary of Changes

For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.


Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
05e7d1027b Fix flaky test test_fork_choice_refresh_old_votes (#19278)
* Fix flaky test test_fork_choice_refresh_old_votes

* Wait for latest voted block to arrive in blockstore

* Wait for block to be full
2021-08-20 14:15:11 -07:00
4d361af976 Add LastFullSnapshotSlot to SnapshotConfig (#19341) 2021-08-20 17:06:53 +00:00
c50b01cb60 Store versioned transactions in the ledger, disabled by default (#19139)
* Add support for versioned transactions, but disable by default

* merge conflicts

* trent's feedback

* bump Cargo.lock

* Fix transaction error encoding

* Rename legacy_transaction method

* cargo clippy

* Clean up casts, int arithmetic, and unused methods

* Check for duplicates in sanitized message conversion

* fix clippy

* fix new test

* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
22674000bd Add EpochSlots frozen state transition (#19112) 2021-08-13 14:21:52 -07:00
0b50bb2b20 Deprecate FeeCalculator returning APIs (#19120) 2021-08-13 09:08:20 -07:00
e9722474eb Move tower storage into its own module 2021-08-11 00:20:46 -07:00
d7ab510229 Move tower save into the VotingService 2021-08-11 00:20:46 -07:00
85befbc8cd Removes bpf_loader_upgradeable_program feature gate. (#19102) 2021-08-11 00:08:57 +02:00
fd937548a0 Move SnapshotArchiveInfo and friends into its own module (#19114) 2021-08-08 07:57:06 -05:00
00890957ee Add snapshot_utils::bank_from_latest_snapshot_archives() (#18983)
While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots.  A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify.  Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled.  Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`.  And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats.  This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
397801a2d8 Extract tower storage details from Tower struct 2021-08-06 10:04:37 -07:00
ca14475085 Add incremental_snapshot_archive_interval_slots to SnapshotConfig (#19026)
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
2021-08-04 14:40:20 -05:00
d2d5f36a3c adds validator flag to allow private ip addresses (#18850) 2021-07-23 15:25:03 +00:00
d1debcd971 Add incremental snapshot utils (#18504)
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
2021-07-22 14:40:37 -05:00