Commit Graph

165 Commits

Author SHA1 Message Date
HaoranYi
0c684721d8 Separate remotely downloaded snapshot archives (#23510)
* seperate remotely downloaded snapshot archives

* add str const for snapshot download dir

* only walk remote sub directory

* move directory creation outside of loop

* move is_remote to traits

* clippy simplify

* clippy

* clippy

* add unittest

* fix local cluster tests

* look for remote snapshot archive in remote foler

* create remote dir in tests

* use snapshot download dir constant

* extract build_remote_dir fn

* fix build

* code review - walking snapshot archived dirs explicitly

* fix build

* fix build

* fix comments

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* clippy

* borrow to avoid copy

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-14 14:03:59 -05:00
ryleung-solana
17b00ad3a4 Add quic-client module (#23166)
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
Brooks Prumo
9bbccbe27c Use AsRef<Path> instead of PathBuf for parameters (#23560) 2022-03-09 16:08:33 -06:00
carllin
bca1d51735 Fix flaky optimistic confirmation tests (#23178) 2022-02-16 16:13:58 -05:00
Ashwin Sekar
ab92578b02 Fix the flaky test test_restart_tower_rollback (#23129)
* Add flag to disable voting until a slot to avoid duplicate voting

* Fix the tower rollback test and remove it from flaky.
2022-02-15 13:19:34 -07:00
Yueh-Hsuan Chiang
9213fcb11b Introduce experimental FIFO-compaction option for shreds in blockstore (#22140) 2022-02-10 11:34:03 -08:00
Justin Starry
d5dec989b9 Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Ashwin Sekar
5acf0f6331 Add feature gate for new vote instruction and plumb through replay (#21683)
* Add feature gate for new vote instruction and plumb through replay

Add tower versions

* Add check for slot hashes history

* Update is_recent check to exclude voting on hard fork root slot

* Move tower rollback test to flaky and ignore it until #22551 lands
2022-02-07 14:06:19 -08:00
Trent Nelson
eac4a6df68 rpc: use minimal mode by default 2022-02-01 19:00:06 -07:00
Justin Starry
d9c259a231 Set the correct root in block commitment cache initialization (#22750)
* Set the correct root in block commitment cache initialization

* clean up test

* bump
2022-01-27 00:48:00 +08:00
Michael Vines
6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Justin Starry
7f20c6149e Refactor: move simple vote parsing to runtime (#22537) 2022-01-20 10:39:21 +08:00
carllin
4ab7d6c23e Filter out outdated slots (#22450)
* Filter out outdated slots

* Fixup error
2022-01-13 19:51:00 -05:00
Ashwin Sekar
eeec1ce2ad Add local cluster test to repro slot hash expiry bug (#21873) 2022-01-10 00:58:21 -05:00
Yueh-Hsuan Chiang
5771c36d3f Rename open_with_access_type() to open_with_options() (#22123) 2022-01-07 12:11:43 -08:00
Justin Starry
0e1afcbb26 Split up local cluster tests into separate CI steps (#22295)
* Split up local cluster tests into separate CI steps

* Update buildkite-pipeline.sh
2022-01-05 14:44:15 +00:00
Yueh-Hsuan Chiang
e8b7f96a89 Add struct BlockstoreOptions (#22121) 2022-01-03 18:30:45 -10:00
behzad nouri
8d980f07ba uses Option<Slot> for SlotMeta.parent_slot (#21808)
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
Ashwin Sekar
f0acf7681e Add vote instructions that directly update on chain vote state (#21531)
* Add vote state instructions

UpdateVoteState and UpdateVoteStateSwitch

* cargo tree

* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
carllin
f493a88258 Fixup flaky tests (#21617)
* Fixup flaky tests

* Fixup listeners
2021-12-06 17:14:38 -05:00
Michael Vines
b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Michael Vines
ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
carllin
b30c94ce55 ClusterInfoVoteListener send only missing votes to BankingStage (#20873) 2021-11-18 15:20:41 -08:00
Michael Keleti
b0ca335463 Rename "trusted" to "known" in validators/ (#21197)
* Replaced trusted with known validator

* Format Convention
2021-11-12 11:57:55 -07:00
sakridge
7bb347faac Tone down the optimistic confirmation printing (#21045) 2021-10-28 15:14:36 +02:00
carllin
44ff30b65b Retry SampleNotDuplicateConfirmed decisions in AncestorHashesService (#20240) 2021-10-15 11:40:03 -07:00
Michael Vines
7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
Brooks Prumo
258c3bca65 Fix copy fn in local-cluster test (#20424) 2021-10-04 19:04:12 -05:00
carllin
2b967202f3 Add tower rollback test (#20158) 2021-10-04 15:21:19 -07:00
Brooks Prumo
4895c69fea Add test for startup processing new roots past full snapshot interval (#19876) 2021-09-20 18:50:29 -05:00
Brooks Prumo
79ade5ec68 Add test_incremental_snapshot_download() to local-cluster (#19746) 2021-09-13 21:44:48 -05:00
Brooks Prumo
62c8bcf565 Add default() to SnapshotConfig (#19776) 2021-09-12 13:44:27 -05:00
Brooks Prumo
9d9482b9d8 Plumb maximum_incremental_snapshot_archives_to_retain (#19640) 2021-09-06 18:01:56 -05:00
Brooks Prumo
5e25ee5ebe Add maximum_incremental_snapshot_archives_to_retain to SnapshotConfig (#19612) 2021-09-03 20:21:32 +00:00
Brooks Prumo
7ab0aec61f Rename maximum_full_snapshot_archives_to_retain (#19610)
To prepare for adding maximum_incremental_snapshot_archives_to_retain,
rename the current field in SnapshotConfig.
2021-09-03 11:28:10 -05:00
Brooks Prumo
1d5a8ebc6a Revert "Add LastFullSnapshotSlot to SnapshotConfig (#19341)" (#19529)
This reverts commit 4d361af976.
2021-08-31 22:03:19 -05:00
steviez
aab2df208a fixup! Fix fork race condition in optimistic violation tower tests (#19192) (#19459) 2021-08-27 00:21:33 -05:00
Ashwin Sekar
73c1ea7de6 Fix fork race condition in optimistic violation tower tests (#19192)
* Fix fork race condition in optimistic violation tower tests

* clippy

* pr comments
2021-08-26 21:41:05 -07:00
Brooks Prumo
6d939811e9 Name snapshots consistently (#19346)
#### Problem

Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:

```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```

#### Summary of Changes

For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.


Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
Ashwin Sekar
05e7d1027b Fix flaky test test_fork_choice_refresh_old_votes (#19278)
* Fix flaky test test_fork_choice_refresh_old_votes

* Wait for latest voted block to arrive in blockstore

* Wait for block to be full
2021-08-20 14:15:11 -07:00
Brooks Prumo
4d361af976 Add LastFullSnapshotSlot to SnapshotConfig (#19341) 2021-08-20 17:06:53 +00:00
Justin Starry
c50b01cb60 Store versioned transactions in the ledger, disabled by default (#19139)
* Add support for versioned transactions, but disable by default

* merge conflicts

* trent's feedback

* bump Cargo.lock

* Fix transaction error encoding

* Rename legacy_transaction method

* cargo clippy

* Clean up casts, int arithmetic, and unused methods

* Check for duplicates in sanitized message conversion

* fix clippy

* fix new test

* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
carllin
22674000bd Add EpochSlots frozen state transition (#19112) 2021-08-13 14:21:52 -07:00
Jack May
0b50bb2b20 Deprecate FeeCalculator returning APIs (#19120) 2021-08-13 09:08:20 -07:00
Michael Vines
e9722474eb Move tower storage into its own module 2021-08-11 00:20:46 -07:00
Michael Vines
d7ab510229 Move tower save into the VotingService 2021-08-11 00:20:46 -07:00
Alexander Meißner
85befbc8cd Removes bpf_loader_upgradeable_program feature gate. (#19102) 2021-08-11 00:08:57 +02:00
Brooks Prumo
fd937548a0 Move SnapshotArchiveInfo and friends into its own module (#19114) 2021-08-08 07:57:06 -05:00
Brooks Prumo
00890957ee Add snapshot_utils::bank_from_latest_snapshot_archives() (#18983)
While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots.  A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify.  Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled.  Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`.  And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats.  This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
Michael Vines
397801a2d8 Extract tower storage details from Tower struct 2021-08-06 10:04:37 -07:00