gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
17,954 Commits 81 Branches 357 Tags
115f376465a237695b4fec2a7fd33b237c6fb6b7
Commit Graph

274 Commits

Author SHA1 Message Date
HaoranYi
0c684721d8 Separate remotely downloaded snapshot archives (#23510) 
* seperate remotely downloaded snapshot archives

* add str const for snapshot download dir

* only walk remote sub directory

* move directory creation outside of loop

* move is_remote to traits

* clippy simplify

* clippy

* clippy

* add unittest

* fix local cluster tests

* look for remote snapshot archive in remote foler

* create remote dir in tests

* use snapshot download dir constant

* extract build_remote_dir fn

* fix build

* code review - walking snapshot archived dirs explicitly

* fix build

* fix build

* fix comments

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* clippy

* borrow to avoid copy

Co-authored-by: Brooks Prumo <brooks@prumo.org>
 2022-03-14 14:03:59 -05:00
Will Hickey
63bf0f66af Bump version to 1.10.3 (#23648) 2022-03-14 11:18:45 -05:00
Yueh-Hsuan Chiang
1e20bd8f9a (LedgerStore) Include storage type as a tag in RocksDB metric reporting (#23523) 
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.

To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type. 
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.

Dependency: #23580
 2022-03-11 15:17:34 -08:00
Brooks Prumo
d20dd21600 Sort tables in Cargo.toml files (#23602) 2022-03-11 11:05:57 -06:00
Will Hickey
b444836a97 Bump version to 1.10.2 (#23597) 2022-03-10 16:41:06 -06:00
ryleung-solana
17b00ad3a4 Add quic-client module (#23166) 
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
 2022-03-09 21:33:05 -05:00
Brooks Prumo
9bbccbe27c Use AsRef<Path> instead of PathBuf for parameters (#23560) 2022-03-09 16:08:33 -06:00
Yueh-Hsuan Chiang
62d2a4cd88 Make ShredStorageType::RocksLevel public (#23272) 
#### Summary of Changes
This PR adds two hidden arguments to the validator that allow users to use RocksDB's FIFO compaction for storing shreds.

        --shred-storage <SHRED_STORAGE>
            EXPERIMENTAL: Controls how RocksDB compacts shreds.  *WARNING*: You will lose your ledger data
            when you switch between options. Possible values are: 'level': stores shreds using RocksDB's default (level)
            compaction. 'fifo': stores shreds under RocksDB's FIFO compaction. This option is more efficient on
            disk-write-bytes of the ledger store. [default: level]  [possible values: level, fifo]

        --shred-storage-size <SHRED_STORAGE_SIZE_BYTES>
            The shred storage size in bytes. The suggested value is 50% of your ledger storage size in bytes. [default:
            268435456000]
 2022-03-03 12:43:58 -08:00
Will Hickey
1a99251498 Bump version to 1.10.1 (#23453) 2022-03-02 13:47:01 -06:00
dependabot[bot]
ce4d579499 chore: bump serial_test from 0.5.1 to 0.6.0 (#23414) 
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.5.1...v0.6.0)

---
updated-dependencies:
- dependency-name: serial_test
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 12:13:40 -07:00
carllin
bca1d51735 Fix flaky optimistic confirmation tests (#23178) 2022-02-16 16:13:58 -05:00
Ashwin Sekar
ab92578b02 Fix the flaky test test_restart_tower_rollback (#23129) 
* Add flag to disable voting until a slot to avoid duplicate voting

* Fix the tower rollback test and remove it from flaky.
 2022-02-15 13:19:34 -07:00
Yueh-Hsuan Chiang
9213fcb11b Introduce experimental FIFO-compaction option for shreds in blockstore (#22140) 2022-02-10 11:34:03 -08:00
Justin Starry
d5dec989b9 Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Ashwin Sekar
5acf0f6331 Add feature gate for new vote instruction and plumb through replay (#21683) 
* Add feature gate for new vote instruction and plumb through replay

Add tower versions

* Add check for slot hashes history

* Update is_recent check to exclude voting on hard fork root slot

* Move tower rollback test to flaky and ignore it until #22551 lands
 2022-02-07 14:06:19 -08:00
Trent Nelson
eac4a6df68 rpc: use minimal mode by default 2022-02-01 19:00:06 -07:00
Justin Starry
d9c259a231 Set the correct root in block commitment cache initialization (#22750) 
* Set the correct root in block commitment cache initialization

* clean up test

* bump
 2022-01-27 00:48:00 +08:00
Michael Vines
6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Justin Starry
7f20c6149e Refactor: move simple vote parsing to runtime (#22537) 2022-01-20 10:39:21 +08:00
carllin
4ab7d6c23e Filter out outdated slots (#22450) 
* Filter out outdated slots

* Fixup error
 2022-01-13 19:51:00 -05:00
Ashwin Sekar
eeec1ce2ad Add local cluster test to repro slot hash expiry bug (#21873) 2022-01-10 00:58:21 -05:00
dependabot[bot]
28275a33d6 Bump tempfile from 3.2.0 to 3.3.0 (#22401) 
* Bump tempfile from 3.2.0 to 3.3.0

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-09 12:26:08 -07:00
Yueh-Hsuan Chiang
5771c36d3f Rename open_with_access_type() to open_with_options() (#22123) 2022-01-07 12:11:43 -08:00
Justin Starry
0e1afcbb26 Split up local cluster tests into separate CI steps (#22295) 
* Split up local cluster tests into separate CI steps

* Update buildkite-pipeline.sh
 2022-01-05 14:44:15 +00:00
Yueh-Hsuan Chiang
e8b7f96a89 Add struct BlockstoreOptions (#22121) 2022-01-03 18:30:45 -10:00
behzad nouri
c9c78622a8 discards serialized gossip crds votes if cannot parse tx (#22129) 2021-12-29 19:31:26 +00:00
behzad nouri
f67ecd5c18 removes unused Cargo dependencies (#22022) 
Also moving some to [dev-dependencies] so that they are not propagated
to other packages which depend on the package.
 2021-12-22 17:01:33 +00:00
behzad nouri
8d980f07ba uses Option<Slot> for SlotMeta.parent_slot (#21808) 
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
 2021-12-14 18:57:11 +00:00
Ashwin Sekar
f0acf7681e Add vote instructions that directly update on chain vote state (#21531) 
* Add vote state instructions

UpdateVoteState and UpdateVoteStateSwitch

* cargo tree

* extract vote state version conversion to common fn
 2021-12-07 16:47:26 -08:00
dependabot[bot]
f6801a4af4 chore: bump itertools from 0.10.1 to 0.10.3 (#21643) 
* chore: bump itertools from 0.10.1 to 0.10.3

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-06 23:23:26 -07:00
carllin
f493a88258 Fixup flaky tests (#21617) 
* Fixup flaky tests

* Fixup listeners
 2021-12-06 17:14:38 -05:00
Trent Nelson
d6f22433d0 Bump version to v1.10.0 2021-12-04 20:17:54 +00:00
Justin Starry
1430b58a6d Remove deprecated slow epoch boundary methods (#21568) 2021-12-03 17:59:10 +00:00
Michael Vines
b8837c04ec Reformat imports to a consistent style for imports 
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
 2021-12-03 09:19:13 -08:00
Michael Vines
dd12d90eac Upgrade to Rust 2021 2021-11-30 20:43:46 -08:00
Michael Vines
ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
carllin
b30c94ce55 ClusterInfoVoteListener send only missing votes to BankingStage (#20873) 2021-11-18 15:20:41 -08:00
Jeff Biseda
d5de0c8e12 add --no-os-network-stats-reporting option (#21296) 2021-11-16 10:26:03 -08:00
Michael Keleti
b0ca335463 Rename "trusted" to "known" in validators/ (#21197) 
* Replaced trusted with known validator

* Format Convention
 2021-11-12 11:57:55 -07:00
sakridge
7bb347faac Tone down the optimistic confirmation printing (#21045) 2021-10-28 15:14:36 +02:00
Trent Nelson
fe098b5ddc rpc-send-tx-svc: add with_config constructor 2021-10-20 13:43:27 -06:00
carllin
44ff30b65b Retry SampleNotDuplicateConfirmed decisions in AncestorHashesService (#20240) 2021-10-15 11:40:03 -07:00
Lijun Wang
d621994fee Accountsdb stream plugin improvement (#20419) 
Support using connection pooling and use multiple threads to do Postgres db operations. The performance is improved from 1500 RPS to 40,000 RPS measured during validator start.

Support multiple plugins at the same time.
 2021-10-08 20:06:58 -07:00
Trent Nelson
767f740305 Bump version to 1.9.0 2021-10-06 17:57:41 -07:00
Michael Vines
7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
Justin Starry
129716f3f0 Optimize stakes cache and rewards at epoch boundaries (#20432) 
* Optimize stakes cache and rewards at epoch boundaries

* Fetch from accounts db

* Add cli flag for disabling epoch boundary optimization
 2021-10-06 00:53:26 -04:00
Brooks Prumo
258c3bca65 Fix copy fn in local-cluster test (#20424) 2021-10-04 19:04:12 -05:00
carllin
2b967202f3 Add tower rollback test (#20158) 2021-10-04 15:21:19 -07:00
Lijun Wang
fe97cb2ddf AccountsDb plugin framework (#20047) 
Summary of Changes

Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows

Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.

The code comprises 4 major parts:

accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
 2021-09-30 14:26:17 -07:00
Brooks Prumo
4895c69fea Add test for startup processing new roots past full snapshot interval (#19876) 2021-09-20 18:50:29 -05:00