gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
18,124 Commits 81 Branches 357 Tags
v1.10
Commit Graph

2906 Commits

Author SHA1 Message Date
Tao Zhu
5e424826ba Persist cost table to blockstore (#18123) 
* Add `ProgramCosts` Column Family to blockstore, implement LedgerColumn; add `delete_cf` to Rocks
* Add ProgramCosts to compaction excluding list alone side with TransactionStatusIndex in one place: `excludes_from_compaction()`

* Write cost table to blockstore after `replay_stage` replayed active banks; add stats to measure persist time
* Deletes program from `ProgramCosts` in blockstore when they are removed from cost_table in memory
* Only try to persist to blockstore when cost_table is changed.
* Restore cost table during validator startup

* Offload `cost_model` related operations from replay main thread to dedicated service thread, add channel to send execute_timings between these threads;
* Move `cost_update_service` to its own module; replay_stage is now decoupled from cost_model.
 2021-07-01 11:32:41 -05:00
dependabot[bot]
05924423c2 chore: bump reqwest from 0.11.2 to 0.11.4 (#18362) 
* chore: bump reqwest from 0.11.2 to 0.11.4

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.2 to 0.11.4.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.2...v0.11.4)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-07-01 09:59:30 -06:00
Brooks Prumo
89a3e4f91e Move SnapshotConfig into its own module (#18331) 
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
 2021-07-01 08:55:26 -05:00
dependabot[bot]
78968d132f chore: bump log from 0.4.11 to 0.4.14 (#18323) 
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-30 18:32:01 +00:00
sakridge
8d9a6deda4 Add repair number per slot (#18082) 2021-06-30 18:20:07 +02:00
dependabot[bot]
e9165232ef chore: bump indexmap from 1.6.2 to 1.7.0 (#18322) 
* chore: bump indexmap from 1.6.2 to 1.7.0

Bumps [indexmap](https://github.com/bluss/indexmap) from 1.6.2 to 1.7.0.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.6.2...1.7.0)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-30 09:25:01 -06:00
Trent Nelson
02b14caa5f test-validator: hold rent constant with --slots-per-epoch 2021-06-30 00:46:12 -06:00
dependabot[bot]
52fd10ce03 chore: bump byteorder from 1.3.4 to 1.4.3 (#18300) 
* chore: bump byteorder from 1.3.4 to 1.4.3

Bumps [byteorder](https://github.com/BurntSushi/byteorder) from 1.3.4 to 1.4.3.
- [Release notes](https://github.com/BurntSushi/byteorder/releases)
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.3.4...1.4.3)

---
updated-dependencies:
- dependency-name: byteorder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Bump expected instruction count

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-29 15:08:04 -06:00
carllin
68c87469c3 Cleanup ReplayStage tests (#18241) 2021-06-28 20:19:42 -07:00
Tao Zhu
9d6f1ebef4 investigate system performance test degradation (#17919) 
* Add stats and counter around cost model ops, mainly:
- calculate transaction cost
- check transaction can fit in a block
- update block cost tracker after transactions are added to block
- replay_stage to update/insert execution cost to table

* Change mutex on cost_tracker to RwLock

* removed cloning cost_tracker for local use, as the metrics show clone is very expensive.

* acquire and hold locks for block of TXs, instead of acquire and release per transaction;

* remove redundant would_fit check from cost_tracker update execution path

* refactor cost checking with less frequent lock acquiring

* avoid many Transaction_cost heap allocation when calculate cost, which
is in the hot path - executed per transaction.

* create hashmap with new_capacity to reduce runtime heap realloc.

* code review changes: categorize stats, replace explicit drop calls, concisely initiate to default

* address potential deadlock by acquiring locks one at time
 2021-06-28 21:34:04 -05:00
dependabot[bot]
7782a0855d chore: bump rand_core from 0.6.2 to 0.6.3 (#18263) 
* chore: bump rand_core from 0.6.2 to 0.6.3

Bumps [rand_core](https://github.com/rust-random/rand) from 0.6.2 to 0.6.3.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/rand_core-0.6.2...rand_core-0.6.3)

---
updated-dependencies:
- dependency-name: rand_core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-28 12:07:27 -06:00
sakridge
5d08bf9aa3 More detailed voting timings in replay stage (#18229) 2021-06-26 17:32:08 +02:00
dependabot[bot]
91576fdd9b chore: bump blake3 from 0.3.7 to 0.3.8 (#18223) 
* chore: bump blake3 from 0.3.7 to 0.3.8

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/0.3.7...0.3.8)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-06-25 11:44:52 -06:00
dependabot[bot]
767c6eb57d chore: bump miow from 0.3.6 to 0.3.7 (#18222) 
* chore: bump miow from 0.3.6 to 0.3.7

Bumps [miow](https://github.com/yoshuawuyts/miow) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/yoshuawuyts/miow/releases)
- [Changelog](https://github.com/yoshuawuyts/miow/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yoshuawuyts/miow/compare/v0.3.6...v0.3.7)

---
updated-dependencies:
- dependency-name: miow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-25 11:08:57 -06:00
Trent Nelson
d269975784 Revert "Clean up build warning" 
This reverts commit 17a173ebb5.
 2021-06-24 19:57:52 -06:00
Michael Vines
314102cb54 Remove redundant JsonRpcConfig::identity_pubkey field 2021-06-22 17:20:11 -07:00
dependabot[bot]
6fc73470df chore: bump raptorq from 1.4.2 to 1.6.3 (#18137) 
Bumps [raptorq](https://github.com/cberner/raptorq) from 1.4.2 to 1.6.3.
- [Release notes](https://github.com/cberner/raptorq/releases)
- [Commits](https://github.com/cberner/raptorq/compare/v1.4.2...v1.6.3)

---
updated-dependencies:
- dependency-name: raptorq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-22 17:28:11 +00:00
dependabot[bot]
2156712768 chore: bump lru from 0.6.1 to 0.6.5 (#18138) 
Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.6.1 to 0.6.5.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.6.1...0.6.5)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-22 17:07:45 +00:00
sakridge
e808f34b0b Add batch stats (#18096) 2021-06-22 15:23:26 +02:00
Michael Vines
3b1517237c Clean up argument names 2021-06-21 21:29:52 -07:00
Michael Vines
84b9de8c18 Shredder no longer holds a keypair 2021-06-21 21:29:52 -07:00
Michael Vines
2435ea3ad8 Remove redundant ReplayStageConfig::my_pubkey field 2021-06-21 21:29:52 -07:00
Michael Vines
51a0007001 serve_repair: Remove internal ContactInfo field duplication 2021-06-21 17:23:49 -07:00
behzad nouri
598093b5db adds shred-version to ip-echo-server response 
When starting a validator, the node initially joins gossip with
shred_verison = 0, until it adopts the entrypoint's shred-version:
https://github.com/solana-labs/solana/blob/9b182f408/validator/src/main.rs#L417

Depending on the load on the entrypoint, this adopting entrypoint
shred-version through gossip sometimes becomes very slow, and causes
several problems in gossip because we have to partially support
shred_version == 0 which is a source of leaking crds values from one
cluster to another. e.g. see
https://github.com/solana-labs/solana/pull/17899
and the other linked issues there.

In order to remove shred_version == 0 from gossip, this commit adds
shred-version to ip-echo-server response. Once the entrypoints are
updated, on validator start-up, if --expected_shred_version is not
specified we will obtain shred-version from the entrypoint using
ip-echo-server.
 2021-06-21 19:37:16 +00:00
dependabot[bot]
d458fac2ff chore: bump bincode from 1.3.1 to 1.3.3 (#18087) 
* chore: bump bincode from 1.3.1 to 1.3.3

Bumps [bincode](https://github.com/servo/bincode) from 1.3.1 to 1.3.3.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/compare/v1.3.1...v1.3.3)

---
updated-dependencies:
- dependency-name: bincode
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-21 16:06:31 +00:00
Jeff Washington (jwash)
ec2f930475 user process.accounts_db_test_hash_calculation for debug_verify hash (#18053) 2021-06-21 10:20:27 -05:00
Michael Vines
4a12c715a3 Drop Error suffix from enum values to avoid the enum_variant_names clippy lint 2021-06-18 23:02:13 +00:00
dependabot[bot]
491e4450cc chore: bump libc from 0.2.96 to 0.2.97 (#18060) 
* chore: bump libc from 0.2.96 to 0.2.97

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.96 to 0.2.97.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.96...0.2.97)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-18 11:51:06 -06:00
Alexander Meißner
789f33e8db chore: cargo fmt 2021-06-18 10:42:46 -07:00
Alexander Meißner
6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Tyera Eulberg
d0511de9a6 chore: bump trees from 0.2.1 to 0.4.2 (#18052) 
* chore: bump trees from 0.2.1 to 0.4.2 (#18041)

Bumps [trees](https://github.com/oooutlk/trees) from 0.2.1 to 0.4.2.
- [Release notes](https://github.com/oooutlk/trees/releases)
- [Commits](https://github.com/oooutlk/trees/commits)

---
updated-dependencies:
- dependency-name: trees
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Accommodate field & type changes

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-17 22:45:09 +00:00
Jeff Washington (jwash)
bab96e878e Revert "chore: bump trees from 0.2.1 to 0.4.2 (#18041)" (#18045) 
This reverts commit f2a7152c0c.
 2021-06-17 14:04:59 -05:00
Lijun Wang
071b1ee3e5 Removed pub from some functions which are actually private to improve encapsulation (#18030) 
Remove the pub marker to improve encapsulation. Readability improvement only, no functional impact.
 2021-06-17 10:14:21 -07:00
dependabot[bot]
f2a7152c0c chore: bump trees from 0.2.1 to 0.4.2 (#18041) 
Bumps [trees](https://github.com/oooutlk/trees) from 0.2.1 to 0.4.2.
- [Release notes](https://github.com/oooutlk/trees/releases)
- [Commits](https://github.com/oooutlk/trees/commits)

---
updated-dependencies:
- dependency-name: trees
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-17 08:07:16 -06:00
Michael Vines
fa04531c7a Extricate RpcCompletedSlotsService from RetransmitStage 2021-06-16 16:20:35 -07:00
Trent Nelson
5bc6c89adc validator: run poh speed test earlier in start up 2021-06-16 21:27:08 +00:00
dependabot[bot]
6f5b248746 chore: bump miow from 0.2.2 to 0.3.6 (#18000) 
* chore: bump miow from 0.2.2 to 0.3.6

Bumps [miow](https://github.com/yoshuawuyts/miow) from 0.2.2 to 0.3.6.
- [Release notes](https://github.com/yoshuawuyts/miow/releases)
- [Changelog](https://github.com/yoshuawuyts/miow/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yoshuawuyts/miow/compare/0.2.2...v0.3.6)

---
updated-dependencies:
- dependency-name: miow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-16 12:30:17 -06:00
Jeff Washington (jwash)
dbd4dc04b0 ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974) 2021-06-15 15:39:22 -05:00
Jeff Washington (jwash)
f558b9b6bf verify bank hash on startup with ledger tool option (#17939) 2021-06-15 11:52:12 -05:00
Jon Cinque
1b1d34da59 Refactor stake program into solana_program (#17906) 
* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt
 2021-06-15 18:04:00 +02:00
behzad nouri
161838655c removes port-based forwarding logic from turbine retransmit (#17716) 
Turbine retransmit logic is based on which socket it received the packet
from (i.e `packet.meta.forward`):
https://github.com/solana-labs/solana/blob/708bbcb00/core/src/retransmit_stage.rs#L467-L470

This can leave the cluster vulnerable to spoofing and selective
propagation of packets; see
https://github.com/solana-labs/solana/issues/6672
https://github.com/solana-labs/solana/pull/7774

This commit identifies if the node is on the "critical path" based on
its index in the shuffled cluster. If so, it forwards the packet to both
neighbors and children; otherwise, the packet is only forwarded to the
children.

The metrics added in
https://github.com/solana-labs/solana/pull/17351
shows that the number of times the index does not match the port is very
rare, and therefore this change should be safe.
 2021-06-15 13:19:41 +00:00
carllin
ccc013e134 Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
Jeff Washington (jwash)
471b34132e add metrics for startup (#17913) 
* add metrics for startup

* roll timings up higher

* fix test

* fix duplicate
 2021-06-14 17:46:49 -05:00
Jeff Washington (jwash)
e6bbd4b3f0 add metrics to handle_snapshot_requests (#17937) 2021-06-14 15:46:19 -05:00
dependabot[bot]
a0872232d3 chore: bump itertools from 0.9.0 to 0.10.1 (#17929) 
* chore: bump itertools from 0.9.0 to 0.10.1

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.9.0 to 0.10.1.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.9.0...v0.10.1)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix versions

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-14 18:32:20 +00:00
dependabot[bot]
a248770a6b chore: bump systemstat from 0.1.7 to 0.1.8 (#17928) 
Bumps [systemstat](https://github.com/unrelentingtech/systemstat) from 0.1.7 to 0.1.8.
- [Release notes](https://github.com/unrelentingtech/systemstat/releases)
- [Commits](https://github.com/unrelentingtech/systemstat/compare/v0.1.7...v0.1.8)

---
updated-dependencies:
- dependency-name: systemstat
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-14 16:48:54 +00:00
sakridge
eeee75c5be Don't use pinned memory when unnecessary (#17832) 
Reports of excessive GPU memory usage and errors
from cudaHostRegister. There are some cases where pinning is
not required.
 2021-06-14 16:10:04 +02:00
dependabot[bot]
a501707058 chore: bump ahash from 0.6.1 to 0.7.4 (#17886) 
Bumps [ahash](https://github.com/tkaitchuck/ahash) from 0.6.1 to 0.7.4.
- [Release notes](https://github.com/tkaitchuck/ahash/releases)
- [Commits](https://github.com/tkaitchuck/ahash/commits)

---
updated-dependencies:
- dependency-name: ahash
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-11 14:25:25 -06:00
sakridge
0feac57cb0 Don't store votes unless we are leader soon (#17803) 2021-06-11 18:29:05 +02:00
dependabot[bot]
2aa7df23b5 chore: bump indexmap from 1.5.1 to 1.6.2 (#17884) 
* chore: bump indexmap from 1.5.1 to 1.6.2

Bumps [indexmap](https://github.com/bluss/indexmap) from 1.5.1 to 1.6.2.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.5.1...1.6.2)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-11 09:50:09 -06:00