Commit Graph

451 Commits

Author SHA1 Message Date
mergify[bot]
f56b25ac29 chore: bump dashmap from 4.0.2 to 5.1.0 (#23372) (#23521)
* chore: bump dashmap from 4.0.2 to 5.1.0

Bumps [dashmap](https://github.com/xacrimon/dashmap) from 4.0.2 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.1.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
(cherry picked from commit 3a0271c113)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-08 14:05:58 +00:00
Will Hickey
84c57dd0a8 Bump version to v1.9.10 (#23304) 2022-02-23 11:30:21 -06:00
Justin Starry
450404f800 v1.9: Enforce tx metadata upload to bigtable (#23212)
* Enforce tx metadata upload with static types (#23028)

* resolve conflicts

* fix test
2022-02-23 12:03:17 +08:00
Will Hickey
08d6b9850d Bump version to 1.9.9 (#23247) 2022-02-19 21:17:11 -06:00
Trent Nelson
c3098e99d1 Bump version to v1.9.8 2022-02-16 21:42:57 -07:00
Will Hickey
2d144afec5 Bump version to 1.9.6 (#23092) 2022-02-11 15:00:06 -06:00
Trent Nelson
8bc4cc90d2 Bump version to 1.9.6 2022-01-21 20:15:43 -07:00
Trent Nelson
cb49ae21b4 Bump version to v1.9.5 2022-01-08 21:17:51 +00:00
mergify[bot]
2f97fee71a Cleanup ledger-tool analyze-storage command (#22310) (#22352)
* Make ledger-tool analyze-storage use Blockstore::open()

Opening a large ledger may require setting a larger open file descriptor
limit. Blockstore::open() does this whereas the underlying Database
object that analyze-storage was opening does not.

* Move key_size call lookup to take advantage of traits

* Fix typo where analyze worked on wrong column

* Make analyze-storage analyze all columns

(cherry picked from commit 9f1f64e384)

Co-authored-by: steviez <steven@solana.com>
2022-01-07 07:47:27 +00:00
mergify[bot]
e7348243b4 [ledger-tool]compare_blocks (#22229) (#22330)
* 1.made load_credentials accept credential path as a parameter. 2.partial implement bigtable comparasion function

* finding missing blocks in bigtables in a specified range

* refactor compare-blocks,add unit test for missing_blocks and fmt

* compare-block fix last block bug

* refactor compare-block and improve wording

* Update ledger-tool/src/bigtable.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* update compare-block command-line description

* style:improve wording/naming/code style

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit d9220652ad)

Co-authored-by: pieceofr <komimi.p@gmail.com>
2022-01-06 08:55:26 +00:00
Will Hickey
ba0188a36d Bump version to 1.9.4 (#22304) 2022-01-05 12:02:36 -06:00
Will Hickey
78e7913352 Bump version to 1.9.3 (#22065) 2021-12-22 11:41:03 -06:00
mergify[bot]
f4521002b9 Clean up demote program write lock feature (backport #21949) (#21969)
* Clean up demote program write lock feature (#21949)

* Clean up demote program write lock feature

* fix test

(cherry picked from commit 6ff0be6a82)

# Conflicts:
#	programs/bpf_loader/src/syscalls.rs
#	runtime/src/accounts.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-12-17 04:45:22 +00:00
Michael Vines
bfa6302985 Bump version to 1.9.2 2021-12-15 16:18:14 -08:00
mergify[bot]
a056fd88cb uses Option<Slot> for SlotMeta.parent_slot (backport #21808) (#21899)
* uses Option<Slot> for SlotMeta.parent_slot (#21808)

SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.

(cherry picked from commit 8d980f07ba)

# Conflicts:
#	ledger-tool/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-14 21:42:57 +00:00
Tao Zhu
4b63d51e3e Bump version to 1.9.1 (#21802) 2021-12-11 12:50:36 +00:00
Michael Vines
b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg
2b269dbe0e Ledger-tool: only require ledger dir when necessary (#21575)
* Don't canonicalize ledger_path unless ledger_path will be used

* Single use statement
2021-12-02 17:23:51 -07:00
Michael Vines
dd12d90eac Upgrade to Rust 2021 2021-11-30 20:43:46 -08:00
dependabot[bot]
a125e7b8c2 chore: bump serde_json from 1.0.71 to 1.0.72 (#21426)
* chore: bump serde_json from 1.0.71 to 1.0.72

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-30 18:03:09 +00:00
Jeff Washington (jwash)
b1b334a17f ledger-tool uses jemalloc like validator (#21351) 2021-11-19 09:48:07 -06:00
Jeff Washington (jwash)
79d21d6805 add --accounts-index-scan-results-limit-mb to allow scans to abort (#21327)
* ScanConfig -> &ScanConfig

* add --accounts-index-scan-results-limit-mb to allow scans to abort

* feedback
2021-11-19 09:00:19 -06:00
dependabot[bot]
89102540b1 chore: bump serde_json from 1.0.70 to 1.0.71 (#21338)
* chore: bump serde_json from 1.0.70 to 1.0.71

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-18 02:58:30 +00:00
Jeff Washington (jwash)
0f69a14247 Add ability to abort scan (#21314) 2021-11-17 13:10:29 -06:00
Jeff Biseda
d5de0c8e12 add --no-os-network-stats-reporting option (#21296) 2021-11-16 10:26:03 -08:00
dependabot[bot]
55ea672041 chore: bump serde_json from 1.0.69 to 1.0.70 (#21285)
* chore: bump serde_json from 1.0.69 to 1.0.70

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-11-15 15:27:47 -07:00
Jeff Washington (jwash)
77a59ca3e3 ledger-tool verify supports --accounts-db-skip-shrink (#21284) 2021-11-15 14:42:56 -06:00
Jeff Washington (jwash)
a043b19988 ledger tool verify launches system monitor to report mem (#21265) 2021-11-15 09:29:30 -06:00
Brooks Prumo
8b2133b078 Refactor total stats calculation into shared fn (#21212) 2021-11-09 19:13:37 -06:00
dependabot[bot]
3c5f505d3e chore: bump serde_json from 1.0.68 to 1.0.69 (#21211)
* chore: bump serde_json from 1.0.68 to 1.0.69

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-09 11:04:40 -07:00
Brooks Prumo
6f3c27f1da Add total accounts stats to ledger-tool accounts (#21187) 2021-11-08 14:45:30 -06:00
Brooks Prumo
2031522809 Add flags to not print account contents (#21178)
It takes a long time to print the contents of all the accounts.  I'll be
adding more to the "accounts" subcommand and would like to skip printing
the account contents.
2021-11-05 08:59:12 -05:00
Justin Starry
140a5f633d Simplify replay vote tracking by using packet metadata (#21112) 2021-11-03 09:02:48 +00:00
dependabot[bot]
855b3ccdc0 chore: bump signal-hook from 0.2.3 to 0.3.10 (#21081)
* chore: bump signal-hook from 0.2.3 to 0.3.10

Bumps [signal-hook](https://github.com/vorner/signal-hook) from 0.2.3 to 0.3.10.
- [Release notes](https://github.com/vorner/signal-hook/releases)
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/signal-hook/compare/v0.2.3...v0.3.10)

---
updated-dependencies:
- dependency-name: signal-hook
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update apis

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-10-30 00:21:54 -06:00
Jeff Washington (jwash)
43ea579f63 add cli for --accounts-hash-num-passes (#20827) 2021-10-25 09:45:46 -05:00
Tao Zhu
c2bfce90b3 - cost_tracker is data member of a bank, it can report metrics when bank is frozen (#20802)
- removed cost_tracker_stats and histogram
- move stats reporting outside of bank freeze
2021-10-24 22:19:23 -05:00
Michael Vines
350bb561eb Clippy 2021-10-23 08:21:20 +00:00
Brooks Prumo
50571685d4 Take incremental snapshots with ledger-tool (#20857) 2021-10-21 21:02:12 -05:00
carllin
b5f21d5e34 Add ledger tool to remove dead slot (#20810) 2021-10-19 20:23:16 -07:00
Tao Zhu
7496b5784b - make cost_tracker a member of bank, remove shared instance from TPU; (#20627)
- decouple cost_model from cost_tracker; allowing one cost_model
  instance being shared within a validator;
- update cost_model api to calculate_cost(&self...)->transaction_cost
2021-10-19 14:37:33 -05:00
sakridge
3b40fa3f44 Add is_full to ledger-tool (#20183) 2021-10-13 12:50:18 +02:00
Tao Zhu
005d6863fd - move cost tracker into bank, so each bank has its own cost tracker; (#20527)
- move related modules to runtime
2021-10-12 08:51:33 -05:00
Jeff Washington (jwash)
a8e000a2a6 add filler accounts to bloat validator and predict failure (#20491)
* add filler accounts to bloat validator and predict failure

* assert no accounts match filler

* cleanup magic numbers

* panic if can't load from snapshot with filler accounts specified

* some renames

* renames

* into_par_iter

* clean filler accts, too
2021-10-11 12:46:27 -05:00
Jeff Washington (jwash)
6cd1a03bb5 AcctIdx: fix ledger-tool arg options (#20513) 2021-10-07 19:43:21 +00:00
Trent Nelson
767f740305 Bump version to 1.9.0 2021-10-06 17:57:41 -07:00
Michael Vines
7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
Tao Zhu
03913f6661 add tx count and thread id to stats, each stat reports and resets when slot changes (#20451) 2021-10-06 00:09:19 -05:00
Justin Starry
250a8503fe Make rewards tracer async friendly (#20452) 2021-10-05 21:30:08 -04:00
Tao Zhu
6ff508c643 add transaction cost histogram metrics (#20350) 2021-10-05 08:57:39 -05:00
Lijun Wang
fe97cb2ddf AccountsDb plugin framework (#20047)
Summary of Changes

Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows

Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.

The code comprises 4 major parts:

accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00