gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
16,362 Commits 81 Branches 357 Tags
58aa2b964b72830291ceaedc900f4e235829c422
Commit Graph

678 Commits

Author SHA1 Message Date
Michael Vines
350bb561eb Clippy 2021-10-23 08:21:20 +00:00
dependabot[bot]
66e431dad5 chore: bump libc from 0.2.103 to 0.2.104 (#20764) 
* chore: bump libc from 0.2.103 to 0.2.104

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-10-19 22:38:02 -06:00
carllin
b5f21d5e34 Add ledger tool to remove dead slot (#20810) 2021-10-19 20:23:16 -07:00
behzad nouri
231b58b5f1 adds more metrics to blockstore insert shreds stats (#20701) 2021-10-17 11:02:34 -04:00
Jeff Washington (jwash)
70b2e5fef2 hash calc uses all threads during startup (#20735) 2021-10-15 17:41:23 -05:00
Tyera Eulberg
e806fa6904 Include token owners in TransactionTokenBalances (#20642) 
* Cache owners in TransactionTokenBalances

* Light cleanup

* Use return struct, and remove pub

* Single-use statements

* Why not, just do the whole crate

* Add metrics

* Make datapoint_debug to prevent spam unless troubleshooting
 2021-10-13 21:46:52 -06:00
sakridge
09e7782d76 Refactor code to get block signatures in get_confirmed_signatures_for_address2 (#20575) 
* Refactor get_confirmed_signatures_for_address2

* Move blockstore benches to ledger where they belong
 2021-10-13 09:55:19 +02:00
Tao Zhu
005d6863fd - move cost tracker into bank, so each bank has its own cost tracker; (#20527) 
- move related modules to runtime
 2021-10-12 08:51:33 -05:00
Jeff Washington (jwash)
a8e000a2a6 add filler accounts to bloat validator and predict failure (#20491) 
* add filler accounts to bloat validator and predict failure

* assert no accounts match filler

* cleanup magic numbers

* panic if can't load from snapshot with filler accounts specified

* some renames

* renames

* into_par_iter

* clean filler accts, too
 2021-10-11 12:46:27 -05:00
Brooks Prumo
5440c1d2e1 SnapshotPackagerService pushes incremental snapshot hashes to CRDS (#20442) 
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!

This commit does two main things:

1. SnapshotPackagerService now knows about incremental snapshot hashes,
   and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
   hashes need to be passed all the way to SnapshotPackagerService so it
   can push these starting hashes to CRDS.  Those values have been piped
   through.

Fixes #20441 and #20423
 2021-10-08 15:14:56 -05:00
Tao Zhu
675fa6993b - update const cost values with data collected by #19627 (#20314) 
- update cost calculation to closely proposed fee schedule #16984
 2021-10-08 14:48:50 -05:00
Trent Nelson
767f740305 Bump version to 1.9.0 2021-10-06 17:57:41 -07:00
Michael Vines
7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
steviez
d8e4840b49 Follow up to blockstore test cleanup (#19656) (#20143) 2021-10-04 10:35:03 -05:00
Lijun Wang
fe97cb2ddf AccountsDb plugin framework (#20047) 
Summary of Changes

Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows

Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.

The code comprises 4 major parts:

accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
 2021-09-30 14:26:17 -07:00
dependabot[bot]
9393654af0 chore: bump libc from 0.2.102 to 0.2.103 (#20280) 
* chore: bump libc from 0.2.102 to 0.2.103

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.102 to 0.2.103.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.102...0.2.103)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-09-28 07:50:46 +00:00
Justin Starry
c71fab6cb3 Add delete subcommand to ledger-tool bigtable (#19931) 
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback
 2021-09-16 23:37:45 +00:00
dependabot[bot]
ea15c0b40b chore: bump libc from 0.2.101 to 0.2.102 (#19924) 
* chore: bump libc from 0.2.101 to 0.2.102

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.101 to 0.2.102.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.101...0.2.102)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-09-16 09:45:01 -06:00
steviez
4fdd70c931 Properly clean temp files in Blockstore::create_new_ledger() (#19589) 
Use TempDir::path() instead of TempDir::into_path(); into_path() returns
the underlying PathBuf and voids the promise to delete the directory
when TempDir goes out of scope.
 2021-09-13 23:13:56 -04:00
Brooks Prumo
97fc64dd0c Call exhaustively_free_unused_resource() instead (#19847) 2021-09-13 18:08:32 -05:00
Brooks Prumo
ef749a2506 Flush accounts cache before calling clean_accounts() (#19839) 2021-09-13 14:56:13 -05:00
Brooks Prumo
3f6eb96d6e Call Bank::update_accounts_hash() before taking bank snapshot (#19765) 
This is a bugfix.

When `load_frozen_forks()` called `snapshot_bank()`, the accounts hash
was not computed.  So then when a snapshot archive was eventually
created, its hash would be all 1s, which is clearly wrong.  The fix is
to update the bank's accounts hash before taking the bank snapshot.
 2021-09-13 08:27:50 -05:00
Brooks Prumo
62c8bcf565 Add default() to SnapshotConfig (#19776) 2021-09-12 13:44:27 -05:00
dependabot[bot]
857fce13f9 chore: bump sha2 from 0.9.6 to 0.9.8 (#19760) 
* chore: bump sha2 from 0.9.6 to 0.9.8

Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.6 to 0.9.8.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.6...sha2-v0.9.8)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-09-11 16:22:40 -06:00
Giles Cope
c3a610374a unused deps 2021-09-11 14:33:53 -07:00
steviez
2dee098b91 Blockstore test cleanup (#19656) 
* Use auto-deleting macro for ledger path
 2021-09-10 07:33:08 -05:00
Jeff Washington (jwash)
456bf15012 AccountsIndexConfig -> AccountsDbConfig (#19687) 2021-09-08 04:30:38 +00:00
Jeff Washington (jwash)
d3f938f0cf Remove Copy from AccountsIndexConfig. Not all types will support it (#19686) 2021-09-07 20:09:40 -05:00
Brooks Prumo
4a5f83d3a7 Add fn to check when to take snapshots (#19682) 2021-09-07 23:26:35 +00:00
Brooks Prumo
a0552e5b46 Make startup aware of Incremental Snapshots (#19600) 2021-09-07 20:43:43 +00:00
Tyera Eulberg
5fa3e5744c Populate memo in blockstore signatures-for-address (#19515) 
* Add TransactionMemos column family

* Traitify extract_memos

* Write TransactionMemos in TransactionStatusService

* Populate memos from column

* Dedupe and add unit test
 2021-09-03 00:28:52 -06:00
Jack May
170927b7c4 nit: Move builtins types to builtins file (#19597) 2021-09-02 21:29:11 -07:00
Brooks Prumo
e9374d32a3 Revert "Make startup aware of Incremental Snapshots (#19550)" (#19599) 
This reverts commit d45ced0a5d.
 2021-09-02 19:14:41 -05:00
Brooks Prumo
d45ced0a5d Make startup aware of Incremental Snapshots (#19550) 2021-09-02 19:05:15 -05:00
Brooks Prumo
1d2f0f6641 Simplify process_blockstore tests (#19553) 
Some of the blockstore_processor tests weren't using
`test_process_blockstore()`, but could.  I updated those tests, and also
changed to using `..` for ignoring, instead of `_`, since I'll be adding
another return value here shortly (thus reducing the need to change
these again).
 2021-09-01 20:13:52 +00:00
behzad nouri
6d9818b8e4 skips retransmit for shreds with unknown slot leader (#19472) 
Shreds' signatures should be verified before they reach retransmit
stage, and if the leader is unknown they should fail signature check.
Therefore retransmit-stage can as well expect to know who the slot
leader is and otherwise just skip the shred.

Blockstore checking signature of recovered shreds before sending them to
retransmit stage:
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/blockstore.rs#L884-L930

Shred signature verifier:
https://github.com/solana-labs/solana/blob/4305d4b7b/core/src/sigverify_shreds.rs#L41-L57
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/sigverify_shreds.rs#L105
 2021-09-01 15:44:26 +00:00
dependabot[bot]
8e4360cf9c chore: bump futures from 0.3.16 to 0.3.17 (#19513) 
* chore: bump futures from 0.3.16 to 0.3.17

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-30 22:12:38 -06:00
dependabot[bot]
dc3d7ad2bd chore: bump sha2 from 0.9.5 to 0.9.6 (#19511) 
* chore: bump sha2 from 0.9.5 to 0.9.6

Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.5 to 0.9.6.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.5...sha2-v0.9.6)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 19:58:26 -06:00
dependabot[bot]
6a53ec28e0 chore: bump serde from 1.0.129 to 1.0.130 (#19497) 
* chore: bump serde from 1.0.129 to 1.0.130

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 18:19:34 +00:00
dependabot[bot]
fbffdf9f3d chore: bump futures-util from 0.3.16 to 0.3.17 (#19498) 
* chore: bump futures-util from 0.3.16 to 0.3.17

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 17:45:47 +00:00
behzad nouri
8ad52fa095 implements copy-on-write for vote-accounts (#19362) 
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186

This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
 2021-08-30 15:54:01 +00:00
dependabot[bot]
e492638b7d chore: bump libc from 0.2.100 to 0.2.101 (#19442) 
* chore: bump libc from 0.2.100 to 0.2.101

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.100 to 0.2.101.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.100...0.2.101)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-08-26 12:38:34 -06:00
Jeff Washington (jwash)
f1d9c014f8 create test function test_process_blockstore (#19432) 2021-08-26 12:48:52 -05:00
dependabot[bot]
46890ac197 chore: bump serde from 1.0.128 to 1.0.129 (#19395) 
* chore: bump serde from 1.0.128 to 1.0.129

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-24 12:32:30 -06:00
Brooks Prumo
f3ce5fa5e8 clippy: branches-sharing-code 
```text
error: all if blocks contain the same code at the end
    --> ledger/src/blockstore.rs:3473:5
     |
3473 | /         Ok(insert_map.get(&slot).unwrap().clone())
3474 | |     }
     | |_____^
     |
     = note: `-D clippy::branches-sharing-code` implied by `-D warnings`
     = note: The end suggestion probably needs some adjustments to use the expression result correctly
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#branches_sharing_code
help: consider moving the end statements out like this
     |
3473 ~     }
3474 +     Ok(insert_map.get(&slot).unwrap().clone())
     |

error: could not compile `solana-ledger` due to previous error
```
 2021-08-23 11:17:21 -05:00
dependabot[bot]
c0c95e88d8 chore: bump libc from 0.2.99 to 0.2.100 (#19371) 
* chore: bump libc from 0.2.99 to 0.2.100

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.99 to 0.2.100.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.99...0.2.100)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-23 09:34:32 -06:00
dependabot[bot]
aea9960c75 chore: bump serde from 1.0.127 to 1.0.128 (#19370) 
* chore: bump serde from 1.0.127 to 1.0.128

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-08-23 09:33:34 -06:00
Brooks Prumo
6d939811e9 Name snapshots consistently (#19346) 
#### Problem

Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:

```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```

#### Summary of Changes

For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.


Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-08-21 15:41:03 -05:00
Tao Zhu
4982dc20f9 replace function with const var for better readability (#19285) 2021-08-19 14:59:53 -05:00
Tao Zhu
d7ba15cde8 Update block max cost limit to fix performance regession (#19276) 
* increase block_cost_max by 10 times to accommodate updated account read and write costs; Otherwise during performace test, banking_stage will start reject and retry transactions in next block, result regressed performance.

* fix typo
 2021-08-18 16:38:44 +00:00