gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
15,398 Commits 81 Branches 357 Tags
b59b55716365be5c69a84d601e046ee37ccdb42c
Commit Graph

15398 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sakridge
592013eaf4 Clean within shrink_all_slots (#19042) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-08-05 23:26:38 -07:00
Jeff Washington (jwash)
0028442e14 rework defaults and construction of accountsdb, accounts, bank (#19083) 
* rework defaults and construction of accountsdb, accounts, bank

* merge issues
 2021-08-05 17:27:13 -05:00
behzad nouri
e4be00fece falls back on working-bank if root-bank::epoch-staked-nodes is none 
bank.get_leader_schedule_epoch(shred_slot)
is one epoch after epoch_schedule.get_epoch(shred_slot).

At epoch boundaries, shred is already one epoch after the root-slot. So
we need epoch-stakes 2 epochs ahead of the root. But the root bank only
has epoch-stakes for one epoch ahead, and as a result looking up epoch
staked-nodes from the root-bank fails.

To be backward compatible with the current master code, this commit
implements a fallback on working-bank if epoch staked-nodes obtained
from the root-bank is none.
 2021-08-05 21:47:33 +00:00
behzad nouri
eaf927cf49 allows only one thread to update cluster-nodes cache entry for an epoch 
If two threads simultaneously call into ClusterNodesCache::get for the
same epoch, and the cache entry is outdated, then both threads recompute
cluster-nodes for the epoch and redundantly overwrite each other.

This commit wraps ClusterNodesCache entries in Arc<Mutex<...>>, so that
when needed only one thread does the computations to update the entry.
 2021-08-05 21:47:33 +00:00
behzad nouri
fb69f45f14 adds fallback & metric for when epoch staked-nodes are none 2021-08-05 21:47:33 +00:00
behzad nouri
50d0e830c9 unifies cluster-nodes computation & caching across turbine stages 
Broadcast-stage is using epoch_staked_nodes based on the same slot that
shreds belong to:
https://github.com/solana-labs/solana/blob/049fb0417/core/src/broadcast_stage/standard_broadcast_run.rs#L208-L228
https://github.com/solana-labs/solana/blob/0cf52e206/core/src/broadcast_stage.rs#L342-L349

But retransmit-stage is using bank-epoch of the working-bank:
https://github.com/solana-labs/solana/blob/19bd30262/core/src/retransmit_stage.rs#L272-L289

So the two are not consistent at epoch boundaries where some nodes may
have a working bank (or similarly a root bank) lagging other nodes. As a
result the node which obtains a packet may construct turbine broadcast
tree inconsistently with its parent node in the tree and so some packets
may fail to reach all nodes in the tree.
 2021-08-05 21:47:33 +00:00
behzad nouri
aa32738dd5 uses cluster-nodes cache in broadcast-stage 
* Current caching mechanism does not update cluster-nodes when the epoch
  (and so epoch staked nodes) changes:
  https://github.com/solana-labs/solana/blob/19bd30262/core/src/broadcast_stage/standard_broadcast_run.rs#L332-L344

* Additionally, the cache update has a concurrency bug in which the
  thread which does compare_and_swap may be blocked when it tries to
  obtain the write-lock on cache, while other threads will keep running
  ahead with the outdated cache (since the atomic timestamp is already
  updated).

In the new ClusterNodesCache, entries are keyed by epoch, and so if
epoch changes cluster-nodes will be recalculated. The time-to-live
eviction policy is also encapsulated and rigidly enforced.
 2021-08-05 21:47:33 +00:00
behzad nouri
30bec3921e uses cluster-nodes cache in retransmit stage 
The new cluster-nodes cache will:
  * ensure cluster-nodes are recalculated if the epoch (and so the epoch
    staked nodes) changes.
  * encapsulate time-to-live eviction policy.
 2021-08-05 21:47:33 +00:00
behzad nouri
ecc1c7957f implements cluster-nodes cache 
Cluster nodes are cached keyed by the respective epoch from which stakes
are obtained, and so if epoch changes cluster-nodes will be recomputed.

A time-to-live eviction policy is enforced to refresh entries in case
gossip contact-infos are updated.
 2021-08-05 21:47:33 +00:00
behzad nouri
44b11154ca sends slots (instead of stakes) through broadcast flow 
Current broadcast code is computing stakes for each slot before sending
them down the channel:
https://github.com/solana-labs/solana/blob/049fb0417/core/src/broadcast_stage/standard_broadcast_run.rs#L208-L228
https://github.com/solana-labs/solana/blob/0cf52e206/core/src/broadcast_stage.rs#L342-L349

Since the stakes are a function of epoch the slot belongs to (and so
does not necessarily change from one slot to another), forwarding the
slot itself would allow better caching downstream.

In addition we need to invalidate the cache if the epoch changes (which
the current code does not do), and that requires to know which slot (and
so epoch) current broadcasted shreds belong to:
https://github.com/solana-labs/solana/blob/19bd30262/core/src/broadcast_stage/standard_broadcast_run.rs#L332-L344
 2021-08-05 21:47:33 +00:00
Brian Anderson
b67ffab370 Add more API documentation for Rust RpcClient (#19021) 
* Add doc links to Transaction API docs

* Add more RpcClient API docs

* Reflow some rpc_client docs

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update sdk/src/transaction.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update RpcClient docs per review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-08-05 15:03:33 -06:00
Jeff Washington (jwash)
e368f10973 add _for_tests to new_no_wallclock_throttle (#19086) 2021-08-05 14:50:25 -05:00
Jeff Washington (jwash)
a9014ceceb Bank::default_for_tests() (#19084) 2021-08-05 11:53:29 -05:00
Jeff Washington (jwash)
24207a09ac remove AccountsIndex::default (#19082) 
* accounts_db calls AccountsDb::new(bins)

* remove AccountsIndex::default
 2021-08-05 11:38:53 -05:00
Jeff Washington (jwash)
5cf28689e6 accounts_db calls AccountsDb::new(bins) (#19068) 2021-08-05 11:15:26 -05:00
behzad nouri
40914de811 updates cluster-slots with root-bank instead of root-slot + bank-forks 
ClusterSlots::update is taking both root-slot and bank-forks only to
later lookup root-bank from bank-forks, which is redundant. Also
potentially by the time bank-forks is locked to obtain root-bank,
root-slot may have already changed and so be inconsistent with the
root-slot passed in as the argument.
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L32-L39
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L122
 2021-08-05 14:43:06 +00:00
behzad nouri
2fc112edcf removes unused code from cluster-slots 2021-08-05 14:43:06 +00:00
Jeff Washington (jwash)
67788ad206 move AccountsIndex upsert into static WriteAccountMapEntry (#18899) 
* rework accounts index to push upsert deeper

* clean up return value of upsert_existing_key

* upsert_existing_key -> update_key_if_exists

* upsert_new_key -> upsert

* upsert_item -> lock_and_update_slot_list

* update_static -> update_slot_list
 2021-08-05 08:45:08 -05:00
Jeff Washington (jwash)
bf16b0517c add _for_tests to setup_bank_and_vote_pubkeys (#19060) 2021-08-05 08:43:35 -05:00
Jeff Washington (jwash)
087db70df6 add traits required by IsCached (#19066) 2021-08-05 08:43:00 -05:00
Jeff Washington (jwash)
14361906ca for all tests, bank::new -> bank::new_for_tests (#19064) 2021-08-05 08:42:38 -05:00
dependabot[bot]
367d5f62ce chore:(deps): bump @solana/spl-token-registry in /explorer (#19077) 
Bumps [@solana/spl-token-registry](https://github.com/solana-labs/token-list) from 0.2.208 to 0.2.210.
- [Release notes](https://github.com/solana-labs/token-list/releases)
- [Changelog](https://github.com/solana-labs/token-list/blob/main/CHANGELOG.md)
- [Commits](https://github.com/solana-labs/token-list/compare/v0.2.208...v0.2.210)

---
updated-dependencies:
- dependency-name: "@solana/spl-token-registry"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-05 08:34:30 +00:00
dependabot[bot]
6e285c42ea chore:(deps): bump @types/node from 16.4.10 to 16.4.12 in /explorer (#19076) 
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 16.4.10 to 16.4.12.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-05 08:30:56 +00:00
dependabot[bot]
80e770dfa0 chore: bump @babel/preset-env from 7.14.9 to 7.15.0 in /web3.js (#19073) 
Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) from 7.14.9 to 7.15.0.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.15.0/packages/babel-preset-env)

---
updated-dependencies:
- dependency-name: "@babel/preset-env"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-05 08:16:48 +00:00
Jeff Washington (jwash)
dfe99efa7c introduce AccountsIndex::default_for_tests() (#19067) 2021-08-04 21:58:53 -05:00
sakridge
5a4979f25f Handle 0-lamport account in index generation (#19041) 
* Handle 0-lamport account in index generation

* rename duplicate to dirty keys

Co-authored-by: Carl Lin <carl@solana.com>
 2021-08-04 23:33:47 +00:00
Jeff Washington (jwash)
bde9b4de94 Bank::new -> Bank::new_for_benches (#19063) 2021-08-04 17:30:43 -05:00
Jeff Washington (jwash)
3280ae3e9f add validator option --accounts-db-skip-shrink (#19028) 
* add validator option --accounts-db-skip-shrink

* typo
 2021-08-04 17:28:33 -05:00
Brooks Prumo
68cc71409e Do not shell out for tar (#19043) 
When making a snapshot archive, we used to shell out and call `tar -S`
for sparse file support.  The tar crate supports sparse files, so no
need to do this anymore.

Fixes #10860
 2021-08-04 17:07:55 -05:00
Brooks Prumo
a1112254a5 Fix wrong old snapshot archives getting purged (#19061) 
I introduced a bug where old snapshot archives were incorrectly purged.
Instead of purged to oldest, I was purged the newest...

The fix is to add a `reverse()` in the purge logic, and I've added a
test to catch this bug in the future.

Fixes #19057
 2021-08-04 16:42:42 -05:00
Jeff Washington (jwash)
0b8d14b0fc move towards account index being dynamically allocated (#19034) 2021-08-04 15:28:35 -05:00
Jeff Washington (jwash)
1ed12a07ab introduce Bank::new_for_tests (#19062) 2021-08-04 15:06:57 -05:00
Brooks Prumo
ca14475085 Add incremental_snapshot_archive_interval_slots to SnapshotConfig (#19026) 
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
 2021-08-04 14:40:20 -05:00
dependabot[bot]
cc27b8a5a7 chore: bump const_format from 0.2.15 to 0.2.17 (#19049) 
Bumps [const_format](https://github.com/rodrimati1992/const_format_crates) from 0.2.15 to 0.2.17.
- [Release notes](https://github.com/rodrimati1992/const_format_crates/releases)
- [Changelog](https://github.com/rodrimati1992/const_format_crates/blob/master/Changelog.md)
- [Commits](https://github.com/rodrimati1992/const_format_crates/commits)

---
updated-dependencies:
- dependency-name: const_format
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-04 13:38:50 -06:00
Trent Nelson
d60ccf64e1 bump recommended maps/nofiles 2021-08-04 11:15:18 -06:00
Jeff Washington (jwash)
6a995f5dfd rename to AccountsDb::new_single_for_tests (#19039) 2021-08-04 11:47:11 -05:00
dependabot[bot]
6d95d679c4 chore: bump curve25519-dalek from 3.1.0 to 3.2.0 (#19051) 
* chore: bump curve25519-dalek from 3.1.0 to 3.2.0

Bumps [curve25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/dalek-cryptography/curve25519-dalek/releases)
- [Changelog](https://github.com/dalek-cryptography/curve25519-dalek/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dalek-cryptography/curve25519-dalek/compare/3.1.0...3.2.0)

---
updated-dependencies:
- dependency-name: curve25519-dalek
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-04 09:14:32 -06:00
dependabot[bot]
999fb4bda9 chore: bump sysctl from 0.4.1 to 0.4.2 (#19052) 
Bumps [sysctl](https://github.com/johalun/sysctl-rs) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/johalun/sysctl-rs/releases)
- [Changelog](https://github.com/johalun/sysctl-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/johalun/sysctl-rs/commits/v0.4.2)

---
updated-dependencies:
- dependency-name: sysctl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-04 09:09:08 -06:00
Jon Cinque
2b33c0c165 stake: Allow stakes with unmatched credits observed to merge (#18985) 
* stake: Allow stakes with unmatched credits observed to merge

* Address feedback

* Remove branch by doing a ceiling in one calc
 2021-08-04 10:43:34 -04:00
Jeff Washington (jwash)
31a620c42b move towards accounts index being dynamic (#19032) 2021-08-04 09:18:05 -05:00
Oliver
06e08c4840 move package_snapshots to AccountsPackagePre ctors (#18997) 
This PR solves #18815. Note that I had to make the snapshot prefix
constants inside `snapshot_utils.rs` public at the crate level in order
to make this work. I'm not sure whether or not introducing this
dependency is entirely good, either way the `snapshot_utils.rs` file
needs a lot of rework so things will move around, I believe this does
the work in the meantime. Any feedback will be greatly appreciated.
 2021-08-04 09:03:03 -05:00
dependabot[bot]
f7be47b8b9 chore:(deps): bump @project-serum/serum in /explorer (#19048) 
Bumps [@project-serum/serum](https://github.com/project-serum/serum-ts) from 0.13.54 to 0.13.55.
- [Release notes](https://github.com/project-serum/serum-ts/releases)
- [Commits](https://github.com/project-serum/serum-ts/commits)

---
updated-dependencies:
- dependency-name: "@project-serum/serum"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-04 08:08:03 +00:00
Alexander Meißner
0a63f65c03 Bumps solana_rbpf to v0.2.14 (#18869) 
* Bumps solana_rbpf to v0.2.14

* Feature gate for verify_mul64_imm_nonzero as discussed in #17520.
 2021-08-04 09:50:28 +02:00
Trent Nelson
9912b73f9f ci: expand acceptable grcov version regex 2021-08-04 07:21:55 +00:00
Trent Nelson
38fb6dcf86 bump rust stable to 1.54, nightly 2021-08-02 2021-08-04 07:21:55 +00:00
Trent Nelson
06a7a9e544 remove superfluous collect()s 2021-08-04 07:21:55 +00:00
Trent Nelson
79b4b83d3c runtime: remove unused allow attrs 2021-08-04 07:21:55 +00:00
Jeff Biseda
0b7ed18cfa recvmmsg IPv6 awareness (#18957) 2021-08-03 20:35:50 -07:00
dependabot[bot]
d8984cb0f4 chore: bump rollup from 2.55.0 to 2.55.1 in /web3.js (#19038) 
Bumps [rollup](https://github.com/rollup/rollup) from 2.55.0 to 2.55.1.
- [Release notes](https://github.com/rollup/rollup/releases)
- [Changelog](https://github.com/rollup/rollup/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rollup/rollup/compare/v2.55.0...v2.55.1)

---
updated-dependencies:
- dependency-name: rollup
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-03 21:24:06 +00:00
dependabot[bot]
13110c16de chore: bump typedoc from 0.21.4 to 0.21.5 in /web3.js (#19037) 
Bumps [typedoc](https://github.com/TypeStrong/TypeDoc) from 0.21.4 to 0.21.5.
- [Release notes](https://github.com/TypeStrong/TypeDoc/releases)
- [Commits](https://github.com/TypeStrong/TypeDoc/compare/v0.21.4...v0.21.5)

---
updated-dependencies:
- dependency-name: typedoc
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-03 21:01:59 +00:00