Commit Graph

2154 Commits

Author SHA1 Message Date
mergify[bot]
477e5d4bff Add --force arg for bigtable upload (#15362)
(cherry picked from commit 98e3e570d2)

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-02-17 02:55:35 +00:00
mergify[bot]
c527e1f2e5 adds an upper bound on cluster-slots size (#15300) (#15357)
https://github.com/solana-labs/solana/issues/14366#issuecomment-769096305
(cherry picked from commit f79c9d4094)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-02-16 22:32:26 +00:00
mergify[bot]
135f47b6be checks that prune-messages have the same inner/outer pubkey (#15352) (#15356)
(cherry picked from commit 076c20f1ca)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-02-16 22:22:29 +00:00
mergify[bot]
f48236837c fill in timing gaps in replay_stage (#14550) (#15197)
* fill in timing gaps in replay_stage

* add replay_stage bank_count metric

* formatting

* handle another gap

* cleanup wait_receive_time to be more straightforward

(cherry picked from commit 935dfdf0f6)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-02-16 09:53:08 +00:00
mergify[bot]
59beb8e548 More configurable rocksdb compaction (#15213) (#15325)
rocksdb compaction can cause long stalls, so
make it more configurable to try and reduce those stalls
and also to coordinate between multiple nodes to not induce
stall at the same time.

(cherry picked from commit 5b8f046c67)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-02-16 01:46:36 +00:00
Trent Nelson
20afb912cd Bump version to 1.5.8 2021-02-13 04:34:36 +00:00
mergify[bot]
e976b1547a Fix flaky test test_concurrent_snapshot_packaging (#15252) (#15281)
(cherry picked from commit 990bb426a9)

Co-authored-by: carllin <carl@solana.com>
2021-02-12 01:22:06 +00:00
mergify[bot]
03ac807756 RPC: add caching to getLargestAccounts (#15154) (#15271)
* introduce get largest accounts cache

* remove cache size and change hash key

* remove eq and hash derivation from commitment config

* add slot to the cache

(cherry picked from commit 4013f91dbe)

Co-authored-by: Josh <josh.hundley@gmail.com>
2021-02-11 21:13:09 +00:00
Michael Vines
cd994f0162 Bump version to 1.5.7 2021-02-10 05:18:39 +00:00
mergify[bot]
01e4d0a1e9 Use spl-token-mint secondary index for relevant getProgramAccounts requests (#15219) (#15224)
(cherry picked from commit 948819dfa8)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-02-10 00:39:55 +00:00
mergify[bot]
7f0f43cb28 Warp timestamp and extend max-allowable-drift for accommodate slow blocks (#15204) (#15222)
* Remove timestamp_correction feature gating

* Remove timestamp_bounding feature gating

* Remove unused deprecated ledger code

* Remove unused deprecated unbounded-timestamp code

* Enable independent adjustment of fast/slow timestamp bounding

* Update timestamp bounds to 25% fast, 80% slow; warp timestamp

* Update bank hash test

* Add PR number to feature

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
(cherry picked from commit da6753b8c0)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-02-10 00:03:26 +00:00
mergify[bot]
d0bf97d25d uses btree-map instead of hash-map for cluster-slots (#15194) (#15220)
retain traverses all values in the hashmap which is slow:
https://github.com/solana-labs/solana/blob/88f22c360/core/src/cluster_slots.rs#L45
btree-map instead allows more efficient prunning there.

In addition there is potential race condition here:
https://github.com/solana-labs/solana/blob/88f22c360/core/src/cluster_slots.rs#L68-L74
If another thread inserts a value at the same slot key between the read
and write lock, current thread will discard the inserted value.

(cherry picked from commit 2758588ddd)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-02-09 23:21:08 +00:00
Michael Vines
c3056734e3 solana-test-validator now uses the BPF JIT by default, --no-bpf-jit to disable 2021-02-09 21:43:00 +00:00
mergify[bot]
b711476811 removes locked pubkey references (#15152) (#15182)
(cherry picked from commit b6f231b60e)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-02-08 03:24:38 +00:00
mergify[bot]
8016f61ce8 use thread pool for non-index hash calculations (#15149) (#15153)
(cherry picked from commit fabecdc86c)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-02-05 21:59:59 +00:00
mergify[bot]
d567a62cc7 caches descendants in bank forks (#15107) (#15148)
(cherry picked from commit 6fd5ec0e4c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-02-05 19:38:36 +00:00
mergify[bot]
36ad03af1f calculate hash from store instead of index (bp #15034) (#15084)
* calculate hash from store instead of index (#15034)

* calculate hash from store instead of index

* restore update hash in abs

(cherry picked from commit 600ff0d915)

* fix merge conflict (#15085)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-02-04 16:58:11 +00:00
mergify[bot]
169403a15e removes pubkey references (#15050) (#15073)
(cherry picked from commit 86467d825a)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-02-04 00:24:58 +00:00
mergify[bot]
a7193ce834 adds flag to disable duplicate instance check (#15006) (#15053)
(cherry picked from commit 0ad063f4e9)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-02-03 08:33:07 -08:00
mergify[bot]
d9fcd84bc2 Add validator flag to opt in to cpi and logs storage (bp #14922) (#14973)
* Add validator flag to opt in to cpi and logs storage (#14922)

* Add validator flag to opt in to cpi and logs storage

* Default TestValidator to opt-in; allow using in multinode-demo

* No clone

Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit cbb8b79a60)

* TestValidator store cpi and logs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-02-02 02:51:35 +00:00
Ryo Onodera
d8d73ff56c Clean up VerifiedVotePackets (#14822)
(cherry picked from commit bd0433c373)
2021-01-29 18:03:41 -08:00
mergify[bot]
b1d5bf30d2 Remove potentially too costly Packets::default() (#14821) (#14915)
* Remove potentially too costly Packets::default()

* Fix test...

* Restore Packets::default()

* Restore Packets::default() more

(cherry picked from commit d6873b82ab)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-01-29 13:52:33 +09:00
mergify[bot]
b62349f081 cli now supports a custodian for stake authorize operations (#14860)
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-28 18:30:43 +00:00
Michael Vines
4378634970 Bump version to 1.5.6 2021-01-27 10:50:56 -08:00
Michael Vines
676da0a836 Ensure sanitary transactions
(cherry picked from commit 04ce33a04e)
2021-01-26 17:03:01 -08:00
mergify[bot]
3c67f71695 Deprecate commitment variants (bp #14797) (#14858)
* Deprecate commitment variants (#14797)

* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments

(cherry picked from commit ffa5c7dcc8)

* Fix 1.5 stake-o-matic

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-01-26 20:49:04 +00:00
behzad nouri
08f7f2546e fixes test_filter_current flakiness (#14816)
(cherry picked from commit d1df9da7d3)
2021-01-25 12:44:46 -08:00
mergify[bot]
231ff7d70d removes redundant epoch stakes cache in retransmit (#14781) (#14817)
Following d6d76219b, staked nodes computed from vote accounts are
already cached in runtime::Stakes, so the caching in retransmit_stage is
redundant.

(cherry picked from commit e1021d9f83)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-01-25 01:11:58 +00:00
mergify[bot]
a154414e65 patches crds vote-index assignment bug (bp #14438) (#14741)
* patches crds vote-index assignment bug (#14438)

If tower is full, old votes are evicted from the front of the deque:
https://github.com/solana-labs/solana/blob/2074e407c/programs/vote/src/vote_state/mod.rs#L367-L373
whereas recent votes if expire are evicted from the back:
https://github.com/solana-labs/solana/blob/2074e407c/programs/vote/src/vote_state/mod.rs#L529-L537

As a result, from a single tower_index scalar, we cannot infer which crds-vote
should be overwritten:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L576

In addition there is an off by one bug in the existing code. tower_index is
bounded by MAX_LOCKOUT_HISTORY - 1:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/consensus.rs#L382
So, it is at most 30, whereas MAX_VOTES is 32:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L29
Which means that this branch is never taken:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L590-L593
so crds table alwasys keeps 29 **oldest** votes by wallclock, and then
only overrides the 30st one each time. (i.e a tally of only two most
recent votes).

(cherry picked from commit 8e581601d6)

* removes unnecessary semicolon

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-01-24 21:24:16 +00:00
mergify[bot]
65a7621b17 broadcasts duplicate shreds through gossip (bp #14699) (#14812)
* broadcasts duplicate shreds through gossip (#14699)

(cherry picked from commit 491b059755)

# Conflicts:
#	core/src/cluster_info.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-01-24 17:45:35 +00:00
mergify[bot]
c9da25836a Upgrade to Rust v1.49.0 (bp #14810) (#14811)
* Upgrade to Rust v1.49.0

(cherry picked from commit cbffab7850)

# Conflicts:
#	core/src/crds_value.rs

* rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-24 04:42:09 +00:00
mergify[bot]
dfbe38b859 Add solana-test-validator --warp-slot argument (bp #14785) (#14796)
* Add convenience function to create a snapshot archive out of any Bank

(cherry picked from commit dd5a2ef05f)

* Add solana-test-validator --warp-slot argument

(cherry picked from commit bf1943e489)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-23 06:38:16 +00:00
mergify[bot]
e127631f8d Add ability to clone accounts from an RPC endpoint (#14784)
(cherry picked from commit cbb9ac19b9)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-22 22:47:29 +00:00
mergify[bot]
c49a8cb67c Rpc: Add custom error for BigTable data not found (#14762) (#14765)
* Expose not-found bigtable error

* Add custom rpc error for bigtable data not found

* Return custom rpc error when bigtable block is not found

* Generalize long-term storage

(cherry picked from commit 71e9958e06)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-01-22 05:56:47 +00:00
mergify[bot]
c8f7719c9e fixes test_filter_current flakiness (bp #14749) (#14761)
* fixes test_filter_current flakiness (#14749)

(cherry picked from commit e4da6761a7)

# Conflicts:
#	core/src/crds_value.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-01-22 01:24:10 +00:00
mergify[bot]
239dc9b0b7 rewrites turbine retransmit peers computation (#14584) (#14742)
(cherry picked from commit b5fd0ed859)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-01-21 14:25:46 +00:00
Jack May
d72c90e475 Bump version to v1.5.5 (#14700) 2021-01-20 20:26:16 +00:00
mergify[bot]
ca262fdeb9 Configure Bigtable's timeout, enabling by default (#14657) (#14669)
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
(cherry picked from commit dcaa025822)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-01-19 15:14:59 +00:00
mergify[bot]
e5d36fcfb3 feature gates turbine retransmit peers patch (#14631) (#14659)
(cherry picked from commit c6ae0667e6)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-01-19 05:38:26 +00:00
mergify[bot]
061965e291 Rename RpcNodeUnhealthy error to NodeUnhealthy, generalize getHealth RPC error object for the future (#14656)
(cherry picked from commit 5d9dc609b1)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-19 05:24:29 +00:00
mergify[bot]
e56681a2f6 Make Bigtable::get_confirmed_blocks inclusive of requested start_slot and end_slot (#14651) (#14655)
* Fix off-by-one error

* Filter out blocks greater than end slot

(cherry picked from commit cbf8ef7480)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-01-19 03:42:59 +00:00
mergify[bot]
8cf23903ce Fix the occasional stuck RPC request (bp #14628) (#14638)
* WIP fix the occasional stuck RPC request

(cherry picked from commit 5cf9094bb9)

* Clean up and add comment

(cherry picked from commit 8d4ab1bab1)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-01-18 19:57:28 +00:00
mergify[bot]
7ac2aae730 More generic accounts purge functions (#14595) (#14640)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit 5f14f45282)

Co-authored-by: carllin <wumu727@gmail.com>
2021-01-18 05:53:40 +00:00
mergify[bot]
a04375e204 Add getSnapshotSlot RPC method (#14632)
(cherry picked from commit 4003f86f04)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-16 20:51:33 +00:00
Michael Vines
7b19d26a6e Add getHealth RPC method 2021-01-16 10:51:54 -08:00
Michael Vines
3c3a3f0b50 Improve solana-test-validator output
(cherry picked from commit 1c2ae15b1d)
2021-01-16 10:14:43 -08:00
mergify[bot]
3d8cadebc0 Use optimistic confirmation in getSignatureStatuses, and various downstream client methods (#14430) (#14611)
* Add optimistically_confirmed field to TransactionStatus

* Update docs

* Convert new field to confirmation_status

* Update docs to confirmationStatus

* Update variants

* Update docs

* Just Confirmed

(cherry picked from commit 9a89689ad3)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-01-15 17:25:04 +00:00
mergify[bot]
bdfffd0151 Add load/execute/store timings (#14561) (#14591)
(cherry picked from commit 907f518f6d)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-01-14 23:48:36 +00:00
Trent Nelson
1b02ec4f6e Bump version to v1.5.4 2021-01-14 04:40:25 +00:00
mergify[bot]
aae51925c1 patches bug in turbine's neighbors computation (#14565) (#14569)
Removing local node's index early from the set here:
https://github.com/solana-labs/solana/blob/e1b59ded4/core/src/retransmit_stage.rs#L346
distorts the order of nodes depending on which node is computing the
turbine fan-out tree, and results in incorrect neighbors computation.

(cherry picked from commit cfcca1cd3c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-01-13 23:52:14 +00:00