a7aa5202ab
bank: fix executor cache metrics ( #22395 )
...
(cherry picked from commit 3b4aad9df1
)
Co-authored-by: Trent Nelson <trent@solana.com >
2022-01-09 05:55:08 +00:00
9aa7821277
Bump version to v1.8.13
2022-01-08 12:27:11 -07:00
fee86726f2
improve multi executor cache addition ( #22381 )
...
(cherry picked from commit 4a9f4e2505
)
Co-authored-by: Jack May <jack@solana.com >
v1.8.12
2022-01-08 12:52:16 +00:00
8a470d3ae8
add excutor creation trace timings
2022-01-08 04:46:31 -07:00
8c6df1f234
bank: prime new executor cache entry use-counts ( #22374 )
...
Co-authored-by: Trent Nelson <trent@solana.com >
2022-01-08 10:14:05 +00:00
7cbfab0958
remove per program timings from blockstore processor ledger replay ( #22370 ) ( #22371 )
...
(cherry picked from commit 813006b33b
)
Co-authored-by: carllin <carl@solana.com >
2022-01-08 08:33:31 +00:00
6130466a21
Executor cache metrics (backport #22332 ) ( #22367 )
...
* Add helper macro for `AddAssign`ing with saturating arithmetic
* bank: Add executors cache metrics
Co-authored-by: Trent Nelson <trent@solana.com >
2022-01-08 02:19:36 +00:00
299a59f458
removes CowCachedExecutors (backport #22343 ) ( #22362 )
...
* removes CowCachedExecutors (#22343 )
Copy-on-write semantics for cached executors can be implemented by a
simple Arc<CachedExecutors> as opposed to CowCachedExecutors:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L244-L247
This will also avoid the need for double locking as in:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3490-L3491
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3525-L3526
(cherry picked from commit c2389fc209
)
# Conflicts:
# runtime/src/bank.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2022-01-07 16:18:00 +00:00
b30c726d22
Add aarch64-apple-darwin publish tarball step ( #22355 )
...
(cherry picked from commit e2aa932e97
)
Co-authored-by: Michael Vines <mvines@gmail.com >
2022-01-07 10:13:46 +00:00
069bb5e32f
Increase timeout of local-cluster-slow CI step
2022-01-07 15:31:28 +08:00
cad0e7f04f
cache executors on failed transactions (backport #22308 ) ( #22327 )
...
* cache executors on failed transactions (#22308 )
(cherry picked from commit 12e160269e
)
# Conflicts:
# program-runtime/src/invoke_context.rs
# programs/bpf_loader/src/lib.rs
# runtime/src/bank.rs
* resolve conflicts
Co-authored-by: Jack May <jack@solana.com >
2022-01-06 15:38:39 -08:00
c90fa6643e
Retain executor cache counts ( #22322 ) ( #22340 )
...
(cherry picked from commit f1e2598baa
)
Co-authored-by: Jack May <jack@solana.com >
2022-01-06 19:11:05 +00:00
54db774203
Split up local cluster tests into separate CI steps (backport #22295 ) ( #22302 )
...
* Split up local cluster tests into separate CI steps (#22295 )
* Split up local cluster tests into separate CI steps
* Update buildkite-pipeline.sh
(cherry picked from commit 0e1afcbb26
)
# Conflicts:
# local-cluster/tests/local_cluster.rs
* resolve conflicts
Co-authored-by: Justin Starry <justin@solana.com >
2022-01-06 15:26:56 +00:00
57b5ce5731
[ledger-tool]compare_blocks ( #22229 ) ( #22329 )
...
* 1.made load_credentials accept credential path as a parameter. 2.partial implement bigtable comparasion function
* finding missing blocks in bigtables in a specified range
* refactor compare-blocks,add unit test for missing_blocks and fmt
* compare-block fix last block bug
* refactor compare-block and improve wording
* Update ledger-tool/src/bigtable.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com >
* update compare-block command-line description
* style:improve wording/naming/code style
Co-authored-by: Tyera Eulberg <teulberg@gmail.com >
(cherry picked from commit d9220652ad
)
Co-authored-by: pieceofr <komimi.p@gmail.com >
2022-01-06 08:47:08 +00:00
e0d933f940
perf: skip updating already cached executors if unmodified (backport #22300 ) ( #22315 )
...
* Skip updating already cached executors if unmodified
(cherry picked from commit f2ed6f09ee
)
# Conflicts:
# program-runtime/src/invoke_context.rs
# runtime/src/bank.rs
* resolve conflicts
Co-authored-by: Justin Starry <justin@solana.com >
2022-01-06 02:04:55 +00:00
e2559f20df
fix(rpc): recreate dead and uncleaned subscriptions ( #22281 ) ( #22293 )
...
(cherry picked from commit c1995c647b
)
Co-authored-by: Nikita <bananaelecitrus@gmail.com >
2022-01-05 10:23:34 +00:00
7b9ca3e9d9
Flip iter operations to keep associated address/header/packets together (backport #22245 ) ( #22256 )
...
* Flip iter operations to keep associated address/header/packets together (#22245 )
Flip iter operations to keep associated address/header/packets together
Before this change, if cast_socket_addr() returned a None for any
address/header pair, the subsequent zip() would misalign the
address/header pair and packet. So, this change zips all three together,
then does filter_map() so keep things aligned.
Additionally, compute total_size inline to avoid running through packets
a second time.
(cherry picked from commit 20b61e28b6
)
# Conflicts:
# streamer/src/recvmmsg.rs
* removes backport merge conflicts
Co-authored-by: steviez <steven@solana.com >
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2022-01-04 16:31:59 +00:00
f0de9b43be
Lower vote-only-mode to 400 ( #22210 ) ( #22271 )
...
(cherry picked from commit 2486e21ffe
)
Co-authored-by: sakridge <sakridge@gmail.com >
2022-01-04 14:17:31 +00:00
6d7b64b140
Fix bug, add error specific timings ( #22225 ) ( #22251 )
...
(cherry picked from commit 005592998d
)
Co-authored-by: carllin <carl@solana.com >
2022-01-04 09:23:49 +00:00
f506851ca1
removes epoch_authorized_voters from VoteTracker (backport #22192 ) ( #22250 )
...
* removes epoch_authorized_voters from VoteTracker (#22192 )
https://github.com/solana-labs/solana/pull/22169
verifies authorized-voter early on in vote-listener pipeline; and so
VoteTracker no longer needs to maintain and check for epoch authorized
voters.
(cherry picked from commit 69d71f8f86
)
# Conflicts:
# core/src/cluster_info_vote_listener.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2022-01-04 02:02:56 +00:00
b70113e201
Correctly set CI_COMMIT when Buildkite provides HEAD instead of a real commit
2022-01-03 17:40:35 -08:00
6093c7a218
Switch from arm64-apple-darwin to aarch64-apple-darwin to align with Rust's target names
2022-01-03 17:27:09 -08:00
b9777e10ee
Add support for arm64-apple-darwin release/channel artifacts
2022-01-03 17:27:05 -08:00
d840d56565
Use experimential docker virtualization framework for arm64
...
(cherry picked from commit ed0b47c6f8
)
2022-01-03 17:27:00 -08:00
afb8df0c62
Correctly set CI_OS_NAME for macOs buildkite agents
2022-01-03 17:26:54 -08:00
84d535b89e
re-calibrate limit based on mainnet data (backport #21995 ) ( #22211 )
...
* re-calibrate limit based on mainnet data, see issue #21917
(cherry picked from commit d743c2917c
)
# Conflicts:
# runtime/src/block_cost_limits.rs
* set secp256k1 cost similar to sigverify
(cherry picked from commit a2a7e91ad6
)
* removes backport merge conflicts
Co-authored-by: Tao Zhu <tao@solana.com >
2022-01-03 19:03:41 +00:00
de1a9f84ac
Bump rbpf to v0.2.21 ( #22216 ) ( #22218 )
...
(cherry picked from commit 9139be89b7
)
2022-01-01 21:43:28 +00:00
22bbd91843
Count compute units even when transaction errors (backport #22182 ) ( #22198 )
...
* Resolve conflicts
* Return err
Co-authored-by: carllin <carl@solana.com >
2022-01-01 13:16:56 +00:00
1babd07faf
Increment execution timings on errors as well (backport #22053 ) ( #22071 )
...
* Increment execution timings on errors as well (#22053 )
(cherry picked from commit 37f6777ceb
)
# Conflicts:
# programs/bpf_loader/src/lib.rs
* Fix conflicts
Co-authored-by: carllin <carl@solana.com >
2022-01-01 13:04:34 +00:00
0592b5568f
Bump local cluster timeout ( #22214 )
2022-01-01 06:02:38 -05:00
ce98feac4e
Exit early on BigTable error ( #22200 ) ( #22208 )
...
(cherry picked from commit 0b1b36f088
)
Co-authored-by: Tyera Eulberg <teulberg@gmail.com >
2022-01-01 02:05:36 +00:00
a53dd611c0
uses enum for shred type (backport #21333 ) ( #22147 )
...
* uses enum for shred type
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167
Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348
The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
(cherry picked from commit 57057f8d39
)
# Conflicts:
# core/src/retransmit_stage.rs
# gossip/src/cluster_info.rs
# ledger/Cargo.toml
# ledger/src/blockstore.rs
# ledger/src/shred.rs
* changes Blockstore::is_shred_duplicate arg type to ShredType
(cherry picked from commit 48dfdfb4d5
)
# Conflicts:
# ledger/src/blockstore.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-31 16:47:45 +00:00
ec970f9d69
checks for authorized voter early on in the vote-listener pipeline (backport #22169 ) ( #22188 )
...
* checks for authorized voter early on in the vote-listener pipeline (#22169 )
Before votes are verified that they are signed by the authorized voter,
they might be dropped in verified-vote-packets code. If there are
enough many spam votes from unauthorized voters, this may potentially
drop valid votes but keep the false ones.
https://github.com/solana-labs/solana/blob/57986f982/core/src/verified_vote_packets.rs#L165-L168
(cherry picked from commit c0c6038654
)
# Conflicts:
# core/src/cluster_info_vote_listener.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-30 17:52:36 +00:00
7793a11b65
Bump solana_rbpf to version v0.2.20 ( #22166 )
2021-12-30 17:34:11 +01:00
4deac1daa4
Update install/src/command.rs
...
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com >
(cherry picked from commit 29edb130cc
)
2021-12-30 07:30:38 -08:00
4c36a93665
Add connect timeout and change overall timeout to None
...
(cherry picked from commit 3c1416091e
)
2021-12-30 07:30:38 -08:00
052309227f
fix: Installer increase download req timeout from 30 seconds to 6 minutes
...
(cherry picked from commit a1912f8400
)
2021-12-30 07:30:38 -08:00
3b895104f3
ClusterInfoVoteListener send only missing votes to BankingStage (backport #20873 ) ( #22135 )
...
* ClusterInfoVoteListener send only missing votes to BankingStage (#20873 )
(cherry picked from commit b30c94ce55
)
# Conflicts:
# core/src/cluster_info_vote_listener.rs
# core/src/verified_vote_packets.rs
# gossip/src/cluster_info.rs
# local-cluster/tests/local_cluster.rs
# runtime/src/bank.rs
* Resolve conflicts
* Remove Select
* Fixup tests
Co-authored-by: carllin <carl@solana.com >
2021-12-30 06:38:49 +00:00
24b0fc8927
get_signatures_for_address does not correctly account for result sets that span local and Bigtable sources (backport #22115 ) ( #22167 )
...
* get_signatures_for_address does not correctly account for result sets that span local and Bigtable sources (#22115 )
* get_signatures_for_address does not correctly account for result sets that span Blockstore and Bigtable.
This causes Bigtable to return `RowNotFound` until the new tx is uploaded.
Check that `before` exists in Bigtable, and if not, set it to `None` to return the full data set.
References #21442
Closes #22110
* Differentiate between before sig not found and no newer signatures
* Dedupe bigtable results to account for potential upload race
Co-authored-by: Tyera Eulberg <tyera@solana.com >
(cherry picked from commit bac6821e19
)
# Conflicts:
# ledger/src/blockstore.rs
* Fix conflicts
Co-authored-by: Omar Kilani <omar.kilani@gmail.com >
Co-authored-by: Tyera Eulberg <tyera@solana.com >
2021-12-29 13:03:32 -07:00
271ae3c2fc
Ignore spl downstream build ( #22173 )
2021-12-29 12:44:46 -07:00
a305fa0472
implements copy-on-write for vote-accounts (backport #19362 ) #22139 ( #22139 )
...
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186
This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-28 21:18:37 +00:00
262b157d21
renames solana_runtime::vote_account::VoteAccount and makes it private (backport #19153 ) ( #22141 )
...
* makes solana_runtime::vote_account::VoteAccount private
VoteAccount is an implementation detail, and should not be public.
Only ArcVoteAccount is the public type.
(cherry picked from commit 1403eaeefc
)
# Conflicts:
# runtime/src/vote_account.rs
* renames solana_runtime::vote_account::VoteAccount
Rename:
VoteAccount -> VoteAccountInner # the private type
ArcVoteAccount -> VoteAccount # the public type
(cherry picked from commit 00e5e12906
)
# Conflicts:
# core/src/progress_map.rs
# ledger/src/blockstore_processor.rs
# ledger/src/staking_utils.rs
# runtime/src/bank.rs
# runtime/src/epoch_stakes.rs
# runtime/src/serde_snapshot/tests.rs
# runtime/src/stakes.rs
# runtime/src/vote_account.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-28 18:26:23 +00:00
bfb02029bf
adds methods to obtain data/coding shreds indices from ErasureMeta (backport #21254 ) ( #22132 )
...
* marks ErasureMeta.size as unused
ErasureMeta.size is unused and redundant:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/blockstore_meta.rs#L57-L58
(cherry picked from commit 7e98b77b33
)
* removes Default implementaion of ErasureConfig
The Default implementation of ErasureConfig is off and never useful:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/erasure.rs#L49-L68
(cherry picked from commit d25d9be555
)
* adds methods to obtain data/coding shreds indices from ErasureMeta
(cherry picked from commit 3fc858eb60
)
# Conflicts:
# ledger/src/blockstore_meta.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-28 01:15:09 +00:00
039244417e
removes redundant args from Shredder::try_recovery (backport #21226 ) ( #22126 )
...
* removes redundant args from Shredder::try_recovery (#21226 )
Shredder::try_recovery is already taking a Vec<Shred> as an argument. All the
other arguments are embedded in the shreds, and are so redundant.
(cherry picked from commit 5fb0ab9d00
)
# Conflicts:
# ledger/src/shred.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-27 21:02:33 +00:00
1e1f383970
rewrites Shredder::try_recovery (backport #21082 ) ( #22125 )
...
* rewrites Shredder::try_recovery (#21082 )
For every missing data-shred, Shredder::try_recovery calls into
new_empty_data_shred which does a redundant serialization into payload
buffer which is then immediately overwritten by the erasure recovery:
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L372-L417
Additionally, the implementation is unnecessary complex hindering
upcoming changes to erasure coding generation
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L814-L938
The commit simplifies the Shredder::try_recovery implementation.
(cherry picked from commit 11a53de0e3
)
# Conflicts:
# ledger/src/erasure.rs
# ledger/src/shred.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-27 16:42:22 +00:00
891b0a5152
optimizes and simplifies SlotMeta::completed_data_indexes ops ( #21059 ) ( #22100 )
...
SlotMeta::completed_data_indexes is defined as a Vec<u32>:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore_meta.rs#L31-L32
which results in inefficient updates:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore.rs#L3245-L3326
This commit changes the type to BTreeSet<u32> for efficient and simpler
updates and lookups.
The change should be backward compatible because Vec<T> and BTreeSet<T>
are both serialized as seq:
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L207-L208
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L216-L217
(cherry picked from commit 484ead01ed
)
Co-authored-by: behzad nouri <behzadnouri@gmail.com >
2021-12-26 17:22:22 +00:00
685e40cbf2
validator: add contact-info query to admin port
2021-12-23 20:23:48 +00:00
17d698d20a
report mem stats ( #21258 ) ( #22066 )
...
(cherry picked from commit f8dcb2f38b
)
# Conflicts:
# Cargo.lock
# core/Cargo.toml
# core/src/system_monitor_service.rs
# runtime/src/bucket_map_holder_stats.rs
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com >
2021-12-23 18:16:56 +00:00
cf34ae7d6f
chore: bump lru from 0.7.0 to 0.7.1 (backport #22018 ) ( #22079 )
...
* chore: bump lru from 0.7.0 to 0.7.1 (#22018 )
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.0...0.7.1 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 69d0b08dd8
)
# Conflicts:
# Cargo.lock
# core/Cargo.toml
# gossip/Cargo.toml
* Fix conflicts
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com >
2021-12-23 02:57:40 +00:00
acd03fc29b
Check file size of snapshot_version when unarchiving snapshot (backport #21925 ) ( #21987 )
2021-12-20 14:22:34 -06:00