gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
11,175 Commits 81 Branches 357 Tags
55b0e9e9c7a1e8aff230c7fba2960b51ed83ee46
Commit Graph

11175 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
mergify[bot]
55b0e9e9c7 builds crds filters in parallel (bp #12360) (#12571) 
* builds crds filters in parallel (#12360)

Based on run-time profiles, the majority time of new_pull_requests is
spent building bloom filters, in hashing and bit-vec ops.

This commit builds crds filters in parallel using rayon constructs. The
added benchmark shows ~5x speedup (4-core machine, 8 threads).

(cherry picked from commit 537bbde22e)

# Conflicts:
#	core/Cargo.toml

* resolves mergify merge conflict

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-09-30 01:08:16 +00:00
mergify[bot]
6d1bea7fb4 Include active stake in 'epoch_rewards' datapoint (#12573) 
(cherry picked from commit 82848d6c73)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-30 01:08:04 +00:00
mergify[bot]
d19ed8816e Query BigTable for block time if does not exist in blockstore (#12560) (#12565) 
(cherry picked from commit 96a7d4dbd8)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-09-29 23:26:03 +00:00
mergify[bot]
af7f48a2fd Track inserted repair shreds (#12455) (#12563) 
(cherry picked from commit ce98088457)

Co-authored-by: sakridge <sakridge@gmail.com>
 2020-09-29 22:57:58 +00:00
Tyera Eulberg
0965389f41 Enable json output from solana feature status (#12554) (#12559) 2020-09-29 21:00:20 +00:00
Trent Nelson
24c60cf3db Bump version to v1.3.15 2020-09-29 20:57:08 +00:00
Michael Vines
c8f4bfca90 Notify but don't abort on unexpected stake account balance v1.3.14 2020-09-29 11:39:11 -07:00
Michael Vines
4a6b65ce53 Switch get_program_accounts to use base64 2020-09-29 18:21:19 +00:00
mergify[bot]
836ed842d6 Increase rpc pubsub max payload to unblock large account notifications (#12548) (#12551) 
(cherry picked from commit 36d55c0667)

Co-authored-by: Justin Starry <justin@solana.com>
 2020-09-29 17:20:31 +00:00
mergify[bot]
966d0f72bb Move process_instruction defs to runtime (#12507) (#12549) 
(cherry picked from commit 2ff983647f)

Co-authored-by: Jack May <jack@solana.com>
 2020-09-29 15:52:38 +00:00
mergify[bot]
a07e90516b separates out ClusterInfo::{gossip,listen} thread-pools (#12535) (#12547) 
https://github.com/solana-labs/solana/pull/12402
moved gossip-work threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/cluster_info.rs#L2330-L2334
to ClusterInfo::new as a new field in the ClusterInfo struct:
https://github.com/solana-labs/solana/blob/35208c5ee/core/src/cluster_info.rs#L249
So that they can be shared between listen and gossip threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/gossip_service.rs#L54-L67

However, in testing https://github.com/solana-labs/solana/pull/12360
it turned out this will cause breakage:
https://buildkite.com/solana-labs/solana/builds/31646
https://buildkite.com/solana-labs/solana/builds/31651
https://buildkite.com/solana-labs/solana/builds/31655
Whereas with separate thread pools all is good. It might be the case
that one thread is slowing down the other by exhausting the thread-pool
whereas with separate thread-pools we get fair scheduling guarantees
from the os.

This commit reverts https://github.com/solana-labs/solana/pull/12402
and instead adds separate thread-pools for listen and gossip threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/gossip_service.rs#L54-L67

(cherry picked from commit 0d5258b6d3)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-09-29 11:27:47 +00:00
mergify[bot]
bd2e09d55a patches bug in Crds::find_old_labels with pubkey specific timeout (#12528) (#12546) 
Current code only returns values which are expired based on the default
timeout. Example from the added unit test:
  - value inserted at time 0
  - pubkey specific timeout = 1
  - default timeout = 3
Then at now = 2, the value is expired, but the function fails to return
the value because it compares with the default timeout.

(cherry picked from commit 57ed4e4657)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-09-29 10:13:13 +00:00
mergify[bot]
655577f9fe feature subcommand: display active stake by feature id when feature activation is not available (#12543) 
(cherry picked from commit 322dbd894f)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-29 06:17:31 +00:00
Trent Nelson
3781ad259b clap-utils: Allow nonce/offline args to be global (bp #12538) 2020-09-29 04:51:33 +00:00
Trent Nelson
5ad5f8b458 cli-output: Add a path to handling --verbose and --quiet display (bp #12531) 2020-09-29 04:44:59 +00:00
mergify[bot]
5b322a995f Rpc -> proper optimistic confirmation (#12514) (#12537) 
* Add service to track the most recent optimistically confirmed bank

* Plumb service into ClusterInfoVoteListener and ReplayStage

* Clean up test

* Use OptimisticallyConfirmedBank in RPC

* Remove superfluous notifications from RpcSubscriptions

* Use crossbeam to avoid mpsc recv_timeout panic

* Review comments

* Remove superfluous last_checked_slots, but pass in OptimisticallyConfirmedBank for complete correctness

(cherry picked from commit 89621adca7)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-09-29 03:49:18 +00:00
mergify[bot]
63d9f32bb4 purges old pending push messages more efficiently (#12522) (#12533) 
(cherry picked from commit c94fe9236f)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-09-29 01:34:58 +00:00
mergify[bot]
a550d82202 Enable commitment arg on solana deploy (#12532) (#12534) 
(cherry picked from commit 35208c5ee7)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-09-28 23:32:50 +00:00
mergify[bot]
4cf69365b2 Port BPFLoader2 activation to FeatureSet (bp #12490) (#12530) 
* Cargo.lock

(cherry picked from commit 6071d0d206)

# Conflicts:
#	Cargo.lock

* Port BPFLoader2 activation to FeatureSet and rework built-in program activation

(cherry picked from commit 31696a1d72)

# Conflicts:
#	core/Cargo.toml
#	genesis-programs/Cargo.toml
#	genesis/Cargo.toml
#	ledger/Cargo.toml
#	local-cluster/Cargo.toml
#	runtime/src/bank.rs

* Add Builtin AbiExample

(cherry picked from commit 833ad20b01)

* Rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-28 23:12:05 +00:00
mergify[bot]
873b4ee830 Add a couple feature tests (#12529) 
(cherry picked from commit 2956cc5aed)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-28 20:31:09 +00:00
mergify[bot]
672d9c9f62 Add feature to resolve spl-token v2 multisig bug (#12525) 
(cherry picked from commit f9a74b51ef)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-28 18:14:20 +00:00
Michael Vines
4bd29c1b32 Add pico-inflation feature 
(cherry picked from commit aa5c008fa8)
 2020-09-28 09:34:35 -07:00
mergify[bot]
72c082f55a Add precompile verification to preflight (#12486) (#12516) 
(cherry picked from commit 6583c8cffe)

Co-authored-by: sakridge <sakridge@gmail.com>
 2020-09-28 06:37:52 +00:00
mergify[bot]
d712a908c2 Port fix_recent_blockhashes_sysvar_delay to FeatureSet (#12503) 
(cherry picked from commit 5d6410c1cb)

# Conflicts:
#	runtime/src/feature_set.rs

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-26 20:26:58 +00:00
mergify[bot]
e3ca1a81b4 Add copy-on-write executor cache (bp #12502) (#12511) 
* Add copy-on-write executor cache (#12502)

* Add copy-on-write executor cache

* Add remove_executor function to the bank

(cherry picked from commit 965f653471)

# Conflicts:
#	runtime/src/bank.rs

* rebase

Co-authored-by: Jack May <jack@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-26 20:26:50 +00:00
mergify[bot]
16bce553e4 Nit: bpf test cleanup (#12401) (#12508) 
(cherry picked from commit 7c4822efb1)

Co-authored-by: Jack May <jack@solana.com>
 2020-09-26 17:53:27 +00:00
mergify[bot]
bc3aa53e02 Runtime feature activation framework (bp #12376) (#12497) 
* Runtime feature activation framework

(cherry picked from commit 93259f0bae)

# Conflicts:
#	runtime/src/bank.rs

* Add feature set identifier to gossiped version information

(cherry picked from commit 35f5f9fc7b)

# Conflicts:
#	Cargo.lock
#	version/Cargo.toml

* Port instructions sysvar and secp256k1 program activation to FeatureSet

(cherry picked from commit c10da16d7b)

# Conflicts:
#	runtime/src/bank.rs
#	runtime/src/message_processor.rs

* Add feature management commands

(cherry picked from commit 93ed0ab2bb)

# Conflicts:
#	Cargo.lock
#	cli/Cargo.toml

* Make test_process_rest_api less fragile

(cherry picked from commit 7526bb96f3)

* Remove id field

(cherry picked from commit cc6ba1e131)

* FeatureSet test

(cherry picked from commit 92406cf9a0)

* cargo fmt

(cherry picked from commit 199940d683)

* cli review feedback

(cherry picked from commit 3a2b8c5e5b)

* Rename active() to is_active()

(cherry picked from commit e39fac9f01)

* Resolve merge conflicts

* Remove continues from compute_active_feature_set()

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-26 17:49:53 +00:00
Michael Vines
6a698af235 Deerror 2020-09-25 22:19:09 -07:00
Michael Vines
7ec38bd71c Improve 'Failed to create snapshot archive' warning message 
(cherry picked from commit 5dcf348098)
 2020-09-25 21:06:05 -07:00
mergify[bot]
8e3882287a Add epoch rewards metric datapoint (bp #12505) (#12509) 
* Add epoch rewards metric datapoint

(cherry picked from commit e50386f928)

# Conflicts:
#	runtime/src/bank.rs

* Update bank.rs

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-26 04:02:23 +00:00
mergify[bot]
0c4074049b Cleanup names, fix line dependent test (#12477) (#12482) 
(cherry picked from commit b8c4b88188)

Co-authored-by: Jack May <jack@solana.com>
 2020-09-26 01:08:55 +00:00
mergify[bot]
250d2ba74a Pre-construct cpi instruction recorders before message processing (#12467) (#12504) 
(cherry picked from commit 1c970bb39f)

Co-authored-by: Justin Starry <justin@solana.com>
 2020-09-26 00:40:32 +00:00
mergify[bot]
b96e0e3d27 Drain the entire compute budget (bp #12478) (#12492) 
* Drain the entire compute budget (#12478)


(cherry picked from commit d00453f747)

* fix conflict

Co-authored-by: Jack May <jack@solana.com>
 2020-09-25 23:22:19 +00:00
mergify[bot]
99b513d905 Bump rust-bpf to v0.2.4 (#12361) (#12501) 
(cherry picked from commit 65049bd112)

Co-authored-by: Jack May <jack@solana.com>
 2020-09-25 22:12:39 +00:00
mergify[bot]
e85c792f70 Add RPC notify and banking keys debug (bp #12396) (#12452) 
* Add RPC notify and banking keys debug (#12396)

(cherry picked from commit 68e5a2ef56)

# Conflicts:
#	core/src/validator.rs

* Rebase

Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-25 21:05:21 +00:00
Trent Nelson
b65a764593 Bump jsonrpc crates to 15.0.0 (bp #12491) 2020-09-25 19:49:10 +00:00
mergify[bot]
a514b0e77b Add ComputeBudget tuner (bp #12476) (#12483) 
* Add ComputeBudget tuner (#12476)

(cherry picked from commit d326512121)

# Conflicts:
#	programs/bpf/Cargo.toml

* fix conflicts

Co-authored-by: Jack May <jack@solana.com>
 2020-09-25 19:13:07 +00:00
mergify[bot]
179dd6ee59 Ignore cargo audit RUSTSEC-2020-0008 (#12489) 
(cherry picked from commit cd5c7f30d5)

Co-authored-by: Jack May <jack@solana.com>
 2020-09-25 10:01:23 -07:00
mergify[bot]
21ba2bad24 Add Signers impl for Vec<Box<dyn Signer>> (#12470) 
(cherry picked from commit 07dfa37cce)

Co-authored-by: Trent Nelson <trent@solana.com>
 2020-09-25 10:45:29 +00:00
mergify[bot]
64b6372f9c cli-output: Add CliTokenAccount type (bp #12466) (#12468) 
* account-decoder: Add string format helpers to UiTokenAmount

(cherry picked from commit bb144bf758)

* cli-output: Add CliTokenAccount type

(cherry picked from commit d95bce2600)

Co-authored-by: Trent Nelson <trent@solana.com>
 2020-09-25 06:05:19 +00:00
mergify[bot]
495ea7cd2f introduce RpcPerfSample and modify getPerformanceSamples output (#12434) (#12464) 
* introduce RpcPerfSample and modify getPerformanceSamples output

* camelCase test results

(cherry picked from commit 1d04c1db94)

Co-authored-by: Josh <josh.hundley@gmail.com>
 2020-09-24 22:45:05 +00:00
mergify[bot]
bb12d65102 Remove legacy inflation activation code (#12460) 
(cherry picked from commit c4aee8c0a0)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-09-24 20:38:41 +00:00
mergify[bot]
72365bb9d2 moves gossip-work thread pool cons to ClusterInfo::new (#12402) (#12458) 
(cherry picked from commit 42f1ef8acb)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-09-24 20:01:03 +00:00
mergify[bot]
c44f6981b1 adds an atomic variant of the bloom filter (#12422) (#12459) 
For crds_gossip_pull, we want to parallelize build_crds_filters, which
requires concurrent writes to bloom filters.

This commit implements a variant of the bloom filter which uses atomics
for its bits vector and so is thread-safe.

(cherry picked from commit bb183938d9)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-09-24 19:49:10 +00:00
mergify[bot]
0213016999 Use publish=false (#12447) (#12453) 
(cherry picked from commit a5c3fc14b3)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-09-24 16:58:18 +00:00
mergify[bot]
4e9e05f311 shards crds values based on their hash prefix (bp #12187) (#12312) 
* shards crds values based on their hash prefix (#12187)

filter_crds_values checks every crds filter against every hash value:
https://github.com/solana-labs/solana/blob/ee646aa7/core/src/crds_gossip_pull.rs#L432
which can be inefficient if the filter's bit-mask only matches small
portion of the entire crds table.

This commit shards crds values into separate tables based on shard_bits
first bits of their hash prefix. Given a (mask, mask_bits) filter,
filtering crds can be done by inspecting only relevant shards.

If CrdsFilter.mask_bits <= shard_bits, then precisely only the crds
values which match (mask, mask_bits) bit pattern are traversed.
If CrdsFilter.mask_bits > shard_bits, then approximately only
1/2^shard_bits of crds values are inspected.

Benchmarking on a gce cluster of 20 nodes, I see ~10% improvement in
generate_pull_responses metric, but with larger clusters, crds table and
2^mask_bits are both larger, so the impact should be more significant.

(cherry picked from commit 9b866d79fb)

* bumps indexmap to 1.6.0

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2020-09-24 16:10:53 +00:00
mergify[bot]
7212bb12ea Record and store invoked instructions in transaction meta (#12311) (#12449) 
* Record invoked instructions and store in transaction meta

* Enable cpi recording if transaction sender is some

* Rename invoked to innerInstructions

(cherry picked from commit 6601ec8f26)

Co-authored-by: Justin Starry <justin@solana.com>
 2020-09-24 15:42:34 +00:00
mergify[bot]
9ff2378948 Remove transaction encoding from storage layer (bp #12404) (#12440) 
* Remove transaction encoding from storage layer (#12404)

(cherry picked from commit 731a943239)

* Bump

Co-authored-by: Justin Starry <justin@solana.com>
 2020-09-24 10:11:27 +00:00
Tyera Eulberg
ec4938a9f3 Bump version to 1.3.14 (#12444) 2020-09-24 07:42:54 +00:00
Tyera Eulberg
41b45ca281 Allow publishing of secp256k1 program 2020-09-24 00:05:11 -06:00