gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
13,613 Commits 81 Branches 357 Tags
25aee12502df871b19ffa23a783d71b53e9e076c
Commit Graph

13613 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
mergify[bot]
25aee12502 retains peer's contact-info when making pull requests (#16715) (#16907) 
ClusterInfo::new_pull_requests has to lookup contact-infos:
https://github.com/solana-labs/solana/blob/a1ef2bd74/core/src/cluster_info.rs#L1663-L1673

when it was already available when making pull requests:
https://github.com/solana-labs/solana/blob/a1ef2bd74/core/src/crds_gossip_pull.rs#L232

(cherry picked from commit 25054bfd35)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 14:54:54 +00:00
mergify[bot]
d8e8528797 removes delayed crds inserts when upserting gossip table (#16806) (#16905) 
It is crucial that VersionedCrdsValue::insert_timestamp does not go
backward in time:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds.rs#L67-L79

Otherwise methods such as get_votes and get_epoch_slots_since will
break, which will break their downstream flow, including vote-listener
and optimistic confirmation:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1197-L1215
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1274-L1298

For that, Crds::new_versioned is intended to be called "atomically" with
Crds::insert_verioned (as the comment already says so):
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds.rs#L126-L129

However, currently this is violated in the code. For example,
filter_pull_responses creates VersionedCrdsValues (with the current
timestamp), then acquires an exclusive lock on gossip, then
process_pull_responses writes those values to the crds table:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L2375-L2392

Depending on the workload and lock contention, the insert_timestamps may
well be in the past when these values finally are inserted into gossip.

To avoid such scenarios, this commit:
  * removes Crds::new_versioned and Crd::insert_versioned.
  * makes VersionedCrdsValue constructor private, only invoked in
    Crds::insert, so that insert_timestamp is populated right before
    insert.

This will improve insert_timestamp monotonicity as long as Crds::insert
is not called with a stalled timestamp. Following commits may further
improve this by calling timestamp() inside Crds::insert, and/or
switching to std::time::Instant which guarantees monotonicity.

(cherry picked from commit 1ac2a8cfa5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 13:36:21 +00:00
mergify[bot]
ed8c796877 moves cluster-info metrics to a separate module (#16883) (#16898) 
(cherry picked from commit b17d5eeaee)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 04:18:46 +00:00
mergify[bot]
ec750cf3eb Add allowed-ip list to faucet (#16891) (#16897) 
(cherry picked from commit 36574c30ef)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-28 03:32:37 +00:00
mergify[bot]
4a35053fba uses current timestamp when flushing local pending push queue (#16808) (#16896) 
local_message_pending_push_queue is recording timestamps at the time the
value is created, and uses that when the pending values are flushed:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L321
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds_gossip.rs#L96-L102

which is then used as the insert_timestamp when inserting values in the
crds table:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds_gossip_push.rs#L183

The flushing may happen 100ms after the values are created (or even
later if there is a lock contention). This will cause non-monotone
insert_timestamps in the crds table (where time goes backward),
hindering the usability of insert_timestamps for other computations.

For example both ClusterInfo::get_votes and get_epoch_slots_since rely
on monotone insert_timestamps when values are inserted into the table:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1197-L1215
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1274-L1298

This commit removes timestamps from local_message_pending_push_queue and
uses current timestamp when flushing the queue.

(cherry picked from commit b468ead1b1)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 01:59:29 +00:00
mergify[bot]
9797178ad1 Refactor remote-wallet path parsing (backport #16798) (#16894) 
* SDK: More conversions for `Pubkey`

(cherry picked from commit 9b7120bf73)

* SDK: More conversion for `DerivationPath`

(cherry picked from commit 722de942ca)

* remote-wallet: Add helpers for locating remote wallets

(cherry picked from commit 64fcb792c2)

* remote-wallet: Plumb `Locator` into `RemoteWalletInfo`

(cherry picked from commit 3d12be29ec)

* remote-wallet: `derivation-path` crate doesn't like empty trailing child indexes

(cherry picked from commit 4ce4f04c58)

* remote-wallet: Move `Locator` to its own module

(cherry picked from commit cac666d035)

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-04-28 01:20:41 +00:00
mergify[bot]
dbc58455df Retain alloc'd and updated data in cpi (backport #16850) (#16890) 
* Retain alloc'd and updated data in cpi (#16850)

(cherry picked from commit 9b3a59f030)

# Conflicts:
#	programs/bpf_loader/src/syscalls.rs
#	sdk/src/feature_set.rs

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
 2021-04-27 23:01:43 +00:00
mergify[bot]
4a3f851e49 Enable multiple payers in accounts-cluster-bench (#16889) (#16892) 
* Enable multiple payer keypairs

* Suppress tx creation if batch size == 0

* Suppress logs when waiting to create txs

* Double airdrop threshold to prevent stall when closing accounts

(cherry picked from commit 283f587afe)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-27 22:43:06 +00:00
mergify[bot]
5a3bf5c90e limits to data_header.size when combining shreds' payloads (backport #16708) (#16870) 
* limits to data_header.size when combining shreds' payloads (#16708)

Shredder::deshred is ignoring data_header.size when combining shreds' payloads:
https://github.com/solana-labs/solana/blob/37b8587d4/ledger/src/shred.rs#L940-L961

Also adding more sanity checks on the alignment of data shreds indices.

(cherry picked from commit 0f3ac51cf1)

# Conflicts:
#	ledger/src/shred.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-27 14:44:58 +00:00
mergify[bot]
de6ec11efc records hash of values purged by expired pull-responses (#16800) (#16871) 
process_pull_responses should record hash of values purged by expired
responses (as well as unexpired ones):
https://github.com/solana-labs/solana/blob/c1829dd00/core/src/crds_gossip_pull.rs#L385-L387

otherwise, these values are not excluded from following pull-requests
(from likely different nodes):
https://github.com/solana-labs/solana/blob/c1829dd00/core/src/crds_gossip_pull.rs#L447-L452

and would waste bandwidth should they be included in subsequent
pull-responses.

(cherry picked from commit 3b8d6b59fb)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-27 13:26:11 +00:00
mergify[bot]
7aec87c086 Add getVoteAccounts RPC method parameter to restrict results to a single vote account (#16859) 
(cherry picked from commit 59fc33635a)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-27 05:43:44 +00:00
mergify[bot]
eabc21c23a block-production subcommand now uses SlotHistory sysvar when possible (#16858) 
(cherry picked from commit b66a68975b)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-27 05:32:59 +00:00
mergify[bot]
713f346211 Fix limit-ledger-size syntax (#16856) (#16857) 
(cherry picked from commit 3af8cb0150)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-27 04:41:35 +00:00
Michael Vines
a81bc0ecf8 solana leader-schedule -um works again 
(cherry picked from commit c2becbc0a8)
 2021-04-26 17:32:05 -07:00
mergify[bot]
a3f1580b8b Update bpf loader info on native-programs docs (#16840) (#16845) 
* Update bpf loader info on native-programs docs

* Link to program deployment docs

(cherry picked from commit 5eb5d9b2f5)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-26 20:41:36 +00:00
mergify[bot]
4f20798654 removes old runtime feature gates in gossip and turbine (#16633) (#16828) 
(cherry picked from commit 9706512115)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-26 18:40:42 +00:00
mergify[bot]
0ecd1755a6 docs: getInflationReward rpc output fields should be in lower camel case (#16802) (#16805) 
(cherry picked from commit ec37a843a4)

Co-authored-by: Josh <josh.hundley@gmail.com>
 2021-04-24 19:37:55 +00:00
mergify[bot]
57dd8a555a Disable flaky test_poh_service (#16772) (#16797) 
(cherry picked from commit 63436cc2bf)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-24 04:38:27 +00:00
Michael Vines
f64cd4a75a Show last vote/root behind distance in solana validators output 
(cherry picked from commit c1829dd00b)
 2021-04-23 20:12:15 -07:00
mergify[bot]
2ce6c86c2a runtime: checked math for Bank::withdraw (#16788) 
(cherry picked from commit be29568318)

# Conflicts:
#	runtime/src/bank.rs

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-04-24 00:25:41 +00:00
Michael Vines
ff9573714b get_packed_len() now correctly handles u32/i32 types 
(cherry picked from commit 1500011fc6)
 2021-04-23 13:52:10 -07:00
mergify[bot]
826111cf79 Restore text wrapping (#16776) (#16780) 
(cherry picked from commit da58f20a99)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-23 17:34:47 +00:00
mergify[bot]
f31f1d0f52 fix reference to Rust Restrictions section (#16763) (#16775) 
(cherry picked from commit e9a616cfc2)

Co-authored-by: strykerin <dacosta.pereirafabio@gmail.com>
 2021-04-23 17:02:27 +00:00
mergify[bot]
e220f7067b docs: fix formatting issue (#16761) (#16774) 
(cherry picked from commit c217ee3a00)

Co-authored-by: strykerin <dacosta.pereirafabio@gmail.com>
 2021-04-23 17:02:18 +00:00
mergify[bot]
d9726e61bc retains crds values if the origin is still active (#16576) (#16771) 
Local timestamps are updated for records associated with a pubkey if the
origin is still active:
https://github.com/solana-labs/solana/blob/c8ed14c64/core/src/crds.rs#L301-L311

However this is done inconsistently on some gossip paths (pull requests
and pull responses) but not all (e.g. push messages). Additionally
update_record_timestamp is inefficient since there can be ~800 values
associated with each pubkey.

This commit updates records timestamps only on contact-infos; and,
instead utilizes origin's timestamp when purging old values.

(cherry picked from commit 2c82f2154d)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-23 16:42:54 +00:00
mergify[bot]
786fa4f22e removes first_coding_index from erasure recovery code (#16646) (#16770) 
first_coding_index is the same as the set_index and is so redundant:
https://github.com/solana-labs/solana/blob/37b8587d4/ledger/src/blockstore_meta.rs#L49-L60

(cherry picked from commit 03194145c0)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-23 13:21:27 +00:00
mergify[bot]
5b74678e37 Ingest votes from gossip into fork choice (#16560) (#16724) 
(cherry picked from commit 4c94f8933f)

Co-authored-by: carllin <carl@solana.com>
 2021-04-23 07:20:10 +00:00
mergify[bot]
d203bd1998 Add TPU client for sending txs to the current leader tpu port (#16736) (#16762) 
* Add TPU client for sending txs to the current leader tpu port

* Update tpu_client.rs

(cherry picked from commit 75b8434b76)

Co-authored-by: Justin Starry <justin@solana.com>
 2021-04-23 02:50:30 +00:00
mergify[bot]
5f5fa38d85 program-test: Add large bootstrap stake for realistic warmups (backport #16739) (#16741) 
* program-test: Add large bootstrap stake for realistic warmups (#16739)

(cherry picked from commit f4214637a9)

# Conflicts:
#	program-test/Cargo.toml

* Fix merge conflict

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
 2021-04-22 23:07:52 +00:00
mergify[bot]
fadf1efa41 Update getLeaderSchedule options (#16749) (#16752) 
(cherry picked from commit 636b5987af)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-22 21:02:46 +00:00
mergify[bot]
0269fffa5a Remove unactivated ristretto syscall (backport #16727) (#16745) 
* Remove unactivated ristretto syscall (#16727)

(cherry picked from commit be4df39a4c)

# Conflicts:
#	programs/bpf/Cargo.lock
#	programs/bpf/rust/ristretto/Cargo.toml
#	programs/bpf/tests/programs.rs
#	programs/bpf_loader/src/syscalls.rs

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
 2021-04-22 18:33:27 +00:00
mergify[bot]
50e441a9ed Update secp instruction link in docs (#16729) (#16733) 
(cherry picked from commit b22c13dcd7)

Co-authored-by: Jack May <jack@solana.com>
 2021-04-22 04:53:38 +00:00
mergify[bot]
9413051053 Clean up "APR" language around inflation rewards (#16732) 
(cherry picked from commit b8b54567b1)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-22 03:29:52 +00:00
mergify[bot]
13e176a633 getLeaderSchedule now supports filtered results based on validator identity (#16731) 
(cherry picked from commit 6004c0abf5)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-22 02:29:01 +00:00
mergify[bot]
9268239c75 Make metrics tests independent of RUST_LOG env var (#16710) (#16730) 
Previously, running the tests with RUST_LOG=none would fail, because the
env logger would set its filter level to reject all log messages, and
incrementing a counter only happens if the global logger has at least
the specified log level. Having the tests behave differently when
RUST_LOG is set is surprising, they should be self-contained.

Nix' buildRustPackage sets RUST_LOG="" to make the build logs less
verbose. I have trouble packaging Solana for Nix because of this, and I
believe making the tests independent of the environment is a good
solution for this.

(cherry picked from commit 3f92abedd5)

Co-authored-by: Ruud van Asseldonk <dev@veniogames.com>
 2021-04-22 01:41:07 +00:00
mergify[bot]
e51d7af847 verify_pubkey() now takes a ref (#16725) 
(cherry picked from commit 91b6888e15)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-21 23:22:13 +00:00
mergify[bot]
147ba1de69 Update float docs (#16695) (#16726) 
(cherry picked from commit bb2b4c7e0b)

Co-authored-by: Jack May <jack@solana.com>
 2021-04-21 22:55:00 +00:00
mergify[bot]
7cc709c82a CLI: Make pay subcommand a proper alias of transfer (#16721) 
(cherry picked from commit 63957f0677)

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-04-21 22:40:11 +00:00
mergify[bot]
a2395e8730 Add --seed support to delegate-stake and withdraw-stake commands (#16717) 
(cherry picked from commit ba9a502e7e)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-21 21:49:39 +00:00
mergify[bot]
ae605f8f02 expands number of erasure coding shreds in the last batch in slots (backport #16484) (#16707) 
* expands number of erasure coding shreds in the last batch in slots (#16484)

Number of parity coding shreds is always less than the number of data
shreds in FEC blocks:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L719

Data shreds are batched in chunks of 32 shreds each:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L714

However the very last batch of data shreds in a slot can be small, in
which case the loss rate can be exacerbated.

This commit expands the number of coding shreds in the last FEC block in
slots to: 64 - number of data shreds; so that FEC blocks are always 64
data and parity coding shreds each.

As a consequence of this, the last FEC block has more parity coding
shreds than data shreds. So for some shred indices we will have a coding
shred but no data shreds. This should not cause any kind of overlapping
FEC blocks as in:
https://github.com/solana-labs/solana/pull/10095
since this is done only for the very last batch in a slot, and the next
slot will reset the shred index.

(cherry picked from commit 37b8587d4e)

# Conflicts:
#	core/benches/shredder.rs
#	ledger/src/shred.rs

* removes backport merge conflicts

* ignore the flaky test for now

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-21 15:25:26 +00:00
mergify[bot]
ea2cc90215 Improve net scripts (backport #16699) (#16700) 
* Pass limit-ledger-size value

(cherry picked from commit 51b748408c)

* Initialize non-bootstrap ndoes with faucet address

(cherry picked from commit 053120e04c)

Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-04-21 08:45:11 +00:00
mergify[bot]
e15ddbb979 Add port and gossip options to solana-test-validator (#16696) (#16698) 
(cherry picked from commit 0924c2d070)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-21 03:46:52 +00:00
mergify[bot]
bbd8bd2e74 Enforce host aligned memory for program regions (backport #16590) (#16683) 
* Enforce host aligned memory for program regions (#16590)

(cherry picked from commit 08d5253651)

# Conflicts:
#	cli/Cargo.toml
#	programs/bpf/Cargo.toml
#	programs/bpf/benches/bpf_loader.rs
#	programs/bpf/tests/programs.rs
#	programs/bpf_loader/Cargo.toml
#	programs/bpf_loader/src/lib.rs

* fix conflicts

Co-authored-by: Jack May <jack@solana.com>
 2021-04-21 01:47:00 +00:00
mergify[bot]
a5794efe16 getVoteAccounts: Limit the length of the epoch_credits array (#16692) 
(cherry picked from commit 34addee882)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-20 22:55:17 +00:00
mergify[bot]
27095378fa Remove unwrap from bpf_loader serialization (#16645) (#16649) 
(cherry picked from commit 2409bb18f3)

Co-authored-by: Jack May <jack@solana.com>
 2021-04-20 16:35:45 +00:00
mergify[bot]
a8836649cb uses current local timestamp when recording purged values (#16675) 
CrdsGossipPull.purged_values is meant to record recently purged values
so that they are excluded from imminent pull requests, until the entire
cluster have synced to the updated value:
https://github.com/solana-labs/solana/blob/c826cddbb/core/src/crds_gossip_pull.rs#L449-L454

However, VersionedCrdsValue.local_timestamp represents the local time
when the value was last updated, and given that crds values may have
different timeouts based on stake, it does not necessarily represent how
recently the value was purged:
https://github.com/solana-labs/solana/blob/c826cddbb/core/src/crds.rs#L75-L76

As such, recording current local timestamp when purging values is more
appropriate. Additionally, purge_purged assumes that the purge_values is
sorted in timestamps when draining the old ones; which is not true if
those timestamps are VersionedCrdsValue.local_timestamp:
https://github.com/solana-labs/solana/blob/c826cddbb/core/src/crds_gossip_pull.rs#L563-L571

(cherry picked from commit bc90e04e64)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-20 12:42:30 +00:00
mergify[bot]
cc81830f13 CLI: Limit stake-history output by default (#16673) 
(cherry picked from commit f91de6a84d)

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-04-20 11:36:15 +00:00
mergify[bot]
558a46f5d5 RPC: use finalized as default pubsub commitment level (#16659) (#16666) 
* RPC: use finalized as default pubsub commitment level

* update docs

* Fix tests

(cherry picked from commit a7e65c0034)

Co-authored-by: Justin Starry <justin@solana.com>
 2021-04-20 09:47:50 +00:00
mergify[bot]
57add5366e Expand a couple docs sections (backport #16664) (#16671) 
* docs: Flesh out address verification in integraion guide

(cherry picked from commit d575450ef0)

* docs: Expand native program descriptions

(cherry picked from commit 12678a819d)

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-04-20 09:33:40 +00:00
mergify[bot]
5057aaddc0 Send votes to next leader's TPU instead of our TPU (#16663) 
(cherry picked from commit c8b474cd0b)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-20 08:45:58 +00:00