gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
13,632 Commits 81 Branches 357 Tags
32fc4e3d0fea14971f3df483337f1ccbb437fc46
Commit Graph

2265 Commits

Author SHA1 Message Date
mergify[bot]
32fc4e3d0f Add keys (backport #17014) (#17015) 
* Rotate keys

(cherry picked from commit b2778f34f5)

* Key rotation

(cherry picked from commit b948a18841)

* Add keys

(cherry picked from commit 6318705607)

Co-authored-by: publish-docs.sh <maintainers@solana.com>
 2021-05-04 01:34:53 +00:00
mergify[bot]
834c96a374 validates gossip addresses before sending pull-requests (backport #16748) (#17009) 
* uses Mutex instead of RwLock for ping_cache

(cherry picked from commit 2231017b35)

* validates gossip addresses before sending pull-requests

IP addresses need to be validated before sending packets to them.
This commit, sends a ping packet to nodes before any pull requests.
Pull requests are then only sent to the nodes which have responded with
the correct hash of their respective ping packet.

(cherry picked from commit 7cea2c4466)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-05-03 19:40:02 +00:00
mergify[bot]
2195d980a2 patches local pending push messages processing (#16833) (#17007) 
process_push_messages writes local pending push messages to the crds
table, but it discards the return value:
https://github.com/solana-labs/solana/blob/cf779c63c/core/src/crds_gossip.rs#L96-L102

In order to exclude outdated values from the next pull-request, we need
to record the hash of values purged/overridden by the local push
messages, otherwise pull-responses will return outdated values back to
the node:
https://github.com/solana-labs/solana/blob/c1829dd00/core/src/crds_gossip_pull.rs#L447-L452

Additionally, gossip packets arrive and are processed out of order. So,
local pending push messages should be flushed *before* generating bloom
filters for pull-requests, preventing pull-responses returning the same
values back to the node itself. This requires flipping order of
generating pull and push messages:
https://github.com/solana-labs/solana/blob/cf779c63c/core/src/cluster_info.rs#L1757-L1762

Both above bugs cause redundant traffic and bandwidth waste in gossip
pull-responses.

(cherry picked from commit a698e34744)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-05-03 17:27:38 +00:00
mergify[bot]
c6c7feb0c2 Retry latest vote if expired (#16735) (#16927) 
(cherry picked from commit b5d30846d6)

Co-authored-by: carllin <carl@solana.com>
 2021-05-03 05:13:29 +00:00
mergify[bot]
cc7fc447a4 Distinguish max replayed and max observed vote (#16936) (#16956) 
(cherry picked from commit 5981399612)

Co-authored-by: carllin <carl@solana.com>
 2021-04-30 00:48:56 +00:00
Michael Vines
49a415414f Add getBlockProduction RPC method 2021-04-28 21:38:53 -07:00
mergify[bot]
25aee12502 retains peer's contact-info when making pull requests (#16715) (#16907) 
ClusterInfo::new_pull_requests has to lookup contact-infos:
https://github.com/solana-labs/solana/blob/a1ef2bd74/core/src/cluster_info.rs#L1663-L1673

when it was already available when making pull requests:
https://github.com/solana-labs/solana/blob/a1ef2bd74/core/src/crds_gossip_pull.rs#L232

(cherry picked from commit 25054bfd35)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 14:54:54 +00:00
mergify[bot]
d8e8528797 removes delayed crds inserts when upserting gossip table (#16806) (#16905) 
It is crucial that VersionedCrdsValue::insert_timestamp does not go
backward in time:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds.rs#L67-L79

Otherwise methods such as get_votes and get_epoch_slots_since will
break, which will break their downstream flow, including vote-listener
and optimistic confirmation:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1197-L1215
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1274-L1298

For that, Crds::new_versioned is intended to be called "atomically" with
Crds::insert_verioned (as the comment already says so):
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds.rs#L126-L129

However, currently this is violated in the code. For example,
filter_pull_responses creates VersionedCrdsValues (with the current
timestamp), then acquires an exclusive lock on gossip, then
process_pull_responses writes those values to the crds table:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L2375-L2392

Depending on the workload and lock contention, the insert_timestamps may
well be in the past when these values finally are inserted into gossip.

To avoid such scenarios, this commit:
  * removes Crds::new_versioned and Crd::insert_versioned.
  * makes VersionedCrdsValue constructor private, only invoked in
    Crds::insert, so that insert_timestamp is populated right before
    insert.

This will improve insert_timestamp monotonicity as long as Crds::insert
is not called with a stalled timestamp. Following commits may further
improve this by calling timestamp() inside Crds::insert, and/or
switching to std::time::Instant which guarantees monotonicity.

(cherry picked from commit 1ac2a8cfa5)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 13:36:21 +00:00
mergify[bot]
ed8c796877 moves cluster-info metrics to a separate module (#16883) (#16898) 
(cherry picked from commit b17d5eeaee)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 04:18:46 +00:00
mergify[bot]
4a35053fba uses current timestamp when flushing local pending push queue (#16808) (#16896) 
local_message_pending_push_queue is recording timestamps at the time the
value is created, and uses that when the pending values are flushed:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L321
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds_gossip.rs#L96-L102

which is then used as the insert_timestamp when inserting values in the
crds table:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/crds_gossip_push.rs#L183

The flushing may happen 100ms after the values are created (or even
later if there is a lock contention). This will cause non-monotone
insert_timestamps in the crds table (where time goes backward),
hindering the usability of insert_timestamps for other computations.

For example both ClusterInfo::get_votes and get_epoch_slots_since rely
on monotone insert_timestamps when values are inserted into the table:
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1197-L1215
https://github.com/solana-labs/solana/blob/ec37a843a/core/src/cluster_info.rs#L1274-L1298

This commit removes timestamps from local_message_pending_push_queue and
uses current timestamp when flushing the queue.

(cherry picked from commit b468ead1b1)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-28 01:59:29 +00:00
mergify[bot]
5a3bf5c90e limits to data_header.size when combining shreds' payloads (backport #16708) (#16870) 
* limits to data_header.size when combining shreds' payloads (#16708)

Shredder::deshred is ignoring data_header.size when combining shreds' payloads:
https://github.com/solana-labs/solana/blob/37b8587d4/ledger/src/shred.rs#L940-L961

Also adding more sanity checks on the alignment of data shreds indices.

(cherry picked from commit 0f3ac51cf1)

# Conflicts:
#	ledger/src/shred.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-27 14:44:58 +00:00
mergify[bot]
de6ec11efc records hash of values purged by expired pull-responses (#16800) (#16871) 
process_pull_responses should record hash of values purged by expired
responses (as well as unexpired ones):
https://github.com/solana-labs/solana/blob/c1829dd00/core/src/crds_gossip_pull.rs#L385-L387

otherwise, these values are not excluded from following pull-requests
(from likely different nodes):
https://github.com/solana-labs/solana/blob/c1829dd00/core/src/crds_gossip_pull.rs#L447-L452

and would waste bandwidth should they be included in subsequent
pull-responses.

(cherry picked from commit 3b8d6b59fb)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-27 13:26:11 +00:00
mergify[bot]
7aec87c086 Add getVoteAccounts RPC method parameter to restrict results to a single vote account (#16859) 
(cherry picked from commit 59fc33635a)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-27 05:43:44 +00:00
mergify[bot]
4f20798654 removes old runtime feature gates in gossip and turbine (#16633) (#16828) 
(cherry picked from commit 9706512115)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-26 18:40:42 +00:00
mergify[bot]
57dd8a555a Disable flaky test_poh_service (#16772) (#16797) 
(cherry picked from commit 63436cc2bf)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-24 04:38:27 +00:00
mergify[bot]
d9726e61bc retains crds values if the origin is still active (#16576) (#16771) 
Local timestamps are updated for records associated with a pubkey if the
origin is still active:
https://github.com/solana-labs/solana/blob/c8ed14c64/core/src/crds.rs#L301-L311

However this is done inconsistently on some gossip paths (pull requests
and pull responses) but not all (e.g. push messages). Additionally
update_record_timestamp is inefficient since there can be ~800 values
associated with each pubkey.

This commit updates records timestamps only on contact-infos; and,
instead utilizes origin's timestamp when purging old values.

(cherry picked from commit 2c82f2154d)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-23 16:42:54 +00:00
mergify[bot]
786fa4f22e removes first_coding_index from erasure recovery code (#16646) (#16770) 
first_coding_index is the same as the set_index and is so redundant:
https://github.com/solana-labs/solana/blob/37b8587d4/ledger/src/blockstore_meta.rs#L49-L60

(cherry picked from commit 03194145c0)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-23 13:21:27 +00:00
mergify[bot]
5b74678e37 Ingest votes from gossip into fork choice (#16560) (#16724) 
(cherry picked from commit 4c94f8933f)

Co-authored-by: carllin <carl@solana.com>
 2021-04-23 07:20:10 +00:00
mergify[bot]
d203bd1998 Add TPU client for sending txs to the current leader tpu port (#16736) (#16762) 
* Add TPU client for sending txs to the current leader tpu port

* Update tpu_client.rs

(cherry picked from commit 75b8434b76)

Co-authored-by: Justin Starry <justin@solana.com>
 2021-04-23 02:50:30 +00:00
mergify[bot]
fadf1efa41 Update getLeaderSchedule options (#16749) (#16752) 
(cherry picked from commit 636b5987af)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-22 21:02:46 +00:00
mergify[bot]
13e176a633 getLeaderSchedule now supports filtered results based on validator identity (#16731) 
(cherry picked from commit 6004c0abf5)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-22 02:29:01 +00:00
mergify[bot]
e51d7af847 verify_pubkey() now takes a ref (#16725) 
(cherry picked from commit 91b6888e15)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-21 23:22:13 +00:00
mergify[bot]
ae605f8f02 expands number of erasure coding shreds in the last batch in slots (backport #16484) (#16707) 
* expands number of erasure coding shreds in the last batch in slots (#16484)

Number of parity coding shreds is always less than the number of data
shreds in FEC blocks:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L719

Data shreds are batched in chunks of 32 shreds each:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L714

However the very last batch of data shreds in a slot can be small, in
which case the loss rate can be exacerbated.

This commit expands the number of coding shreds in the last FEC block in
slots to: 64 - number of data shreds; so that FEC blocks are always 64
data and parity coding shreds each.

As a consequence of this, the last FEC block has more parity coding
shreds than data shreds. So for some shred indices we will have a coding
shred but no data shreds. This should not cause any kind of overlapping
FEC blocks as in:
https://github.com/solana-labs/solana/pull/10095
since this is done only for the very last batch in a slot, and the next
slot will reset the shred index.

(cherry picked from commit 37b8587d4e)

# Conflicts:
#	core/benches/shredder.rs
#	ledger/src/shred.rs

* removes backport merge conflicts

* ignore the flaky test for now

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-21 15:25:26 +00:00
mergify[bot]
e15ddbb979 Add port and gossip options to solana-test-validator (#16696) (#16698) 
(cherry picked from commit 0924c2d070)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-21 03:46:52 +00:00
mergify[bot]
a5794efe16 getVoteAccounts: Limit the length of the epoch_credits array (#16692) 
(cherry picked from commit 34addee882)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-20 22:55:17 +00:00
mergify[bot]
a8836649cb uses current local timestamp when recording purged values (#16675) 
CrdsGossipPull.purged_values is meant to record recently purged values
so that they are excluded from imminent pull requests, until the entire
cluster have synced to the updated value:
https://github.com/solana-labs/solana/blob/c826cddbb/core/src/crds_gossip_pull.rs#L449-L454

However, VersionedCrdsValue.local_timestamp represents the local time
when the value was last updated, and given that crds values may have
different timeouts based on stake, it does not necessarily represent how
recently the value was purged:
https://github.com/solana-labs/solana/blob/c826cddbb/core/src/crds.rs#L75-L76

As such, recording current local timestamp when purging values is more
appropriate. Additionally, purge_purged assumes that the purge_values is
sorted in timestamps when draining the old ones; which is not true if
those timestamps are VersionedCrdsValue.local_timestamp:
https://github.com/solana-labs/solana/blob/c826cddbb/core/src/crds_gossip_pull.rs#L563-L571

(cherry picked from commit bc90e04e64)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-20 12:42:30 +00:00
mergify[bot]
558a46f5d5 RPC: use finalized as default pubsub commitment level (#16659) (#16666) 
* RPC: use finalized as default pubsub commitment level

* update docs

* Fix tests

(cherry picked from commit a7e65c0034)

Co-authored-by: Justin Starry <justin@solana.com>
 2021-04-20 09:47:50 +00:00
mergify[bot]
5057aaddc0 Send votes to next leader's TPU instead of our TPU (#16663) 
(cherry picked from commit c8b474cd0b)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-20 08:45:58 +00:00
Michael Vines
a1b0f2f681 Increase test timeout 2021-04-19 04:12:16 +00:00
Michael Vines
f59d4f29d9 clippy 2021-04-19 04:12:16 +00:00
mergify[bot]
25491780df Remove unnecessary clone (#16621) 
(cherry picked from commit 6907a2366e)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-17 18:31:18 +00:00
Trent Nelson
4e94446fc3 Bump version to v1.6.7 2021-04-16 23:31:30 +00:00
mergify[bot]
b3488e0139 Cli: move airdrop to rpc requests (#16557) (#16564) 
* Add recent_blockhash to requestAirdrop

* Move tx confirmation to separate method

* Add RpcClient airdrop methods

* Request cli airdrop via RpcClient

* Pass optional faucet_addr into TestValidator and fix tests

* Update client/src/rpc_client.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
(cherry picked from commit 7dfb51c0b4)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-15 07:35:19 +00:00
mergify[bot]
719db7eed0 uses timeouts based on stake for filtering pull responses (#16549) (#16551) 
filter_pull_responses is using default timeout when discarding pull
responses (except for ContactInfo):
https://github.com/solana-labs/solana/blob/f804ce63c/core/src/crds_gossip_pull.rs#L349-L350

But purging code uses timeouts based on stake:
https://github.com/solana-labs/solana/blob/f804ce63c/core/src/cluster_info.rs#L1867-L1870

So the crds value will not be purged from the sender's table and will be
sent again over the next pull request.

(cherry picked from commit d92721aab9)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-14 21:43:48 +00:00
mergify[bot]
4ddb72a32d prioritizes contact-infos in pull responses (#16541) (#16550) 
Expired crds values where the contact-info does not exist are wasted:
https://github.com/solana-labs/solana/blob/f804ce63c/core/src/crds_gossip_pull.rs#L353-L378
and then are sent again over the next pull-request.

Also, the stake of the first response (which can be anything) is used to
weight all pull-responses to a node, while the rest of responses can
have different stake.
https://github.com/solana-labs/solana/blob/f804ce63c/core/src/cluster_info.rs#L2231

(cherry picked from commit f35a6a8be0)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-14 20:14:22 +00:00
mergify[bot]
ff1171338f Fix channel panic in tests (#16503) (#16543) 
* Fix channel panic

* Add exit signal to PohRecorder because Crossbeam doesnt drop objects inside dropped channel

(cherry picked from commit f0c150cfb9)

Co-authored-by: carllin <carl@solana.com>
 2021-04-14 19:04:31 +00:00
Michael Vines
4ef3a679a4 Bump version to v1.6.6 2021-04-14 10:27:02 -07:00
mergify[bot]
3bb8016a40 Remove blake3 from bpf program dependencies (#16506) (#16509) 
(cherry picked from commit f641429056)

Co-authored-by: Justin Starry <justin@solana.com>
 2021-04-13 11:18:26 +00:00
Justin Starry
579065443a v1.6: Use blake3 message hash in status cache (#16507) 2021-04-13 16:57:20 +08:00
mergify[bot]
8ee294639a validator: Add authorized-voter add/remove-all commands (bp #16492) (#16496) 
* Clean up build warning

(cherry picked from commit 17a173ebb5)

* Add authorized-voter add/remove-all commands

(cherry picked from commit 2229b70c4e)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-04-13 00:07:06 +00:00
mergify[bot]
37c2b68677 poll checking for new record in poh service after every batch of hashes instead of busy waiting (#16167) (#16486) 
* poll waiting in poh service after every batch of hashes

* clippy

(cherry picked from commit 414c7070cb)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-04-12 19:07:23 +00:00
mergify[bot]
d9944c8ae3 TransactionRecorder uses unique channel so we can use Recv instead of RecvTimeout (#16195) (#16485) 
* time

* new channel each call

* new channel every time

(cherry picked from commit 5eff23db0c)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-04-12 18:56:22 +00:00
mergify[bot]
6c8bbdca0a Allow fork choice to support multiple versions of a slot (#16266) (#16480) 
(cherry picked from commit dc7030ffaa)

Co-authored-by: carllin <carl@solana.com>
 2021-04-12 09:14:02 +00:00
mergify[bot]
60fba7be75 Track gossip vote updates per hash for replay stage (#16421) (#16468) 
* Track gossip vote updates per hash for replay stage

(cherry picked from commit 99b3aab703)

Co-authored-by: carllin <carl@solana.com>
 2021-04-11 09:33:28 +00:00
mergify[bot]
24075ceeff Fill in not-yet-finalized block-time if possible (#16460) (#16463) 
(cherry picked from commit 8bc0bdd40b)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-09 21:48:16 +00:00
mergify[bot]
f7ef1e68b0 patches bug in banking stage where buffered packets are never retained (#16276) (#16458) 
banking_stage::handle_forwarding is retaining buffered packets with
empty index, so nothing is held:
https://github.com/solana-labs/solana/blob/6f3926b64/core/src/banking_stage.rs#L520

(cherry picked from commit 701fc93343)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-04-09 18:44:32 +00:00
mergify[bot]
723e7f11b9 Simplify some pattern-matches (#16402) (#16446) 
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).

(cherry picked from commit b08cff9e77)

Co-authored-by: François Garillot <4142+huitseeker@users.noreply.github.com>
 2021-04-08 20:45:01 +00:00
mergify[bot]
f7211d3c07 Cli: use get_inflation_rewards and limit epochs queried (#16408) (#16444) 
* Fix block-with-limit when not finalized blocks found

* Enable confirmed commitment in getInflationReward

* Use get_inflation_rewards in cli

* Line up rewards output

* Add range validator

* Change cli epoch arg -> num epochs

* Add solana inflation rewards subcommand

* Consolidate epoch rewards meta

(cherry picked from commit bb9d2fd07a)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-04-08 18:16:04 +00:00
mergify[bot]
eadab5e2f0 No wallclock throttle tests (#16396) (#16399) 
(cherry picked from commit 1219842a96)

Co-authored-by: carllin <carl@solana.com>
 2021-04-07 11:05:51 +00:00
mergify[bot]
a2857928a4 Rpc: introduce get_inflation_reward rpc call (#16278) (#16410) 
* feat: introduce get_inflation_reward rpc call

* fix: style suggestions

* fix: more style changes and match how other rpc functions are defined

* feat: get reward for a single epoch

* feat: default to the most recent epoch

* fix: don't factor out get_confirmed_block

* style: introduce from impl for RpcEncodingConfigWrapper

* style: bring commitment into variable

* feat: support multiple pubkeys for get_inflation_reward

* feat: add get_inflation_reward to rpc client

* feat: return rewards in order

* fix: rename pubkeys to addresses

* docs: introduce jsonrpc docs for get_inflation_reward

* style: early return in map (not sure which is more idiomatic)

* fix: call the rpc client function args addresses as well

* fix: style

* fix: filter out only addresses we care about

* style: make this more idiomatic

* fix: change rpc client epoch to optional and include some docs edits

* feat: filter out rent rewards in get_inflation_reward

* feat: add option epoch config param to get_inflation_reward

* feat: rpc client get_inflation_reward takes epoch instead of config and some filter staking and voting rewards

(cherry picked from commit e501fa5f0b)

Co-authored-by: Josh <josh.hundley@gmail.com>
 2021-04-07 02:26:45 +00:00