2190 Commits

Author SHA1 Message Date
mergify[bot]
bd8e3182ae requires stakes for propagating crds values through gossip (backport #15561) (#16773)
* requires stakes for propagating crds values through gossip (#15561)

(cherry picked from commit f2865dfd63)

# Conflicts:
#	core/src/cluster_info.rs
#	sdk/src/feature_set.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-23 17:55:15 +00:00
Michael Vines
7f2977a3b0 Bump version to 1.5.20 2021-04-20 17:04:26 +00:00
mergify[bot]
936ff7424e makes turbine peer computation consistent between broadcast and retransmit (#14910) (#16653)
get_broadcast_peers is using tvu_peers:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/broadcast_stage.rs#L362-L370
which is potentially inconsistent with retransmit_peers:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/cluster_info.rs#L1332-L1345

Also, the leader does not include its own contact-info when broadcasting
shreds:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/cluster_info.rs#L1324
but on the retransmit side, slot leader is removed only _after_ neighbors and
children are computed:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/retransmit_stage.rs#L383-L384
So the turbine broadcast tree is different between the two stages.

This commit:
* Removes retransmit_peers. Broadcast and retransmit stages will use tvu_peers
  consistently.
* Retransmit stage removes slot leader _before_ computing children and
  neighbors.

(cherry picked from commit 570fd3f810)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-20 12:44:50 +00:00
mergify[bot]
23e114e077 buffers data shreds to make larger erasure coded sets (#15849) (#16674)
Broadcast stage batches up to 8 entries:
https://github.com/solana-labs/solana/blob/79280b304/core/src/broadcast_stage/broadcast_utils.rs#L26-L29
which will be serialized into some number of shreds and chunked into FEC
sets of at most 32 shreds each:
https://github.com/solana-labs/solana/blob/79280b304/ledger/src/shred.rs#L576-L597
So depending on the size of entries, FEC sets can be small, which may
aggravate loss rate.
For example 16 FEC sets of 2:2 data/code shreds each have higher loss
rate than one 32:32 set.

This commit broadcasts data shreds immediately, but also buffers them
until it has a batch of 32 data shreds, at which point 32 coding shreds
are generated and broadcasted.

(cherry picked from commit 4f82b897bc)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-20 12:40:04 +00:00
mergify[bot]
6268fa32c6 RPC: use finalized as default pubsub commitment level (backport #16659) (#16665)
* RPC: use finalized as default pubsub commitment level (#16659)

* RPC: use finalized as default pubsub commitment level

* update docs

* Fix tests

(cherry picked from commit a7e65c0034)

# Conflicts:
#	core/tests/rpc.rs

* fix conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-04-20 10:07:16 +00:00
Kristofer Peterson
7df2525972 Refactored ShortU16Visitor::visit_seq() to reject overflows, extra leading zeros and ensure one-to-one encoding. 2021-04-20 03:01:52 -06:00
mergify[bot]
0c2faa3168 Cli: move airdrop to rpc requests (#16557) (#16563)
* Add recent_blockhash to requestAirdrop

* Move tx confirmation to separate method

* Add RpcClient airdrop methods

* Request cli airdrop via RpcClient

* Pass optional faucet_addr into TestValidator and fix tests

* Update client/src/rpc_client.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-04-15 18:12:07 +00:00
mergify[bot]
ef3c33ad06 Remove blake3 from bpf program dependencies (#16506) (#16528)
(cherry picked from commit f641429056)

Co-authored-by: Justin Starry <justin@solana.com>
2021-04-14 03:22:12 +00:00
Justin Starry
02762ff785 v1.5: Use blake3 message hash in status cache (#16513)
* v1.5: Use blake3 message hash in status cache

* bump bpf ix count
2021-04-14 09:52:24 +08:00
mergify[bot]
a6b346d876 removes OrderedIterator and transaction batch iteration order (backport #16153) (#16510)
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-13 12:47:44 +00:00
mergify[bot]
4276591eb9 limits number of unique pubkeys in the crds table (bp #15539) (#16475)
* limits number of unique pubkeys in the crds table (#15539)

(cherry picked from commit 56923c91bf)

# Conflicts:
#	core/src/cluster_info.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-12 00:36:41 +00:00
mergify[bot]
127e7407e4 Fill in not-yet-finalized block-time if possible (#16460) (#16462)
(cherry picked from commit 8bc0bdd40b)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-04-09 21:36:44 +00:00
mergify[bot]
11ab894256 [easy, cleanup] Simplify some pattern-matches (bp #16402) (#16445)
* Simplify some pattern-matches (#16402)

When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).

(cherry picked from commit b08cff9e77)

# Conflicts:
#	accounts-cluster-bench/src/main.rs
#	core/src/rpc.rs
#	runtime/src/accounts_hash.rs
#	runtime/src/message_processor.rs

* Fix conflicts

Co-authored-by: François Garillot <4142+huitseeker@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-04-08 20:01:20 +00:00
mergify[bot]
f10f57e89f Cli: use get_inflation_rewards and limit epochs queried (#16408) (#16443)
* Fix block-with-limit when not finalized blocks found

* Enable confirmed commitment in getInflationReward

* Use get_inflation_rewards in cli

* Line up rewards output

* Add range validator

* Change cli epoch arg -> num epochs

* Add solana inflation rewards subcommand

* Consolidate epoch rewards meta

(cherry picked from commit bb9d2fd07a)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-04-08 19:05:59 +00:00
mergify[bot]
c7629d595b Remove wallclock throttle from BankingStage tests (bp #16396) (#16398)
* No wallclock throttle tests (#16396)

(cherry picked from commit 1219842a96)

# Conflicts:
#	core/src/banking_stage.rs

* Resolve conflicts

Co-authored-by: carllin <carl@solana.com>
2021-04-07 16:05:26 +00:00
mergify[bot]
492a02d737 Rpc: introduce get_inflation_reward rpc call (bp #16278) (#16409)
* Rpc: introduce get_inflation_reward rpc call (#16278)

* feat: introduce get_inflation_reward rpc call

* fix: style suggestions

* fix: more style changes and match how other rpc functions are defined

* feat: get reward for a single epoch

* feat: default to the most recent epoch

* fix: don't factor out get_confirmed_block

* style: introduce from impl for RpcEncodingConfigWrapper

* style: bring commitment into variable

* feat: support multiple pubkeys for get_inflation_reward

* feat: add get_inflation_reward to rpc client

* feat: return rewards in order

* fix: rename pubkeys to addresses

* docs: introduce jsonrpc docs for get_inflation_reward

* style: early return in map (not sure which is more idiomatic)

* fix: call the rpc client function args addresses as well

* fix: style

* fix: filter out only addresses we care about

* style: make this more idiomatic

* fix: change rpc client epoch to optional and include some docs edits

* feat: filter out rent rewards in get_inflation_reward

* feat: add option epoch config param to get_inflation_reward

* feat: rpc client get_inflation_reward takes epoch instead of config and some filter staking and voting rewards

(cherry picked from commit e501fa5f0b)

# Conflicts:
#	client/src/rpc_client.rs
#	core/src/rpc.rs

* fix: resolve cherry-pick conflicts

* fix: change bool_to_option filter_map to filter + map

* style: use filter_map in place of filter + map

Co-authored-by: Josh <josh.hundley@gmail.com>
2021-04-07 06:09:27 +00:00
mergify[bot]
624f9790bd validator: Use a const for wait for supermajority threshold (#16391)
(cherry picked from commit 7a2a39093d)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-06 05:02:25 +00:00
mergify[bot]
a23fb497ec Cluster info shred spies (bp #16389) (#16394)
* cluster-info: Don't subtract non-shred spies from node count

(cherry picked from commit b6b08706b9)

* cluster-info: Get rid of some integer math while we're here

(cherry picked from commit b71875df61)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-06 01:25:16 +00:00
mergify[bot]
b4bb062a2e Wait for 90 percent of stake before starting (#16340) (#16343)
(cherry picked from commit 3429785d9b)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-04-03 22:34:37 +00:00
mergify[bot]
194a07862f makes test_pull_request_time_pruning smaller (#16128) (#16338)
(cherry picked from commit b041b55028)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-03 14:53:59 +00:00
mergify[bot]
e043825de2 limits CrdsGossipPull::pull_request_time size (bp #15793) (#16334)
* limits CrdsGossipPull::pull_request_time size (#15793)

There is no pruning logic on CrdsGossipPull::pull_request_time
https://github.com/solana-labs/solana/blob/79ac1997d/core/src/crds_gossip_pull.rs#L172-L174
potentially allowing this to take too much memory.

Additionally, CrdsGossipPush::last_pushed_to is pruning recent push
timestamps:
https://github.com/solana-labs/solana/blob/79ac1997d/core/src/crds_gossip_push.rs#L275-L279
instead of the older ones.

Co-authored-by: Nathan Hawkins <utsl@utsl.org>
(cherry picked from commit a6c23648cb)

# Conflicts:
#	core/src/cluster_info.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-03 13:29:45 +00:00
Trent Nelson
c32bd40aa4 Bump version to v1.5.19 2021-04-01 20:23:50 +00:00
mergify[bot]
7e480df9fa Rpc: enable getConfirmedSignaturesForAddress2 to return confirmed (not yet finalized) data (#16281) (#16292)
* Update blockstore method to allow return of unfinalized signature

* Support confirmed sigs in getConfirmedSignaturesForAddress2

* Add deprecated comments

* Update docs

* Enable confirmed transaction-history in cli

* Return real confirmation_status; fill in not-yet-finalized block time if possible

(cherry picked from commit da27acabcc)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-04-01 11:29:51 -06:00
mergify[bot]
708a45e5ed indexes epoch slots in crds table (#15459) (#16287)
ClusterInfo::get_epoch_slots_since scans the entire crds table to obtain
epoch-slots inserted since a timestamp:
https://github.com/solana-labs/solana/blob/013daa8f4/core/src/cluster_info.rs#L1245-L1262
The alternative is to index epoch-slots in crds table ordered by their
insert timestamp.

(cherry picked from commit 5a9896706c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-04-01 14:21:00 +00:00
mergify[bot]
afd5b9f09b Rpc: fix getConfirmedTransaction slot (#16288) (#16289)
* Fix transaction blockstore apis

* Update blockstore apis in rpc

(cherry picked from commit 18bd47dbe1)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-04-01 08:47:03 +00:00
Tyera Eulberg
ac4722afd7 Bump version to 1.5.18 2021-03-29 23:00:20 -07:00
mergify[bot]
09843a28d9 Rpc: enable getConfirmedBlocks and getConfirmedBlocksWithLimit to return confirmed (not yet finalized) data (bp #16161) (#16197)
* Rpc: enable getConfirmedBlocks and getConfirmedBlocksWithLimit to return confirmed (not yet finalized) data (#16161)

* Add commitment config capabilities

* Use rpc limit if no end_slot provided

* Limit to actually finalized blocks

* Support confirmed blocks in getConfirmedBlocks and getConfirmedBlocksWithLimit

* Update docs

* Add client plumbing

* Rename config enum

(cherry picked from commit 60ed8e2892)

# Conflicts:
#	client/src/rpc_config.rs
#	core/src/rpc.rs

* Fix conflicts

* Future-aware enum name

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-03-29 22:11:46 +00:00
mergify[bot]
fb08b41513 Rpc: enable getConfirmedBlock and getConfirmedTransaction to return confirmed (not yet finalized) data (bp #16142) (#16159)
* Rpc: enable getConfirmedBlock and getConfirmedTransaction to return confirmed (not yet finalized) data (#16142)

* Add Blockstore block and tx apis that allow unrooted responses

* Add TransactionStatusMessage, and send on bank freeze; also refactor TransactionStatusSender

* Track highest slot with tx-status writes complete

* Rename and unpub fn

* Add commitment to GetConfirmed input configs

* Support confirmed blocks in getConfirmedBlock

* Support confirmed txs in getConfirmedTransaction

* Update sigs-for-addr2 comment

* Enable confirmed block in cli

* Enable confirmed transaction in cli

* Review comments

* Rename blockstore method

(cherry picked from commit 433f1ead1c)

# Conflicts:
#	core/src/replay_stage.rs
#	core/src/rpc.rs
#	core/src/validator.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-03-27 05:45:00 +00:00
mergify[bot]
579575fe84 Add timeout for local cluster partition tests (bp #16123) (#16136)
* Add timeout for local cluster partition tests (#16123)

* Add timeout for local cluster partition tests

* fix optimistic conf test logs

* Bump instruction count assertions

(cherry picked from commit e817a6db00)

# Conflicts:
#	local-cluster/Cargo.toml
#	local-cluster/tests/local_cluster.rs
#	programs/bpf/tests/programs.rs

* Fix conflicts

* Revert instruction count assertion changes

Co-authored-by: Justin Starry <justin@solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-03-26 00:15:08 +00:00
mergify[bot]
fc36a0c5ba Support getBlockTime for unfinalized blocks (#16103) (#16109)
(cherry picked from commit a8ef29df27)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-03-25 06:05:41 +00:00
mergify[bot]
3317a14bd5 rpc: add getSlotLeaders method (#16057) (#16078)
(cherry picked from commit e7fd7d46cf)

Co-authored-by: Justin Starry <justin@solana.com>
2021-03-24 03:20:57 +00:00
Tyera Eulberg
f356a05e96 Bump version to 1.5.17 (#16043) 2021-03-19 18:56:05 -06:00
Tyera Eulberg
c3c4ce48af Make getStakeActivation response consistent for undelegated accounts (#16039) 2021-03-19 15:51:40 -06:00
mergify[bot]
a5f8635fdc rpc: Add config options limiting getConfirmedBlock response data (bp #15970) (#15994)
* rpc: Add config options limiting getConfirmedBlock response data (#15970)

* Add new confirmed block struct

* Add RpcConfirmedBlockConfig options

* Configure block response based on new options

* Add client api, use in cli fetch_epoch_rewards

* Update docs

* Apply review suggestions

(cherry picked from commit aa54c468ea)

# Conflicts:
#	core/src/rpc.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-03-18 20:51:53 +00:00
mergify[bot]
9c596cfd6c Allow unbounded wallclock processing time in tests (#15961) (#15965)
(cherry picked from commit f548a04fae)

Co-authored-by: carllin <carl@solana.com>
2021-03-18 00:09:08 +00:00
mergify[bot]
e4032ec87f Ignore flaky test_banking_stage_entries_only and test_banking_stage_entryfication (#15958)
(cherry picked from commit 8a9b51952e)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-17 20:25:13 +00:00
Michael Vines
be05c8b121 Bump version to 1.5.16 2021-03-16 13:29:52 -07:00
mergify[bot]
191e51b01d Wallclock BankingStage Throttle (#15731) (#15889)
(cherry picked from commit c1ba265dd9)

Co-authored-by: carllin <carl@solana.com>
2021-03-16 08:40:03 +00:00
Michael Vines
9a151259b2 =1.5.15 2021-03-16 00:32:15 -07:00
mergify[bot]
437f0bb8c6 Rpc: support extended config for getConfirmedBlock (bp #15827) (#15832)
* Rpc: support extended config for getConfirmedBlock (#15827)

* Add rpc confirmed-block config wrapper to support struct of extended config

* Update docs

* Make config wrapper generic and use in getConfirmedTransaction as well

* Update/clean confirmed-tx docs

(cherry picked from commit 5b2da19c93)

# Conflicts:
#	core/src/rpc.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-03-13 00:02:26 +00:00
mergify[bot]
f511625887 Add more slot update notifications (bp #15734) (#15821)
* Add more slot update notifications (#15734)

* Add more slot update notifications

* fix merge

* Address feedback and add integration test

* switch to datapoint

* remove unused shred method

* fix clippy

* new thread for rpc completed slots

* remove extra constant

* fixes

* rely on channel closing

* fix check

(cherry picked from commit 918d04e3f0)

* fix tests

* fix fmt

Co-authored-by: Justin Starry <justin@solana.com>
2021-03-12 15:25:45 +00:00
mergify[bot]
4c3bfcaedf Remove old feature: simple_capitalization (bp #15763) (#15814)
* Remove old feature: simple_capitalization (#15763)

* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit

(cherry picked from commit 4bbeb9c033)

# Conflicts:
#	runtime/src/accounts_db.rs
#	runtime/src/bank.rs

* Fix conflicts

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-03-12 05:21:55 +00:00
mergify[bot]
6e214bbc04 Docs fixups (bp #15801) (#15802)
* docs: add docs links for crates published to crates.io

(cherry picked from commit 24d18b3cf2)

# Conflicts:
#	core/Cargo.toml
#	measure/Cargo.toml
#	programs/bpf/rust/finalize/Cargo.toml

* docs: add rust client api entry

(cherry picked from commit 3e6c7c4a3e)

* docs: rename 'deployed programs' section to 'on-chain programs'

(cherry picked from commit 0e452c8d91)

* docs: 'builtins' -> 'runtime facilities'

(cherry picked from commit 9c8be34906)

* docs: stabilize spl token jsonrpc methods

(cherry picked from commit 45190f6281)

* docs: deprecate lastvalidslot field of jsonrpc getfees

(cherry picked from commit c4ee1ab710)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-03-11 21:49:14 +00:00
mergify[bot]
58a9639810 Add tracer key for tracing transaction path through the network (#15732) (#15771)
(cherry picked from commit 2bee9435f3)

Co-authored-by: carllin <carl@solana.com>
2021-03-09 04:44:16 +00:00
mergify[bot]
74b13605c0 Report datapoint on number of retransmit shreds (#15694) (#15769)
(cherry picked from commit 331c45decf)

Co-authored-by: carllin <carl@solana.com>
2021-03-09 03:08:36 +00:00
Tyera Eulberg
9fbc03d517 Bump version to 1.5.15 (#15768) 2021-03-09 01:48:54 +00:00
mergify[bot]
c60e21b900 PoH batch size calibration (#15717) (#15747)
(cherry picked from commit d09112fa6d)

# Conflicts:
#	local-cluster/src/validator_configs.rs

Co-authored-by: sakridge <sakridge@gmail.com>
2021-03-06 04:29:54 +00:00
mergify[bot]
30781bb7b1 consolidate DEFAULT_HASHES_PER_TICK (#14463) (#15748)
(cherry picked from commit 773b21b34e)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-03-06 02:59:43 +00:00
mergify[bot]
2114864626 Convert blockstore TransactionStatus column family to protobufs (bp #15733) (#15737)
* Convert blockstore TransactionStatus column family to protobufs (#15733)

* Prevent panic if TransactionStatus can't be deserialized

* Convert Blockstore TransactionStatus column to protobuf

* Add compatability test

(cherry picked from commit 7e65289729)

# Conflicts:
#	ledger/Cargo.toml

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-03-05 17:37:54 +00:00
Michael Vines
d3672ca23b Bump version to 1.5.14 2021-03-03 19:01:25 -08:00