gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
15,711 Commits 81 Branches 357 Tags
d3c80be7d5ccaa9b46b2d183f1fb18c58aafe2a8
Commit Graph

130 Commits

Author SHA1 Message Date
behzad nouri
7a8807b8bb retransmits shreds recovered from erasure codes 
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.

This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.
 2021-08-17 13:44:10 +00:00
behzad nouri
3efccbffab sends shreds (instead of packets) to retransmit stage 
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
 2021-08-17 13:44:10 +00:00
behzad nouri
8198a7eae1 adds packet/shred count stats to window-service 
Adding back these metrics from the earlier commit which removed them
from retransmit stage.
 2021-08-17 13:44:10 +00:00
behzad nouri
b64eeb7729 removes erroneous uses of &Arc<...> from window-service 2021-08-13 17:26:31 +00:00
behzad nouri
d57398a959 removes repeated bank-forks locking in window-service 
Window service is repeatedly locking bank-forks to look-up working-bank
for every single shred:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L597-L606

This commit updates shred_filter signature in recv_window so that where
we already obtain the lock on bank-forks, we can also look-up
working-bank once for all packets:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L256-L277
 2021-08-13 17:26:31 +00:00
Jeff Washington (jwash)
14361906ca for all tests, bank::new -> bank::new_for_tests (#19064) 2021-08-05 08:42:38 -05:00
Jeff Biseda
9255ae334d drop outstanding_requests lock before sending repair requests (#18893) 2021-07-28 19:30:43 -07:00
carllin
c0704d4ec9 Plumb signal from replay to ancestor hashes service (#18880) 2021-07-26 20:59:00 -07:00
carllin
1ee64afb12 Introduce AncestorHashesService (#18812) 2021-07-23 16:54:47 -07:00
behzad nouri
d2d5f36a3c adds validator flag to allow private ip addresses (#18850) 2021-07-23 15:25:03 +00:00
behzad nouri
7d56fa8363 sends packets in batches from sigverify-stage (#18446) 
sigverify-stage is breaking batches to single-item vectors before
sending them down the channel:
https://github.com/solana-labs/solana/blob/d451363dc/core/src/sigverify_stage.rs#L88-L92

Also simplifying window-service code, reducing number of nested branches.
 2021-07-22 14:49:21 +00:00
carllin
588c0464b8 Add sampling logic and DuplicateSlotRepairStatus module (#18721) 2021-07-21 11:15:08 -07:00
carllin
8a846b048e Add AncestorHashesRepair type (#18681) 2021-07-15 19:29:53 -07:00
sakridge
7f2254225e Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
jbiseda
a86ced0bac generate deterministic seeds for shreds (#17950) 
* generate shred seed from leader pubkey

* clippy

* clippy

* review

* review 2

* fmt

* review

* check

* review

* cleanup

* fmt
 2021-07-07 08:21:12 -07:00
sakridge
8d9a6deda4 Add repair number per slot (#18082) 2021-06-30 18:20:07 +02:00
Michael Vines
84b9de8c18 Shredder no longer holds a keypair 2021-06-21 21:29:52 -07:00
Michael Vines
4a12c715a3 Drop Error suffix from enum values to avoid the enum_variant_names clippy lint 2021-06-18 23:02:13 +00:00
Alexander Meißner
6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
carllin
52dccc656a Purge slots greater than new last index (#16071) 2021-05-26 16:12:57 -07:00
Tyera Eulberg
9a5330b7eb Move gossip modules into solana-gossip crate (#17352) 
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
 2021-05-26 09:15:46 -06:00
Tao Zhu
0781fe1b4f Upgrade Rust to 1.52.0 (#17096) 
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script

* upgrade to Rust 1.52.1

* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
 2021-05-19 09:31:47 -05:00
behzad nouri
81ad795d46 removes position field in coding-shred-header 
CodingShredHeader.position is equal to
  ShredCommonHeader.index - ShredCommonHeader.fec_set_index
and is so redundant. The extra position field can add bugs if not
consistent with index and fec_set_index.
 2021-05-10 13:20:56 +00:00
behzad nouri
37b8587d4e expands number of erasure coding shreds in the last batch in slots (#16484) 
Number of parity coding shreds is always less than the number of data
shreds in FEC blocks:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L719

Data shreds are batched in chunks of 32 shreds each:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L714

However the very last batch of data shreds in a slot can be small, in
which case the loss rate can be exacerbated.

This commit expands the number of coding shreds in the last FEC block in
slots to: 64 - number of data shreds; so that FEC blocks are always 64
data and parity coding shreds each.

As a consequence of this, the last FEC block has more parity coding
shreds than data shreds. So for some shred indices we will have a coding
shred but no data shreds. This should not cause any kind of overlapping
FEC blocks as in:
https://github.com/solana-labs/solana/pull/10095
since this is done only for the very last batch in a slot, and the next
slot will reset the shred index.
 2021-04-21 12:47:50 +00:00
sakridge
8e69dd42c1 Add non-default repair nonce values (#16512) 
* Track outstanding nonces in repair

* Rework outstanding requests to use lru cache and randomize nonces

Co-authored-by: Carl <carl@solana.com>
 2021-04-20 09:37:33 -07:00
carllin
52703badfa Setup ReplayStage confirmation scaffolding for duplicate slots (#9698) 2021-03-24 23:41:52 -07:00
behzad nouri
491b059755 broadcasts duplicate shreds through gossip (#14699) 2021-01-24 15:47:43 +00:00
sakridge
aa2751e614 Check shred type in is_duplicate (#14050) 2020-12-10 18:20:08 -08:00
Michael Vines
7bc073defe Run codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand 2020-10-21 19:08:13 -07:00
sakridge
ce98088457 Track inserted repair shreds (#12455) 2020-09-29 14:13:21 -07:00
carllin
1c1a3f979d Detect and notify when deserializable shreds are available (#11816) 
* Add logic to check for complete data ranges

* Add RPC signature notification

Co-authored-by: Carl <carl@solana.com>
 2020-09-01 22:06:06 -07:00
carllin
7a14e359d7 Switch to using weighted repair in RepairService (#10735) 
* Plumb votes into repair service

* Remove refactoring

* Fix tests

* Switch to using RepairWeight for generating repairs

* Revert "Weight repair slots based on vote stake (#10741)"

This reverts commit cabd0a09c3.

* Update logging

Co-authored-by: Carl <carl@solana.com>
 2020-07-10 05:52:54 +00:00
sakridge
cabd0a09c3 Weight repair slots based on vote stake (#10741) 
* Weight repair slots based on vote stake

* Add test
 2020-06-22 20:27:45 -07:00
Greg Fitzgerald
6ee222363e Move BankForks to solana_runtime (#10637) 
* Move BankForks to solana_runtime

* Update imports
 2020-06-17 15:27:03 +00:00
Kristofer Peterson
e23340d89e Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445) 
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
 2020-06-09 09:38:14 +09:00
carllin
97f2bcff69 master: Add nonce to shreds repairs, add shred data size to header (#10109) 
* Add nonce to shreds/repairs

* Add data shred size to header

Co-authored-by: Carl <carl@solana.com>
 2020-05-19 12:38:18 -07:00
Kristofer Peterson
58ef02f02b 9951 clippy errors in the test suite (#10030) 
automerge
 2020-05-15 09:35:43 -07:00
Jack May
eb1acaf927 Remove archiver and storage program (#9992) 
automerge
 2020-05-14 18:22:47 -07:00
carllin
bab3502260 Push down cluster_info lock (#9594) 
* Push down cluster_info lock

* Rework budget decrement

Co-authored-by: Carl <carl@solana.com>
 2020-04-21 12:54:45 -07:00
Michael Vines
f3afe5c99c Remove dead code (#9404) 
automerge
 2020-04-09 13:09:59 -07:00
sakridge
ed036b978d Accumulate blockstore metrics and submit every 2s (#9075) 2020-03-26 12:51:41 -07:00
carllin
076fef5e57 Update Cluster Slots to support multiple threads (#9071) 
Co-authored-by: Carl <carl@solana.com>
 2020-03-25 18:09:19 -07:00
Michael Vines
18c1f0dfe9 Remove stub core/src/genesis_utils.rs (#8999) 2020-03-21 10:54:40 -07:00
anatoly yakovenko
9cedeb0a8d Pull streamer out into its own module. (#8917) 
automerge
 2020-03-17 23:30:23 -07:00
Tyera Eulberg
ab361a8073 Rename KeypairUtil to Signer (#8360) 
automerge
 2020-02-20 13:28:55 -08:00
carllin
76e20015a4 Add separate thread to check for and store duplicate slot proofs (#7834) 2020-01-16 15:27:54 -08:00
Greg Fitzgerald
b5dba77056 Rename blocktree to blockstore (#7757) 
automerge
 2020-01-13 13:13:52 -08:00
Pankaj Garg
87b2525e03 Limit maximum number of shreds in a slot to 32K (#7584) 
* Limit maximum number of shreds in a slot to 32K

* mark dead slot replay as fatal error
 2019-12-30 07:42:09 -08:00
carllin
e98132fd76 Move window insertion to separate thread (#7397) 
* Move window insertion to separate thread
 2019-12-19 00:15:49 -08:00
Pankaj Garg
d94041e98d Allow coding shred index to be different than data shred index (#7438) 
* Allow coding shred index to be different than data shred index

* move fec_set_index to shred's common header

* fix bench
 2019-12-12 16:50:29 -08:00