Commit Graph

674 Commits

Author SHA1 Message Date
Michael Vines
6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Yueh-Hsuan Chiang
fe7543c31a (Ledger Store) APIs for obtaining physical size of all data and coding shreds (#22443) 2022-01-19 19:31:19 -08:00
Tao Zhu
a724fa2347 Add hidden cli option to allow validator reports replayed transaction cost metrics (#22369)
* add hidden cli option to allow validator reports replayed transaction cost detail metrics

* Update validator/src/main.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

* - rebase master, using unbounded instead of channel; dowgrade to datapoint_trace

* removed cli arg, prefer log at trace

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-01-15 00:31:21 +00:00
Justin Starry
f804ccdece Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
carllin
eca8d21249 log internals (#22493) 2022-01-13 19:50:46 -05:00
Jeff Biseda
8b66625c95 convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
carllin
813006b33b remove per program timings from blockstore processor ledger replay (#22370) 2022-01-08 01:52:08 -05:00
Yueh-Hsuan Chiang
5771c36d3f Rename open_with_access_type() to open_with_options() (#22123) 2022-01-07 12:11:43 -08:00
steviez
207825d30b Minimize boilerplate code around Rocks column families (#22345) 2022-01-06 23:39:09 -06:00
Justin Starry
52d12cc802 Add runtime support for address table lookups (#22223)
* Add support for address table lookups in runtime

* feedback

* feedback
2022-01-07 11:59:09 +08:00
Trent Nelson
7d32909e17 move ExecuteTimings from runtime::bank to program_runtime::timings 2022-01-06 03:56:46 -07:00
Justin Starry
45458e7139 Refactor: Improve type safety and readability of transaction execution (#22215)
* Refactor Bank::load_and_execute_transactions

* Refactor: improve type safety of TransactionExecutionResult

* Add enum for extra type safety in execution results

* feedback
2022-01-05 10:15:15 +08:00
behzad nouri
01a096adc8 adds bitflags to Packet.Meta
Instead of a separate bool type for each flag, all the flags can be
encoded in a type-safe bitflags encoded in a single u8:
https://github.com/solana-labs/solana/blob/d6ec103be/sdk/src/packet.rs#L19-L31
2022-01-04 13:53:40 +00:00
Jeff Biseda
ca8fef5855 retransmit consecutive leader blocks (#22157) 2022-01-04 00:24:16 -08:00
Yueh-Hsuan Chiang
e8b7f96a89 Add struct BlockstoreOptions (#22121) 2022-01-03 18:30:45 -10:00
Justin Starry
2a00382d71 Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
carllin
33d0b5e011 Revert "Count compute units even when transaction errors (#22059)" (#22174)
This reverts commit eaa8c67bde.
2021-12-30 02:42:32 -05:00
Omar Kilani
bac6821e19 get_signatures_for_address does not correctly account for result sets that span local and Bigtable sources (#22115)
* get_signatures_for_address does not correctly account for result sets that span Blockstore and Bigtable.

This causes Bigtable to return `RowNotFound` until the new tx is uploaded.

Check that `before` exists in Bigtable, and if not, set it to `None` to return the full data set.

References #21442
Closes #22110

* Differentiate between before sig not found and no newer signatures

* Dedupe bigtable results to account for potential upload race

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-12-29 10:25:10 -07:00
carllin
eaa8c67bde Count compute units even when transaction errors (#22059) 2021-12-28 17:05:11 -05:00
Yueh-Hsuan Chiang
b89cd8cd1a Avoid cloning Vec<Entry> when calling entries_to_test_shreds() (#22093) 2021-12-24 12:32:43 -08:00
Yueh-Hsuan Chiang
bf8fbf8383 Add code comment for handle_chaining in blockstore.rs (#21876) 2021-12-21 22:36:24 -08:00
Yueh-Hsuan Chiang
3155a04189 Add comment block for insert_shreds_handle_duplicate in blockstore.rs (#21877) 2021-12-21 22:36:13 -08:00
behzad nouri
65d59f4ef0 tracks erasure coding shreds' indices explicitly (#21822)
The indices for erasure coding shreds are tied to data shreds:
https://github.com/solana-labs/solana/blob/90f41fd9b/ledger/src/shred.rs#L921

However with the upcoming changes to erasure schema, there will be more
erasure coding shreds than data shreds and we can no longer infer coding
shreds indices from data shreds.

The commit adds constructs to track coding shreds indices explicitly.
2021-12-19 22:37:55 +00:00
behzad nouri
89d66c3210 removes next_shred_index from return value of entries to shreds api (#21961)
next-shred-index is already readily available from returned data shreds.
The commit simplifies the api for upcoming changes to erasure coding
schema which will require explicit tracking of indices for coding shreds
as well as data shreds.
2021-12-17 15:01:55 +00:00
behzad nouri
efd64a3862 simplifies ShredIndex api (#21932) 2021-12-16 19:17:32 +00:00
behzad nouri
8183f28636 adds ErasureSetId identifying erasure coding sets of shreds (#21928) 2021-12-16 14:18:55 +00:00
Yueh-Hsuan Chiang
882f886450 Add comment block for commit_slot_meta_working_set in blockstore.rs (#21852) 2021-12-15 13:12:50 -08:00
Yueh-Hsuan Chiang
e374fb1d60 Add code comment for get_slot_meta_entry in blockstore.rs (#21844) 2021-12-15 13:12:38 -08:00
Yueh-Hsuan Chiang
71b12b1f56 Add comment for clear_unconfirmed_slot() in blockstore.rs (#21837) 2021-12-15 00:37:09 -08:00
Yueh-Hsuan Chiang
e476e17abf Add code comment for check_insert_data_shred in blockstore.rs (#21845) 2021-12-15 00:36:11 -08:00
behzad nouri
8d980f07ba uses Option<Slot> for SlotMeta.parent_slot (#21808)
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
behzad nouri
4ceb2689f5 adds ShredId uniquely identifying each shred (#21820) 2021-12-14 17:34:02 +00:00
Ikko Ashimine
e19c7923c3 chore: fix typo in shred.rs (#21890)
begining -> beginning
2021-12-14 11:19:27 -05:00
steviez
4a9d7318d1 Rework test parameters to be shreds instead of entries (#21780)
The number of shreds that result from a given number of entries is
variable and in our test case, somewhat unintuitive to think about when
trying to determine how much data we're pushing into the blockstore. So,
this change converts the unit of test parameters from entries to shreds.

This change also cleans up some variable naming for clarity and prints.
2021-12-13 23:34:43 -06:00
behzad nouri
e08139f949 uses Option<u64> for SlotMeta.last_index (#21775)
SlotMeta.last_index may be unknown and current code is using u64::MAX to
indicate that:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L169-L174

This lacks type-safety and can introduce bugs if not always checked for
Several instances of slot_meta.last_index + 1 are also subject to
overflow.

This commit updates the type to Option<u64>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-11 14:47:20 +00:00
Justin Starry
254ef3e7b6 Rename Packets to PacketBatch (#21794) 2021-12-11 09:44:15 -05:00
behzad nouri
49ba09b333 adds back ErasureMeta::first_coding_index field (#21623)
https://github.com/solana-labs/solana/pull/16646
removed first_coding_index since the field is currently redundant and
always equal to fec_set_index.
However, with upcoming changes to erasure coding schema, this will no
longer be the same as fec_set_index and so requires a separate field to
represent.
2021-12-10 20:08:04 +00:00
behzad nouri
8063273d09 adds more sanity checks to shreds (#21675) 2021-12-09 16:43:57 +00:00
behzad nouri
cd17f63d81 adds back position field to coding-shred-header (#21600)
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.
2021-12-05 14:42:09 +00:00
Michael Vines
b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
ryleung-solana
8cf36e5cb0 Add GPU sigverify for verify path (#20851)
Allows the use of GPU acceleration in verifying the signatures in Entry's after deserialization in the replay stage

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
Co-authored-by: Ryan Leung <ryan.leung@solana.com>
2021-11-30 21:16:13 -05:00
Yueh-Hsuan Chiang
38fcfb7542 Persist coding shreds (#21214) 2021-11-30 09:18:36 -08:00
Michael Vines
e31b469f56 Quash M1 build warning 2021-11-29 22:42:09 -08:00
Michael Vines
ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
Tyera Eulberg
9e043d2055 Use max_complete_transaction_status_slot in BigTableUploadService (#21401)
* Use max_complete_transaction_status_slot in BigTableUploadService

* Use node root to limit BigTableUploadService
2021-11-23 17:36:13 -07:00
behzad nouri
dd338b6c9f changes Shred::parent return type to Option<Slot> (#21370)
Shred::parent can return garbage if the struct fields are invalid:
https://github.com/solana-labs/solana/blob/8a50b6302/ledger/src/shred.rs#L446-L453

The commit adds more sanity checks and changes the return type to Option<Slot>.
2021-11-23 14:45:26 +00:00
Jeff Biseda
2ed7e3af89 prioritize slot repairs for unknown last index and close to completion (#21070) 2021-11-19 19:17:30 -08:00
behzad nouri
7da2df7d8c adds methods to obtain shreds' erasure coded block and index (#21325) 2021-11-19 20:01:15 +00:00
behzad nouri
48dfdfb4d5 changes Blockstore::is_shred_duplicate arg type to ShredType 2021-11-19 14:16:39 +00:00
behzad nouri
57057f8d39 uses enum for shred type
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167

Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348

The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
2021-11-19 14:16:39 +00:00