gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
16,834 Commits 81 Branches 357 Tags
v1.9.1
Commit Graph

717 Commits

Author SHA1 Message Date
Michael Vines
fb955bd4ec Update Cargo.toml 2021-12-14 14:18:20 -08:00
Michael Vines
5c3fbb384f Futures 0.3.18 has been yanked, back off to .17 
(cherry picked from commit 2a6dcb2ffd)

# Conflicts:
#	ledger/Cargo.toml
 2021-12-14 14:18:20 -08:00
mergify[bot]
a056fd88cb uses Option<Slot> for SlotMeta.parent_slot (backport #21808) (#21899) 
* uses Option<Slot> for SlotMeta.parent_slot (#21808)

SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.

(cherry picked from commit 8d980f07ba)

# Conflicts:
#	ledger-tool/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-12-14 21:42:57 +00:00
mergify[bot]
2f1816d1db adds ShredId uniquely identifying each shred (backport #21820) (#21897) 
* adds ShredId uniquely identifying each shred (#21820)

(cherry picked from commit 4ceb2689f5)

# Conflicts:
#	ledger/src/blockstore.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-12-14 21:03:08 +00:00
mergify[bot]
9956afb2bd uses Option<u64> for SlotMeta.last_index (#21775) (#21806) 
SlotMeta.last_index may be unknown and current code is using u64::MAX to
indicate that:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L169-L174

This lacks type-safety and can introduce bugs if not always checked for
Several instances of slot_meta.last_index + 1 are also subject to
overflow.

This commit updates the type to Option<u64>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.

(cherry picked from commit e08139f949)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-12-11 17:39:05 +00:00
mergify[bot]
01941cf3de Rename Packets to PacketBatch (backport #21794) (#21805) 
* Rename Packets to PacketBatch (#21794)

(cherry picked from commit 254ef3e7b6)

# Conflicts:
#	core/src/verified_vote_packets.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
 2021-12-11 16:53:23 +00:00
Tao Zhu
4b63d51e3e Bump version to 1.9.1 (#21802) 2021-12-11 12:50:36 +00:00
mergify[bot]
e71ea19e60 adds back ErasureMeta::first_coding_index field (#21623) (#21785) 
https://github.com/solana-labs/solana/pull/16646
removed first_coding_index since the field is currently redundant and
always equal to fec_set_index.
However, with upcoming changes to erasure coding schema, this will no
longer be the same as fec_set_index and so requires a separate field to
represent.

(cherry picked from commit 49ba09b333)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-12-10 23:14:10 +00:00
mergify[bot]
8cacf82cb8 adds more sanity checks to shreds (#21675) (#21734) 
(cherry picked from commit 8063273d09)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-12-09 18:44:43 +00:00
mergify[bot]
b4c8e095bd adds back position field to coding-shred-header (#21600) (#21620) 
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.

(cherry picked from commit cd17f63d81)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-12-05 16:40:22 +00:00
Michael Vines
b8837c04ec Reformat imports to a consistent style for imports 
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
 2021-12-03 09:19:13 -08:00
Michael Vines
dd12d90eac Upgrade to Rust 2021 2021-11-30 20:43:46 -08:00
ryleung-solana
8cf36e5cb0 Add GPU sigverify for verify path (#20851) 
Allows the use of GPU acceleration in verifying the signatures in Entry's after deserialization in the replay stage

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
Co-authored-by: Ryan Leung <ryan.leung@solana.com>
 2021-11-30 21:16:13 -05:00
Yueh-Hsuan Chiang
38fcfb7542 Persist coding shreds (#21214) 2021-11-30 09:18:36 -08:00
Michael Vines
e31b469f56 Quash M1 build warning 2021-11-29 22:42:09 -08:00
Michael Vines
ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
dependabot[bot]
3c2276fd09 chore: bump futures from 0.3.17 to 0.3.18 (#21412) 
* chore: bump futures from 0.3.17 to 0.3.18

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-23 21:55:17 -07:00
Tyera Eulberg
9e043d2055 Use max_complete_transaction_status_slot in BigTableUploadService (#21401) 
* Use max_complete_transaction_status_slot in BigTableUploadService

* Use node root to limit BigTableUploadService
 2021-11-23 17:36:13 -07:00
behzad nouri
dd338b6c9f changes Shred::parent return type to Option<Slot> (#21370) 
Shred::parent can return garbage if the struct fields are invalid:
https://github.com/solana-labs/solana/blob/8a50b6302/ledger/src/shred.rs#L446-L453

The commit adds more sanity checks and changes the return type to Option<Slot>.
 2021-11-23 14:45:26 +00:00
dependabot[bot]
191cb62c37 chore: bump libc from 0.2.107 to 0.2.108 (#21377) 
* chore: bump libc from 0.2.107 to 0.2.108

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.107 to 0.2.108.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.107...0.2.108)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-22 17:02:34 -07:00
Jeff Biseda
2ed7e3af89 prioritize slot repairs for unknown last index and close to completion (#21070) 2021-11-19 19:17:30 -08:00
behzad nouri
7da2df7d8c adds methods to obtain shreds' erasure coded block and index (#21325) 2021-11-19 20:01:15 +00:00
behzad nouri
48dfdfb4d5 changes Blockstore::is_shred_duplicate arg type to ShredType 2021-11-19 14:16:39 +00:00
behzad nouri
57057f8d39 uses enum for shred type 
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167

Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348

The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
 2021-11-19 14:16:39 +00:00
Justin Starry
66fa062f13 rename process_entries to indicate it's only for tests (#21321) 2021-11-17 20:53:40 +01:00
Yueh-Hsuan Chiang
3aa49e2c69 Keep all erasure coding shreds even on successful recovery (#20968) (#21052) 
Problem
Blockstore currently removes erasure shreds if the data shreds are
successfully recovered on insert, which is an issue if we want to
serve coding shreds over repair.

Summary of Changes
This diff keeps all coding shreds even on successful recovery and
changes change the signature of prev_inserted_codes to immutable
reference to ensure its immunity.

Fixes #20968
 2021-11-16 11:26:34 -08:00
behzad nouri
3fc858eb60 adds methods to obtain data/coding shreds indices from ErasureMeta 2021-11-13 17:08:05 +00:00
behzad nouri
d25d9be555 removes Default implementaion of ErasureConfig 
The Default implementation of ErasureConfig is off and never useful:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/erasure.rs#L49-L68
 2021-11-13 17:08:05 +00:00
behzad nouri
7e98b77b33 marks ErasureMeta.size as unused 
ErasureMeta.size is unused and redundant:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/blockstore_meta.rs#L57-L58
 2021-11-13 17:08:05 +00:00
behzad nouri
5fb0ab9d00 removes redundant args from Shredder::try_recovery (#21226) 
Shredder::try_recovery is already taking a Vec<Shred> as an argument. All the
other arguments are embedded in the shreds, and are so redundant.
 2021-11-10 21:19:03 +00:00
Kirill Fomichev
b4d09388aa Disable JIT compiler on unsupported platforms (#21215) 2021-11-09 11:31:21 +00:00
dependabot[bot]
c2754b8a51 chore: bump libc from 0.2.106 to 0.2.107 (#21202) 
* chore: bump libc from 0.2.106 to 0.2.107

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.106 to 0.2.107.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.106...0.2.107)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-08 12:14:12 -07:00
behzad nouri
11a53de0e3 rewrites Shredder::try_recovery (#21082) 
For every missing data-shred, Shredder::try_recovery calls into
new_empty_data_shred which does a redundant serialization into payload
buffer which is then immediately overwritten by the erasure recovery:
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L372-L417

Additionally, the implementation is unnecessary complex hindering
upcoming changes to erasure coding generation
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L814-L938

The commit simplifies the Shredder::try_recovery implementation.
 2021-11-04 16:15:51 +00:00
dependabot[bot]
ca51ea3e04 chore: bump libc from 0.2.105 to 0.2.106 (#21093) 
* chore: bump libc from 0.2.105 to 0.2.106

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.105 to 0.2.106.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.105...0.2.106)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-01 14:17:45 -06:00
behzad nouri
484ead01ed optimizes and simplifies SlotMeta::completed_data_indexes ops (#21059) 
SlotMeta::completed_data_indexes is defined as a Vec<u32>:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore_meta.rs#L31-L32

which results in inefficient updates:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore.rs#L3245-L3326

This commit changes the type to BTreeSet<u32> for efficient and simpler
updates and lookups.

The change should be backward compatible because Vec<T> and BTreeSet<T>
are both serialized as seq:
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L207-L208
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L216-L217
 2021-10-31 12:56:25 +00:00
sakridge
4642a2c856 Optimize test_recovery_with_expanded_coding_shreds (#20849) 2021-10-28 11:38:08 +02:00
Tyera Eulberg
f0de3e9bf0 chore: bump tonic from 0.5.2 to 0.6.1 (#21035) 
* Bump tonic and prost crates

* connect_lazy no longer infallible
 2021-10-28 06:59:36 +00:00
Justin Starry
036d7fcc81 Clean up sanitized tx creation for tests (#21006) 2021-10-27 18:09:16 +01:00
dependabot[bot]
cb1947bea2 chore: bump libc from 0.2.104 to 0.2.105 (#20972) 
* chore: bump libc from 0.2.104 to 0.2.105

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.104 to 0.2.105.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.104...0.2.105)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-10-26 00:14:58 -06:00
Michael Vines
350bb561eb Clippy 2021-10-23 08:21:20 +00:00
dependabot[bot]
66e431dad5 chore: bump libc from 0.2.103 to 0.2.104 (#20764) 
* chore: bump libc from 0.2.103 to 0.2.104

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-10-19 22:38:02 -06:00
carllin
b5f21d5e34 Add ledger tool to remove dead slot (#20810) 2021-10-19 20:23:16 -07:00
behzad nouri
231b58b5f1 adds more metrics to blockstore insert shreds stats (#20701) 2021-10-17 11:02:34 -04:00
Jeff Washington (jwash)
70b2e5fef2 hash calc uses all threads during startup (#20735) 2021-10-15 17:41:23 -05:00
Tyera Eulberg
e806fa6904 Include token owners in TransactionTokenBalances (#20642) 
* Cache owners in TransactionTokenBalances

* Light cleanup

* Use return struct, and remove pub

* Single-use statements

* Why not, just do the whole crate

* Add metrics

* Make datapoint_debug to prevent spam unless troubleshooting
 2021-10-13 21:46:52 -06:00
sakridge
09e7782d76 Refactor code to get block signatures in get_confirmed_signatures_for_address2 (#20575) 
* Refactor get_confirmed_signatures_for_address2

* Move blockstore benches to ledger where they belong
 2021-10-13 09:55:19 +02:00
Tao Zhu
005d6863fd - move cost tracker into bank, so each bank has its own cost tracker; (#20527) 
- move related modules to runtime
 2021-10-12 08:51:33 -05:00
Jeff Washington (jwash)
a8e000a2a6 add filler accounts to bloat validator and predict failure (#20491) 
* add filler accounts to bloat validator and predict failure

* assert no accounts match filler

* cleanup magic numbers

* panic if can't load from snapshot with filler accounts specified

* some renames

* renames

* into_par_iter

* clean filler accts, too
 2021-10-11 12:46:27 -05:00
Brooks Prumo
5440c1d2e1 SnapshotPackagerService pushes incremental snapshot hashes to CRDS (#20442) 
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!

This commit does two main things:

1. SnapshotPackagerService now knows about incremental snapshot hashes,
   and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
   hashes need to be passed all the way to SnapshotPackagerService so it
   can push these starting hashes to CRDS.  Those values have been piped
   through.

Fixes #20441 and #20423
 2021-10-08 15:14:56 -05:00
Tao Zhu
675fa6993b - update const cost values with data collected by #19627 (#20314) 
- update cost calculation to closely proposed fee schedule #16984
 2021-10-08 14:48:50 -05:00