Commit Graph

2737 Commits

Author SHA1 Message Date
mergify[bot]
be0bcd85ed tracks erasure coding shreds' indices explicitly (#21822) (#22094)
The indices for erasure coding shreds are tied to data shreds:
https://github.com/solana-labs/solana/blob/90f41fd9b/ledger/src/shred.rs#L921

However with the upcoming changes to erasure schema, there will be more
erasure coding shreds than data shreds and we can no longer infer coding
shreds indices from data shreds.

The commit adds constructs to track coding shreds indices explicitly.

(cherry picked from commit 65d59f4ef0)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-23 19:38:50 +00:00
Will Hickey
78e7913352 Bump version to 1.9.3 (#22065) 2021-12-22 11:41:03 -06:00
mergify[bot]
1a2823b875 chore: bump lru from 0.7.0 to 0.7.1 (#22018) (#22056)
Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.0...0.7.1)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 69d0b08dd8)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-21 16:27:43 -07:00
mergify[bot]
e3a4b98432 removes Select in favor of recv_timeout/try_iter (#21981) (#22001)
crossbeam_channel::Select::ready_timeout might return with success spuriously.

(cherry picked from commit 7476dfeec0)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-18 19:37:07 +00:00
mergify[bot]
91657ba8fe new net-stats require a new table (#21996) (#22000)
(cherry picked from commit 3fe942ab30)

Co-authored-by: Jeff Biseda <jbiseda@gmail.com>
2021-12-18 10:26:16 +00:00
mergify[bot]
35ee48bec9 RPC Block Subscription (backport #21787) (#21992)
* RPC Block Subscription (#21787)

* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
(cherry picked from commit 76098dd42a)

# Conflicts:
#	Cargo.lock
#	client-test/Cargo.toml
#	rpc/src/rpc_subscriptions.rs

* Fix conflicts

Co-authored-by: segfaultdoctor <seg@jito.network>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-12-18 01:43:37 +00:00
mergify[bot]
e572678176 removes next_shred_index from return value of entries to shreds api (#21961) (#21980)
next-shred-index is already readily available from returned data shreds.
The commit simplifies the api for upcoming changes to erasure coding
schema which will require explicit tracking of indices for coding shreds
as well as data shreds.

(cherry picked from commit 89d66c3210)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-17 17:57:57 +00:00
mergify[bot]
f4521002b9 Clean up demote program write lock feature (backport #21949) (#21969)
* Clean up demote program write lock feature (#21949)

* Clean up demote program write lock feature

* fix test

(cherry picked from commit 6ff0be6a82)

# Conflicts:
#	programs/bpf_loader/src/syscalls.rs
#	runtime/src/accounts.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-12-17 04:45:22 +00:00
Michael Vines
bfa6302985 Bump version to 1.9.2 2021-12-15 16:18:14 -08:00
mergify[bot]
41ea597256 Fix subtraction overflow (#21871) (#21901)
(cherry picked from commit cb395abff7)

Co-authored-by: carllin <carl@solana.com>
2021-12-14 23:22:47 +00:00
mergify[bot]
a056fd88cb uses Option<Slot> for SlotMeta.parent_slot (backport #21808) (#21899)
* uses Option<Slot> for SlotMeta.parent_slot (#21808)

SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.

(cherry picked from commit 8d980f07ba)

# Conflicts:
#	ledger-tool/src/main.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-14 21:42:57 +00:00
mergify[bot]
2f1816d1db adds ShredId uniquely identifying each shred (backport #21820) (#21897)
* adds ShredId uniquely identifying each shred (#21820)

(cherry picked from commit 4ceb2689f5)

# Conflicts:
#	ledger/src/blockstore.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-14 21:03:08 +00:00
mergify[bot]
9956afb2bd uses Option<u64> for SlotMeta.last_index (#21775) (#21806)
SlotMeta.last_index may be unknown and current code is using u64::MAX to
indicate that:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L169-L174

This lacks type-safety and can introduce bugs if not always checked for
Several instances of slot_meta.last_index + 1 are also subject to
overflow.

This commit updates the type to Option<u64>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.

(cherry picked from commit e08139f949)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-11 17:39:05 +00:00
mergify[bot]
01941cf3de Rename Packets to PacketBatch (backport #21794) (#21805)
* Rename Packets to PacketBatch (#21794)

(cherry picked from commit 254ef3e7b6)

# Conflicts:
#	core/src/verified_vote_packets.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2021-12-11 16:53:23 +00:00
Tao Zhu
4b63d51e3e Bump version to 1.9.1 (#21802) 2021-12-11 12:50:36 +00:00
mergify[bot]
8cacf82cb8 adds more sanity checks to shreds (#21675) (#21734)
(cherry picked from commit 8063273d09)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-09 18:44:43 +00:00
mergify[bot]
b4c8e095bd adds back position field to coding-shred-header (#21600) (#21620)
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.

(cherry picked from commit cd17f63d81)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-12-05 16:40:22 +00:00
Jeff Biseda
9c6b95e1e1 fix distance calculation in get_closest_completion (#21601) 2021-12-03 22:36:46 -08:00
dependabot[bot]
446089edb4 chore: bump retain_mut from 0.1.4 to 0.1.5 (#21599)
Bumps [retain_mut](https://github.com/upsuper/retain_mut) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/upsuper/retain_mut/releases)
- [Commits](https://github.com/upsuper/retain_mut/compare/v0.1.4...v0.1.5)

---
updated-dependencies:
- dependency-name: retain_mut
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-03 22:11:42 -07:00
Justin Starry
1430b58a6d Remove deprecated slow epoch boundary methods (#21568) 2021-12-03 17:59:10 +00:00
Michael Vines
b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Alexander Meißner
b78f5b6032 Refactor: Cleanup InstructionProcessor (#21404)
* Moves create_message(), native_invoke() and process_cross_program_instruction()
from the InstructionProcessor to the InvokeContext so that they can have a useful "self" parameter.

* Moves InstructionProcessor into InvokeContext and Bank.

* Moves ExecuteDetailsTimings into its own file.

* Moves Executor into invoke_context.rs

* Moves PreAccount into its own file.

* impl AbiExample for BuiltinPrograms
2021-12-01 08:54:42 +01:00
Michael Vines
dd12d90eac Upgrade to Rust 2021 2021-11-30 20:43:46 -08:00
dependabot[bot]
de8edad30b chore: bump etcd-client from 0.7.2 to 0.8.1 (#21513)
Bumps [etcd-client](https://github.com/etcdv3/etcd-client) from 0.7.2 to 0.8.1.
- [Release notes](https://github.com/etcdv3/etcd-client/releases)
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.7.2...0.8.1)

---
updated-dependencies:
- dependency-name: etcd-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-30 18:16:32 +00:00
dependabot[bot]
a125e7b8c2 chore: bump serde_json from 1.0.71 to 1.0.72 (#21426)
* chore: bump serde_json from 1.0.71 to 1.0.72

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-30 18:03:09 +00:00
Michael Vines
ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
Tao Zhu
9edfc5936d Refactor accounts.rs with Justin's comments to improve lock accounts (#21406)
with results code path.
- fix a bug that could unlock accounts that weren't locked
- add test to the refactored function
- skip enumerating transaction accounts if qos results is an error
- add #[must_use] annotation
- avoid clone error in results
- add qos error code to unlock_accounts match statement
- remove unnecessary AbiExample
2021-11-23 21:17:55 +00:00
Lijun Wang
c29838fce1 Accountsdb plugin transaction part 3: Transaction Notifier (#21374)
The TransactionNotifierInterface interface for notifying transactions.
Changes to transaction_status_service to notify the notifier of the transaction data.
Interface to query the plugin's interest in transaction data
2021-11-23 09:55:53 -08:00
Tao Zhu
2602e7c3bc Fix flaky test (#21402)
* the async test is flaky on ci

* fix unstable test by increasing stats repoting time
2021-11-23 09:47:17 -06:00
behzad nouri
dd338b6c9f changes Shred::parent return type to Option<Slot> (#21370)
Shred::parent can return garbage if the struct fields are invalid:
https://github.com/solana-labs/solana/blob/8a50b6302/ledger/src/shred.rs#L446-L453

The commit adds more sanity checks and changes the return type to Option<Slot>.
2021-11-23 14:45:26 +00:00
Tao Zhu
cd5a39ee43 the async test is flaky on ci (#21365) 2021-11-22 18:16:20 -06:00
Jeff Washington (jwash)
87831e7f8d start system monitor earlier in validator so we get memory stats at startup (#21372) 2021-11-22 14:37:17 -06:00
sakridge
f31ca8ba8c Report cluster slots size (#21380) 2021-11-22 17:47:58 +01:00
Jeff Biseda
2ed7e3af89 prioritize slot repairs for unknown last index and close to completion (#21070) 2021-11-19 19:17:30 -08:00
sakridge
0bda0c3e0c Add bank drop service (#21322) 2021-11-19 17:20:18 +01:00
behzad nouri
48dfdfb4d5 changes Blockstore::is_shred_duplicate arg type to ShredType 2021-11-19 14:16:39 +00:00
behzad nouri
57057f8d39 uses enum for shred type
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167

Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348

The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
2021-11-19 14:16:39 +00:00
carllin
b30c94ce55 ClusterInfoVoteListener send only missing votes to BankingStage (#20873) 2021-11-18 15:20:41 -08:00
Tao Zhu
0ca255220e - Encapsulate QoS Service metrics reporting within QosServioce, so client (#21191)
code (eg banking_stage) doesn't need to worry about it.
- Remove dead cost_* stats from banking_stage, clean up call path.
2021-11-18 15:35:30 -06:00
dependabot[bot]
89102540b1 chore: bump serde_json from 1.0.70 to 1.0.71 (#21338)
* chore: bump serde_json from 1.0.70 to 1.0.71

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-18 02:58:30 +00:00
Lijun Wang
89c45a57f8 Refactor slot status notification to decouple from accounts notifications (#21308)
Problem

Slot status can be used of in other scenarios in addition to account information such as transactions, blocks. The current implementation is too tightly coupled.

Summary of Changes

Decouple the slot status notification from accounts notification. Created a new slot status notification module.
2021-11-17 17:11:38 -08:00
Justin Starry
66fa062f13 rename process_entries to indicate it's only for tests (#21321) 2021-11-17 20:53:40 +01:00
Jeff Biseda
d5de0c8e12 add --no-os-network-stats-reporting option (#21296) 2021-11-16 10:26:03 -08:00
sakridge
398af132a5 More set_root metrics (#21286) 2021-11-15 16:28:18 -07:00
dependabot[bot]
55ea672041 chore: bump serde_json from 1.0.69 to 1.0.70 (#21285)
* chore: bump serde_json from 1.0.69 to 1.0.70

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-11-15 15:27:47 -07:00
Jeff Washington (jwash)
f2bd9947cc mem stats: rescale from kb to bytes (#21282) 2021-11-15 14:42:41 -06:00
Jeff Washington (jwash)
f8dcb2f38b report mem stats (#21258) 2021-11-13 00:59:41 +00:00
Michael Keleti
b0ca335463 Rename "trusted" to "known" in validators/ (#21197)
* Replaced trusted with known validator

* Format Convention
2021-11-12 11:57:55 -07:00
Tao Zhu
11153e1f87 refactor cost calculation (#21062)
* - cache calculated transaction cost to allow sharing;
- atomic cost tracking op;
- only lock accounts for transactions eligible for current block;
- moved qos service and stats reporting to its own model;
- add cost_weight default to neutral (as 1), vote has zero weight;

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update core/src/qos_service.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update core/src/qos_service.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-11-12 01:04:53 -06:00
behzad nouri
5fb0ab9d00 removes redundant args from Shredder::try_recovery (#21226)
Shredder::try_recovery is already taking a Vec<Shred> as an argument. All the
other arguments are embedded in the shreds, and are so redundant.
2021-11-10 21:19:03 +00:00