2774 Commits

Author SHA1 Message Date
0269bf0f19 Bump version to 1.9.17 2022-04-15 09:30:21 +00:00
2ac42ef4ec Remove duplicate increment (#24219) (#24225)
(cherry picked from commit ff3b6d2b8b)

Co-authored-by: carllin <carl@solana.com>
2022-04-14 13:24:33 -07:00
643ae053ab Bump version to v1.9.16 2022-04-09 03:08:56 +00:00
e02542003d fix merge conflicts 2022-04-08 19:22:26 -05:00
671e9cbac1 Address review comments
(cherry picked from commit a058f348a2)

# Conflicts:
#	runtime/src/cost_tracker.rs
2022-04-08 19:22:26 -05:00
55e64910f7 Unittest for cost tracker after process_and_record_transactions
(cherry picked from commit 2ed29771f2)
2022-04-08 19:22:26 -05:00
9ae13f26e5 Adjustments to cost_tracker updates
- don't store pending tx signatures and costs in CostTracker
- apply tx costs to global state immediately again
- go from commit_or_cancel to update_or_remove, where the cost tracker
  is either updated with the true costs for successful tx, or the costs
  of a retryable tx is removed
- move the function into qos_service and hold the cost tracker lock for
  the whole loop

(cherry picked from commit 924b8ea1eb)

# Conflicts:
#	core/src/qos_service.rs
#	runtime/src/cost_tracker.rs
2022-04-08 19:22:26 -05:00
6cb6b9206f - Only commit successfully executed transactions' cost to cost_tracker;
- In-fly transactions are pended in cost_tracker until being committed
  or cancelled;

(cherry picked from commit 9e07272af8)

# Conflicts:
#	core/src/qos_service.rs
#	runtime/src/cost_model.rs
#	runtime/src/cost_tracker.rs
2022-04-08 19:22:26 -05:00
7615585464 v1.9: Bump tonic, tonic-build, prost, and etcd-client (#24159)
* Bump tonic, prost, and etcd-client

* Restore doc ignores

* Restore clippy ignore
2022-04-08 10:22:05 -06:00
aa26edb7fe Bump version to 1.9.15 (#23954) 2022-03-26 11:33:49 -05:00
8859f35b26 Bump version to 1.9.14 (#23703)
* Bump version to 1.9.14

* Bump openssl-src
2022-03-16 11:21:18 -06:00
8b0576d954 Rename AccountsDb plugins to Geyser plugins (backport #23604) (#23668)
* Rename AccountsDb plugins to Geyser plugins (#23604)

(cherry picked from commit 102dd68a03)

# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	core/Cargo.toml
#	core/src/replay_stage.rs
#	core/src/tvu.rs
#	geyser-plugin-interface/Cargo.toml
#	geyser-plugin-manager/Cargo.toml
#	geyser-plugin-manager/src/geyser_plugin_service.rs
#	geyser-plugin-manager/src/slot_status_notifier.rs
#	validator/src/bin/solana-test-validator.rs
#	validator/src/main.rs

* Fix conflicts

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-03-15 05:04:17 +00:00
3ed915dcc9 Bump version to 1.9.13 (#23614) 2022-03-11 10:09:26 -06:00
3875bc91ab Revert "chore: bump dashmap from 4.0.2 to 5.1.0 (#23372) (#23521)" (#23596)
This reverts commit f56b25ac29.
2022-03-11 09:12:46 +01:00
49952e05cf Report even if slot begins and ends in process_buffered_packets() (#23549) (#23576)
(cherry picked from commit 588414a776)

Co-authored-by: carllin <carl@solana.com>
2022-03-10 07:59:26 +00:00
4a100fbe3b Bump version to 1.9.12 (#23577) 2022-03-09 23:22:45 -06:00
607a98e9d0 remove persist_cost_table code 2022-03-09 21:06:00 -07:00
86a97563b5 Patch validator from loading persisted program costs 2022-03-09 21:06:00 -07:00
4e5d9885da Revert exponential moving average cost model changes (backport #23541) (#23543)
* Revert "fix tests after merge"

This reverts commit ba2d83f580.

(cherry picked from commit 0a17edcc1f)

* Revert "1. Persist to blockstore less frequently;"

This reverts commit 7aa1fb4e24.

(cherry picked from commit c878c9e2cb)

# Conflicts:
#	core/src/cost_update_service.rs
#	core/src/tvu.rs
#	runtime/src/cost_model.rs

* Revert "use EMA in place of Welford"

This reverts commit 6587dbfa47.

(cherry picked from commit 9acbfa5eb1)

* Revert "- estimate a program cost as 2 standard deviation above mean"

This reverts commit a25ac1c988.

(cherry picked from commit 5a0cd05866)

# Conflicts:
#	core/src/cost_update_service.rs
#	runtime/src/cost_model.rs

* fix merge conflicts

Co-authored-by: Carl Lin <carl@solana.com>
Co-authored-by: Tao Zhu <tao@solana.com>
2022-03-09 03:55:36 +00:00
714cf0eff2 solana-validator set-identity no longer writes a tower file unnecessarily (#23542)
(cherry picked from commit b719d6a2ad)

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-09 01:55:14 +00:00
de694402ca Bump version to 1.9.11 2022-03-08 10:53:04 -08:00
f56b25ac29 chore: bump dashmap from 4.0.2 to 5.1.0 (#23372) (#23521)
* chore: bump dashmap from 4.0.2 to 5.1.0

Bumps [dashmap](https://github.com/xacrimon/dashmap) from 4.0.2 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.1.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
(cherry picked from commit 3a0271c113)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-08 14:05:58 +00:00
84c57dd0a8 Bump version to v1.9.10 (#23304) 2022-02-23 11:30:21 -06:00
450404f800 v1.9: Enforce tx metadata upload to bigtable (#23212)
* Enforce tx metadata upload with static types (#23028)

* resolve conflicts

* fix test
2022-02-23 12:03:17 +08:00
68d846c7a9 Revert "Add simulation detection countermeasure (backport #22880) (#23143)" (#23262)
This reverts commit 0fdbec9735.
2022-02-21 13:38:40 -07:00
08d6b9850d Bump version to 1.9.9 (#23247) 2022-02-19 21:17:11 -06:00
02f8651a9c Fix the flaky test test_restart_tower_rollback (backport #23129) (#23155)
* Fix the flaky test test_restart_tower_rollback (#23129)

* Add flag to disable voting until a slot to avoid duplicate voting

* Fix the tower rollback test and remove it from flaky.

(cherry picked from commit ab92578b02)

* Resolve conflicts

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2022-02-17 20:31:27 +00:00
0fdbec9735 Add simulation detection countermeasure (backport #22880) (#23143)
* Add simulation detection countermeasure (#22880)

* Add simulation detection countermeasures

* Add program and test using TestValidator

* Remove incinerator deposit

* Remove incinerator

* Update Cargo.lock

* Add more features to simulation bank

* Update Cargo.lock per rebase

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
(cherry picked from commit c42b80f099)

# Conflicts:
#	programs/bpf/Cargo.lock
#	programs/bpf/Cargo.toml

* Update Cargo.lock

Co-authored-by: Michael Vines <mvines@gmail.com>
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2022-02-17 14:45:24 +00:00
f629c71849 Add slot-based timing metrics (backport #23097) (#23210)
* Add execute timings (#23097)

(cherry picked from commit 619335df1a)

# Conflicts:
#	core/src/banking_stage.rs

* resolve conflicts

Co-authored-by: carllin <carl@solana.com>
2022-02-17 10:57:50 +00:00
c3098e99d1 Bump version to v1.9.8 2022-02-16 21:42:57 -07:00
8b12749f02 forward_buffered_packets return packet count in error path (#23167) (#23187)
(cherry picked from commit 115d71536b)

Co-authored-by: Jeff Biseda <jbiseda@gmail.com>
2022-02-16 13:01:16 -08:00
3fd78ac6ea shrink batches when over 80% of the space is wasted (backport #23066) (#23189)
* shrink batches when over 80% of the space is wasted (#23066)

* shrink batches when over 80% of the space is wasted

(cherry picked from commit 83d31c9e65)

# Conflicts:
#	core/benches/sigverify_stage.rs
#	core/src/sigverify_stage.rs
#	perf/src/sigverify.rs

* fixup!

Co-authored-by: anatoly yakovenko <anatoly@solana.com>
2022-02-16 19:30:16 +00:00
41bbc11a46 flag end-of-slot when poh bank is gone (backport #23069) (#23174)
* flag end-of-slot when poh bank is gone

(cherry picked from commit 03bf66a51b)

# Conflicts:
#	core/src/banking_stage.rs

* merge fix

Co-authored-by: Tao Zhu <tao@solana.com>
2022-02-16 18:08:03 +00:00
8963724ed6 solana-validator set-identity now supports the --require-tower flag 2022-02-15 21:09:44 -08:00
3e3fb4e296 Introduce slot-specific packet metrics (backport #22906) (#23077)
* Introduce slot-specific packet metrics (#22906)

(cherry picked from commit 2f9e30a1f7)

# Conflicts:
#	core/benches/banking_stage.rs
#	core/src/banking_stage.rs
#	core/src/qos_service.rs

* Resolve conflicdts

Co-authored-by: carllin <carl@solana.com>
2022-02-13 05:44:38 +00:00
2d144afec5 Bump version to 1.9.6 (#23092) 2022-02-11 15:00:06 -06:00
d64eebb799 estimate a program cost as 2 standard deviation above mean (backport #22286) (#23019)
* - estimate a program cost as 2 standard deviation above mean
- replaced get_average / get_mode with get_default to assign max units to unknown program

(cherry picked from commit a25ac1c988)

# Conflicts:
#	runtime/src/cost_model.rs

* use EMA in place of Welford

(cherry picked from commit 6587dbfa47)

* 1. Persist to blockstore less frequently;
2. reduce alpha for EMA to 1 percent to have roughly 200 data points for estimatio

(cherry picked from commit 7aa1fb4e24)

# Conflicts:
#	core/src/cost_update_service.rs
#	core/src/tvu.rs
#	runtime/src/cost_model.rs

* fix tests after merge

(cherry picked from commit ba2d83f580)

* fix merge

Co-authored-by: Tao Zhu <tao@solana.com>
2022-02-09 22:16:55 +00:00
df16a37ab5 bench should update leader schedule cache (#22991) (#22998)
(cherry picked from commit e52e48076e)

Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
2022-02-08 15:05:57 +00:00
dd6c365bd9 Resolve conflicts (#22905)
Co-authored-by: carllin <carl@solana.com>
2022-02-05 06:47:18 +00:00
9ea025315e removes VoteTracker::new in favor of VoteTracker::default (#22941) (#22946)
VoteTracker::new does not need a bank and is so redundant:
https://github.com/solana-labs/solana/blob/5a230f418/core/src/cluster_info_vote_listener.rs#L103-L107
(cherry picked from commit 27aaf9df85)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-02-04 20:59:01 +00:00
c43cef79b5 Add quic port for accepting transactions (#22753) (#22937)
using quinn library

streamer: Sign TLS cert with validator identity key

Handle multiple incoming chunks

(cherry picked from commit 5a230f418d)

Co-authored-by: sakridge <sakridge@gmail.com>
2022-02-04 20:53:27 +00:00
f7091811d4 Allow buffered packets be consumed if bank is active, regardless leader schedule 2022-02-03 16:56:27 -06:00
15ef1827bf push live packets straight to buffer, leader only process packets from buffer 2022-02-03 16:56:27 -06:00
69e207ca58 rpc: use minimal mode by default (backport #22734) (#22879)
* rpc: use minimal mode by default

(cherry picked from commit eac4a6df68)

# Conflicts:
#	local-cluster/tests/local_cluster.rs

* test-validator-bin: reinstate full rpc method set

Co-authored-by: Trent Nelson <trent@solana.com>
2022-02-03 08:25:49 +00:00
fb8db79e63 adds reverse lookup index to cluster-nodes (#22892) (#22894)
retransmit has to exclude slot leader from set of nodes for each shred;
which currently requires a linear scan:
https://github.com/solana-labs/solana/blob/e3b137066/core/src/cluster_nodes.rs#L238-L242

This commit adds a reverse lookup index to avoid linear scan.

(cherry picked from commit dccbddad80)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-02-02 21:15:51 +00:00
237347847b caches WeightedShuffle struct in ClusterNodes (#22877) (#22889)
Instead of reconstructing WeightedShuffle struct for each shred
broadcast or retransmit, we can use the same struct with minimal
mutations.

(cherry picked from commit e3b137066d)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-02-02 16:54:57 +00:00
c715bc93cf removes Rng field from WeightedShuffle struct (#22850) (#22868)
(cherry picked from commit 45e09664b8)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-02-01 17:33:52 +00:00
4dfbb4347c includes zero weighted entries in WeightedShuffle (#22829) (#22847)
Current WeightedShuffle implementation excludes zero weighted entries
from the shuffle:
https://github.com/solana-labs/solana/blob/13e631dcf/gossip/src/weighted_shuffle.rs#L29-L30

Though mathematically this might make more sense, for our use-cases
(turbine specifically), this results in less efficient code:
https://github.com/solana-labs/solana/blob/13e631dcf/core/src/cluster_nodes.rs#L409-L430

This commit changes the implementation so that zero weighted indices are
also included in the shuffle but appear only at the end after non-zero
weighted indices.

(cherry picked from commit 604ca9316c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-01-31 18:16:40 +00:00
2368e09d89 Add vote account address to vote subscription
(cherry picked from commit 331b953551)

# Conflicts:
#	core/src/cluster_info_vote_listener.rs
#	rpc/src/rpc_pubsub.rs
#	rpc/src/rpc_subscriptions.rs
2022-01-27 11:19:26 -08:00
b0e0410003 Set the correct root in block commitment cache initialization (#22750) (#22757)
* Set the correct root in block commitment cache initialization

* clean up test

* bump

(cherry picked from commit d9c259a231)

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-27 03:44:59 +00:00