* adds back position field to coding-shred-header (#21600)
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.
(cherry picked from commit cd17f63d81)
# Conflicts:
# core/src/window_service.rs
# ledger/src/blockstore.rs
# ledger/src/shred.rs
* removes backport merge conflicts
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
* Add function for changing thread's nice value
Linux only.
* Add validator option to change niceness of snapshot packager thread
* Add validator option to change niceness of RPC server threads
Fixes https://github.com/solana-labs/solana/issues/14556
* Run `./scripts/cargo-for-all-lock-files.sh tree`
* Add missing websocket methods to rust RPC PubSub client (#21065)
- Added accountSubscribe, programSubscribe, slotSubscribe and rootSubscribe to rust RpcClient
- Removed duplication on cleanup threads
- Moved RPCVote from rpc/ to client/rpc_response
(cherry picked from commit a0f9e0e8ee)
# Conflicts:
# Cargo.lock
# client-test/Cargo.toml
# core/tests/client.rs
* Fix conflicts
* Make test result not depend on TestValidator setup
Co-authored-by: Manuel Gil <manugildev@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
* - cost_tracker is data member of a bank, it can report metrics when bank is frozen (#20802)
- removed cost_tracker_stats and histogram
- move stats reporting outside of bank freeze
(cherry picked from commit c2bfce90b3)
# Conflicts:
# Cargo.lock
# core/src/banking_stage.rs
# core/src/replay_stage.rs
# core/src/tvu.rs
# ledger-tool/src/main.rs
# programs/bpf/Cargo.lock
# runtime/Cargo.toml
# runtime/src/cost_tracker.rs
* manual fix merge conflicts
Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
* - make cost_tracker a member of bank, remove shared instance from TPU; (#20627)
- decouple cost_model from cost_tracker; allowing one cost_model
instance being shared within a validator;
- update cost_model api to calculate_cost(&self...)->transaction_cost
(cherry picked from commit 7496b5784b)
# Conflicts:
# core/src/banking_stage.rs
# ledger-tool/src/main.rs
# runtime/src/bank.rs
# runtime/src/cost_model.rs
# runtime/src/cost_tracker.rs
* manual fix merge conflicts
Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
Rather than pass in individual features, pass in the entire feature set
so that we can add the ed25519 program feature in a later commit.
(cherry picked from commit 0f62771f42)
Conflicts:
banks-server/src/banks_server.rs
core/src/banking_stage.rs
programs/secp256k1/src/lib.rs
rpc/src/rpc.rs
runtime/src/bank.rs
sdk/src/transaction.rs
sdk/src/transaction/sanitized.rs
* - move cost tracker into bank, so each bank has its own cost tracker; (#20527)
- move related modules to runtime
(cherry picked from commit 005d6863fd)
# Conflicts:
# Cargo.lock
# core/benches/banking_stage.rs
# core/src/banking_stage.rs
# core/src/lib.rs
# core/src/tvu.rs
# ledger-tool/src/main.rs
# ledger/src/blockstore_processor.rs
# programs/bpf/Cargo.lock
# runtime/Cargo.toml
# runtime/src/cost_model.rs
* manual fix merge conflicts
Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
* Accountsdb stream plugin improvement (#20419)
Support using connection pooling and use multiple threads to do Postgres db operations. The performance is improved from 1500 RPS to 40,000 RPS measured during validator start.
Support multiple plugins at the same time.
* Fixed a fmt issue
Merge AccountsDb plugin framework to v1.8 (#20518)
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
