* Add compute budget program as a noop (#20992)
(cherry picked from commit 1e2bef76e3)
# Conflicts:
# sdk/src/feature_set.rs
* resolve conflicts
Co-authored-by: Jack May <jack@solana.com>
Solang requires a method for verify ed25519 signatures. Add a new
builtin program at address Ed25519SigVerify111111111111111111111111111
which takes any number of ed25519 signature, public key, and message.
If any of the signatures fails to verify, an error is returned.
The changes for the web3.js package will go into another commit, since
the tests test against a released solana node. Adding web3.js ed25519
testing will break CI.
(cherry picked from commit b491354e51)
Conflicts:
Cargo.lock
Cargo.toml
programs/bpf/Cargo.lock
runtime/Cargo.toml
sdk/src/feature_set.rs
sdk/src/transaction.rs
sdk/src/transaction/sanitized.rs
This consists of:
- syscalls
- passing return data from invoked to invoker
- printing to stable log
- rust and C SDK changes
(cherry picked from commit 53b47b87b2)
* Cost Model to limit transactions which are not parallelizeable (#16694)
* * Add following to banking_stage:
1. CostModel as immutable ref shared between threads, to provide estimated cost for transactions.
2. CostTracker which is shared between threads, tracks transaction costs for each block.
* replace hard coded program ID with id() calls
* Add Account Access Cost as part of TransactionCost. Account Access cost are weighted differently between read and write, signed and non-signed.
* Establish instruction_execution_cost_table, add function to update or insert instruction cost, unit tested. It is read-only for now; it allows Replay to insert realtime instruction execution costs to the table.
* add test for cost_tracker atomically try_add operation, serves as safety guard for future changes
* check cost against local copy of cost_tracker, return transactions that would exceed limit as unprocessed transaction to be buffered; only apply bank processed transactions cost to tracker;
* bencher to new banking_stage with max cost limit to allow cost model being hit consistently during bench iterations
* replay stage feed back program cost (#17731)
* replay stage feeds back realtime per-program execution cost to cost model;
* program cost execution table is initialized into empty table, no longer populated with hardcoded numbers;
* changed cost unit to microsecond, using value collected from mainnet;
* add ExecuteCostTable with fixed capacity for security concern, when its limit is reached, programs with old age AND less occurrence will be pushed out to make room for new programs.
* investigate system performance test degradation (#17919)
* Add stats and counter around cost model ops, mainly:
- calculate transaction cost
- check transaction can fit in a block
- update block cost tracker after transactions are added to block
- replay_stage to update/insert execution cost to table
* Change mutex on cost_tracker to RwLock
* removed cloning cost_tracker for local use, as the metrics show clone is very expensive.
* acquire and hold locks for block of TXs, instead of acquire and release per transaction;
* remove redundant would_fit check from cost_tracker update execution path
* refactor cost checking with less frequent lock acquiring
* avoid many Transaction_cost heap allocation when calculate cost, which
is in the hot path - executed per transaction.
* create hashmap with new_capacity to reduce runtime heap realloc.
* code review changes: categorize stats, replace explicit drop calls, concisely initiate to default
* address potential deadlock by acquiring locks one at time
* Persist cost table to blockstore (#18123)
* Add `ProgramCosts` Column Family to blockstore, implement LedgerColumn; add `delete_cf` to Rocks
* Add ProgramCosts to compaction excluding list alone side with TransactionStatusIndex in one place: `excludes_from_compaction()`
* Write cost table to blockstore after `replay_stage` replayed active banks; add stats to measure persist time
* Deletes program from `ProgramCosts` in blockstore when they are removed from cost_table in memory
* Only try to persist to blockstore when cost_table is changed.
* Restore cost table during validator startup
* Offload `cost_model` related operations from replay main thread to dedicated service thread, add channel to send execute_timings between these threads;
* Move `cost_update_service` to its own module; replay_stage is now decoupled from cost_model.
* log warning when channel send fails (#18391)
* Aggregate cost_model into cost_tracker (#18374)
* * aggregate cost_model into cost_tracker, decouple it from banking_stage to prevent accidental deadlock. * Simplified code, removed unused functions
* review fixes
* update ledger tool to restore cost table from blockstore (#18489)
* update ledger tool to restore cost model from blockstore when compute-slot-cost
* Move initialize_cost_table into cost_model, so the function can be tested and shared between validator and ledger-tool
* refactor and simplify a test
* manually fix merge conflicts
* Per-program id timings (#17554)
* more manual fixing
* solve a merge conflict
* featurize cost model
* more merge fix
* cost model uses compute_unit to replace microsecond as cost unit
(#18934)
* Reject blocks for costs above the max block cost (#18994)
* Update block max cost limit to fix performance regession (#19276)
* replace function with const var for better readability (#19285)
* Add few more metrics data points (#19624)
* periodically report sigverify_stage stats (#19674)
* manual merge
* cost model nits (#18528)
* Accumulate consumed units (#18714)
* tx wide compute budget (#18631)
* more manual merge
* ignore zerorize drop security
* - update const cost values with data collected by #19627
- update cost calculation to closely proposed fee schedule #16984
* add transaction cost histogram metrics (#20350)
* rebase to 1.7.15
* add tx count and thread id to stats (#20451)
each stat reports and resets when slot changes
* remove cost_model feature_set
* ignore vote transactions from cost model
Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Jeff Biseda <jbiseda@gmail.com>
Co-authored-by: Jack May <jack@solana.com>
* Add separate vote processing tpu port
* Add feature to send to tpu vote port
* Add vote rejecting sigverify mode
* use packet.meta.is_simple_vote_tx in place of deserialization
* consolidate code that identifies vote tx atcommon path for cpu and gpu
* new key for feature set
* banking forward tpu vote
* add tpu vote port to dockerfile and other review changes
* Simplify thread id compare
* fix a test; updated cluster_info ABI change
Co-authored-by: Tao Zhu <tao@solana.com>
* Check seed length before trying to cal program address (#19699)
(cherry picked from commit 778b2adbea)
* resolve conflicts
Co-authored-by: Jack May <jack@solana.com>
* stake: Advance `credits_observed` on activation epoch (#19309)
* stake: Advance `credits_observed` on activation epoch
* Add test for merging stakes just after activation
(cherry picked from commit 2c3bdedea3)
# Conflicts:
# runtime/src/bank.rs
* Fix merge issues
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
* stake: Remove v2 program references (#19308)
* stake: Remove v2 program references
* Remove stake v2 feature, along with stake rewrite
(cherry picked from commit 73aa004c59)
# Conflicts:
# runtime/src/bank.rs
* Fix merge conflict
* Fix uses of old `stake` function
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
* system-program: Move lamports == 0 check on transfers
* Address feedback
* Update stake split to explicitly allocate + assign
* Update stake tests referring to split instruction
* Revert whitespace
* Update split instruction index in test
* Remove unnecessary `assign_with_seed` from `split_with_seed`
* Fix stake instruction parser
* Update test to allow splitting into account with lamports
(cherry picked from commit 8f5e773caf)
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
* Add missing ProgramError to InstructionError mappings
* add note
* Clarify process of adding new program error
(cherry picked from commit 83b9a046d1)
Co-authored-by: Jack May <jack@solana.com>
* Always bail if program modifies a ro account (#17569)
(cherry picked from commit a3240aebde)
* resolve conflicts
* nudge
Co-authored-by: Jack May <jack@solana.com>
* feat: allow SetAuthority in CLI calls
* chore: clippy match_like_matches_macro
* chore: clippy match_like_matches_macro
* chore: rename CLI to CPI
* chore: move check for cpi authorised instruction to syscalls
* chore: add set_upgrade_authority cpi test
* chore: assert upgrade authority was changed
* feat: gate set_upgrade_authority via cpi with a feature
* chore: move feature to the end of the list
* chore: remove white spaces
* chore: remove white spaces
* chore: update comment to rerun build
* Drop write lock on sysvars
* adds env var for demoting sysvar write lock demotion
* moves demote logic to is_writable
* feature gates sysvar write lock demotion
* adds builtins to write lock demotion
* adds system program id to builtins
* adds Feature111...
* adds an abi-freeze test
* mvines set of builtin program keys
Co-authored-by: Michael Vines <mvines@gmail.com>
* update tests
* adds bpf loader keys
* Add test sysvar
* Plumb demote_sysvar to is_writable
* more plumbing of demote_sysvar_write_locks to is_writable
* patches test_program_bpf_instruction_introspection
* hard codes demote_sysvar_write_locks to false for serialization/encoding methods
* Revert "hard codes demote_sysvar_write_locks to false for serialization/encoding methods"
This reverts commit ae3e2d2e777437bddd753933097a210dcbc1b1fc.
* change the hardcoded ones to demote_sysvar_write_locks=true
* Use data_as_mut_slice
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
