Commit Graph

16987 Commits

Author SHA1 Message Date
d0cf5bb721 Bump thread_local (#22711) (#22714)
(cherry picked from commit 1c10677f82)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-01-24 12:10:10 -07:00
fb54991901 Fix typos (#22700)
Fix typos

(cherry picked from commit fd0f5e4d12)

Co-authored-by: tanliwei <tanliwei@users.noreply.github.com>
2022-01-24 03:23:40 +00:00
9995a54be7 fix: flag was incorrect in doc (#22698)
(cherry picked from commit 714a344937)

Co-authored-by: Arash <arash@backbone.link>
2022-01-23 21:24:58 +00:00
d9a5f714e1 Refactor: Rename variables and helper method to PohRecorder (#22676) (#22688)
* Refactor: Rename leader_first_tick_height field

* Refactor: add `PohRecorder::slot_for_tick_height` helper

* Refactor: Add type for poh leader status

(cherry picked from commit 1240217a73)

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-23 15:34:44 +08:00
620a80b581 sigverify -- dedupe bloom filter too slow followups 2022-01-21 23:59:41 -07:00
b354dae249 Perf: Only check executors cache for executable bpf program ids (backport #22624) (#22629)
* Perf: Only check executors cache for executable bpf program ids (#22624)

* Only check executors cache for executable bpf program ids

* switch to native loader check

* clean up tests

* fix tests

* clippy

(cherry picked from commit 7d34a7acac)

# Conflicts:
#	runtime/src/bank.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-22 14:08:15 +08:00
af7ed83285 Document transaction module (backport #22440) (#22664)
* Document transaction module (#22440)

* Document transaction module

* example_mocks is only for feature = full

(cherry picked from commit 8dd62854fa)

# Conflicts:
#	sdk/src/transaction/mod.rs

* Fix conflicts

Co-authored-by: Brian Anderson <andersrb@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-01-21 22:58:44 -07:00
8bc4cc90d2 Bump version to 1.9.6 2022-01-21 20:15:43 -07:00
39a4cc95dc v1.9: Impl get_/set_return_data syscalls for ProgramTest (#22652)
* Remove &mut self from set_return_data

* Impl get_/set_return_data for program-test SyscallStubs

* Add return_data program-test
v1.9.5
2022-01-21 18:03:27 -07:00
187ed6a387 Remove unused fields from Bank (backport #22491) (#22630)
* Remove unused fields from Bank (#22491)

(cherry picked from commit 9977396d8f)

# Conflicts:
#	runtime/src/serde_snapshot/future.rs

* fixup the backport

Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-01-22 00:13:32 +00:00
91bc44931f Add hidden cli option to allow validator reports replayed transaction cost metrics (backport #22369) (#22519)
* Add hidden cli option to allow validator reports replayed transaction cost metrics (#22369)

* add hidden cli option to allow validator reports replayed transaction cost detail metrics

* Update validator/src/main.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

* - rebase master, using unbounded instead of channel; dowgrade to datapoint_trace

* removed cli arg, prefer log at trace

Co-authored-by: Michael Vines <mvines@gmail.com>
(cherry picked from commit a724fa2347)

# Conflicts:
#	core/src/tvu.rs

* fix conflict

Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
2022-01-21 15:05:41 -07:00
35ca3182ba Add estimated and actual block cost units metrics (backport #22326) (#22517)
* Add estimated and actual block cost units metrics (#22326)

* - report cost details for transactions selected to be packed into block;
- report estimated execution units packed into block, and actual units and time after execution

* revert reporting per-transaction details

* rollup transaction cost details (eg signature cost, wirte lock, data cost and execution costs) into block stats

* change naming from units to cu, use struct to replace tuple

(cherry picked from commit 1309a9cea0)

# Conflicts:
#	core/src/banking_stage.rs
#	core/src/qos_service.rs

* fix conflicts

Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
2022-01-21 15:05:19 -07:00
24345d8e63 Update introduction.md (#22623) (#22625)
A few fixes for grammatical and spelling issues.

(cherry picked from commit 373f200ab8)

Co-authored-by: filip <44206832+filipkujawa@users.noreply.github.com>
2022-01-21 13:31:47 -07:00
bf45f5b88e Faster dedup v1.9 (#22638)
Faster dedup port of #22607
2022-01-21 11:21:28 -08:00
2ddb5b27c1 Refactor: move instructions sysvar serialization out of Message (#22544) (#22595)
(cherry picked from commit 7ba57e7a7c)

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-21 13:45:47 +08:00
7f10fd6a21 Refactor: move compute budget runtime logic into solana-program-runtime (backport #22543) (#22545)
* Refactor: move compute budget runtime logic into solana-program-runtime (#22543)

(cherry picked from commit cc76a73c49)

# Conflicts:
#	programs/bpf/tests/programs.rs
#	sdk/src/compute_budget.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-21 13:45:01 +08:00
a0a881594a Speed up packet dedup and fix benches (#22592) (#22613)
* Speed up packet dedup and fix benches

* fix tests

* allow int arithmetic in bench

(cherry picked from commit a2d251ce1e)

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-20 22:51:58 +00:00
e9e35fd7bd system-monitor-service: support percentages from bigger numbers (#22598)
(cherry picked from commit cca3dbc76d)

Co-authored-by: Trent Nelson <trent@solana.com>
2022-01-20 11:37:52 +00:00
66b94b86a9 banking-stage: remove unused stats fields 2022-01-20 06:09:10 +00:00
59f406d78a Refactor: move simple vote parsing to runtime (backport #22537) (#22587)
* Refactor: move simple vote parsing to runtime (#22537)

(cherry picked from commit 7f20c6149e)

# Conflicts:
#	core/src/cluster_info_vote_listener.rs
#	core/src/verified_vote_packets.rs
#	programs/vote/src/vote_transaction.rs
#	rpc/src/rpc_subscriptions.rs
#	runtime/src/bank.rs
#	runtime/src/bank_utils.rs
#	runtime/src/vote_sender_types.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-20 04:51:50 +00:00
dbf9a32883 Optimize packet dedup (#22571) (#22585)
* Use bloom filter to dedup packets

* dedup first

* Update bloom/src/bloom.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* fixup

* fixup

* fixup

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
(cherry picked from commit d343713f61)

# Conflicts:
#	Cargo.lock
#	core/Cargo.toml
#	core/src/banking_stage.rs
#	core/src/sigverify_stage.rs
#	gossip/Cargo.toml
#	perf/Cargo.toml
#	programs/bpf/Cargo.lock
#	runtime/Cargo.toml

Co-authored-by: anatoly yakovenko <anatoly@solana.com>
2022-01-20 02:51:49 +00:00
37e9076db0 Add PacketBatch packet_indexes stat (#22564) (#22575)
* collect stats on packet batch indicies

* cleanup

* cleanup

* cleanup

* change name

(cherry picked from commit 650882217c)

# Conflicts:
#	core/src/banking_stage.rs

Co-authored-by: buffalu <85544055+buffalu@users.noreply.github.com>
2022-01-20 00:05:05 +00:00
f77ea5f324 improves sigverify discard_excess_packets performance (backport #22577) (#22580)
* improves sigverify discard_excess_packets performance (#22577)

As shown by the added benchmark, current code does worse if there is a
spam address plus a lot of unique addresses.

on current master:
test bench_packet_discard_many_senders  ... bench:   1,997,960 ns/iter (+/- 103,715)
test bench_packet_discard_mixed_senders ... bench:  14,256,116 ns/iter (+/- 534,865)
test bench_packet_discard_single_sender ... bench:   1,306,809 ns/iter (+/- 61,992)

with this commit:
test bench_packet_discard_many_senders  ... bench:   1,644,025 ns/iter (+/- 83,715)
test bench_packet_discard_mixed_senders ... bench:   1,089,789 ns/iter (+/- 86,324)
test bench_packet_discard_single_sender ... bench:     955,234 ns/iter (+/- 55,953)

(cherry picked from commit dcf44d2523)

# Conflicts:
#	core/src/sigverify_stage.rs

* removes mergify merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-01-19 20:14:10 +00:00
c9df037dae Track discard time of excess packets in sigverify (#22554) (#22570)
* discard time histogram

* closer to the if

* update

(cherry picked from commit e616a7ebfc)

Co-authored-by: anatoly yakovenko <anatoly@solana.com>
2022-01-19 01:52:30 +00:00
2b87d99479 Use VecDeque instead of Vec in sigverify stage (#22538) (#22550)
avoid bad performance of remove(0) for a single sender

(cherry picked from commit 49443406fd)

# Conflicts:
#	core/src/sigverify_stage.rs

Co-authored-by: sakridge <sakridge@gmail.com>
2022-01-19 01:46:34 +00:00
2546ef4ad6 metrics for generate new bank forks (#22492) (#22548)
* metrics for generate new bank forks

* fixed

* Apply suggestions from code review

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* --fixup

* fixup!

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
(cherry picked from commit 2d94e6e5d3)

# Conflicts:
#	core/src/replay_stage.rs

Co-authored-by: anatoly yakovenko <anatoly@solana.com>
2022-01-19 01:45:25 +00:00
96ae795758 Add more details about vote account key rotation (#22539)
(cherry picked from commit 901b2881fb)

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-01-17 09:16:25 +00:00
9bddb4e437 vote account withdraw authority may change the authorized voter 2022-01-15 23:46:10 -08:00
4079f12a3e Perf: Store deserialized sysvars in the sysvars cache (backport #22455) (#22480)
* Perf: Store deserialized sysvars in the sysvars cache (#22455)

* Perf: Store deserialized sysvars in sysvars cache

* add bench

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-15 08:48:34 +00:00
e121b94524 Bugfix/block subscribe (#22516) (#22525)
* use correct operation name

* require enable_rpc_transaction_history flag when enabling block_subscription

Co-authored-by: Zano <segfaultdoctor@protonmail.com>
(cherry picked from commit 7171b3a3ac)

Co-authored-by: segfaultdoctor <zano@jito.wtf>
2022-01-15 05:05:21 +00:00
a7623ad18c Fetch sysvars from invoke context for vote program (backport #22444) (#22469)
* Fetch sysvars from invoke context for vote program (#22444)

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-15 03:56:00 +00:00
054e475c6c ABI changed - added two more vote_cost related fields to cost_tracker 2022-01-14 10:49:43 -06:00
7a421fe602 Port counting vote CUs to block cost to v1.9 2022-01-14 10:49:43 -06:00
2ef0b85829 docs: fix get fee for message docs (#22501) (#22504)
(cherry picked from commit f12a8fcd73)

Co-authored-by: Yihau Chen <a122092487@gmail.com>
2022-01-14 09:00:22 +00:00
a6b7a3b7ff Refactor: move sysvar cache to new module (backport #22448) (#22461)
* Refactor: move sysvar cache to new module

(cherry picked from commit 7171c95bdd)

# Conflicts:
#	Cargo.lock
#	program-runtime/Cargo.toml
#	program-runtime/src/invoke_context.rs
#	programs/bpf/Cargo.lock
#	programs/bpf_loader/src/syscalls.rs
#	programs/stake/src/stake_instruction.rs
#	programs/vote/src/vote_instruction.rs
#	runtime/src/message_processor.rs

* resolve conflicts

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-14 08:43:26 +00:00
9d69f2b324 Bank::get_fee_for_message is now nonce aware (backport #22494) (#22499)
* `Bank::get_fee_for_message` is now nonce aware

(cherry picked from commit 4c577d7f8c)

# Conflicts:
#	runtime/src/bank.rs
#	sdk/program/src/message/sanitized.rs

* Resolve conflicts

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-01-14 03:25:10 +00:00
4f82a4ba1f log internals (#22493) (#22497)
(cherry picked from commit eca8d21249)

Co-authored-by: carllin <carl@solana.com>
2022-01-14 02:33:36 +00:00
ed0b30efcc nit: Traceable balance checks (#22462) (#22489)
(cherry picked from commit 1632ee03da)

Co-authored-by: Jack May <jack@solana.com>
2022-01-13 19:09:00 +00:00
4ee6bc9a93 downgrade individual per-program-timing to trace to reduce writes to influx (#22471)
(cherry picked from commit 6614727be8)

Co-authored-by: Tao Zhu <tao@solana.com>
2022-01-13 02:37:47 +00:00
676c43b9d2 Fixed a merge issue (#22464)
Removed AddressLookupError
2022-01-12 13:56:39 -08:00
b1d8296498 Update docs vis-a-vis prohibition of RentPaying accounts (#22438) (#22458)
* Rent-exempt docs for exchange integrations

* Remove discussion of rent-paying accounts from developing docs

* Improve verbiage

(cherry picked from commit b27333e52d)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-01-12 19:58:05 +00:00
34984ed16e v1.9: Only examine explicit tx accounts for rent state (#22442)
* Add failing test

* Fix: only examine accounts explicitly included in a tx
2022-01-11 20:55:10 -07:00
f4d1577337 Refactor: consolidate memo extraction for each message version (#22422) (#22435)
(cherry picked from commit 35a5dd9c45)

Co-authored-by: Justin Starry <justin@solana.com>
2022-01-12 00:23:03 +00:00
58dcc451a9 Prevent rent-paying account creation (backport #22292) (#22428)
* Prevent rent-paying account creation (#22292)

* Fixup typo

* Add new feature

* Add new TransactionError

* Add framework for checking account state before and after transaction processing

* Fail transactions that leave new rent-paying accounts

* Only check rent-state of writable tx accounts

* Review comments: combine process_result success behavior; log and metrics before feature activation

* Fix tests that assume rent-exempt accounts are okay

* Remove test no longer relevant

* Remove native/sysvar special case

* Move metrics submission to report legacy->legacy rent paying transitions as well

(cherry picked from commit 637e366b18)

# Conflicts:
#	runtime/src/bank.rs
#	runtime/src/lib.rs

* Fix conflicts and rework for TransactionRefCells

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-01-11 23:17:03 +00:00
f0695ef6d9 optimizes ReadOnlyAccountsCache LRU eviction implementation (backport #22403) (#22426)
* optimizes ReadOnlyAccountsCache LRU eviction implementation (#22403)

ReadOnlyAccountsCache is using a background thread, table scan and sort
to implement LRU eviction policy:
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L66-L73
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L186-L191
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L222

DashMap internally locks each shard when accessed; so a table scan in
the background thread can create a lot of lock contention.

This commit adds an index-list queue containing cached keys in the order
that they are accessed. Each hash-map entry also includes its index into
this queue.
When an item is first entered into the cache, it is added to the end of
the queue. Also each time an entry is looked up from the cache it is
moved to the end of queue. As a result, items in the queue are always
sorted in the order that they have last been accessed. When doing LRU
eviction, cache entries are evicted from the front of the queue.
Using index-list, all queue operations above are O(1) with low overhead
and so above achieves an efficient implementation of LRU cache eviction
policy.

(cherry picked from commit a49ef49f87)

# Conflicts:
#	Cargo.lock
#	programs/bpf/Cargo.lock
#	runtime/Cargo.toml
#	runtime/src/accounts_db.rs

* removes backport merge conflicts

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-01-11 19:09:57 +00:00
41b0d6cca3 limits gossip vote stats to the top most voted slots (#22416) (#22418)
(cherry picked from commit 49da347d84)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2022-01-10 23:14:57 +00:00
ae77a52c97 wrap create executor timings datapoint in a module (#22398)
(cherry picked from commit 428575f9ae)

Co-authored-by: Trent Nelson <trent@solana.com>
2022-01-09 06:41:44 +00:00
133314e58c bank: fix executor cache metrics (#22396)
(cherry picked from commit 3b4aad9df1)

Co-authored-by: Trent Nelson <trent@solana.com>
2022-01-09 06:06:35 +00:00
cb49ae21b4 Bump version to v1.9.5 2022-01-08 21:17:51 +00:00
a9ebba5643 Clarify docs of minimum_balance (#22385) (#22387)
(cherry picked from commit 0f94e1d3a2)

Co-authored-by: Evan Conrad <evan@roomservice.dev>
2022-01-08 20:07:57 +00:00