Commit Graph

332 Commits

Author SHA1 Message Date
Jeff Washington (jwash)
87e577db4c Revert "use cost model to limit new account creation (#21369)"
This reverts commit 90f41fd9b7.
2021-12-12 15:00:35 -06:00
Jeff Washington (jwash)
90f41fd9b7 use cost model to limit new account creation (#21369)
* use cost model to limit new account creation

* handle every system instruction

* remove &

* simplify match

* simplify match

* add datapoint for account data size

* add postgres error handling

* handle accounts:unlock_accounts
2021-12-12 14:57:18 -06:00
Jeff Washington (jwash)
8d1e5ac294 refactor and test scan abort code (#21390) 2021-12-08 14:09:34 -06:00
Michael Vines
b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Jack May
976eb81d4f Cleanup the bank's use of nonces (#21246) 2021-12-02 09:57:05 -08:00
Tao Zhu
9edfc5936d Refactor accounts.rs with Justin's comments to improve lock accounts (#21406)
with results code path.
- fix a bug that could unlock accounts that weren't locked
- add test to the refactored function
- skip enumerating transaction accounts if qos results is an error
- add #[must_use] annotation
- avoid clone error in results
- add qos error code to unlock_accounts match statement
- remove unnecessary AbiExample
2021-11-23 21:17:55 +00:00
sakridge
c4d68063c7 Add timing for accounts add_root (#21379) 2021-11-22 18:29:45 +01:00
Jeff Washington (jwash)
79d21d6805 add --accounts-index-scan-results-limit-mb to allow scans to abort (#21327)
* ScanConfig -> &ScanConfig

* add --accounts-index-scan-results-limit-mb to allow scans to abort

* feedback
2021-11-19 09:00:19 -06:00
Jack May
03c36d240a Nonce naming cleanup (#21336) 2021-11-18 16:07:17 -08:00
Jeff Washington (jwash)
0f69a14247 Add ability to abort scan (#21314) 2021-11-17 13:10:29 -06:00
Tao Zhu
11153e1f87 refactor cost calculation (#21062)
* - cache calculated transaction cost to allow sharing;
- atomic cost tracking op;
- only lock accounts for transactions eligible for current block;
- moved qos service and stats reporting to its own model;
- add cost_weight default to neutral (as 1), vote has zero weight;

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update core/src/qos_service.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update core/src/qos_service.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-11-12 01:04:53 -06:00
Justin Starry
aaccba8377 Refactor RentDebits to use hashmap instead of vec 2021-11-04 15:02:24 +00:00
Justin Starry
036d7fcc81 Clean up sanitized tx creation for tests (#21006) 2021-10-27 18:09:16 +01:00
Jack May
bfbbc53dac Divorce the runtime from FeeCalculator (#20737) 2021-10-22 14:32:40 -07:00
Jack May
a8098f37d0 add checked instructions sysvar api (#20790) 2021-10-19 21:01:58 -07:00
Jeff Washington (jwash)
46bf7d4a4a AcctIdx: support 2^24 bins (#20739) 2021-10-18 14:05:16 -05:00
Jeff Washington (jwash)
70b2e5fef2 hash calc uses all threads during startup (#20735) 2021-10-15 17:41:23 -05:00
Jack May
da45be366a Remove blockhash from fee calculation (#20641) 2021-10-13 13:10:58 -07:00
Alexander Meißner
1d813ea078 Refactor: Make program_id always last in program chain (#20598)
* Replaces program_id field in InvokeContextStackFrame by index.

* Swaps order of program account and programdata account.

* Removes program_id parameter from InvokeContext::push().
2021-10-13 08:58:20 +02:00
Jeff Washington (jwash)
a8e000a2a6 add filler accounts to bloat validator and predict failure (#20491)
* add filler accounts to bloat validator and predict failure

* assert no accounts match filler

* cleanup magic numbers

* panic if can't load from snapshot with filler accounts specified

* some renames

* renames

* into_par_iter

* clean filler accts, too
2021-10-11 12:46:27 -05:00
Michael Vines
7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
Lijun Wang
fe97cb2ddf AccountsDb plugin framework (#20047)
Summary of Changes

Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows

Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.

The code comprises 4 major parts:

accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
Tyera Eulberg
2cd9dc99b6 Restore ability for programs to upgrade themselves (#20265)
* Make helper associated fn

* Add feature definition

* Add handling to preserve program-id write lock when upgradeable loader is present; restore bpf upgrade-self test

* Use single feature
2021-09-28 09:59:08 -06:00
Jeff Washington (jwash)
681e8728a1 get rid of logs left in test (#20116) 2021-09-22 12:39:48 -05:00
Jeff Washington (jwash)
4dc2f08198 AcctIdx: hold_range_in_memory (#19955) 2021-09-17 22:19:29 +00:00
Alexander Meißner
88c1b8f047 Index loaders / executable accounts (#19469)
* Appends loaders / executable_accounts to accounts in transaction loading.

* Adds indices to loaders / executable_accounts.

* Moves MessageProcessor::create_keyed_accounts() into InvokeContext::push().

* Removes "executable_accounts",
now referenced by transaction wide index into "accounts".

* Removes create_pre_accounts() from InstructionProcessor,
as it is already in MessageProcessor.

* Collect program account indices directly in load_executable_accounts().
2021-09-10 08:36:21 +02:00
Tyera Eulberg
38bbb77989 Return error if Transaction contains writable executable or ProgramData accounts (#19629)
* Return error if Transaction locks an executable as writable

* Return error if a ProgramData account is writable but the upgradable loader isn't present

* Remove unreachable clause

* Fixup bpf tests

* Review comments

* Add new TransactionError

* Disallow writes to any upgradeable-loader account when loader not present; remove is_upgradeable_loader_present exception for all other executables
2021-09-08 15:21:52 -06:00
Jeff Washington (jwash)
456bf15012 AccountsIndexConfig -> AccountsDbConfig (#19687) 2021-09-08 04:30:38 +00:00
Tyera Eulberg
decec3cd8b Demote write locks on transaction program ids (#19593)
* Add feature

* Demote write lock on program ids

* Fixup bpf tests

* Update MappedMessage::is_writable

* Comma nit

* Review comments
2021-09-04 03:05:30 +00:00
Jeff Washington (jwash)
82a6bbe068 add Debug trait to range for accounts index (#19531) 2021-09-01 08:13:56 -05:00
Jack May
5cf8d8795b Feature cleanup (#19528) 2021-08-31 14:51:26 -07:00
Jeff Washington (jwash)
02b050e0f5 replace AccountsIndex btree with hashmap of 8k bins (#19212) 2021-08-26 18:12:43 -05:00
Jeff Washington (jwash)
98bc694606 hash calculation adds really old slots to dirty_stores (#19434) 2021-08-26 14:32:43 -05:00
jon-chuang
3e5ba8dcaa bug: sysvar::Instructions is not owned by Sysvar1111111111111111111111111111111111111 (#19242)
* Fix instructions sysvar owner

* Update feature switch address

Co-authored-by: Justin Starry <justin@solana.com>
2021-08-20 08:32:28 -07:00
Justin Starry
c50b01cb60 Store versioned transactions in the ledger, disabled by default (#19139)
* Add support for versioned transactions, but disable by default

* merge conflicts

* trent's feedback

* bump Cargo.lock

* Fix transaction error encoding

* Rename legacy_transaction method

* cargo clippy

* Clean up casts, int arithmetic, and unused methods

* Check for duplicates in sanitized message conversion

* fix clippy

* fix new test

* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
Jeff Washington (jwash)
7c70f2158b accounts_index_bins to AccountsIndexConfig (#19257)
* accounts_index_bins to AccountsIndexConfig

* rename param bins -> config

* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
Jack May
0b50bb2b20 Deprecate FeeCalculator returning APIs (#19120) 2021-08-13 09:08:20 -07:00
Brooks Prumo
5fb6b341c4 Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots (#18870)
* Handle cleaning zero-lamport accounts

Handle cleaning zero-lamport accounts in slots higher than the last full
snapshot slot.  This is part of the Incremental Snapshot work.

Fixes #18825
2021-08-12 15:56:08 -05:00
Jeff Washington (jwash)
651343688d Use option for account index bucket parameter (#19150) 2021-08-10 11:32:25 -05:00
Jeff Washington (jwash)
47e0d9aa95 plumb more accounts_index bins (#19123) 2021-08-10 05:45:46 -05:00
Jeff Washington (jwash)
c7b986220e Accounts::new_with_config_for_tests (#19104) 2021-08-09 09:01:12 -05:00
Jeff Washington (jwash)
f771063275 get rid of Accounts new and default (#19100) 2021-08-06 15:52:27 -05:00
Jeff Washington (jwash)
41f4973f0d accounts default refactoring stragglers (#19097) 2021-08-06 12:36:42 -05:00
Jeff Washington (jwash)
0028442e14 rework defaults and construction of accountsdb, accounts, bank (#19083)
* rework defaults and construction of accountsdb, accounts, bank

* merge issues
2021-08-05 17:27:13 -05:00
Trent Nelson
21bc43ed58 nonce: Unify NonceError with SystemError 2021-07-30 14:47:14 -06:00
Jack May
0167daa116 cleanup feature consistent_recent_blockhashes_sysvar (#18932) 2021-07-28 10:25:33 -07:00
Justin Starry
d166b9856a Move transaction sanitization earlier in the pipeline (#18655)
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
Trent Nelson
3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
Trent Nelson
568660b402 Revert "Remove feature switch for secp256k1 program (#18467)"
This reverts commit fd574dcb3b.
2021-07-15 18:43:55 +00:00
Justin Starry
e806d31224 Optimize Message::is_non_loader_key method (#18579) 2021-07-12 10:42:22 -05:00