1589 Commits

Author SHA1 Message Date
mergify[bot]
4ff482cd47 Fail simulation if transaction contains duplicate accounts (#18304) (#18334)
(cherry picked from commit b08f8bd1b0)

Co-authored-by: Justin Starry <justin@solana.com>
2021-06-30 21:35:56 +00:00
mergify[bot]
a10a0824eb refactor untar_snapshot_in to push parallelism deeper for further refactoring (#18310) (#18312)
(cherry picked from commit ce53b84cdc)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-30 17:59:57 +00:00
mergify[bot]
7e40a051a4 bin accounts index (#18111) (#18280)
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-29 09:14:10 -05:00
mergify[bot]
d1b26cb267 reduce pubkey copies on insert path (#18240) (#18276)
(cherry picked from commit 1f1e54b9d8)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-28 16:31:48 -05:00
Jack May
6a0a03415c Don't update if already an executable
(cherry picked from commit 2fbedd834f)
2021-06-27 12:24:00 -06:00
mergify[bot]
b7d04cf7b9 replace account index bulk insert with iterator (#18198) (#18235)
(cherry picked from commit e06376664b)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-25 17:55:42 -05:00
mergify[bot]
41179b1282 metric for accounts index insertion time (#18202) (#18226)
(cherry picked from commit f2a2581259)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-25 13:36:29 -05:00
mergify[bot]
c99460ba15 remove unnecessary copies from accounts index code paths (#18196) (#18212)
(cherry picked from commit f9fccdee85)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-25 02:23:23 +00:00
mergify[bot]
769fcb7f50 untar in parallel (#18184) (#18210)
* untar in parallel

* make enum for 'ignore' return value

(cherry picked from commit 31ec986ea1)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-24 19:35:21 -05:00
mergify[bot]
60812790e1 rework dirty_pubkeys from insert_new_if_missing_into_primary_index (#18200) (#18208)
(cherry picked from commit 4b314be5bd)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-24 17:55:45 -05:00
mergify[bot]
09b612b130 add min/max bin size metrics to hash calculation (#18155) (#18181)
(cherry picked from commit 77c3ffe4cf)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-23 20:04:15 +00:00
mergify[bot]
26fdf3fb07 Add batch stats (#18096) (#18142)
(cherry picked from commit e808f34b0b)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-06-23 12:43:37 -05:00
mergify[bot]
f41f3f6b51 apply 'debug_do_not_add_builtins' to feature activations (#18110) (#18150)
(cherry picked from commit fbc94d84c8)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-23 17:34:19 +00:00
mergify[bot]
bc08351a0a capture sort time in hash calculation (#18118) (#18153)
(cherry picked from commit d3ee73e151)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-23 15:49:28 +00:00
mergify[bot]
088b3893c3 Keep track of dirty stores on remove accounts to clean (#17601) (#18173)
* Keep track of dirty stores on remove accounts to clean

and not zero_lamport key set

* Only dirty when count==0?

* Add another clean

(cherry picked from commit 3b1738c000)

Co-authored-by: sakridge <sakridge@gmail.com>
2021-06-23 10:23:44 +00:00
mergify[bot]
aca66674d3 correctly calculate hash_total for hash/lamport calculation (#18144) (#18146)
(cherry picked from commit d5f9f3b7ce)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-22 21:21:10 +00:00
Trent Nelson
597429ab3e Bump version to v1.7.4 2021-06-22 19:57:34 +00:00
Trent Nelson
fcabaa7eff sdk: refactor pda generation 2021-06-22 11:36:05 -06:00
mergify[bot]
6f376489a5 Handle removing slots during account scans (#17471) (#17953)
(cherry picked from commit ccc013e134)

Co-authored-by: carllin <carl@solana.com>
2021-06-22 00:59:12 +00:00
mergify[bot]
0cbf7bef1e Move stake_weighted_timestamp module (#18114) (#18120)
* Move timestamp module into runtime

* Less public

* Remove unused enum

(cherry picked from commit 19fe1dd463)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-06-22 00:11:12 +00:00
mergify[bot]
0f87e598f6 refactor generate_index_for_slot (#17984) (#18116)
(cherry picked from commit 2087f5da94)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-21 18:25:56 -05:00
mergify[bot]
090c801cc6 eliminate flatten and sort in hash calculation (#17802) (#18115)
* eliminate flatten and sort in hash calculation

* reduce critical section time

* remove now no-longer necessary test code

* conflict with reset bins to 0 pr

(cherry picked from commit bf97627021)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-21 17:47:47 -05:00
mergify[bot]
c676b7a473 refactor reconstruct_accountsdb_from_fields (#17987) (#18100)
(cherry picked from commit ae82e4e25b)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-21 15:14:45 -05:00
mergify[bot]
97a7f747fb remove unused code (#17981) (#18048)
(cherry picked from commit 20e714b3d0)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-21 10:12:50 -05:00
mergify[bot]
cef9e0de0c generate_index timings (#17983) (#18049)
(cherry picked from commit 1feaaf009d)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-21 10:12:27 -05:00
mergify[bot]
0e7512a225 Fix Nightly Clippy Warnings (backport #18065) (#18070)
* chore: cargo +nightly clippy --fix -Z unstable-options

(cherry picked from commit 6514096a67)

# Conflicts:
#	core/src/banking_stage.rs
#	core/src/cost_model.rs
#	core/src/cost_tracker.rs
#	core/src/execute_cost_table.rs
#	core/src/replay_stage.rs
#	core/src/tvu.rs
#	ledger-tool/src/main.rs
#	programs/bpf_loader/build.rs
#	rbpf-cli/src/main.rs
#	sdk/cargo-build-bpf/src/main.rs
#	sdk/cargo-test-bpf/src/main.rs
#	sdk/src/secp256k1_instruction.rs

* chore: cargo fmt

(cherry picked from commit 789f33e8db)

* Updates BPF program assert_instruction_count tests.

(cherry picked from commit c1e03f3410)

# Conflicts:
#	programs/bpf/tests/programs.rs

* Resolve conflicts

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-18 20:02:48 +00:00
mergify[bot]
c65c580b20 introduce LockMapType for accounts_index (#18021) (#18051)
(cherry picked from commit 0a81c37fce)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-18 01:39:27 +00:00
Stephen Akridge
d159ae9342 Bump version to v1.7.3 2021-06-17 15:34:50 -06:00
mergify[bot]
a540af1ca7 refactor rebuild_bank_from_snapshots (backport #17988) (#18006)
* refactor rebuild_bank_from_snapshots (#17988)

(cherry picked from commit bb7413c9f9)

# Conflicts:
#	runtime/src/snapshot_utils.rs

* merge errors

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-17 19:26:10 +00:00
mergify[bot]
e9c234d89f Make account shrink configurable #17544 (backport #17778) (#18013)
* Make account shrink configurable #17544 (#17778)

1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544

(cherry picked from commit 269d995832)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/accounts_db.rs
#	runtime/src/snapshot_utils.rs

* fix some merge errors

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-17 17:32:03 +00:00
mergify[bot]
962a2126b5 Log more info on runtime account verification errors (#17861) (#17879)
(cherry picked from commit a1fab0c5ca)

Co-authored-by: Jack May <jack@solana.com>
2021-06-17 01:55:37 +00:00
mergify[bot]
5c495ad1b0 ledger tool limit_load_slot_count_from_snapshot avoids assert failures (backport #17974) (#18008)
* ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974)

(cherry picked from commit dbd4dc04b0)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/blockstore_processor.rs
#	runtime/benches/accounts.rs
#	runtime/src/bank.rs

* fix merge errors

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-17 01:32:50 +00:00
mergify[bot]
30bbc1350d Startup optimization in shrink - don't shrink non-shrinkable slots (backport #17405) (#17792)
* Skip shrink when it doesn't save anything (#17405)

(cherry picked from commit 14c52ab018)

# Conflicts:
#	runtime/src/accounts_db.rs

* fix merge error

Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-16 23:28:03 +00:00
mergify[bot]
2f0f1fd5f5 break out generate index reporting (#17980) (#18015)
(cherry picked from commit 71796f4951)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 22:02:19 +00:00
mergify[bot]
c28e6ebc4c do not use index for verify_bank_hash_and_lamports (#17812) (#18010)
(cherry picked from commit adc683956f)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 21:33:30 +00:00
mergify[bot]
4ed0fcdde6 avoid unnecessary empty arrays when binning (#17944) (#18011)
(cherry picked from commit eee5414c64)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 20:48:18 +00:00
mergify[bot]
296a8ade63 set hash bins to 65k (#17912) (#18009)
(cherry picked from commit 55ee3b5f2f)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 20:31:26 +00:00
mergify[bot]
8492031fd0 refactor generate_index process_storage_slot (#17982) (#18007)
(cherry picked from commit 7ca04d6a86)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 20:14:49 +00:00
mergify[bot]
bff7259111 parallel storage -> accounts folder (#17955) (#18004)
(cherry picked from commit 7de79425ce)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 15:51:00 +00:00
mergify[bot]
67e1814581 nit: rename measure (#17946) (#17995)
(cherry picked from commit 5ecb30ff58)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 14:51:38 +00:00
mergify[bot]
12e92dd59d verify bank hash on startup with ledger tool option (backport #17939) (#17996)
* verify bank hash on startup with ledger tool option (#17939)

(cherry picked from commit f558b9b6bf)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/snapshot_utils.rs

* fix merge errors

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-16 14:46:34 +00:00
mergify[bot]
61573756f8 add metrics for startup (#17913) (#17975)
* add metrics for startup

    * roll timings up higher

    * fix test

    * fix duplicate

    (cherry picked from commit 471b34132e)

    # Conflicts:
    #       ledger/src/bank_forks_utils.rs
    #       runtime/src/snapshot_utils.rs
conflicts because #17778 is not present.

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-16 01:40:56 +00:00
mergify[bot]
fe5fed1182 Refactor bank_forks_utils::load() and some snapshot logic (#17492) (#17979)
Refactor a few functions that are on the load-from-snapshot path, to facilitate
adding in incremental snapshots more easily.

Additionally, add some tests and doc comments.

(cherry picked from commit 1953543274)

Co-authored-by: Brooks Prumo <brooks@solana.com>
2021-06-15 22:03:49 +00:00
Jon Cinque
0c90307677 v1.7: Stake refactor (Manual backport of #17906) (#17978)
* Refactor stake program into solana_program (#17906)

* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt

* Run cargo fmt post cherry-pick
2021-06-15 23:43:22 +02:00
mergify[bot]
cdd2a51f1f remove unused parameters from CalculateHashIntermediate (#17949) (#17976)
(cherry picked from commit 4d8ffdcc11)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-15 19:15:07 +00:00
mergify[bot]
0dbe3434f0 refactor so hash verify can be done by more callers (#17941) (#17947)
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-15 16:49:32 +00:00
mergify[bot]
8b5ba771ad name arguments to help with confusion (#17942) (#17948)
(cherry picked from commit 7fde9b6ff0)

    # Conflicts:
    #       runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-15 14:46:07 +00:00
mergify[bot]
d6f17517cb calculate_capitalization uses hash calculation (#17443) (#17932)
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-15 05:03:06 +00:00
mergify[bot]
15b2f280e3 use trait to simplify and consolidate cumulative code (#17852) (#17931)
(cherry picked from commit 2dc6969858)

    # Conflicts:
    #       runtime/src/accounts_hash.rs

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-15 04:29:13 +00:00
mergify[bot]
60b43a1ddf Pass iterator for SortedStorages::new_with_slots (#17811) (#17847)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit 2a9b127029)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-15 00:01:53 +00:00