gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
14,860 Commits 81 Branches 357 Tags
f8fccc7e91e077bbbcf77fa8e5b33e381be6e14c
Commit Graph

14860 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
mergify[bot]
e9c234d89f Make account shrink configurable #17544 (backport #17778) (#18013) 
* Make account shrink configurable #17544 (#17778)

1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544

(cherry picked from commit 269d995832)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/accounts_db.rs
#	runtime/src/snapshot_utils.rs

* fix some merge errors

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
v1.7.2 		2021-06-17 17:32:03 +00:00
mergify[bot]
b472dac6b3 validator: expose max active pubsub subscriptions to CLI (#18036) 
(cherry picked from commit 5efc48fc69)

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-06-17 08:02:20 +00:00
mergify[bot]
523dac1be3 metrics: Don't unwrap client instantiation errors (#18019) 
(cherry picked from commit 5cc073420a)

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-06-17 07:08:36 +00:00
mergify[bot]
962a2126b5 Log more info on runtime account verification errors (#17861) (#17879) 
(cherry picked from commit a1fab0c5ca)

Co-authored-by: Jack May <jack@solana.com>
 2021-06-17 01:55:37 +00:00
mergify[bot]
5c495ad1b0 ledger tool limit_load_slot_count_from_snapshot avoids assert failures (backport #17974) (#18008) 
* ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974)

(cherry picked from commit dbd4dc04b0)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/blockstore_processor.rs
#	runtime/benches/accounts.rs
#	runtime/src/bank.rs

* fix merge errors

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
 2021-06-17 01:32:50 +00:00
mergify[bot]
f633f34e43 Clean up remove_all_authorized_voters() (#18029) 
(cherry picked from commit f1ebbbab8f)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-06-17 00:28:41 +00:00
mergify[bot]
bacf1b9acc accounts-cluster-bench, close all accounts in a sliding window (#17993) (#18026) 
(cherry picked from commit ed18add7d3)

Co-authored-by: carllin <carl@solana.com>
 2021-06-16 23:53:31 +00:00
mergify[bot]
4df9da5c48 validator: run poh speed test earlier in start up (#18024) 
(cherry picked from commit 5bc6c89adc)

Co-authored-by: Trent Nelson <trent@solana.com>
 2021-06-16 23:28:15 +00:00
mergify[bot]
30bbc1350d Startup optimization in shrink - don't shrink non-shrinkable slots (backport #17405) (#17792) 
* Skip shrink when it doesn't save anything (#17405)

(cherry picked from commit 14c52ab018)

# Conflicts:
#	runtime/src/accounts_db.rs

* fix merge error

Co-authored-by: sakridge <sakridge@gmail.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
 2021-06-16 23:28:03 +00:00
mergify[bot]
2f0f1fd5f5 break out generate index reporting (#17980) (#18015) 
(cherry picked from commit 71796f4951)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 22:02:19 +00:00
mergify[bot]
c28e6ebc4c do not use index for verify_bank_hash_and_lamports (#17812) (#18010) 
(cherry picked from commit adc683956f)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 21:33:30 +00:00
mergify[bot]
6479c11e9a Don't store votes unless we are leader soon (#17803) (#17894) 
(cherry picked from commit 0feac57cb0)

Co-authored-by: sakridge <sakridge@gmail.com>
 2021-06-16 21:28:06 +00:00
mergify[bot]
4ed0fcdde6 avoid unnecessary empty arrays when binning (#17944) (#18011) 
(cherry picked from commit eee5414c64)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 20:48:18 +00:00
mergify[bot]
296a8ade63 set hash bins to 65k (#17912) (#18009) 
(cherry picked from commit 55ee3b5f2f)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 20:31:26 +00:00
mergify[bot]
a84953ccfd Improve program deployment error message (#17717) (#17725) 
(cherry picked from commit 39654d3fa5)

Co-authored-by: Jack May <jack@solana.com>
 2021-06-16 20:16:00 +00:00
mergify[bot]
8492031fd0 refactor generate_index process_storage_slot (#17982) (#18007) 
(cherry picked from commit 7ca04d6a86)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 20:14:49 +00:00
mergify[bot]
bff7259111 parallel storage -> accounts folder (#17955) (#18004) 
(cherry picked from commit 7de79425ce)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 15:51:00 +00:00
mergify[bot]
67e1814581 nit: rename measure (#17946) (#17995) 
(cherry picked from commit 5ecb30ff58)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 14:51:38 +00:00
mergify[bot]
12e92dd59d verify bank hash on startup with ledger tool option (backport #17939) (#17996) 
* verify bank hash on startup with ledger tool option (#17939)

(cherry picked from commit f558b9b6bf)

# Conflicts:
#	core/tests/snapshots.rs
#	ledger/src/bank_forks_utils.rs
#	runtime/src/snapshot_utils.rs

* fix merge errors

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
 2021-06-16 14:46:34 +00:00
mergify[bot]
4ee366edfa Properly handle block_height in Bigtable bincode deserialization (#17990) (#17994) 
* Default block_height on eof

* Add comment to prevent future errors

(cherry picked from commit c57d1b44ef)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-06-16 05:41:38 +00:00
mergify[bot]
61573756f8 add metrics for startup (#17913) (#17975) 
* add metrics for startup

    * roll timings up higher

    * fix test

    * fix duplicate

    (cherry picked from commit 471b34132e)

    # Conflicts:
    #       ledger/src/bank_forks_utils.rs
    #       runtime/src/snapshot_utils.rs
conflicts because #17778 is not present.

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-16 01:40:56 +00:00
mergify[bot]
fe5fed1182 Refactor bank_forks_utils::load() and some snapshot logic (#17492) (#17979) 
Refactor a few functions that are on the load-from-snapshot path, to facilitate
adding in incremental snapshots more easily.

Additionally, add some tests and doc comments.

(cherry picked from commit 1953543274)

Co-authored-by: Brooks Prumo <brooks@solana.com>
 2021-06-15 22:03:49 +00:00
Jon Cinque
0c90307677 v1.7: Stake refactor (Manual backport of #17906) (#17978) 
* Refactor stake program into solana_program (#17906)

* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt

* Run cargo fmt post cherry-pick
 2021-06-15 23:43:22 +02:00
mergify[bot]
cdd2a51f1f remove unused parameters from CalculateHashIntermediate (#17949) (#17976) 
(cherry picked from commit 4d8ffdcc11)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-15 19:15:07 +00:00
mergify[bot]
0dbe3434f0 refactor so hash verify can be done by more callers (#17941) (#17947) 
Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-15 16:49:32 +00:00
mergify[bot]
ef205593c5 removes port-based forwarding logic from turbine retransmit (#17716) (#17973) 
Turbine retransmit logic is based on which socket it received the packet
from (i.e `packet.meta.forward`):
https://github.com/solana-labs/solana/blob/708bbcb00/core/src/retransmit_stage.rs#L467-L470

This can leave the cluster vulnerable to spoofing and selective
propagation of packets; see
https://github.com/solana-labs/solana/issues/6672
https://github.com/solana-labs/solana/pull/7774

This commit identifies if the node is on the "critical path" based on
its index in the shuffled cluster. If so, it forwards the packet to both
neighbors and children; otherwise, the packet is only forwarded to the
children.

The metrics added in
https://github.com/solana-labs/solana/pull/17351
shows that the number of times the index does not match the port is very
rare, and therefore this change should be safe.

(cherry picked from commit 161838655c)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-06-15 15:16:20 +00:00
mergify[bot]
8b5ba771ad name arguments to help with confusion (#17942) (#17948) 
(cherry picked from commit 7fde9b6ff0)

    # Conflicts:
    #       runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
 2021-06-15 14:46:07 +00:00
mergify[bot]
991f99b239 Bump spl-token to v3.1.1 (backport #17951) (#17958) 
* Bump spl-token to v3.1.1 (#17951)

(cherry picked from commit b7de369992)

# Conflicts:
#	Cargo.lock
#	account-decoder/Cargo.toml
#	accounts-cluster-bench/Cargo.toml
#	programs/bpf/Cargo.lock
#	rpc/Cargo.toml
#	tokens/Cargo.toml

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-15 08:29:54 +00:00
mergify[bot]
d6f17517cb calculate_capitalization uses hash calculation (#17443) (#17932) 
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-15 05:03:06 +00:00
mergify[bot]
15b2f280e3 use trait to simplify and consolidate cumulative code (#17852) (#17931) 
(cherry picked from commit 2dc6969858)

    # Conflicts:
    #       runtime/src/accounts_hash.rs

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
 2021-06-15 04:29:13 +00:00
mergify[bot]
60b43a1ddf Pass iterator for SortedStorages::new_with_slots (#17811) (#17847) 
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit 2a9b127029)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-15 00:01:53 +00:00
mergify[bot]
d6b83e3b0a add metrics to handle_snapshot_requests (#17937) (#17945) 
(cherry picked from commit e6bbd4b3f0)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-14 22:56:14 +00:00
mergify[bot]
0446f89d22 binary search in slices for hashing (#17755) (#17804) 
(cherry picked from commit 817d48be21)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-14 17:32:05 -05:00
mergify[bot]
54bc3e606e Fix typo in docs (#17920) (#17938) 
(cherry picked from commit 3657469826)

Co-authored-by: Sarat Limawongpranee <innneang@users.noreply.github.com>
 2021-06-14 12:04:02 -06:00
mergify[bot]
fed90cfbe8 Fix accounts index panic in purge_exact (#17757) (#17935) 
(cherry picked from commit c2191d885d)

Co-authored-by: sakridge <sakridge@gmail.com>
 2021-06-14 17:41:59 +00:00
mergify[bot]
e2e41a29eb Don't use pinned memory when unnecessary (#17832) (#17934) 
Reports of excessive GPU memory usage and errors
from cudaHostRegister. There are some cases where pinning is
not required.

(cherry picked from commit eeee75c5be)

Co-authored-by: sakridge <sakridge@gmail.com>
 2021-06-14 16:30:51 +00:00
mergify[bot]
3b813db42f minor fixes to punctuation and typos (#17881) (#17923) 
* fix minor typos and punctuation

* fix minor typos and punctuation

* rewording for clarity and typo corrections

* rewording for clarity and typo corrections

* rewording for clarity and typo corrections

Co-authored-by: Gregg Dourgarian <greggd@aidacreative.com>
(cherry picked from commit 54155f875a)

Co-authored-by: Haik Dulgarian <greggd@tempworks.com>
 2021-06-14 06:56:39 +00:00
mergify[bot]
16b1a4d003 short cuts expiration check if origin's contact-info is still valid (#17918) (#17921) 
Crds::find_old_labels can skip checking values timestamps if the
origin's contact info hasn't expired yet:
https://github.com/solana-labs/solana/blob/985280ec0/gossip/src/crds.rs#L394-L408

(cherry picked from commit cca46308bc)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-06-13 21:12:24 +00:00
mergify[bot]
b51ea3ca0c excludes epoch-slots from nodes with unknown or different shred version (#17899) (#17916) 
Inspecting TDS gossip table shows that crds values of nodes with
different shred-versions are creeping in. Their epoch-slots are
accumulated in ClusterSlots causing bogus slots very far from current
root which are not purged and so cause ClusterSlots keep consuming more
memory:
https://github.com/solana-labs/solana/issues/17789
https://github.com/solana-labs/solana/issues/14366#issuecomment-769896036
https://github.com/solana-labs/solana/issues/14366#issuecomment-832754654

This commit updates ClusterInfo::get_epoch_slots, and discards entries
from nodes with unknown or different shred-version.

Follow up commits will patch gossip not to waste bandwidth and memory
over crds values of nodes with different shred-version.

(cherry picked from commit 985280ec0b)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
 2021-06-13 15:56:05 +00:00
mergify[bot]
dc76675644 Don't require non-existent param for keygen new (#17896) (#17897) 
(cherry picked from commit 22c356d24c)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2021-06-11 22:08:46 +00:00
mergify[bot]
274a238a00 Port unconfirmed duplicate tracking logic from ProgressMap to ForkChoice (#17779) (#17889) 
(cherry picked from commit c8535be0e1)

Co-authored-by: carllin <carl@solana.com>
 2021-06-11 11:41:59 +00:00
mergify[bot]
10507f0ade Account for duplicate before a bank is frozen or replayed (#17866) (#17883) 
(cherry picked from commit afafa624a3)

Co-authored-by: carllin <carl@solana.com>
 2021-06-11 07:06:42 +00:00
mergify[bot]
af2a6106da Check for undefined symbols in .so and warn about run-time errors (#17850) (#17880) 
(cherry picked from commit c684e2bdc0)

Co-authored-by: Dmitri Makarov <dmakarov@users.noreply.github.com>
 2021-06-11 00:03:17 +00:00
mergify[bot]
120a7e433f Warn about InstructionError meta (#17864) (#17878) 
(cherry picked from commit fa6bdd2d12)

Co-authored-by: Jack May <jack@solana.com>
 2021-06-10 23:49:03 +00:00
Trent Nelson
98e34f07df programs/config: Disallow duplicate signers 2021-06-10 06:54:18 +00:00
mergify[bot]
738df79394 Add local cluster tests that broadcast duplicate slots (#13995) (#17863) 
* Add duplicate node local cluster test

* fix clippy

* remove dupe test

(cherry picked from commit 050bb5446d)

Co-authored-by: Justin Starry <justin@solana.com>
 2021-06-10 03:17:44 +00:00
mergify[bot]
98e9b6e70b refactor: create type (#17818) (#17848) 
(cherry picked from commit 576e3d95f7)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-09 08:38:04 +00:00
mergify[bot]
c9318f8dc2 Wrap long lines (#17842) 
(cherry picked from commit e5e7390d44)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-06-09 08:10:34 +00:00
mergify[bot]
78f3606e30 test: simple CalculateHashIntermediate refactoring (#17813) (#17846) 
(cherry picked from commit 01ef2a5c4a)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
 2021-06-08 21:47:40 +00:00
mergify[bot]
97f4d098e1 Add more info about how to safely change the identity of a staked validator (#17843) 
(cherry picked from commit 193813d49a)

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-06-08 19:25:41 +00:00