437 Commits

Author SHA1 Message Date
Michael Vines
f5eb5a3ba6 Bump version to v1.4.13 2020-11-26 06:59:28 +00:00
Tyera Eulberg
fd9e003ae1 Bump version to 1.4.12 (#13792) 2020-11-24 22:53:36 +00:00
Tyera Eulberg
7099d07fd3 Bump version to 1.4.11 (#13780) 2020-11-24 06:30:35 +00:00
mergify[bot]
6dc62bfb77 Fix fragile tests in prep of stake rewrite pr (#13654) (#13721)
* Fix fragile tests in prep of stake rewrite pr

* Restore BOOTSTRAP_VALIDATOR_LAMPORTS where appropriate

* Further clean up

* Further clean up

* Aligh with other call site change

* Remove false warn!

* fix ci!

(cherry picked from commit b74d7b5758)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-20 09:34:46 +00:00
Tyera Eulberg
855cf9a362 Bump version to 1.4.10 (#13657) 2020-11-18 01:29:58 +00:00
mergify[bot]
ea0a3521ed Fix assertion failure (#13626) (#13631)
Co-authored-by: Carl Lin <carl@solana.com>
(cherry picked from commit afc1b59475)

Co-authored-by: carllin <wumu727@gmail.com>
2020-11-17 07:01:10 +00:00
Trent Nelson
b838aba840 Bump version to 1.4.9 2020-11-13 17:50:10 -07:00
mergify[bot]
f6b65b033e Fix overflow in entry tick verification (bp #13572) (#13581)
* Fix overflow in entry hash count verification

(cherry picked from commit d611337394)

* clippy

(cherry picked from commit 01a4889b53)

Co-authored-by: Justin Starry <justin@solana.com>
2020-11-13 16:49:05 +00:00
Ryo Onodera
e14c2f94f4 Bump version to v1.4.8 (#13552) 2020-11-12 17:43:31 +00:00
mergify[bot]
2cb9ca5966 Fix slow/stuck unstaking due to toggling in epoch (#13501) (#13535)
* Fix slow/stuck unstaking due to toggling in epoch

* nits

* nits

* Add stake_program_v2 feature status check to cli

Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit 89b474e192)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-11 22:27:10 +00:00
Tyera Eulberg
c9e646b86b Bump version to v1.4.7 (#13488) 2020-11-10 05:55:26 +00:00
Michael Vines
e738bf1c9a Bump version to v1.4.6 2020-11-07 02:49:14 +00:00
mergify[bot]
b15d826476 Allow feature builtins to overwrite existing builtins (#13403) (#13420)
* Allow feature builtins to overwrite existing builtins

* Add feature_builtin ActivationType

* Correctly retain idempotent for replacing case

* Fix test

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
(cherry picked from commit bc62313c66)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-11-05 16:51:25 +00:00
Michael Vines
5877427389 Bump version to v1.4.5 2020-11-01 17:05:45 +00:00
mergify[bot]
b28d10d46f Add bank timestamp bounding (bp #13120) (#13331)
* Add bounding feature

(cherry picked from commit 96b8aa8bd1)

* Repurpose unused as Clock::epoch_start_timestamp; add gated update

(cherry picked from commit 0049ab69fb)

* Add bounded timestamp-estimation method

(cherry picked from commit 80db6c0980)

* Use bounded timestamp-correction when feature enabled

(cherry picked from commit 90778615f6)

* Prevent block times from ever going backward

(cherry picked from commit eb2560e782)

* Sample votes from ancestors back to root

(cherry picked from commit 4260b3b416)

* Add Clock sysvar details, update struct docs

(cherry picked from commit 3a1e125ce3)

* Add design proposal and update validator-timestamp-oracle

(cherry picked from commit a3912bc084)

* Adapt to feature::create_account

Co-authored-by: Tyera Eulberg <tyera@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2020-11-01 06:02:34 +00:00
mergify[bot]
40a3885d3b Native/builtin programs now receive an InvokeContext (bp #13286) (#13298)
* Native/builtin programs now receive an InvokeContext

(cherry picked from commit df8dab9d2b)

* Remove MessageProcessor::loaders

(cherry picked from commit 2664a1f7ef)

* Remove Entrypoint type

(cherry picked from commit 225bed11c7)

* Remove programs clone()

(cherry picked from commit 33884d847a)

* Add sol_log_compute_units syscall

(cherry picked from commit 66e51a7363)

* Add Bank::set_bpf_compute_budget()

(cherry picked from commit 7d686b72a0)

* Rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-30 07:47:17 +00:00
mergify[bot]
69b3f10207 move Account to solana-sdk (bp #13198) (#13269)
* move Account to solana-sdk (#13198)

(cherry picked from commit c458d4b213)

# Conflicts:
#	programs/bpf/benches/bpf_loader.rs

* resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
2020-10-29 19:16:52 +00:00
mergify[bot]
cf896dbeee Use bank timestamp to populate Blockstore::blocktime_cf when correction active (#13158) (#13160)
(cherry picked from commit 39686ef098)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-26 20:34:15 +00:00
Michael Vines
597618846b Bump version to v1.4.4 2020-10-24 22:10:32 +00:00
Trent Nelson
53b074aa35 Bump version to 1.4.3 2020-10-23 04:20:28 +00:00
mergify[bot]
edfbd8d65a Add replacements for Pubkey::new_rand()/Hash::new_rand() (bp #12987) (#13076)
* Add pubkey_new_rand(), mark Pubkey::new_rand() deprecated

(cherry picked from commit 0e68ed6a8d)

* Add hash_new_rand(), mark Hash::new_rand() as deprecated

(cherry picked from commit 76f11c7dae)

* Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand`

(cherry picked from commit 7bc073defe)

# Conflicts:
#	programs/bpf/benches/bpf_loader.rs
#	runtime/benches/accounts.rs
#	runtime/src/accounts.rs

* Run `codemod --extensions rs Hash::new_rand solana_sdk:#️⃣:new_rand`

(cherry picked from commit 17c391121a)

* Remove unused pubkey::Pubkey imports

(cherry picked from commit 959880db60)

# Conflicts:
#	runtime/src/accounts_index.rs

* Resolve conflicts

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-22 05:08:01 +00:00
mergify[bot]
5236acf4b0 Add ledger-tool dead-slots and improve purge a lot (#13065) (#13071)
* Add ledger-tool dead-slots and improve purge a lot

* Reduce batch size...

* Add --dead-slots-only and fixed purge ordering

(cherry picked from commit 0776fa05c7)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-10-21 18:57:22 +00:00
mergify[bot]
d05bfa08c7 improves threads' utilization in processing gossip packets (#12962) (#13023)
ClusterInfo::process_packets handles incoming packets in a thread_pool:
https://github.com/solana-labs/solana/blob/87311cce7/core/src/cluster_info.rs#L2118-L2134

However, profiling runtime shows that threads are not well utilized and
a lot of the processing is done sequentially.

This commit redistributes the work done in parallel. Testing on a gce
cluster shows 20%+ improvement in processing gossip packets with much
smaller variations.

(cherry picked from commit 75d62ca095)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2020-10-20 19:59:35 +00:00
mergify[bot]
f36cfb92f7 Convert Blockstore Rewards cf to protobuf (bp #12860) (#12935)
* Convert Blockstore Rewards cf to protobuf (#12860)

* Add Blockstore protobuf cf type

* Add Rewards message to proto and make generated pub

* Convert Rewards cf to ProtobufColumn

* Add bench

* Adjust tags

* Move solana proto definitions and conversion methods to new crate

(cherry picked from commit 359707c85e)

# Conflicts:
#	Cargo.lock
#	ledger/Cargo.toml
#	storage-bigtable/Cargo.toml

* v1.4-ify

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-10-16 01:27:37 +00:00
Trent Nelson
f3c986385f Bump version to 1.4.2 2020-10-15 20:26:29 +00:00
Trent Nelson
6e91996606 Bump version to 1.4.1 2020-10-14 03:05:04 +00:00
mergify[bot]
ed06e8b85d Pacify cargo audit (bp #12797) (#12799)
* Bump reqwest/rayon to get past `cargo audit`

(cherry picked from commit 8a119c1483)

# Conflicts:
#	dos/Cargo.toml
#	download-utils/Cargo.toml
#	metrics/Cargo.toml

* Switch to tempfile

(cherry picked from commit d3b0f87a49)

* Rework cargo audit ignores

(cherry picked from commit 2301dcf973)

* Cargo.lock

(cherry picked from commit 859eb606da)

* rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-12 04:29:55 +00:00
mergify[bot]
6b9b107ead Fix various ledger-tool error due to no builtins (bp #12759) (#12766)
* Fix various ledger-tool error due to no builtins (#12759)

* Fix various ledger-tool error due to no builtins

* Add missing file...

(cherry picked from commit 1f4bcf70b0)

# Conflicts:
#	core/Cargo.toml
#	ledger/Cargo.toml

* Rebase

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2020-10-10 07:10:38 +00:00
mergify[bot]
567a1cb944 Correct Bank timestamp drift every slot (#12737) (#12777)
* Move timestamp helper to sdk

* Add Bank method for getting timestamp estimate

* Return sysvar info from Bank::clock

* Add feature-gated timestamp correction

* Rename unix_timestamp method to be more descriptive

* Review comments

* Add timestamp metric

(cherry picked from commit b028c47d2b)

# Conflicts:
#	runtime/src/feature_set.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-10-10 00:42:44 +00:00
mergify[bot]
7a53ca18a6 Store program logs in blockstore / bigtable (TransactionWithStatusMeta) (#12678) (#12735)
* introduce store program logs in blockstore / bigtable

* fix test, transaction logs created for successful transactions

* fix test for legacy bincode implementation around log_messages

* only api nodes should record logs

* truncate transaction logs to 100KB

* refactor log truncate for improved coverage

(cherry picked from commit 8f5431551e)

Co-authored-by: Josh <josh.hundley@gmail.com>
2020-10-08 20:19:26 +00:00
sakridge
71a308affd Use datapoint instead of print for is_full signal (#12673) 2020-10-07 13:36:54 -07:00
sakridge
ce98088457 Track inserted repair shreds (#12455) 2020-09-29 14:13:21 -07:00
Michael Vines
31696a1d72 Port BPFLoader2 activation to FeatureSet and rework built-in program activation 2020-09-28 12:50:19 -07:00
dependabot[bot]
0718501c72 chore: bump rocksdb from 0.14.0 to 0.15.0
Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.14.0...v0.15.0)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-28 09:52:31 -07:00
Michael Vines
c10da16d7b Port instructions sysvar and secp256k1 program activation to FeatureSet 2020-09-25 11:40:36 -07:00
Justin Starry
6601ec8f26 Record and store invoked instructions in transaction meta (#12311)
* Record invoked instructions and store in transaction meta

* Enable cpi recording if transaction sender is some

* Rename invoked to innerInstructions
2020-09-24 22:36:22 +08:00
Justin Starry
731a943239 Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
sakridge
68e5a2ef56 Add RPC notify and banking keys debug (#12396) 2020-09-23 18:46:42 -07:00
Josh
65a6bfad09 Add blockstore column to store performance sampling data (#12251)
* Add blockstore column to store performance sampling data

* introduce timer and write performance metrics to blockstore

* introduce getRecentPerformanceSamples rpc

* only run on rpc nodes enabled with transaction history

* add unit tests for get_recent_performance_samples

* remove RpcResponse from rpc call

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* Add PerfSamples to purge/compaction

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* remove duplicate constants

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-09-22 12:26:32 -07:00
Ryo Onodera
cb8661bd49 Persistent tower (#10718)
* Save/restore Tower

* Avoid unwrap()

* Rebase cleanups

* Forcibly pass test

* Correct reconcilation of votes after validator resume

* d b g

* Add more tests

* fsync and fix test

* Add test

* Fix fmt

* Debug

* Fix tests...

* save

* Clarify error message and code cleaning around it

* Move most of code out of tower save hot codepath

* Proper comment for the lack of fsync on tower

* Clean up

* Clean up

* Simpler type alias

* Manage tower-restored ancestor slots without banks

* Add comment

* Extract long code blocks...

* Add comment

* Simplify returned tuple...

* Tweak too aggresive log

* Fix typo...

* Add test

* Update comment

* Improve test to require non-empty stray restored slots

* Measure tower save and dump all tower contents

* Log adjust and add threshold related assertions

* cleanup adjust

* Properly lower stray restored slots priority...

* Rust fmt

* Fix test....

* Clarify comments a bit and add TowerError::TooNew

* Further clean-up arround TowerError

* Truly create ancestors by excluding last vote slot

* Add comment for stray_restored_slots

* Add comment for stray_restored_slots

* Use BTreeSet

* Consider root_slot into post-replay adjustment

* Tweak logging

* Add test for stray_restored_ancestors

* Reorder some code

* Better names for unit tests

* Add frozen_abi to SavedTower

* Fold long lines

* Tweak stray ancestors and too old slot history

* Re-adjust error conditon of too old slot history

* Test normal ancestors is checked before stray ones

* Fix conflict, update tests, adjust behavior a bit

* Fix test

* Address review comments

* Last touch!

* Immediately after creating cleaning pr

* Revert stray slots

* Revert comment...

* Report error as metrics

* Revert not to panic! and ignore unfixable test...

* Normalize lockouts.root_slot more strictly

* Add comments for panic! and more assertions

* Proper initialize root without vote account

* Clarify code and comments based on review feedback

* Fix rebase

* Further simplify based on assured tower root

* Reorder code for more readability

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-09-19 14:03:54 +09:00
carllin
3533e11786 Fix blockstore processor squash (#12319) 2020-09-17 21:33:08 -07:00
sakridge
3930cb865a Add keccak-secp256k1 instruction (#11839)
* Implement keccak-secp256k1 instruction

Verifies eth addreses with ecrecover function

* Move secp256k1 test
2020-09-15 18:23:21 -07:00
Ryo Onodera
de4a613610 Check bank capitalization (#11927)
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
Tyera Eulberg
05db41fe9c Cache block time in Blockstore (#11955)
* Add blockstore column to cache block times

* Add method to cache block time

* Add service to cache block time

* Update rpc getBlockTime to use new method, and refactor blockstore slightly

* Return block_time with confirmed block, if available

* Add measure and warning to cache-block-time
2020-09-09 09:33:14 -06:00
Ryo Onodera
c274e26eb8 Prevent unbound memory growth by blockstore_processor (#12110)
* Prevent unbound memory growth by blockstore_processor

* Promote log to info! considering infrequency

* Exclude the time of freeing from interval...

* Skip not-shrinkable slots even if forced

* Add comment
2020-09-09 12:05:10 +09:00
Ryo Onodera
53b8ea4464 Rename to ClusterType and restore devnet compat. (#12068)
* Rename to ClusterType and restore devnet compat.

* De-duplicate parse code and add comments

* Adjust default Devnet genesis & reduce it in tests
2020-09-08 23:55:09 +09:00
Michael Vines
cd3c134b58 Remove dead code 2020-09-04 16:01:49 -07:00
Michael Vines
d3611f74c8 Add exit flag for bigtable upload operations 2020-09-04 16:01:49 -07:00
Michael Vines
bafdcf24f5 Add BigTableUploadService 2020-09-04 16:01:49 -07:00
Michael Vines
bc7731b969 Add BigTableUploadService 2020-09-04 16:01:49 -07:00