Commit Graph

14702 Commits

Author SHA1 Message Date
d06ca605cf Bump jsonrpc crates (backport #17797) (#17805)
* Bump jsonrpc crates (#17797)

* Bump jsonrpc crates

* Update error text

* gitignore for rpc

(cherry picked from commit 2e998ed11d)

# Conflicts:
#	Cargo.lock
#	validator/Cargo.toml

* Fix conflicts

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-07 20:15:23 +00:00
334e11e4b9 rework hash calculation to not keep slot and write version (#17685) (#17794)
* rework hash calculation to not keep slot and write version

* refactor functions and add tests

* always use multiple slot code path

(cherry picked from commit b5bb91b50f)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-07 15:31:18 +00:00
fd68b4e7a8 Support out of band dumping of unrooted slots in AccountsDb (#17269) (#17777)
* Accounts dumping logic

* Add test for interaction between cache flush and remove_unrooted_slot()

* Update comments

* Rename

* renaming

* Add more comments

* Renaming

* Fixup test and bad check

(cherry picked from commit bbcdf073ba)

Co-authored-by: carllin <carl@solana.com>
2021-06-07 09:37:55 +00:00
e5ea16fad8 system-program: Remove zero lamport check on transfers (#17726) (#17764)
* system-program: Move lamports == 0 check on transfers

* Address feedback

* Update stake split to explicitly allocate + assign

* Update stake tests referring to split instruction

* Revert whitespace

* Update split instruction index in test

* Remove unnecessary `assign_with_seed` from `split_with_seed`

* Fix stake instruction parser

* Update test to allow splitting into account with lamports

(cherry picked from commit 8f5e773caf)

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-06-06 18:27:29 +00:00
6d5a4b5cce Document ProgramTest::new and fix ProgramTest::add_program (#17754) (#17768)
* document ProgramTest::new

* simplify ProgramTest::new doc-string

* make ProgramTest::add_program noisier

`add_program` (and `new`, implicitly) now prints a warning when the user
supplies a bogus program name to a ProgramTest and invokes `test-bpf`.

Additionally, it is now impossible to ask for a regular `test` and for
the generated ProgramTest to load BPF code instead of native code.
Previously, this was caused by a precedence issue: BPF code would always
be preferred over native if the program name was valid, regardless of
user choice.

(cherry picked from commit 2aaf55795f)

Co-authored-by: xuoe <alex@psi.io>
2021-06-06 07:03:13 +00:00
ffb6b5a23b add data point for cap mismatch (#17746) (#17751)
(cherry picked from commit f6fb8906c7)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-04 20:39:46 +00:00
e247625025 Create solana-poh and move remaining rpc modules to solana-rpc (backport #17698) (#17745)
* Create solana-poh and move remaining rpc modules to solana-rpc (#17698)

* Create solana-poh crate

* Move BigTableUploadService to solana-ledger

* Add solana-rpc to workspace

* Move dependencies to solana-rpc

* Move remaining rpc modules to solana-rpc

* Single use statement solana-poh

* Single use statement solana-rpc

(cherry picked from commit 544b3c0d17)

# Conflicts:
#	Cargo.lock
#	banking-bench/Cargo.toml
#	core/Cargo.toml
#	core/benches/banking_stage.rs
#	local-cluster/Cargo.toml
#	rpc/Cargo.toml
#	stake-monitor/Cargo.toml
#	validator/Cargo.toml

* Fix conflicts & versions

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-04 18:19:08 +00:00
67c07aa5d3 increase bin count for hash scan (#17562) (#17747)
(cherry picked from commit 5197454fea)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-04 18:12:48 +00:00
0de1ce0c2c adds fallback logic if retransmit multicast fails (#17714) (#17742)
In retransmit-stage, based on the packet.meta.seed and resulting
children/neighbors, each packet is sent to a different set of peers:
https://github.com/solana-labs/solana/blob/708bbcb00/core/src/retransmit_stage.rs#L421-L457

However, current code errors out as soon as a multicast call fails,
which will skip all the remaining packets:
https://github.com/solana-labs/solana/blob/708bbcb00/core/src/retransmit_stage.rs#L467-L470

This can exacerbate packets loss in turbine.

This commit:
  * keeps iterating over retransmit packets for loop even if some
    intermediate sends fail.
  * adds a fallback to UdpSocket::send_to if multicast fails.

Recent discord chat:
https://discord.com/channels/428295358100013066/689412830075551748/849530845052403733

(cherry picked from commit be957f25c9)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-06-04 16:20:44 +00:00
59f4fba05c simplify test construction (#17686) (#17693)
(cherry picked from commit 83ceedc091)

# Conflicts:
#	runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-04 15:57:11 +00:00
c0c764377c excludes caller's crds values from pull responses (#17542) (#17744)
If the crds entry belongs to the caller itself, then the caller will
always have the more recent version of it, regardless of it being
filtered out by the bloom filter or not.

The exception is node-instance types which are meant to detect duplicate
running instances, and those are exempted.

(cherry picked from commit 7cf6e66ddd)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-06-04 15:54:00 +00:00
c9bc059637 writes epoch-slots to crds table synchronously (#17719) (#17743)
epoch-slots may be overwritten before they are written to crds table:
https://github.com/solana-labs/solana/issues/17711

This commit writes new epoch-slots to crds table synchronously with
push_epoch_slots. The functions is still not thread-safe as commented in
the code, however currently only one threads is invoking this code.

(cherry picked from commit 60b0a13444)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2021-06-04 15:34:39 +00:00
78147d48e4 use slots returned from get_snapshot_storages to sort (#17638) (#17695)
* use slots returned from get_snapshot_storages to sort

    * add tests

    (cherry picked from commit 9388aaca15)

    # Conflicts:
    #       runtime/src/accounts_db.rs
    #       runtime/src/sorted_storages.rs

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-04 10:14:37 -05:00
5e7db52087 Avoid full-range compactions with periodic filtered b.g. ones (backport #16697) (#17741)
* Avoid full-range compactions with periodic filtered b.g. ones (#16697)

* Update rocksdb to v0.16.0

* Promote the infrequent and important log to info!

* Force background compaction by ttl without manual compaction

* Fix test

* Support no compaction mode in test_ledger_cleanup_compaction

* Fix comment

* Make compaction_interval customizable

* Avoid major compaction with periodic filtering...

* Adress lazy_static, special cfs and range check

* Clean up a bit and add comment

* Add comment

* More comments...

* Config code cleanup

* Add comment

* Use .conflicts_with()

* Nullify unneeded delete_range ops for special CFs

* Some clean ups

* Clarify the locking intention

* Ensure special CFs' consistency with PurgeType::CompactionFilter

* Fix comment

* Fix bad copy paste

* Fix various types...

* Don't use tuples

* Add a unit test for compaction_filter

* Fix typo...

* Remove flag and just use new behavior always

* Fix wrong condition negation...

* Doc. about no set_last_purged_slot in purge_slots

* Write a test and fix off-by-one bug....

* Apply suggestions from code review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Follow up to github review suggestions

* Fix line-wrapping

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
(cherry picked from commit 1f97b2365f)

# Conflicts:
#	ledger/src/blockstore_db.rs

* Fix conflict

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-06-04 14:38:02 +00:00
ae42413d57 support bin divisions up to 65536 (#17563) (#17692)
* support bin divisions up to 65536

* add tests

(cherry picked from commit db8811eacd)

# Conflicts:
#	runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-04 06:33:00 +00:00
d433bd3d84 remove slot lookup (#17691) (#17694)
(cherry picked from commit 4bd32d891f)

    # Conflicts:
    #       runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2021-06-04 06:09:46 +00:00
58dd6dc227 add info for vm.max_map_count incorrectly set. (#17727) (#17730)
(cherry picked from commit 81bafd9daf)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-04 05:02:13 +00:00
893df9b277 Rename ValidatorExit and move to sdk (#17728) (#17729)
(cherry picked from commit 3a647c4bea)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-06-04 04:38:49 +00:00
17dc13760b sort storages by slot before scan (#17411) (#17724)
* sort storages by slot before scan

* fix return value

(cherry picked from commit ef5169ff24)

# Conflicts:
#	runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-04 03:21:20 +00:00
498bf911eb Add missing ProgramError to InstructionError mappings (#16231) (#17723)
* Add missing ProgramError to InstructionError mappings

* add note

* Clarify process of adding new program error

(cherry picked from commit 83b9a046d1)

Co-authored-by: Jack May <jack@solana.com>
2021-06-04 00:25:25 +00:00
96de58b3a4 reduce copy (#17672) (#17688)
(cherry picked from commit d802eb303c)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-03 22:26:31 +00:00
9b61fa24c7 rework test for flexibility (#17592) (#17681)
* rework test for flexibility

* respond to pr feedback

(cherry picked from commit 886898eabf)

# Conflicts:
#	runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-03 21:25:58 +00:00
e9be3cf6bc implement ancestors as rolling bit field (#17482) (#17673)
(cherry picked from commit eec996ba41)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-03 18:47:37 +00:00
ea44a71914 parallel get_snapshot_storages (#17589) (#17679)
(cherry picked from commit 738cc9549f)

# Conflicts:
#	runtime/src/accounts_db.rs
#	runtime/src/bank.rs
#	runtime/src/serde_snapshot/tests.rs

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-03 18:32:04 +00:00
a00fbbf5ca add metric for collecting storages (#17527) (#17675)
(cherry picked from commit 72bb271a94)

# Conflicts:
#	runtime/src/accounts_db.rs
#	runtime/src/snapshot_utils.rs

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-03 05:34:32 +00:00
eb1a04af65 refactor SnapshotStorage helpers to prepare for later changes (#17560) (#17678)
(cherry picked from commit 654918ab27)

# Conflicts:
#	runtime/src/accounts_db.rs

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-03 05:24:11 +00:00
d1fbf77f3f RollingBitField: bug fixes and add tests (#17525) (#17674)
* RollingBitField: bug fixes and add tests

* respond to feedback

(cherry picked from commit 8924fbf6a0)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-03 05:07:02 +00:00
04fbf73a29 remove antiquated assert (#17643) (#17689)
(cherry picked from commit 07bac27ac7)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-02 21:53:33 +00:00
db70eb3160 minor test code cleanup (#17645) (#17680)
(cherry picked from commit 107af52deb)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-02 20:14:23 +00:00
cd974c26b6 add check_hash to non-index hash calculation (#17558) (#17676)
(cherry picked from commit 55c22d3b76)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-02 20:00:50 +00:00
53e0f5d61c remove unnecessary clone (#17559) (#17677)
(cherry picked from commit 1d02dba06f)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
2021-06-02 19:59:55 +00:00
e864bf4898 Fix CPI recursion depth (#17659) (#17670)
* Fix CPI recursion depth

(cherry picked from commit 80e5b24b38)

Co-authored-by: Jack May <jack@solana.com>
2021-06-02 11:06:50 +00:00
81d12b7644 Fix test instruction count (#17655) (#17660)
(cherry picked from commit 4e0db50e4e)

Co-authored-by: Jack May <jack@solana.com>
2021-06-02 09:20:11 +00:00
309fcd6270 Fix test instruction count (#17655) (#17660)
(cherry picked from commit 4e0db50e4e)

Co-authored-by: Jack May <jack@solana.com>
2021-06-02 08:51:45 +00:00
938112e449 fix getBlock API reference (#17661) (#17662)
(cherry picked from commit 10c6e771b5)

Co-authored-by: oncecsc <74547046+oncecsc@users.noreply.github.com>
2021-06-02 07:32:15 +00:00
3e012ea69e Cli: Decode known program custom errors returned from simulation (#17652) (#17658)
(cherry picked from commit b7f98ea18a)

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-06-02 07:03:46 +00:00
975c942ea7 Bump version to v1.7.1 2021-06-02 05:21:14 +00:00
2798271da0 Add memory operation syscalls (backport #16447) (#17648)
* Add memory operation syscalls (#16447)

(cherry picked from commit 2b50529265)

# Conflicts:
#	programs/bpf/Cargo.lock

* Resolve conflicts

Co-authored-by: Jack May <jack@solana.com>
v1.7.0
2021-06-01 18:43:50 -07:00
dc258cebab Add create-stake command to solana-tokens CLI (#17550) (#17649)
* Add create-stake command to solana-tokens CLI

* Add --unlocked-sol arg

Thanks @CriesofCarrots!

(cherry picked from commit 1b7f8777d6)

Co-authored-by: Greg Fitzgerald <greg@solana.com>
2021-06-02 01:16:48 +00:00
4b8c5194c7 Purge expired BlockHeight data from blockstore (backport #17634) (#17641)
* Purge expired BlockHeight data from blockstore (#17634)

* Purge expired BlockHeight data from blockstore

* Also call compact_storage and add comment....

(cherry picked from commit 96cdbfdcc0)

# Conflicts:
#	ledger/src/blockstore_db.rs

* Fix conflict

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-02 01:04:11 +00:00
3c7c6dacfb Add BPF rustc option to reduce the optimizations to safer level (#17590) (#17594)
(cherry picked from commit 831e87c65d)

Co-authored-by: Dmitri Makarov <dmakarov@users.noreply.github.com>
2021-05-31 19:15:56 +00:00
e36337a764 Make the sys-tuner oneliner actually copy-pastable (#17615) (#17619)
* Make the sys-tuner oneliner actually copy-pastable

* Use `command -v`

(cherry picked from commit 41975016b9)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-05-31 18:13:47 +00:00
a49856b898 Rework #17486 (backport #17566) (#17597)
* Revert "Improve missing default signer error messaging (#17486)"

This reverts commit 6d40d0d141.

(cherry picked from commit ca8c1c6c42)

* Improve missing default filepath signer error messaging

(cherry picked from commit 06a926f2f4)

* CI: temporarily skip spl downstream build

(cherry picked from commit d01b4f80f9)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-05-31 17:28:06 +00:00
8ca2f52041 Make initialize public (#17605) (#17607)
(cherry picked from commit 2896fc3987)

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-05-31 08:51:05 +00:00
2f7f243022 Always bail if program modifies a ro account (backport #17569) (#17584)
* Always bail if program modifies a ro account (#17569)

(cherry picked from commit a3240aebde)

* resolve conflicts

* nudge

Co-authored-by: Jack May <jack@solana.com>
2021-05-28 20:34:10 +00:00
7e443770d7 test-validator: add an arg to control faucet genesis balance (#17582)
(cherry picked from commit 974a96738a)

Co-authored-by: Trent Nelson <trent@solana.com>
2021-05-28 17:50:44 +00:00
8ec09884b8 Revert bpf-tools to version 1.8 because of a codegen bug suspicion (#17568) (#17577)
(cherry picked from commit 2316ddb90a)

Co-authored-by: Dmitri Makarov <dmakarov@users.noreply.github.com>
2021-05-28 11:10:14 +00:00
88c7e636d6 Bump console from 0.11.3 to 0.14.1 (#16301) (#17552)
* Bump console from 0.11.3 to 0.14.1

Bumps [console](https://github.com/mitsuhiko/console) from 0.11.3 to 0.14.1.
- [Release notes](https://github.com/mitsuhiko/console/releases)
- [Changelog](https://github.com/mitsuhiko/console/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/console/compare/0.11.3...0.14.1)

Signed-off-by: dependabot[bot] <support@github.com>

* Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
(cherry picked from commit ec1a307a7c)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-27 21:30:17 +00:00
add3fd479d chore: bump chrono-humanize from 0.1.1 to 0.2.1 (#16895) (#17549)
* chore: bump chrono-humanize from 0.1.1 to 0.2.1

Bumps [chrono-humanize](https://gitlab.com/imp/chrono-humanize-rs) from 0.1.1 to 0.2.1.
- [Release notes](https://gitlab.com/imp/chrono-humanize-rs/tags)
- [Commits](https://gitlab.com/imp/chrono-humanize-rs/compare/0.1.1...0.2.1)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
(cherry picked from commit 4f74c77146)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-27 20:37:42 +00:00
70410536b9 Bump serde_bytes from 0.11.4 to 0.11.5 (#16299) (#17546)
* Bump serde_bytes from 0.11.4 to 0.11.5

Bumps [serde_bytes](https://github.com/serde-rs/bytes) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/serde-rs/bytes/releases)
- [Commits](https://github.com/serde-rs/bytes/compare/0.11.4...0.11.5)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
(cherry picked from commit 6e2ae68643)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-27 20:04:59 +00:00