gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
13,651 Commits 81 Branches 357 Tags
4e5ef6bce24b69a08bae937103d06f153938d28c
Commit Graph

511 Commits

Author SHA1 Message Date
carllin
4e5ef6bce2 Add cluster state verifier logging (#16330) 
* Add cluster state verifier logging

* Add duplicate-slots iterator to ledger tool
 2021-04-02 21:48:44 -07:00
Tyera Eulberg
da27acabcc Rpc: enable getConfirmedSignaturesForAddress2 to return confirmed (not yet finalized) data (#16281) 
* Update blockstore method to allow return of unfinalized signature

* Support confirmed sigs in getConfirmedSignaturesForAddress2

* Add deprecated comments

* Update docs

* Enable confirmed transaction-history in cli

* Return real confirmation_status; fill in not-yet-finalized block time if possible
 2021-04-01 04:35:57 +00:00
Tyera Eulberg
18bd47dbe1 Rpc: fix getConfirmedTransaction slot (#16288) 
* Fix transaction blockstore apis

* Update blockstore apis in rpc
 2021-03-31 21:04:00 -06:00
behzad nouri
3f63ed9a72 removes OrderedIterator and transaction batch iteration order (#16153) 
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817

However load_and_execute_transactions is iterating over
  lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.

If iteratorion_order is [1, 2, 0], and i is 0, then:
  lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
  transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).

This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
 2021-03-31 23:59:19 +00:00
Tyera Eulberg
433f1ead1c Rpc: enable getConfirmedBlock and getConfirmedTransaction to return confirmed (not yet finalized) data (#16142) 
* Add Blockstore block and tx apis that allow unrooted responses

* Add TransactionStatusMessage, and send on bank freeze; also refactor TransactionStatusSender

* Track highest slot with tx-status writes complete

* Rename and unpub fn

* Add commitment to GetConfirmed input configs

* Support confirmed blocks in getConfirmedBlock

* Support confirmed txs in getConfirmedTransaction

* Update sigs-for-addr2 comment

* Enable confirmed block in cli

* Enable confirmed transaction in cli

* Review comments

* Rename blockstore method
 2021-03-26 16:47:35 -06:00
Jeff Washington (jwash)
4f4cffbd03 Throttle PoH ticks by cumulative slot time (#16139) 
* Throttle PoH ticks by cumulative slot time

* respond to pr feedback

* saturating sub

* updated comment
 2021-03-26 18:54:16 +00:00
Justin Starry
e7fd7d46cf rpc: add getSlotLeaders method (#16057) 2021-03-23 17:48:54 +00:00
behzad nouri
4f82b897bc buffers data shreds to make larger erasure coded sets (#15849) 
Broadcast stage batches up to 8 entries:
https://github.com/solana-labs/solana/blob/79280b304/core/src/broadcast_stage/broadcast_utils.rs#L26-L29
which will be serialized into some number of shreds and chunked into FEC
sets of at most 32 shreds each:
https://github.com/solana-labs/solana/blob/79280b304/ledger/src/shred.rs#L576-L597
So depending on the size of entries, FEC sets can be small, which may
aggravate loss rate.
For example 16 FEC sets of 2:2 data/code shreds each have higher loss
rate than one 32:32 set.

This commit broadcasts data shreds immediately, but also buffers them
until it has a batch of 32 data shreds, at which point 32 coding shreds
are generated and broadcasted.
 2021-03-23 14:52:38 +00:00
carllin
d76ad33597 Handle blockstore insert dup checks (#16051) 2021-03-22 16:18:22 -07:00
Tyera Eulberg
654449ce91 Bump tokio to 1.1 (#15926) 2021-03-16 21:32:50 +00:00
behzad nouri
3b85cbc504 renames is_last_in_fec_set back to is_last_data (#15848) 
https://github.com/solana-labs/solana/pull/10095
renamed is_last_data to is_last_in_fec_set. However, the code shows that
this is actually meant to indicate where the serialized data is
complete:
https://github.com/solana-labs/solana/blob/420174d3d/ledger/src/shred.rs#L599-L600
https://github.com/solana-labs/solana/blob/420174d3d/ledger/src/shred.rs#L229-L231

There are multiple FEC sets for each `&[Entry]` serialized and this flag
does not represent shreds last in FEC sets (only the very last one by
overlap). So the name is wrong and confusing
 2021-03-16 10:09:16 +00:00
Michael Vines
a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Michael Vines
0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Justin Starry
918d04e3f0 Add more slot update notifications (#15734) 
* Add more slot update notifications

* fix merge

* Address feedback and add integration test

* switch to datapoint

* remove unused shred method

* fix clippy

* new thread for rpc completed slots

* remove extra constant

* fixes

* rely on channel closing

* fix check
 2021-03-12 21:44:06 +08:00
Trent Nelson
24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
Jeff Washington (jwash)
1135ffd595 mut data refs as slice (#15782) 2021-03-10 15:28:03 -06:00
Jeff Washington (jwash)
8a3135d17b Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
Jeff Washington (jwash)
9448540ea6 log timing after ledger process (#15716) 2021-03-05 11:01:52 -06:00
Tyera Eulberg
7e65289729 Convert blockstore TransactionStatus column family to protobufs (#15733) 
* Prevent panic if TransactionStatus can't be deserialized

* Convert Blockstore TransactionStatus column to protobuf

* Add compatability test
 2021-03-05 09:05:35 -07:00
Jeff Washington (jwash)
be35c1c1b7 add execute detail timings (#15638) 2021-03-03 17:07:45 -06:00
sakridge
f1223fb783 Lower blockstore processor error severity (#15578) 2021-03-01 14:57:37 -08:00
Trent Nelson
ef63ffce7d Bump ed25519-dalek to 1.0.1 2021-03-01 21:10:46 +00:00
Michael Vines
33eaa2b238 Sort forks in "ledger processed..." log message 2021-02-28 17:32:50 -08:00
Michael Vines
e92cff3efa create-snapshot subcommad now accepts the ROOT keyword 2021-02-26 16:40:10 -08:00
carllin
c2e8814dce Add limit and shrink policy for recycler (#15320) 2021-02-24 00:15:58 -08:00
Michael Vines
5df36aec7d Pacify clippy 2021-02-19 20:08:41 -08:00
Tyera Eulberg
170cb792eb Return blockstore error if previous_blockhash cannot be determined (#15382) 
* Return blockstore error if previous_blockhash cannot be determined

* Add require_previous_blockshash flag
 2021-02-18 01:04:52 +00:00
Trent Nelson
7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Michael Vines
9ba69a7381 Adapt to fs_extra 1.2.0 2021-02-17 12:44:58 -08:00
Tyera Eulberg
98e3e570d2 Add --force arg for bigtable upload 2021-02-16 17:24:07 -08:00
behzad nouri
e403aeaf05 adds an inverted index to leader schedule (#15249) 
next_leader_slot is doing a linear search for slots in which a pubkey is
the leader:
https://github.com/solana-labs/solana/blob/e59a24d9f/ledger/src/leader_schedule_cache.rs#L123-L157
This can be done more efficiently by adding an inverted index to leader
schedule.
 2021-02-15 00:52:52 +00:00
Tyera Eulberg
0812931c38 Log if unsanitary transactions are read from blockstore (#15319) 2021-02-14 06:32:43 +00:00
Tyera Eulberg
da6753b8c0 Warp timestamp and extend max-allowable-drift for accommodate slow blocks (#15204) 
* Remove timestamp_correction feature gating

* Remove timestamp_bounding feature gating

* Remove unused deprecated ledger code

* Remove unused deprecated unbounded-timestamp code

* Enable independent adjustment of fast/slow timestamp bounding

* Update timestamp bounds to 25% fast, 80% slow; warp timestamp

* Update bank hash test

* Add PR number to feature

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-02-09 15:49:00 -07:00
sakridge
bbae23358c ledger-tool cleanup and additions (#15179) 
* Plumb allow-dead-slots to ledger-tool verify

* ledger-tool cleanup and add some useful missing args

Print root slots and how many unrooted past last root.
 2021-02-06 17:26:42 -08:00
sakridge
774416a546 Ledger purge printing (#15176) 2021-02-06 15:45:08 -08:00
Ryo Onodera
c69027e5aa chore: bump serde from 1.0.118 to 1.0.122 (#15126) 
* chore: bump serde from 1.0.118 to 1.0.122

* Update programs/bpf/Cargo.lock
 2021-02-05 19:23:24 +09:00
Tyera Eulberg
d1563f0ccd Bump tonic, prost, tarpc, tokio (#15013) 
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
 2021-02-05 00:21:53 -07:00
Michael Vines
42bf6dc2ab Revert hard nofile limit back to 500000 2021-02-03 19:43:55 +00:00
Ryo Onodera
8376781ec8 Don't squash caught errors, please (#15046) 
* Don't squash caught errors, please

* Update blockstore.rs

* Update blockstore.rs
 2021-02-03 14:42:34 +00:00
Trent Nelson
894b412aef docs: bump nofiles recommendations to match maps 2021-02-02 21:59:21 -07:00
dependabot[bot]
1df93fa2be chore: bump serde from 1.0.112 to 1.0.118 (#14828) 
* chore: bump serde from 1.0.112 to 1.0.122

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.122.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.122)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update frozen_abi digest following serde update

* Revert "chore: bump serde from 1.0.112 to 1.0.122"

This reverts commit a3ef4442a4.

* Revert "[auto-commit] Update all Cargo lock files"

This reverts commit c41c3b005f.

* chore: bump serde from 1.0.112 to 1.0.118

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.118)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Remove serum-dex pinning

* blind commit!

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2021-02-02 23:28:16 +09:00
Tyera Eulberg
cbb8b79a60 Add validator flag to opt in to cpi and logs storage (#14922) 
* Add validator flag to opt in to cpi and logs storage

* Default TestValidator to opt-in; allow using in multinode-demo

* No clone

Co-authored-by: Carl Lin <carl@solana.com>
 2021-02-01 14:00:51 -07:00
Michael Vines
cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
sakridge
191193289f Upgrade sha2 to 0.9.3 (#14746) 2021-01-22 22:25:22 -08:00
Josh
1de6d28eaf Add block_time to getConfirmedSignaturesForAddress2 and getConfirmedTransaction (#14572) 
* add block_time to get_confirmed_signatures_for_address2 and protobuf implementation for tx_by_addr

* add tests for convert

* update cargo lock

* run cargo format after rebase

* introduce legacy TransactionByAddrInfo

* move LegacyTransactionByAddrInfo back to storage-bigtable
 2021-01-20 22:10:35 -08:00
dependabot[bot]
4ecc0f4fa6 chore: bump rayon from 1.4.1 to 1.5.0 (#14177) 
* chore: bump rayon from 1.4.1 to 1.5.0

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.4.1...rayon-core-v1.5.0)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-01-15 18:07:35 +09:00
sakridge
907f518f6d Add load/execute/store timings (#14561) 2021-01-14 14:14:16 -08:00
Ryo Onodera
e95ebcf864 Don't stop to find newer cluster-confirmed roots (#14557) 
* Don't stop to find newer cluster-confirmed roots

* Fix and add new tests

* nits
 2021-01-13 15:47:06 +00:00
carllin
6dfad0652f Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
sakridge
d8105bb7d7 Add rocskdb high priority threads (#14515) 
Without them, memtable writes can stall on compactions.
 2021-01-11 13:32:02 -08:00