2355 Commits

Author SHA1 Message Date
81f4fd56c7 Log the repairee pubkey when unable to serve them (#5222)
automerge
2019-07-22 14:13:29 -07:00
0d8f5379a0 Add time units 2019-07-22 08:22:21 -07:00
a07b17b9b5 Drop older slots in the ledger (#5188)
* Add facility to delete blocktree columns in range

* Add ledger cleanup service

* Add local_cluster test
2019-07-20 13:13:55 -07:00
3f54c0f1a6 Update struct order arbitrarily to match rpc output (#5197) 2019-07-19 11:45:04 -06:00
8b69998379 Lower recovery messages (#5181) 2019-07-19 09:20:14 -07:00
adfb8ff2a1 Add getEpochInfo() and getLeaderSchedule() RPC methods (#5189)
* Add getLeaderSchedule() RPC method

* Add getEpochInfo() RPC method

* Add JSON RPC docs
2019-07-19 07:31:18 -07:00
afa05acb32 more replay_stage grooming (#5163) 2019-07-18 14:54:27 -07:00
a3a91ba222 Fix misleading variable name (#5176)
automerge
2019-07-18 14:07:32 -07:00
751b54b60b Skip sleeping in replay stage if a bank was recently processed (#5161)
* Skip sleeping in replay stage if a bank was recently processed

* Remove return
2019-07-18 12:04:53 -07:00
6ad9dc18d8 Add ability to prune ledger (#5128)
* Add utility to prune the ledger

* Add tests

* Fix clippy

* Fix off by one

* Rework to force delete every column

* Minor fixup
2019-07-17 14:42:29 -07:00
027ebb6670 no more OUT_DIR (#5139)
* no more OUT_DIR

* no more OUT_DIR

* more information about failure
2019-07-17 14:27:58 -07:00
0ffd91df27 groom poh_recorder (#5127)
* groom poh_recorder

* fixup

* nits

* slot() from the outside means "the slot the recorder is working on"

* remove redundant check

* review comments, put next_tick back in the "is reset" check

* remove redundant check
2019-07-17 14:10:15 -07:00
10d85f8366 Add weighted shuffle support for values upto u64::MAX (#5151)
automerge
2019-07-17 12:44:28 -07:00
f88c72c41e stress tweaks (#5140) 2019-07-16 22:04:40 -07:00
bf5bce50a4 Fix stake pruning test (#5124) 2019-07-16 13:20:03 -04:00
fe87c05423 fix transaction_count (#5110)
* fix transaction_count

* add sig count to bank hash
2019-07-15 13:42:59 -07:00
40faaef9da Revert "Logging (#5017)" (#5096)
This reverts commit b50a3bae72.
2019-07-14 18:48:15 -07:00
0d27515d09 tmp dirs target to farf (#5079) 2019-07-12 18:28:42 -07:00
1c966aac25 Facility to generate a blocktree prune list using ledger tool (#5041)
automerge
2019-07-12 16:58:13 -07:00
7aecb87bce Add a version field to blobs (#5057) 2019-07-12 13:43:19 -07:00
ebcdc06dc3 Restore ledger-tool print and json commands (#5048)
* Restore ledger-tool print and  json commands

* Remove obsolete read_ledger()
2019-07-11 20:33:36 -07:00
0a36a78133 Fix replicator segment selection (#5046) 2019-07-11 18:31:41 -07:00
5698d48dc8 merkle-tree: Make instantiation a little less painful (#5037)
automerge
2019-07-11 15:15:08 -07:00
4c90898f0b Dynamic erasure set configuration (#5018)
* Use local erasure session to create/broadcast coding blobs

* Individual session for each recovery (as the config might be different)

* address review comments

* new constructors for session and coding generator

* unit test for dynamic erasure config
2019-07-11 13:58:33 -07:00
b2c776eabc Fix getProgramAccounts RPC (#5024)
* Use scan_accounts to load accounts by program_id

* Add bank test

* Use get_program_accounts in RPC
2019-07-11 12:58:28 -06:00
e4926e4110 Set exit when replicator run exits (#5016) 2019-07-10 16:27:18 -07:00
b50a3bae72 Logging (#5017)
* Add logging to replay_stage

* locktower logging
2019-07-10 15:52:31 -07:00
35ec7a5156 Decouple turns from segments in PoRep (#5004)
* Decouple Segments from Turns in Storage

* Get replicator local cluster tests running in a reasonable amount of time

* Fix unused imports

* Document new RPC APIs

* Check for exit while polling
2019-07-10 13:33:29 -07:00
a383ea532f Implement new Index Column (#4827)
* Implement new Index Column

* Correct slicing of blobs

* Mark coding blobs as coding when they're recovered

* Prevent broadcast stages from mixing coding and data blobs in blocktree

* Mark recovered blobs as present in the index

* Fix indexing error in recovery

* Fix broken tests, and some bug fixes

* increase min stack size for coverage runs
2019-07-10 11:08:17 -07:00
f537482c86 remove set_leader from cluster_info (#4998) 2019-07-09 22:06:47 -07:00
b8e7736af2 Move SLOTS_PER_SEGMENT to genesis (#4992)
automerge
2019-07-09 16:48:40 -07:00
f777a1a74c groom replay_stage and poh_recorder (#4961)
* groom replay_stage and poh_recorder

* fixup

* fixup

* don't freeze() parent, need to review bank_forks and maybe vote...
2019-07-09 15:36:30 -07:00
22ef3c7c54 Blob verify (#4951)
* Ensure signable data is not out of range

* Add a broadcast stage that puts bad sizes in blobs

* Resign blob after modifyign size

* Remove assertions that fail when size != meta.size
2019-07-08 18:21:52 -07:00
417e8d5064 fix blocktree_processor test_process_entries_stress (#4967) 2019-07-08 18:11:58 -07:00
1feb9bea21 Harden Merkle Tree against second pre-image attacks (#4925)
* merkle-tree: Harden against second pre-image attacks

* core/chacha: Bump test golden hash
2019-07-08 19:00:06 -06:00
c5ba2e0883 bank_forks test stability (#4959)
automerge
2019-07-08 15:55:49 -07:00
eb4edd75e6 make bank commit_credits non public (#4944)
* make bank commit_credits non pub

* track down create() failure

* move bank_client to process_transaction(), which commits credits
2019-07-08 15:37:54 -07:00
bb6bcd79c0 Handle replicator errors without panicking (#4955)
* Handle replicator errors without panicking

* Unwelcome println
2019-07-08 12:43:35 -07:00
ef7022d638 Refactor replicators to not block on startup (#4932)
* Refactor replicators to not block on startup

* Ignore setup failure
2019-07-08 10:17:25 -07:00
2aac094f63 Ensure blobs are deserializable without unwrapping (#4948)
* Return result from deserializing blobs in blocktree instead of assuming deserialization will succeed

* Mark bad deserialization as dead fork

* Add test for corrupted blobs in blocktree and replay_stage
2019-07-07 14:37:12 -07:00
2ec5d2c7f5 start local_cluster nodes from genesis blocks (#4928) 2019-07-03 14:03:52 -07:00
12ef0c25b5 change vote commission to u8 (from u32) (#4887)
automerge
2019-07-02 14:18:11 -07:00
3615445a12 Broadcast run for injecting fake blobs in turbine (#4889)
* Broadcast run for injecting fake blobs in turbine

* address review comments

* new local cluster test that uses fake blob broadcast

* added a test to make sure tvu_peers ordering is guaranteed
2019-07-01 17:54:03 -07:00
65adce65fa Always send pull responses to the origin addr (#4894) 2019-07-01 16:49:05 -07:00
c1953dca8f Cleanup some of banking stage (#4878)
* Add committable transactions that cause errors like InstructionErrors back to retryable list on MaxHeightReached

* Remove unnecessary logic

* Add comments/renaming for clarity
2019-07-01 12:14:40 -07:00
d7ed3b8024 Add RPC api to return program accounts (#4876)
automerge
2019-06-29 09:59:07 -07:00
a89589a1d5 Add Measure abstraction over measuring time intervals (#4851)
Allows one to swap in different implementations. This provides
the normal Insant::now() -> .elapsed() path.
2019-06-29 15:34:49 +02:00
75b494d4a3 Lower warn to info, fetch from validator root instead of root + 1 (#4870)
* Lower warn to info, fetch from validator root instead of root + 1

* b/c -> because
2019-06-28 16:17:20 -07:00
0fa1af5d47 Cleanup num_threads() and batch_limit numbers (#4852) 2019-06-28 10:55:24 +02:00
66552d7047 Credit-Only Accounts: Cache account balance for thread-safe load/store (#4691)
* Implement CreditOnlyLocks

* Update credit-only atomic on account load

* Update credit-only atomic after bank.freeze_lock; store credits if all credit-only lock references are dropped

* Commit credit-only credits on bank freeze

* Update core to CreditAccountLocks

* Impl credit-only in System Transfer

* Rework CreditAccountLocks, test, and fix bugs

* Review comments: Pass CreditAccountLocks by reference; Tighten up insert block

* Only store credits on completed slot

* Check balance in bench_exchange funding to ensure commit_credits has completed

* Add is_debitable info to KeyedAccount meta to pass into programs

* Reinstate CreditOnlyLocks check on lock_account

* Rework CreditAccountLocks to remove strong_count usage

* Add multi-threaded credit-only locks test

* Improve RwLocks usage

* Review comments: panic if bad things happen; tighter code

* Assert lock_accounts race does not happen

* Revert panic if bad things happen; not a bad thing
2019-06-27 17:25:10 -04:00