Commit Graph

2273 Commits

Author SHA1 Message Date
40914de811 updates cluster-slots with root-bank instead of root-slot + bank-forks
ClusterSlots::update is taking both root-slot and bank-forks only to
later lookup root-bank from bank-forks, which is redundant. Also
potentially by the time bank-forks is locked to obtain root-bank,
root-slot may have already changed and so be inconsistent with the
root-slot passed in as the argument.
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L32-L39
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L122
2021-08-05 14:43:06 +00:00
2fc112edcf removes unused code from cluster-slots 2021-08-05 14:43:06 +00:00
bf16b0517c add _for_tests to setup_bank_and_vote_pubkeys (#19060) 2021-08-05 08:43:35 -05:00
14361906ca for all tests, bank::new -> bank::new_for_tests (#19064) 2021-08-05 08:42:38 -05:00
3280ae3e9f add validator option --accounts-db-skip-shrink (#19028)
* add validator option --accounts-db-skip-shrink

* typo
2021-08-04 17:28:33 -05:00
1ed12a07ab introduce Bank::new_for_tests (#19062) 2021-08-04 15:06:57 -05:00
ca14475085 Add incremental_snapshot_archive_interval_slots to SnapshotConfig (#19026)
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
2021-08-04 14:40:20 -05:00
06a7a9e544 remove superfluous collect()s 2021-08-04 07:21:55 +00:00
03353d500f Actively manage dead slots in AncestorHashesService (#18912) 2021-08-02 14:33:28 -07:00
049fb0417f allows sendmmsg api taking owned values (as well as references) (#18999)
Current signature of api in sendmmsg requires a slice of inner
references:
https://github.com/solana-labs/solana/blob/fe1ee4980/streamer/src/sendmmsg.rs#L130-L152

That forces the call-site to convert owned values to references even
though doing so is redundant and adds an extra level of indirection:
https://github.com/solana-labs/solana/blob/fe1ee4980/core/src/repair_service.rs#L291

This commit expands the api using AsRef and Borrow traits to allow
calling the method with owned values (as well as references like
before).
2021-07-30 20:58:49 +00:00
5d297ccf96 Cost model uses compute_unit to replace microsecond as cost unit (#18934)
* wip - cost_update_services to log both us and cu for each instruction to determine possible ratio

* replace microsecond with compute_unit as cost unit
2021-07-29 22:19:36 +00:00
da480bdb5f Fix unstable retransmit-num_nodes (#18970) 2021-07-29 17:32:32 +00:00
d06dc6c8a6 shares cluster-nodes between retransmit threads (#18947)
cluster_nodes and last_peer_update are not shared between retransmit
threads, as each thread have its own value:
https://github.com/solana-labs/solana/blob/65ccfed86/core/src/retransmit_stage.rs#L476-L477

Additionally, with shared references, this code:
https://github.com/solana-labs/solana/blob/0167daa11/core/src/retransmit_stage.rs#L315-L328
has a concurrency bug where the thread which does compare_and_swap,
updates cluster_nodes much later after other threads have run with
outdated cluster_nodes for a while. In particular, the write-lock there
may block.
2021-07-29 16:20:15 +00:00
71f6d839f9 validator: remove disused cuda config argument 2021-07-29 03:08:52 +00:00
8ed0cd0fff validator: check target CPU features earlier 2021-07-29 03:08:52 +00:00
c435f7b3e3 validator: add avx2 runtime check 2021-07-29 03:08:52 +00:00
e641f257ef test-validator: move feature check earlier in startup 2021-07-29 03:08:52 +00:00
59641623d1 Improve check for Apple M1 silicon under Rosetta 2021-07-29 03:08:52 +00:00
9255ae334d drop outstanding_requests lock before sending repair requests (#18893) 2021-07-28 19:30:43 -07:00
84e78316b1 Write helper for multithread update (#18808) 2021-07-29 03:16:36 +02:00
f1b9f97aef remove avx error on macos (#18923) 2021-07-27 16:34:04 -07:00
c0704d4ec9 Plumb signal from replay to ancestor hashes service (#18880) 2021-07-26 20:59:00 -07:00
1ee64afb12 Introduce AncestorHashesService (#18812) 2021-07-23 16:54:47 -07:00
d2d5f36a3c adds validator flag to allow private ip addresses (#18850) 2021-07-23 15:25:03 +00:00
611af87fdb Really start caching by fixing swapped CAS... (#18842) 2021-07-23 10:17:19 +09:00
d1debcd971 Add incremental snapshot utils (#18504)
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
2021-07-22 14:40:37 -05:00
7d56fa8363 sends packets in batches from sigverify-stage (#18446)
sigverify-stage is breaking batches to single-item vectors before
sending them down the channel:
https://github.com/solana-labs/solana/blob/d451363dc/core/src/sigverify_stage.rs#L88-L92

Also simplifying window-service code, reducing number of nested branches.
2021-07-22 14:49:21 +00:00
61865c0ee0 solana-validator set-identity now loads the tower file for the new identity 2021-07-21 22:22:08 -07:00
588c0464b8 Add sampling logic and DuplicateSlotRepairStatus module (#18721) 2021-07-21 11:15:08 -07:00
bbd22f06f4 implements generic lookups into gossip crds table (#18765)
This commit adds CrdsEntry trait which allows generic lookups into crds
table. For example to get ContactInfo or LowestSlot associated with a
Pubkey, the lookup code would be respectively:
   crds.get::<&ContactInfo>(pubkey)
   crds.get::<&LowestSlot>(pubkey)
2021-07-21 12:16:26 +00:00
ce467bea20 Add frozen hashes and marking DuplicateConfirmed in blockstore to state machine (#18648) 2021-07-18 17:04:25 -07:00
e316586516 excludes private ip addresses 2021-07-16 20:05:48 -06:00
ae5ad5cf9b sendmmsg cleanup #18589
Rationalize usage of sendmmsg(2). Skip packets which failed to send and track failures.
2021-07-16 14:36:49 -07:00
ca71ca3d6d Accumulate consumed units (#18714) 2021-07-16 12:40:12 -07:00
d166b9856a Move transaction sanitization earlier in the pipeline (#18655)
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
8a846b048e Add AncestorHashesRepair type (#18681) 2021-07-15 19:29:53 -07:00
3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
568660b402 Revert "Remove feature switch for secp256k1 program (#18467)"
This reverts commit fd574dcb3b.
2021-07-15 18:43:55 +00:00
0f8bcf65af Add voting service (#18552) 2021-07-15 16:35:51 +02:00
cf31afdd6a makes CrdsGossip thread-safe (#18615) 2021-07-14 22:27:17 +00:00
b30b32300d solana-validator set-identity now works for voting validators 2021-07-14 09:42:35 -07:00
62d864559f Tower cleanup: reduce fn visibility, remove unnecessary new_with_key() 2021-07-14 09:42:35 -07:00
7f2254225e Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
c90af3cd63 removes id from push_lowest_slot args (#18645)
push_lowest_slot cannot sign the new crds-value unless the id (pubkey)
argument passed-in is the same pubkey as in ClusterInfo::keypair(), in
which case the id argument is redundant:
https://github.com/solana-labs/solana/blob/bb41cf346/gossip/src/cluster_info.rs#L824-L845

Additionally, the lookup is done with self.id(), but insert is done with
the id argument, which is logically a bug.
2021-07-13 22:32:59 +00:00
350baece21 Explicitly sanitize program id indexes before usage
1. check transaction has valid program_id before using it to avoid possible panic;
2. change calculate_cost function signature to return Result;
3. add CostModelError enum, update return type from Result<_, str> to Result<_, CostModelError>
2021-07-13 17:29:22 -05:00
4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
175083c4c1 Add updated duplicate broadcast test (#18506) 2021-07-10 22:22:07 -07:00
e9ace3a0d5 cost model nits (#18528) 2021-07-09 12:55:31 -07:00
fd574dcb3b Remove feature switch for secp256k1 program (#18467)
* Remove feature switch for secp256k1 program

* fix tests
2021-07-09 10:08:03 -05:00
4d3e301ee4 Introduce slot dumping to ReplayStage (#18160) 2021-07-08 19:07:32 -07:00