Add bank timestamp bounding (bp #13120) (#13331)

* Add bounding feature

(cherry picked from commit 96b8aa8bd1)

* Repurpose unused as Clock::epoch_start_timestamp; add gated update

(cherry picked from commit 0049ab69fb)

* Add bounded timestamp-estimation method

(cherry picked from commit 80db6c0980)

* Use bounded timestamp-correction when feature enabled

(cherry picked from commit 90778615f6)

* Prevent block times from ever going backward

(cherry picked from commit eb2560e782)

* Sample votes from ancestors back to root

(cherry picked from commit 4260b3b416)

* Add Clock sysvar details, update struct docs

(cherry picked from commit 3a1e125ce3)

* Add design proposal and update validator-timestamp-oracle

(cherry picked from commit a3912bc084)

* Adapt to feature::create_account

Co-authored-by: Tyera Eulberg <tyera@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>

docs/sidebars.js

@@ -173,6 +173,7 @@ module.exports = {
         ],
       },
       "implemented-proposals/abi-management",
+      "implemented-proposals/bank-timestamp-correction",
       "implemented-proposals/commitment",
       "implemented-proposals/cross-program-invocation",
       "implemented-proposals/durable-tx-nonces",

docs/src/apps/sysvars.md

@@ -20,6 +20,28 @@ epoch, and estimated wall-clock Unix timestamp. It is updated every slot.
 
 - Address: `SysvarC1ock11111111111111111111111111111111`
 - Layout: [Clock](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/clock/struct.Clock.html)
+- Fields:
+  - `slot`: the current slot
+  - `epoch_start_timestamp`: the Unix timestamp of the first slot in this epoch. In the first slot of an epoch, this timestamp is identical to the `unix_timestamp` (below).
+  - `epoch`: the current epoch
+  - `leader_schedule_epoch`: the most recent epoch for which the leader schedule has already been generated
+  - `unix_timestamp`: the Unix timestamp of this slot.
+
+  Each slot has an estimated duration based on Proof of History. But in reality,
+  slots may elapse faster and slower than this estimate. As a result, the Unix
+  timestamp of a slot is generated based on oracle input from voting validators.
+  This timestamp is calculated as the stake-weighted median of timestamp
+  estimates provided by votes, bounded by the expected time elapsed since the
+  start of the epoch.
+
+  More explicitly: for each slot, the most recent vote timestamp provided by
+  each validator is used to generate a timestamp estimate for the current slot
+  (the elapsed slots since the vote timestamp are assumed to be
+  Bank::ns_per_slot). Each timestamp estimate is associated with the stake
+  delegated to that vote account to create a distribution of timestamps by
+  stake. The median timestamp is used as the `unix_timestamp`, unless the
+  elapsed time since the `epoch_start_timestamp` has deviated from the expected
+  elapsed time by more than 25%.
 
 ## EpochSchedule
 

docs/src/implemented-proposals/bank-timestamp-correction.md

@@ -0,0 +1,79 @@
+---
+title: Bank Timestamp Correction
+---
+
+Each Bank has a timestamp that is stashed in the Clock sysvar and used to assess
+time-based stake account lockups. However, since genesis, this value has been
+based on a theoretical slots-per-second instead of reality, so it's quite
+inaccurate. This poses a problem for lockups, since the accounts will not
+register as lockup-free on (or anytime near) the date the lockup is set to
+expire.
+
+Block times are already being estimated to cache in Blockstore and long-term
+storage using a [validator timestamp oracle](validator-timestamp-oracle.md);
+this data provides an opportunity to align the bank timestamp more closely with
+real-world time.
+
+The general outline of the proposed implementation is as follows:
+
+- Correct each Bank timestamp using the validator-provided timestamp.
+- Update the validator-provided timestamp calculation to use a stake-weighted
+  median, rather than a stake-weighted mean.
+- Bound the timestamp correction so that it cannot deviate too far from the
+  expected theoretical estimate
+
+## Timestamp Correction
+
+On every new Bank, the runtime calculates a realistic timestamp estimate using
+validator timestamp-oracle data. The Bank timestamp is corrected to this value
+if it is greater than or equal to the previous Bank's timestamp. That is, time
+should not ever go backward, so that locked up accounts may be released by the
+correction, but once released, accounts can never be relocked by a time
+correction.
+
+### Calculating Stake-Weighted Median Timestamp
+
+In order to calculate the estimated timestamp for a particular Bank, the runtime
+first needs to get the most recent vote timestamps from the active validator
+set. The `Bank::vote_accounts()` method provides the vote accounts state, and
+these can be filtered to all accounts whose most recent timestamp is for an
+ancestor slot back to the current root. This should guarantee 2/3+ of the
+current cluster stake is represented, since by definition, roots must be
+confirmed by 2/3+ stake.
+
+From each vote timestamp, an estimate for the current Bank is calculated using
+the epoch's target ns_per_slot for any delta between the Bank slot and the
+timestamp slot. Each timestamp estimate is  is associated with the stake
+delegated to that vote account, and all the timestamps are collected to create a
+stake-weighted timestamp distribution.
+
+From this set, the stake-weighted median timestamp -- that is, the timestamp at
+which 50% of the stake estimates a greater-or-equal timestamp and 50% of the
+stake estimates a lesser-or-equal timestamp -- is selected as the potential
+corrected timestamp.
+
+This stake-weighted median timestamp is preferred over the stake-weighted mean
+because the multiplication of stake by proposed timestamp in the mean
+calculation allows a node with very small stake to still have a large effect on
+the resulting timestamp by proposing a timestamp that is very large or very
+small. For example, using the previous `calculate_stake_weighted_timestamp()`
+method, a node with 0.00003% of the stake proposing a timestamp of `i64::MAX`
+can shift the timestamp forward 97k years!
+
+### Bounding Timestamps
+
+In addition to preventing time moving backward, we can prevent malicious
+activity by bounding the corrected timestamp to an acceptable level of deviation
+from the theoretical expected time.
+
+This proposal suggests that each timestamp be allowed to deviate up to 25% from
+the expected time since the start of the epoch.
+
+In order to calculate the timestamp deviation, each Bank needs to log the
+`epoch_start_timestamp` in the Clock sysvar. This value is set to the
+`Clock::unix_timestamp` on the first slot of each epoch.
+
+Then, the runtime compares the expected elapsed time since the start of the
+epoch with the proposed elapsed time based on the corrected timestamp. If the
+corrected elaped time is within +/- 25% of expected, the corrected timestamp is
+accepted. Otherwise, it is bounded to the acceptable deviation.

docs/src/implemented-proposals/validator-timestamp-oracle.md

@@ -48,19 +48,11 @@ Vote vector (`Vote::slots.iter().max()`). It is signed by the validator's
 identity keypair as a usual Vote. In order to enable this reporting, the Vote
 struct needs to be extended to include a timestamp field, `timestamp: Option<UnixTimestamp>`, which will be set to `None` in most Votes.
 
-This proposal suggests that Vote instructions with `Some(timestamp)` be issued
-every 30min, which should be short enough to prevent block times drifting very
-much, without adding too much transaction overhead to the cluster. Validators
-can convert this time to a slot interval using the `slots_per_year` value that
-is stored in each bank.
-
-```text
-let seconds_in_30min = 1800;
-let timestamp_interval = (slots_per_year / SECONDS_PER_YEAR) * seconds_in_30min;
-```
-
-Votes with `Some(timestamp)` should be triggered in `replay_stage::handle_votable_bank()`
-when `bank.slot() % timestamp_interval == 0`.
+As of https://github.com/solana-labs/solana/pull/10630, validators submit a
+timestamp every vote. This enables implementation of a block time caching
+service that allows nodes to calculate the estimated timestamp immediately after
+the block is rooted, and cache that value in Blockstore. This provides
+persistent data and quick queries, while still meeting requirement 1) above.
 
 ### Vote Accounts