gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add bank timestamp bounding (bp #13120) (#13331)

Browse Source 
* Add bounding feature

(cherry picked from commit 96b8aa8bd1)

* Repurpose unused as Clock::epoch_start_timestamp; add gated update

(cherry picked from commit 0049ab69fb)

* Add bounded timestamp-estimation method

(cherry picked from commit 80db6c0980)

* Use bounded timestamp-correction when feature enabled

(cherry picked from commit 90778615f6)

* Prevent block times from ever going backward

(cherry picked from commit eb2560e782)

* Sample votes from ancestors back to root

(cherry picked from commit 4260b3b416)

* Add Clock sysvar details, update struct docs

(cherry picked from commit 3a1e125ce3)

* Add design proposal and update validator-timestamp-oracle

(cherry picked from commit a3912bc084)

* Adapt to feature::create_account

Co-authored-by: Tyera Eulberg <tyera@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
This commit is contained in:
mergify[bot]
2020-11-01 06:02:34 +00:00
committed by GitHub
parent b6dc48da75
commit b28d10d46f
9 changed files with 795 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/sidebars.js
View File

@ -173,6 +173,7 @@ module.exports = {
],
},
"implemented-proposals/abi-management",
"implemented-proposals/bank-timestamp-correction",
"implemented-proposals/commitment",
"implemented-proposals/cross-program-invocation",
"implemented-proposals/durable-tx-nonces",

22
docs/src/apps/sysvars.md
View File

@ -20,6 +20,28 @@ epoch, and estimated wall-clock Unix timestamp. It is updated every slot.
- Address: `SysvarC1ock11111111111111111111111111111111`
- Layout: [Clock](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/clock/struct.Clock.html)
- Fields:
- `slot`: the current slot
- `epoch_start_timestamp`: the Unix timestamp of the first slot in this epoch. In the first slot of an epoch, this timestamp is identical to the `unix_timestamp` (below).
- `epoch`: the current epoch
- `leader_schedule_epoch`: the most recent epoch for which the leader schedule has already been generated
- `unix_timestamp`: the Unix timestamp of this slot.
Each slot has an estimated duration based on Proof of History. But in reality,
slots may elapse faster and slower than this estimate. As a result, the Unix
timestamp of a slot is generated based on oracle input from voting validators.
This timestamp is calculated as the stake-weighted median of timestamp
estimates provided by votes, bounded by the expected time elapsed since the
start of the epoch.
More explicitly: for each slot, the most recent vote timestamp provided by
each validator is used to generate a timestamp estimate for the current slot
(the elapsed slots since the vote timestamp are assumed to be
Bank::ns_per_slot). Each timestamp estimate is associated with the stake
delegated to that vote account to create a distribution of timestamps by
stake. The median timestamp is used as the `unix_timestamp`, unless the
elapsed time since the `epoch_start_timestamp` has deviated from the expected
elapsed time by more than 25%.
## EpochSchedule

79
docs/src/implemented-proposals/bank-timestamp-correction.md Normal file
View File

@ -0,0 +1,79 @@
---
title: Bank Timestamp Correction
---
Each Bank has a timestamp that is stashed in the Clock sysvar and used to assess
time-based stake account lockups. However, since genesis, this value has been
based on a theoretical slots-per-second instead of reality, so it's quite
inaccurate. This poses a problem for lockups, since the accounts will not
register as lockup-free on (or anytime near) the date the lockup is set to
expire.
Block times are already being estimated to cache in Blockstore and long-term
storage using a [validator timestamp oracle](validator-timestamp-oracle.md);
this data provides an opportunity to align the bank timestamp more closely with
real-world time.
The general outline of the proposed implementation is as follows:
- Correct each Bank timestamp using the validator-provided timestamp.
- Update the validator-provided timestamp calculation to use a stake-weighted
median, rather than a stake-weighted mean.
- Bound the timestamp correction so that it cannot deviate too far from the
expected theoretical estimate
## Timestamp Correction
On every new Bank, the runtime calculates a realistic timestamp estimate using
validator timestamp-oracle data. The Bank timestamp is corrected to this value
if it is greater than or equal to the previous Bank's timestamp. That is, time
should not ever go backward, so that locked up accounts may be released by the
correction, but once released, accounts can never be relocked by a time
correction.
### Calculating Stake-Weighted Median Timestamp
In order to calculate the estimated timestamp for a particular Bank, the runtime
first needs to get the most recent vote timestamps from the active validator
set. The `Bank::vote_accounts()` method provides the vote accounts state, and
these can be filtered to all accounts whose most recent timestamp is for an
ancestor slot back to the current root. This should guarantee 2/3+ of the
current cluster stake is represented, since by definition, roots must be
confirmed by 2/3+ stake.
From each vote timestamp, an estimate for the current Bank is calculated using
the epoch's target ns_per_slot for any delta between the Bank slot and the
timestamp slot. Each timestamp estimate is is associated with the stake
delegated to that vote account, and all the timestamps are collected to create a
stake-weighted timestamp distribution.
From this set, the stake-weighted median timestamp -- that is, the timestamp at
which 50% of the stake estimates a greater-or-equal timestamp and 50% of the
stake estimates a lesser-or-equal timestamp -- is selected as the potential
corrected timestamp.
This stake-weighted median timestamp is preferred over the stake-weighted mean
because the multiplication of stake by proposed timestamp in the mean
calculation allows a node with very small stake to still have a large effect on
the resulting timestamp by proposing a timestamp that is very large or very
small. For example, using the previous `calculate_stake_weighted_timestamp()`
method, a node with 0.00003% of the stake proposing a timestamp of `i64::MAX`
can shift the timestamp forward 97k years!
### Bounding Timestamps
In addition to preventing time moving backward, we can prevent malicious
activity by bounding the corrected timestamp to an acceptable level of deviation
from the theoretical expected time.
This proposal suggests that each timestamp be allowed to deviate up to 25% from
the expected time since the start of the epoch.
In order to calculate the timestamp deviation, each Bank needs to log the
`epoch_start_timestamp` in the Clock sysvar. This value is set to the
`Clock::unix_timestamp` on the first slot of each epoch.
Then, the runtime compares the expected elapsed time since the start of the
epoch with the proposed elapsed time based on the corrected timestamp. If the
corrected elaped time is within +/- 25% of expected, the corrected timestamp is
accepted. Otherwise, it is bounded to the acceptable deviation.

18
docs/src/implemented-proposals/validator-timestamp-oracle.md
View File

@ -48,19 +48,11 @@ Vote vector (`Vote::slots.iter().max()`). It is signed by the validator's
identity keypair as a usual Vote. In order to enable this reporting, the Vote
struct needs to be extended to include a timestamp field, `timestamp: Option<UnixTimestamp>`, which will be set to `None` in most Votes.
This proposal suggests that Vote instructions with `Some(timestamp)` be issued
every 30min, which should be short enough to prevent block times drifting very
much, without adding too much transaction overhead to the cluster. Validators
can convert this time to a slot interval using the `slots_per_year` value that
is stored in each bank.
```text
let seconds_in_30min = 1800;
let timestamp_interval = (slots_per_year / SECONDS_PER_YEAR) * seconds_in_30min;
```
Votes with `Some(timestamp)` should be triggered in `replay_stage::handle_votable_bank()`
when `bank.slot() % timestamp_interval == 0`.
As of https://github.com/solana-labs/solana/pull/10630, validators submit a
timestamp every vote. This enables implementation of a block time caching
service that allows nodes to calculate the estimated timestamp immediately after
the block is rooted, and cache that value in Blockstore. This provides
persistent data and quick queries, while still meeting requirement 1) above.
### Vote Accounts

16
ledger/src/blockstore.rs
View File

@ -33,7 +33,9 @@ use solana_sdk::{
program_utils::limited_deserialize,
pubkey::Pubkey,
signature::{Keypair, Signature, Signer},
stake_weighted_timestamp::{calculate_stake_weighted_timestamp, TIMESTAMP_SLOT_RANGE},
stake_weighted_timestamp::{
calculate_stake_weighted_timestamp, EstimateType, TIMESTAMP_SLOT_RANGE,
},
timing::timestamp,
transaction::Transaction,
};
@ -1638,9 +1640,15 @@ impl Blockstore {
}
let mut calculate_timestamp = Measure::start("calculate_timestamp");
let stake_weighted_timestamp =
calculate_stake_weighted_timestamp(&unique_timestamps, stakes, slot, slot_duration)
.ok_or(BlockstoreError::EmptyEpochStakes)?;
let stake_weighted_timestamp = calculate_stake_weighted_timestamp(
&unique_timestamps,
stakes,
slot,
slot_duration,
EstimateType::Unbounded,
None,
)
.ok_or(BlockstoreError::EmptyEpochStakes)?;
calculate_timestamp.stop();
datapoint_info!(
"blockstore-get-block-time",

262
runtime/src/bank.rs
View File

@ -59,7 +59,9 @@ use solana_sdk::{
signature::{Keypair, Signature},
slot_hashes::SlotHashes,
slot_history::SlotHistory,
stake_weighted_timestamp::{calculate_stake_weighted_timestamp, TIMESTAMP_SLOT_RANGE},
stake_weighted_timestamp::{
calculate_stake_weighted_timestamp, EstimateType, DEPRECATED_TIMESTAMP_SLOT_RANGE,
},
system_transaction,
sysvar::{self},
timing::years_as_slots,
@ -741,7 +743,7 @@ impl Bank {
}
bank.update_stake_history(None);
}
bank.update_clock();
bank.update_clock(None);
bank.update_rent();
bank.update_epoch_schedule();
bank.update_recent_blockhashes();
@ -844,7 +846,7 @@ impl Bank {
new.update_slot_hashes();
new.update_rewards(parent.epoch());
new.update_stake_history(Some(parent.epoch()));
new.update_clock();
new.update_clock(Some(parent.epoch()));
new.update_fees();
if !new.fix_recent_blockhashes_sysvar_delay() {
new.update_recent_blockhashes();
@ -1073,26 +1075,74 @@ impl Bank {
.unwrap_or_default()
}
fn update_clock(&self) {
fn update_clock(&self, parent_epoch: Option<Epoch>) {
let mut unix_timestamp = self.unix_timestamp_from_genesis();
if self
.feature_set
.is_active(&feature_set::timestamp_correction::id())
{
if let Some(timestamp_estimate) = self.get_timestamp_estimate() {
let (estimate_type, epoch_start_timestamp) =
if let Some(timestamp_bounding_activation_slot) = self
.feature_set
.activated_slot(&feature_set::timestamp_bounding::id())
{
// This check avoids a chicken-egg problem with epoch_start_timestamp, which is
// needed for timestamp bounding, but isn't yet corrected for the activation slot
let epoch_start_timestamp = if self.slot() > timestamp_bounding_activation_slot
{
let epoch = if let Some(epoch) = parent_epoch {
epoch
} else {
self.epoch()
};
let first_slot_in_epoch =
self.epoch_schedule.get_first_slot_in_epoch(epoch);
Some((first_slot_in_epoch, self.clock().epoch_start_timestamp))
} else {
None
};
(EstimateType::Bounded, epoch_start_timestamp)
} else {
(EstimateType::Unbounded, None)
};
if let Some(timestamp_estimate) =
self.get_timestamp_estimate(estimate_type, epoch_start_timestamp)
{
if timestamp_estimate > unix_timestamp {
datapoint_info!(
"bank-timestamp-correction",
("from_genesis", unix_timestamp, i64),
("corrected", timestamp_estimate, i64),
);
unix_timestamp = timestamp_estimate
unix_timestamp = timestamp_estimate;
let ancestor_timestamp = self.clock().unix_timestamp;
if self
.feature_set
.is_active(&feature_set::timestamp_bounding::id())
&& timestamp_estimate < ancestor_timestamp
{
unix_timestamp = ancestor_timestamp;
}
}
}
}
let epoch_start_timestamp = if self
.feature_set
.is_active(&feature_set::timestamp_bounding::id())
{
// On epoch boundaries, update epoch_start_timestamp
if parent_epoch.is_some() && parent_epoch.unwrap() != self.epoch() {
unix_timestamp
} else {
self.clock().epoch_start_timestamp
}
} else {
Self::get_unused_from_slot(self.slot, self.unused) as i64
};
let clock = sysvar::clock::Clock {
slot: self.slot,
unused: Self::get_unused_from_slot(self.slot, self.unused),
epoch_start_timestamp,
epoch: self.epoch_schedule.get_epoch(self.slot),
leader_schedule_epoch: self.epoch_schedule.get_leader_schedule_epoch(self.slot),
unix_timestamp,
@ -1427,7 +1477,11 @@ impl Bank {
self.update_recent_blockhashes_locked(&blockhash_queue);
}
fn get_timestamp_estimate(&self) -> Option<UnixTimestamp> {
fn get_timestamp_estimate(
&self,
estimate_type: EstimateType,
epoch_start_timestamp: Option<(Slot, UnixTimestamp)>,
) -> Option<UnixTimestamp> {
let mut get_timestamp_estimate_time = Measure::start("get_timestamp_estimate");
let recent_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = self
.vote_accounts()
@ -1435,7 +1489,13 @@ impl Bank {
.filter_map(|(pubkey, (_, account))| {
VoteState::from(&account).and_then(|state| {
let timestamp_slot = state.last_timestamp.slot;
if self.slot().checked_sub(timestamp_slot)? <= TIMESTAMP_SLOT_RANGE as u64 {
if (self
.feature_set
.is_active(&feature_set::timestamp_bounding::id())
&& self.ancestors.contains_key(&timestamp_slot))
|| self.slot().checked_sub(timestamp_slot)?
<= DEPRECATED_TIMESTAMP_SLOT_RANGE as u64
{
Some((
pubkey,
(state.last_timestamp.slot, state.last_timestamp.timestamp),
@ -1454,6 +1514,8 @@ impl Bank {
stakes,
self.slot(),
slot_duration,
estimate_type,
epoch_start_timestamp,
);
get_timestamp_estimate_time.stop();
datapoint_info!(
@ -9638,7 +9700,7 @@ mod tests {
let validator_vote_keypairs1 = ValidatorVoteKeypairs::new_rand();
let validator_keypairs = vec![&validator_vote_keypairs0, &validator_vote_keypairs1];
let GenesisConfigInfo {
genesis_config,
mut genesis_config,
mint_keypair: _,
voting_keypair: _,
} = create_genesis_config_with_vote_accounts(
@ -9646,8 +9708,15 @@ mod tests {
&validator_keypairs,
vec![10_000; 2],
);
genesis_config
.accounts
.remove(&feature_set::timestamp_bounding::id())
.unwrap();
let mut bank = Bank::new(&genesis_config);
assert_eq!(bank.get_timestamp_estimate(), Some(0));
assert_eq!(
bank.get_timestamp_estimate(EstimateType::Unbounded, None),
Some(0)
);
let recent_timestamp: UnixTimestamp = bank.unix_timestamp_from_genesis();
update_vote_account_timestamp(
@ -9668,7 +9737,7 @@ mod tests {
&validator_vote_keypairs1.vote_keypair.pubkey(),
);
assert_eq!(
bank.get_timestamp_estimate(),
bank.get_timestamp_estimate(EstimateType::Unbounded, None),
Some(recent_timestamp + additional_secs / 2)
);
@ -9677,14 +9746,17 @@ mod tests {
}
let adjustment = (bank.ns_per_slot as u64 * bank.slot()) / 1_000_000_000;
assert_eq!(
bank.get_timestamp_estimate(),
bank.get_timestamp_estimate(EstimateType::Unbounded, None),
Some(recent_timestamp + adjustment as i64 + additional_secs / 2)
);
for _ in 0..7 {
bank = new_from_parent(&Arc::new(bank));
}
assert_eq!(bank.get_timestamp_estimate(), None);
assert_eq!(
bank.get_timestamp_estimate(EstimateType::Unbounded, None),
None
);
}
#[test]
@ -9699,6 +9771,10 @@ mod tests {
.accounts
.remove(&feature_set::timestamp_correction::id())
.unwrap();
genesis_config
.accounts
.remove(&feature_set::timestamp_bounding::id())
.unwrap();
let bank = Bank::new(&genesis_config);
let recent_timestamp: UnixTimestamp = bank.unix_timestamp_from_genesis();
@ -9714,10 +9790,10 @@ mod tests {
// Bank::new_from_parent should not adjust timestamp before feature activation
let mut bank = new_from_parent(&Arc::new(bank));
let clock =
from_account::<sysvar::clock::Clock>(&bank.get_account(&sysvar::clock::id()).unwrap())
.unwrap();
assert_eq!(clock.unix_timestamp, bank.unix_timestamp_from_genesis());
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis()
);
// Request `timestamp_correction` activation
bank.store_account(
@ -9733,15 +9809,131 @@ mod tests {
// Now Bank::new_from_parent should adjust timestamp
let bank = Arc::new(new_from_parent(&Arc::new(bank)));
let clock =
from_account::<sysvar::clock::Clock>(&bank.get_account(&sysvar::clock::id()).unwrap())
.unwrap();
assert_eq!(
clock.unix_timestamp,
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis() + additional_secs
);
}
#[test]
fn test_timestamp_bounding_feature() {
let leader_pubkey = solana_sdk::pubkey::new_rand();
let GenesisConfigInfo {
mut genesis_config,
voting_keypair,
..
} = create_genesis_config_with_leader(5, &leader_pubkey, 3);
let slots_in_epoch = 32;
genesis_config
.accounts
.remove(&feature_set::timestamp_bounding::id())
.unwrap();
genesis_config.epoch_schedule = EpochSchedule::new(slots_in_epoch);
let bank = Bank::new(&genesis_config);
let recent_timestamp: UnixTimestamp = bank.unix_timestamp_from_genesis();
let additional_secs = 1;
update_vote_account_timestamp(
BlockTimestamp {
slot: bank.slot(),
timestamp: recent_timestamp + additional_secs,
},
&bank,
&voting_keypair.pubkey(),
);
// Bank::new_from_parent should allow unbounded timestamp before activation
let mut bank = new_from_parent(&Arc::new(bank));
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis() + additional_secs
);
// Bank::new_from_parent should not allow epoch_start_timestamp to be set before activation
bank.update_clock(Some(0));
assert_eq!(
bank.clock().epoch_start_timestamp,
Bank::get_unused_from_slot(bank.slot(), bank.unused) as i64
);
// Request `timestamp_bounding` activation
let feature = Feature { activated_at: None };
bank.store_account(
&feature_set::timestamp_bounding::id(),
&feature::create_account(&feature, 42),
);
for _ in 0..30 {
bank = new_from_parent(&Arc::new(bank));
}
// Refresh vote timestamp
let recent_timestamp: UnixTimestamp = bank.unix_timestamp_from_genesis();
let additional_secs = 1;
update_vote_account_timestamp(
BlockTimestamp {
slot: bank.slot(),
timestamp: recent_timestamp + additional_secs,
},
&bank,
&voting_keypair.pubkey(),
);
// Advance to epoch boundary to activate
bank = new_from_parent(&Arc::new(bank));
// Bank::new_from_parent is bounding, but should not use epoch_start_timestamp in activation slot
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis() + additional_secs
);
assert_eq!(
bank.clock().epoch_start_timestamp,
bank.unix_timestamp_from_genesis() + additional_secs
);
// Past activation slot, bounding should use epoch_start_timestamp in activation slot
bank = new_from_parent(&Arc::new(bank));
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis()
);
for _ in 0..30 {
bank = new_from_parent(&Arc::new(bank));
}
// Refresh vote timestamp
let recent_timestamp: UnixTimestamp = bank.unix_timestamp_from_genesis();
let additional_secs = 20;
update_vote_account_timestamp(
BlockTimestamp {
slot: bank.slot(),
timestamp: recent_timestamp + additional_secs,
},
&bank,
&voting_keypair.pubkey(),
);
// Advance to epoch boundary
bank = new_from_parent(&Arc::new(bank));
// Past activation slot, bounding should use previous epoch_start_timestamp on epoch boundary slots
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis() // Plus estimated offset + 25%
+ ((slots_in_epoch as u32 * Duration::from_nanos(bank.ns_per_slot as u64))
.as_secs()
* 25
/ 100) as i64,
);
assert_eq!(
bank.clock().epoch_start_timestamp,
bank.clock().unix_timestamp
);
}
#[test]
fn test_update_clock_timestamp() {
let leader_pubkey = solana_sdk::pubkey::new_rand();
@ -9750,13 +9942,13 @@ mod tests {
voting_keypair,
..
} = create_genesis_config_with_leader(5, &leader_pubkey, 3);
let bank = Bank::new(&genesis_config);
let mut bank = Bank::new(&genesis_config);
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis()
);
bank.update_clock();
bank.update_clock(None);
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis()
@ -9770,7 +9962,7 @@ mod tests {
&bank,
&voting_keypair.pubkey(),
);
bank.update_clock();
bank.update_clock(None);
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis()
@ -9784,7 +9976,7 @@ mod tests {
&bank,
&voting_keypair.pubkey(),
);
bank.update_clock();
bank.update_clock(None);
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis()
@ -9798,11 +9990,27 @@ mod tests {
&bank,
&voting_keypair.pubkey(),
);
bank.update_clock();
bank.update_clock(None);
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis() + 1
);
// Timestamp cannot go backward from ancestor Bank to child
bank = new_from_parent(&Arc::new(bank));
update_vote_account_timestamp(
BlockTimestamp {
slot: bank.slot(),
timestamp: bank.unix_timestamp_from_genesis() - 1,
},
&bank,
&voting_keypair.pubkey(),
);
bank.update_clock(None);
assert_eq!(
bank.clock().unix_timestamp,
bank.unix_timestamp_from_genesis()
);
}
fn setup_bank_with_removable_zero_lamport_account() -> Arc<Bank> {

9
sdk/program/src/clock.rs
View File

@ -79,14 +79,15 @@ pub type UnixTimestamp = i64;
pub struct Clock {
/// the current network/bank Slot
pub slot: Slot,
/// unused
pub unused: u64,
/// the timestamp of the first Slot in this Epoch
pub epoch_start_timestamp: UnixTimestamp,
/// the bank Epoch
pub epoch: Epoch,
/// the future Epoch for which the leader schedule has
/// most recently been calculated
pub leader_schedule_epoch: Epoch,
/// computed from genesis creation time and network time
/// in slots, drifts!
/// originally computed from genesis creation time and network time
/// in slots (drifty); corrected using validator timestamp oracle as of
/// timestamp_correction and timestamp_bounding features
pub unix_timestamp: UnixTimestamp,
}

5
sdk/src/feature_set.rs
View File

@ -78,6 +78,10 @@ pub mod pull_request_ping_pong_check {
solana_sdk::declare_id!("5RzEHTnf6D7JPZCvwEzjM19kzBsyjSU3HoMfXaQmVgnZ");
}
pub mod timestamp_bounding {
solana_sdk::declare_id!("8FyEA6ABYiMxX7Az6AopQN3mavLD8Rz3N4bvKnbbBFFq");
}
lazy_static! {
/// Map of feature identifiers to user-visible description
pub static ref FEATURE_NAMES: HashMap<Pubkey, &'static str> = [
@ -99,6 +103,7 @@ lazy_static! {
(sol_log_compute_units_syscall::id(), "sol_log_compute_units syscall (#13243)"),
(pubkey_log_syscall_enabled::id(), "pubkey log syscall"),
(pull_request_ping_pong_check::id(), "ping-pong packet check #12794"),
(timestamp_bounding::id(), "add timestamp-correction bounding #13120"),
/*************** ADD NEW FEATURES HERE ***************/
]
.iter()

435
sdk/src/stake_weighted_timestamp.rs
View File

@ -5,15 +5,50 @@ use solana_sdk::{
clock::{Slot, UnixTimestamp},
pubkey::Pubkey,
};
use std::{collections::HashMap, time::Duration};
use std::{
collections::{BTreeMap, HashMap},
time::Duration,
};
pub const TIMESTAMP_SLOT_RANGE: usize = 16;
pub const TIMESTAMP_SLOT_RANGE: usize = 32;
pub const DEPRECATED_TIMESTAMP_SLOT_RANGE: usize = 16; // Deprecated. Remove in the Solana v1.6.0 timeframe
const MAX_ALLOWABLE_DRIFT_PERCENTAGE: u32 = 25;
pub enum EstimateType {
Bounded,
Unbounded, // Deprecated. Remove in the Solana v1.6.0 timeframe
}
pub fn calculate_stake_weighted_timestamp(
unique_timestamps: &HashMap<Pubkey, (Slot, UnixTimestamp)>,
stakes: &HashMap<Pubkey, (u64, Account)>,
slot: Slot,
slot_duration: Duration,
estimate_type: EstimateType,
epoch_start_timestamp: Option<(Slot, UnixTimestamp)>,
) -> Option<UnixTimestamp> {
match estimate_type {
EstimateType::Bounded => calculate_bounded_stake_weighted_timestamp(
unique_timestamps,
stakes,
slot,
slot_duration,
epoch_start_timestamp,
),
EstimateType::Unbounded => calculate_unbounded_stake_weighted_timestamp(
unique_timestamps,
stakes,
slot,
slot_duration,
),
}
}
fn calculate_unbounded_stake_weighted_timestamp(
unique_timestamps: &HashMap<Pubkey, (Slot, UnixTimestamp)>,
stakes: &HashMap<Pubkey, (u64, Account)>,
slot: Slot,
slot_duration: Duration,
) -> Option<UnixTimestamp> {
let (stake_weighted_timestamps_sum, total_stake) = unique_timestamps
.iter()
@ -36,6 +71,67 @@ pub fn calculate_stake_weighted_timestamp(
}
}
fn calculate_bounded_stake_weighted_timestamp(
unique_timestamps: &HashMap<Pubkey, (Slot, UnixTimestamp)>,
stakes: &HashMap<Pubkey, (u64, Account)>,
slot: Slot,
slot_duration: Duration,
epoch_start_timestamp: Option<(Slot, UnixTimestamp)>,
) -> Option<UnixTimestamp> {
let mut stake_per_timestamp: BTreeMap<UnixTimestamp, u128> = BTreeMap::new();
let mut total_stake = 0;
for (vote_pubkey, (timestamp_slot, timestamp)) in unique_timestamps.iter() {
let offset = slot.saturating_sub(*timestamp_slot) as u32 * slot_duration;
let estimate = timestamp + offset.as_secs() as i64;
let stake = stakes
.get(&vote_pubkey)
.map(|(stake, _account)| stake)
.unwrap_or(&0);
stake_per_timestamp
.entry(estimate)
.and_modify(|stake_sum| *stake_sum += *stake as u128)
.or_insert(*stake as u128);
total_stake += *stake as u128;
}
if total_stake == 0 {
return None;
}
let mut stake_accumulator = 0;
let mut estimate = 0;
// Populate `estimate` with stake-weighted median timestamp
for (timestamp, stake) in stake_per_timestamp.into_iter() {
stake_accumulator += stake;
if stake_accumulator > total_stake / 2 {
estimate = timestamp;
break;
}
}
// Bound estimate by `MAX_ALLOWABLE_DRIFT_PERCENTAGE` since the start of the epoch
if let Some((epoch_start_slot, epoch_start_timestamp)) = epoch_start_timestamp {
let poh_estimate_offset = slot.saturating_sub(epoch_start_slot) as u32 * slot_duration;
let estimate_offset =
Duration::from_secs(estimate.saturating_sub(epoch_start_timestamp) as u64);
let max_allowable_drift = poh_estimate_offset * MAX_ALLOWABLE_DRIFT_PERCENTAGE / 100;
if estimate_offset > poh_estimate_offset
&& estimate_offset - poh_estimate_offset > max_allowable_drift
{
// estimate offset since the start of the epoch is higher than
// `MAX_ALLOWABLE_DRIFT_PERCENTAGE`
estimate = epoch_start_timestamp
+ poh_estimate_offset.as_secs() as i64
+ max_allowable_drift.as_secs() as i64;
} else if estimate_offset < poh_estimate_offset
&& poh_estimate_offset - estimate_offset > max_allowable_drift
{
// estimate offset since the start of the epoch is lower than
// `MAX_ALLOWABLE_DRIFT_PERCENTAGE`
estimate = epoch_start_timestamp + poh_estimate_offset.as_secs() as i64
- max_allowable_drift.as_secs() as i64;
}
}
Some(estimate)
}
#[cfg(test)]
pub mod tests {
use super::*;
@ -95,7 +191,7 @@ pub mod tests {
.cloned()
.collect();
assert_eq!(
calculate_stake_weighted_timestamp(
calculate_unbounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
@ -138,7 +234,7 @@ pub mod tests {
.cloned()
.collect();
assert_eq!(
calculate_stake_weighted_timestamp(
calculate_unbounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
@ -147,4 +243,335 @@ pub mod tests {
Some(recent_timestamp + expected_offset as i64)
);
}
#[test]
fn test_calculate_bounded_stake_weighted_timestamp_uses_median() {
let recent_timestamp: UnixTimestamp = 1_578_909_061;
let slot = 5;
let slot_duration = Duration::from_millis(400);
let pubkey0 = solana_sdk::pubkey::new_rand();
let pubkey1 = solana_sdk::pubkey::new_rand();
let pubkey2 = solana_sdk::pubkey::new_rand();
let pubkey3 = solana_sdk::pubkey::new_rand();
let pubkey4 = solana_sdk::pubkey::new_rand();
// Test low-staked outlier(s)
let stakes: HashMap<Pubkey, (u64, Account)> = [
(
pubkey0,
(sol_to_lamports(1.0), Account::new(1, 0, &Pubkey::default())),
),
(
pubkey1,
(sol_to_lamports(1.0), Account::new(1, 0, &Pubkey::default())),
),
(
pubkey2,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
(
pubkey3,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
(
pubkey4,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
]
.iter()
.cloned()
.collect();
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (5, 0)),
(pubkey1, (5, recent_timestamp)),
(pubkey2, (5, recent_timestamp)),
(pubkey3, (5, recent_timestamp)),
(pubkey4, (5, recent_timestamp)),
]
.iter()
.cloned()
.collect();
let unbounded = calculate_unbounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
)
.unwrap();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
None,
)
.unwrap();
assert_eq!(bounded - unbounded, 527); // timestamp w/ 0.00003% of the stake can shift the timestamp backward 8min
assert_eq!(bounded, recent_timestamp); // low-staked outlier cannot affect bounded timestamp
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (5, recent_timestamp)),
(pubkey1, (5, i64::MAX)),
(pubkey2, (5, recent_timestamp)),
(pubkey3, (5, recent_timestamp)),
(pubkey4, (5, recent_timestamp)),
]
.iter()
.cloned()
.collect();
let unbounded = calculate_unbounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
)
.unwrap();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
None,
)
.unwrap();
assert_eq!(unbounded - bounded, 3074455295455); // timestamp w/ 0.00003% of the stake can shift the timestamp forward 97k years!
assert_eq!(bounded, recent_timestamp); // low-staked outlier cannot affect bounded timestamp
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (5, 0)),
(pubkey1, (5, i64::MAX)),
(pubkey2, (5, recent_timestamp)),
(pubkey3, (5, recent_timestamp)),
(pubkey4, (5, recent_timestamp)),
]
.iter()
.cloned()
.collect();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
None,
)
.unwrap();
assert_eq!(bounded, recent_timestamp); // multiple low-staked outliers cannot affect bounded timestamp if they don't shift the median
// Test higher-staked outlier(s)
let stakes: HashMap<Pubkey, (u64, Account)> = [
(
pubkey0,
(
sol_to_lamports(1_000_000.0), // 1/3 stake
Account::new(1, 0, &Pubkey::default()),
),
),
(
pubkey1,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
(
pubkey2,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
]
.iter()
.cloned()
.collect();
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (5, 0)),
(pubkey1, (5, i64::MAX)),
(pubkey2, (5, recent_timestamp)),
]
.iter()
.cloned()
.collect();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
None,
)
.unwrap();
assert_eq!(bounded, recent_timestamp); // outlier(s) cannot affect bounded timestamp if they don't shift the median
let stakes: HashMap<Pubkey, (u64, Account)> = [
(
pubkey0,
(
sol_to_lamports(1_000_001.0), // 1/3 stake
Account::new(1, 0, &Pubkey::default()),
),
),
(
pubkey1,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
]
.iter()
.cloned()
.collect();
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> =
[(pubkey0, (5, 0)), (pubkey1, (5, recent_timestamp))]
.iter()
.cloned()
.collect();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
None,
)
.unwrap();
assert_eq!(recent_timestamp - bounded, 1578909061); // outliers > 1/2 of available stake can affect timestamp
}
#[test]
fn test_calculate_bounded_stake_weighted_timestamp_poh() {
let epoch_start_timestamp: UnixTimestamp = 1_578_909_061;
let slot = 20;
let slot_duration = Duration::from_millis(400);
let poh_offset = (slot * slot_duration).as_secs();
let acceptable_delta = (MAX_ALLOWABLE_DRIFT_PERCENTAGE * poh_offset as u32 / 100) as i64;
let poh_estimate = epoch_start_timestamp + poh_offset as i64;
let pubkey0 = solana_sdk::pubkey::new_rand();
let pubkey1 = solana_sdk::pubkey::new_rand();
let pubkey2 = solana_sdk::pubkey::new_rand();
let stakes: HashMap<Pubkey, (u64, Account)> = [
(
pubkey0,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
(
pubkey1,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
(
pubkey2,
(
sol_to_lamports(1_000_000.0),
Account::new(1, 0, &Pubkey::default()),
),
),
]
.iter()
.cloned()
.collect();
// Test when stake-weighted median is too high
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (slot as u64, poh_estimate + acceptable_delta + 1)),
(pubkey1, (slot as u64, poh_estimate + acceptable_delta + 1)),
(pubkey2, (slot as u64, poh_estimate + acceptable_delta + 1)),
]
.iter()
.cloned()
.collect();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
Some((0, epoch_start_timestamp)),
)
.unwrap();
assert_eq!(bounded, poh_estimate + acceptable_delta);
// Test when stake-weighted median is too low
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (slot as u64, poh_estimate - acceptable_delta - 1)),
(pubkey1, (slot as u64, poh_estimate - acceptable_delta - 1)),
(pubkey2, (slot as u64, poh_estimate - acceptable_delta - 1)),
]
.iter()
.cloned()
.collect();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
Some((0, epoch_start_timestamp)),
)
.unwrap();
assert_eq!(bounded, poh_estimate - acceptable_delta);
// Test stake-weighted median within bounds
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (slot as u64, poh_estimate + acceptable_delta)),
(pubkey1, (slot as u64, poh_estimate + acceptable_delta)),
(pubkey2, (slot as u64, poh_estimate + acceptable_delta)),
]
.iter()
.cloned()
.collect();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
Some((0, epoch_start_timestamp)),
)
.unwrap();
assert_eq!(bounded, poh_estimate + acceptable_delta);
let unique_timestamps: HashMap<Pubkey, (Slot, UnixTimestamp)> = [
(pubkey0, (slot as u64, poh_estimate - acceptable_delta)),
(pubkey1, (slot as u64, poh_estimate - acceptable_delta)),
(pubkey2, (slot as u64, poh_estimate - acceptable_delta)),
]
.iter()
.cloned()
.collect();
let bounded = calculate_bounded_stake_weighted_timestamp(
&unique_timestamps,
&stakes,
slot as Slot,
slot_duration,
Some((0, epoch_start_timestamp)),
)
.unwrap();
assert_eq!(bounded, poh_estimate - acceptable_delta);
}
}