gaspersic/solana
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add doc content and feedback (#13563)

Browse Source
This commit is contained in:
Jack May
2020-11-13 10:18:04 -08:00
committed by GitHub
parent 01a4889b53
commit 887b0e4b72
23 changed files with 754 additions and 804 deletions
Download Patch File Download Diff File Expand all files Collapse all files

79
docs/src/developing/programming-model/accounts.md
View File

@@ -13,12 +13,13 @@ tells the runtime who is allowed to access the data and how.
Unlike a file, the account includes metadata for the lifetime of the file. That
lifetime is expressed in "tokens", which is a number of fractional native
tokens, called _lamports_. Accounts are held in validator memory and pay
["rent"](#rent) to stay there. Each validator periodically scans all
accounts and collects rent. Any account that drops to zero lamports is purged.
["rent"](#rent) to stay there. Each validator periodically scans all accounts
and collects rent. Any account that drops to zero lamports is purged. Accounts
can also be marked [rent-exempt](#rent-exemption) if they contain a sufficnet
number of lamports.
In the same way that a Linux user uses a path to look up a file, a Solana client
uses an _address_ to look up an account. The address is usually a 256-bit public
key.
uses an _address_ to look up an account. The address is a 256-bit public key.
## Signers
@@ -32,19 +33,22 @@ then use that information to make authority decisions.
## Read-only
Transactions can mark some accounts as _read-only accounts_. The runtime permits
read-only accounts to be read concurrently by multiple programs. If a program
attempts to modify a read-only account, the transaction is rejected by the
runtime.
Transactions can [indicate](transactions.md#message-header-format) that some of
the accounts it references be treated as _read-only accounts_ in order to enable
parallel account processing between transactions. The runtime permits read-only
accounts to be read concurrently by multiple programs. If a program attempts to
modify a read-only account, the transaction is rejected by the runtime.
## Executable
If an account is marked "executable" in its metadata, it can be used by a
_loader_ to run programs. For example, a BPF-compiled program is marked
executable by the BPF loader during deployment once the loader has determined
that the BPF bytecode in the account's data is valid. No program is allowed to
modify the contents of an executable account once deployed and executable mark
is permanent.
If an account is marked "executable" in its metadata then it is considered a
program which can be executed by including the account's public key an
instruction's [program id](transactions.md#program-id). Accounts are marked as
executable during a successful program deployment process by the loader that
owns the account. For example, during BPF program deployment, once the loader
has determined that the BPF bytecode in the account's data is valid, the loader
permanently marks the program account as executable. Once executable, the
runtime enforces that the account's data (the program) is immutable.
## Creating
@@ -56,7 +60,7 @@ megabytes.
An account address can be any arbitrary 256 bit value, and there are mechanisms
for advanced users to create derived addresses
(`SystemProgram::CreateAccountWithSeed`,
[`Pubkey::CreateProgramAddress`](program-derived-addresses.md)).
[`Pubkey::CreateProgramAddress`](calling-between-programs.md#program-derived-addresses)).
Accounts that have never been created via the system program can also be passed
to programs. When an instruction references an account that hasn't been
@@ -73,52 +77,13 @@ operation the program controls or performs.
A created account is initialized to be _owned_ by a built-in program called the
System program and is called a _system account_ aptly. An account includes
"owner" metadata. The owner is a program ID. The runtime grants the program
write access to the account if its ID matches the owner. For the case of the
"owner" metadata. The owner is a program id. The runtime grants the program
write access to the account if its id matches the owner. For the case of the
System program, the runtime allows clients to transfer lamports and importantly
_assign_ account ownership, meaning changing owner to different program ID. If
_assign_ account ownership, meaning changing owner to different program id. If
an account is not owned by a program, the program is only permitted to read its
data and credit the account.
## Runtime Capability of Programs
The runtime only permits the owner program to debit the account or modify its
data. The program then defines additional rules for whether the client can
modify accounts it owns. In the case of the System program, it allows users to
transfer lamports by recognizing transaction signatures. If it sees the client
signed the transaction using the keypair's _private key_, it knows the client
authorized the token transfer.
In other words, the entire set of accounts owned by a given program can be
regarded as a key-value store where a key is the account address and value is
program-specific arbitrary binary data. A program author can decide how to
manage the program's whole state as possibly many accounts.
After the runtime executes each of the transaction's instructions, it uses the
account metadata to verify that the access policy was not violated. If a program
violates the policy, the runtime discards all account changes made by all
instructions in the transaction and marks the transaction as failed.
### Policy
After a program has processed an instruction the runtime verifies that the
program only performed operations it was permitted to, and that the results
adhere to the runtime policy.
The policy is as follows:
- Only the owner of the account may change owner.
- And only if the account is writable.
- And only if the data is zero-initialized or empty.
- An account not assigned to the program cannot have its balance decrease.
- The balance of read-only and executable accounts may not change.
- Only the system program can change the size of the data and only if the system
program owns the account.
- Only the owner may change account data.
- And if the account is writable.
- And if the account is not executable.
- Executable is one-way (false->true) and only the account owner may set it.
- No one modification to the rent_epoch associated with this account.
## Rent
Keeping accounts alive on Solana incurs a storage cost called _rent_ because the

281
docs/src/developing/programming-model/calling-between-programs.md Normal file
View File

@@ -0,0 +1,281 @@
---
title: Calling Between Programs
---
## Cross-Program Invocations
The Solana runtime allows programs to call each other via a mechanism called
cross-program invocation. Calling between programs is achieved by one program
invoking an instruction of the other. The invoking program is halted until the
invoked program finishes processing the instruction.
For example, a client could create a transaction that modifies two accounts,
each owned by separate on-chain programs:
```rust,ignore
let message = Message::new(vec![
token_instruction::pay(&alice_pubkey),
acme_instruction::launch_missiles(&bob_pubkey),
]);
client.send_and_confirm_message(&[&alice_keypair, &bob_keypair], &message);
```
A client may to instead allow the `acme` program to conveniently invoke `token`
instructions on the client's behalf:
```rust,ignore
let message = Message::new(vec![
acme_instruction::pay_and_launch_missiles(&alice_pubkey, &bob_pubkey),
]);
client.send_and_confirm_message(&[&alice_keypair, &bob_keypair], &message);
```
Given two on-chain programs `token` and `acme`, each implementing instructions
`pay()` and `launch_missiles()` respectively, acme can be implemented with a
call to a function defined in the `token` module by issuing a cross-program
invocation:
```rust,ignore
mod acme {
use token_instruction;
fn launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
...
}
fn pay_and_launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
let alice_pubkey = accounts[1].key;
let instruction = token_instruction::pay(&alice_pubkey);
invoke(&instruction, accounts)?;
launch_missiles(accounts)?;
}
```
`invoke()` is built into Solana's runtime and is responsible for routing the
given instruction to the `token` program via the instruction's `program_id`
field.
Note that `invoke` requires the caller to pass all the accounts required by the
instruction being invoked. This means that both the executable account (the
ones that matches the instruction's program id) and the accounts passed to the
instruction procesor.
Before invoking `pay()`, the runtime must ensure that `acme` didn't modify any
accounts owned by `token`. It does this by applying the runtime's policy to the
current state of the accounts at the time `acme` calls `invoke` vs. the initial
state of the accounts at the beginning of the `acme`'s instruction. After
`pay()` completes, the runtime must again ensure that `token` didn't modify any
accounts owned by `acme` by again applying the runtime's policy, but this time
with the `token` program ID. Lastly, after `pay_and_launch_missiles()`
completes, the runtime must apply the runtime policy one more time, where it
normally would, but using all updated `pre_*` variables. If executing
`pay_and_launch_missiles()` up to `pay()` made no invalid account changes,
`pay()` made no invalid changes, and executing from `pay()` until
`pay_and_launch_missiles()` returns made no invalid changes, then the runtime
can transitively assume `pay_and_launch_missiles()` as whole made no invalid
account changes, and therefore commit all these account modifications.
### Instructions that require privileges
The runtime uses the privileges granted to the caller program to determine what
privileges can be extended to the callee. Privileges in this context refer to
signers and writable accounts. For example, if the instruction the caller is
processing contains a signer or writable account, then the caller can invoke an
instruction that also contains that signer and/or writable account.
This privilege extension relies on the fact that programs are immutable. In the
case of the `acme` program, the runtime can safely treat the transaction's
signature as a signature of a `token` instruction. When the runtime sees the
`token` instruction references `alice_pubkey`, it looks up the key in the `acme`
instruction to see if that key corresponds to a signed account. In this case, it
does and thereby authorizes the `token` program to modify Alice's account.
### Program signed accounts
Programs can issue instructions that contain signed accounts that were not
signed in the original transaction by using [Program derived
addresses](#program-derived-addresses).
To sign an account with program derived addresses, a program may
`invoke_signed()`.
```rust,ignore
invoke_signed(
&instruction,
accounts,
&[&["First addresses seed"],
&["Second addresses first seed", "Second addresses second seed"]],
)?;
```
### Call Depth
Cross-program invocations allow programs to invoke other programs directly but
the depth is constrained currently to 4.
### Reentrancy
Reentrancy is currently limited to direct self recursion capped at a fixed
depth. This restriction prevents situations where a program might invoke another
from an intermediary state without the knowledge that it might later be called
back into. Direct recursion gives the program full control of its state at the
point that it gets called back.
## Program Derived Addresses
Program derived addresses allow programmaticly generated signature to be used
when [calling between programs](#cross-program-invocations).
Using a program derived address, a program may be given the authority over an
account and later transfer that authority to another. This is possible because
the program can act as the signer in the transaction that gives authority.
For example, if two users want to make a wager on the outcome of a game in
Solana, they must each transfer their wager's assets to some intermediary that
will honor their agreement. Currently, there is no way to implement this
intermediary as a program in Solana because the intermediary program cannot
transfer the assets to the winner.
This capability is necessary for many DeFi applications since they require
assets to be transferred to an escrow agent until some event occurs that
determines the new owner.
- Decentralized Exchanges that transfer assets between matching bid and ask
orders.
- Auctions that transfer assets to the winner.
- Games or prediction markets that collect and redistribute prizes to the
winners.
Program derived address:
1. Allow programs to control specific addresses, called program addresses, in
such a way that no external user can generate valid transactions with
signatures for those addresses.
2. Allow programs to programmatically sign for programa addresses that are
present in instructions invoked via [Cross-Program Invocations](#cross-program-invocations).
Given the two conditions, users can securely transfer or assign the authority of
on-chain assets to program addresses and the program can then assign that
authority elsewhere at its discretion.
### Private keys for program addresses
A Program address does not lie on the ed25519 curve and therefore has no valid
private key associated with it, and thus generating a signature for it is
impossible. While it has no private key of its own, it can be used by a program
to issue an instruction that includes the Program address as a signer.
### Hash-based generated program addresses
Program addresses are deterministically derived from a collection of seeds and a
program id using a 256-bit pre-image resistant hash function. Program address
must not lie on the ed25519 curve to ensure there is no associated private key.
During generation an error will be returned if the address is found to lie on
the curve. There is about a 50/50 change of this happening for a given
collection of seeds and program id. If this occurs a different set of seeds or
a seed bump (additional 8 bit seed) can be used to find a valid program address
off the curve.
Deterministic program addresses for programs follow a similar derivation path as
Accounts created with `SystemInstruction::CreateAccountWithSeed` which is
implemented with `system_instruction::create_address_with_seed`.
For reference that implementation is as follows:
```rust,ignore
pub fn create_address_with_seed(
base: &Pubkey,
seed: &str,
program_id: &Pubkey,
) -> Result<Pubkey, SystemError> {
if seed.len() > MAX_ADDRESS_SEED_LEN {
return Err(SystemError::MaxSeedLengthExceeded);
}
Ok(Pubkey::new(
hashv(&[base.as_ref(), seed.as_ref(), program_id.as_ref()]).as_ref(),
))
}
```
Programs can deterministically derive any number of addresses by using seeds.
These seeds can symbolically identify how the addresses are used.
From `Pubkey`::
```rust,ignore
/// Generate a derived program address
/// * seeds, symbolic keywords used to derive the key
/// * program_id, program that the address is derived for
pub fn create_program_address(
seeds: &[&[u8]],
program_id: &Pubkey,
) -> Result<Pubkey, PubkeyError>
```
### Using program addresses
Clients can use the `create_program_address` function to generate a destination
address.
```rust,ignore
// deterministically derive the escrow key
let escrow_pubkey = create_program_address(&[&["escrow"]], &escrow_program_id);
// construct a transfer message using that key
let message = Message::new(vec![
token_instruction::transfer(&alice_pubkey, &escrow_pubkey, 1),
]);
// process the message which transfer one 1 token to the escrow
client.send_and_confirm_message(&[&alice_keypair], &message);
```
Programs can use the same function to generate the same address. In the function
below the program issues a `token_instruction::transfer` from a program address
as if it had the private key to sign the transaction.
```rust,ignore
fn transfer_one_token_from_escrow(
program_id: &Pubkey,
keyed_accounts: &[KeyedAccount]
) -> Result<()> {
// User supplies the destination
let alice_pubkey = keyed_accounts[1].unsigned_key();
// Deterministically derive the escrow pubkey.
let escrow_pubkey = create_program_address(&[&["escrow"]], program_id);
// Create the transfer instruction
let instruction = token_instruction::transfer(&escrow_pubkey, &alice_pubkey, 1);
// The runtime deterministically derives the key from the currently
// executing program ID and the supplied keywords.
// If the derived address matches a key marked as signed in the instruction
// then that key is accepted as signed.
invoke_signed(&instruction, &[&["escrow"]])?
}
```
### Instructions that require signers
The addresses generated with `create_program_address` are indistinguishable from
any other public key. The only way for the runtime to verify that the address
belongs to a program is for the program to supply the seeds used to generate the
address.
The runtime will internally call `create_program_address`, and compare the
result against the addresses supplied in the instruction.
## Examples
Refer to [Developing with
Rust](developing/deployed-programs/../../../deployed-programs/developing-rust.md#examples)
and [Developing with
C](developing/deployed-programs/../../../deployed-programs/developing-c.md#examples)
for examples of how to use cross-program invocation.

61
docs/src/developing/programming-model/compute-budget.md
View File

@@ -1,61 +0,0 @@
---
title: "Compute budget"
---
To prevent a program from abusing computation resources each instruction in a
transaction is given a compute budget. The budget consists of computation units
that are consumed as the program performs various operations and bounds that the
program may not exceed. When the program consumes its entire budget or exceeds
a bound then the runtime halts the program and returns an error.
The following operations incur a compute cost:
- Executing BPF instructions
- Calling system calls
- logging
- creating program addresses
- cross-program invocations
- ...
For cross-program invocations the programs invoked inherit the budget of their
parent. If an invoked program consume the budget or exceeds a bound the entire
invocation chain and the parent are halted.
The current [compute
budget](https://github.com/solana-labs/solana/blob/d3a3a7548c857f26ec2cb10e270da72d373020ec/sdk/src/process_instruction.rs#L65)
can be found in the Solana SDK.
For example, if the current budget is:
```rust
max_units: 200,000,
log_units: 100,
log_u64_units: 100,
create_program address units: 1500,
invoke_units: 1000,
max_invoke_depth: 4,
max_call_depth: 64,
stack_frame_size: 4096,
log_pubkey_units: 100,
```
Then the program
- Could execute 200,000 BPF instructions if it does nothing else
- Could log 2,000 log messages
- Can not exceed 4k of stack usage
- Can not exceed a BPF call depth of 64
- Cannot exceed 4 levels of cross-program invocations.
Since the compute budget is consumed incrementally as the program executes the
total budget consumption will be a combination of the various costs of the
operations it performs.
At runtime a program may log how much of the compute budget remains. See
[debugging](developing/deployed-programs/debugging.md#monitoring-compute-budget-consumption)
for more information.
The budget values are conditional on feature enablement, take a look the compute
budget's
[new](https://github.com/solana-labs/solana/blob/d3a3a7548c857f26ec2cb10e270da72d373020ec/sdk/src/process_instruction.rs#L97)
function to find out how the budget is constructed. An understanding of how
[features](runtime-features.md) work and what features are enabled on the
cluster being used are required to determine the current budget's values.

155
docs/src/developing/programming-model/cpi.md
View File

@@ -1,155 +0,0 @@
---
title: Cross-Program Invocation
---
## Problem
In today's implementation, a client can create a transaction that modifies two
accounts, each owned by a separate on-chain program:
```rust,ignore
let message = Message::new(vec![
token_instruction::pay(&alice_pubkey),
acme_instruction::launch_missiles(&bob_pubkey),
]);
client.send_and_confirm_message(&[&alice_keypair, &bob_keypair], &message);
```
However, the current implementation does not allow the `acme` program to
conveniently invoke `token` instructions on the client's behalf:
```rust,ignore
let message = Message::new(vec![
acme_instruction::pay_and_launch_missiles(&alice_pubkey, &bob_pubkey),
]);
client.send_and_confirm_message(&[&alice_keypair, &bob_keypair], &message);
```
Currently, there is no way to create instruction `pay_and_launch_missiles` that
executes `token_instruction::pay` from the `acme` program. A possible workaround
is to extend the `acme` program with the implementation of the `token` program
and create `token` accounts with `ACME_PROGRAM_ID`, which the `acme` program is
permitted to modify. With that workaround, `acme` can modify token-like accounts
created by the `acme` program, but not token accounts created by the `token`
program.
## Solution
The goal of this design is to modify Solana's runtime such that an on-chain
program can invoke an instruction from another program.
Given two on-chain programs `token` and `acme`, each implementing instructions
`pay()` and `launch_missiles()` respectively, we would ideally like to implement
the `acme` module with a call to a function defined in the `token` module:
```rust,ignore
mod acme {
use token;
fn launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
...
}
fn pay_and_launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
token::pay(&accounts[1..])?;
launch_missiles(accounts)?;
}
```
The above code would require that the `token` crate be dynamically linked so
that a custom linker could intercept calls and validate accesses to
`accounts`. Even though the client intends to modify both `token` and
`acme` accounts, only `token` program is permitted to modify the `token`
account, and only the `acme` program is allowed to modify the `acme` account.
Backing off from that ideal direct cross-program call, a slightly more verbose
solution is to allow `acme` to invoke `token` by issuing a token instruction via
the runtime.
```rust,ignore
mod acme {
use token_instruction;
fn launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
...
}
fn pay_and_launch_missiles(accounts: &[AccountInfo]) -> Result<()> {
let alice_pubkey = accounts[1].key;
let instruction = token_instruction::pay(&alice_pubkey);
invoke(&instruction, accounts)?;
launch_missiles(accounts)?;
}
```
`invoke()` is built into Solana's runtime and is responsible for routing the
given instruction to the `token` program via the instruction's `program_id`
field.
Note that `invoke` requires the caller to pass all the accounts required by the
instruction being invoked. This means that both the executable account (the
ones that matches the instruction's program id) and the accounts passed to the
instruction procesor.
Before invoking `pay()`, the runtime must ensure that `acme` didn't modify any
accounts owned by `token`. It does this by applying the runtime's policy to the
current state of the accounts at the time `acme` calls `invoke` vs. the initial
state of the accounts at the beginning of the `acme`'s instruction. After
`pay()` completes, the runtime must again ensure that `token` didn't modify any
accounts owned by `acme` by again applying the runtime's policy, but this time
with the `token` program ID. Lastly, after `pay_and_launch_missiles()`
completes, the runtime must apply the runtime policy one more time, where it
normally would, but using all updated `pre_*` variables. If executing
`pay_and_launch_missiles()` up to `pay()` made no invalid account changes,
`pay()` made no invalid changes, and executing from `pay()` until
`pay_and_launch_missiles()` returns made no invalid changes, then the runtime
can transitively assume `pay_and_launch_missiles()` as whole made no invalid
account changes, and therefore commit all these account modifications.
### Instructions that require privileges
The runtime uses the privileges granted to the caller program to determine what
privileges can be extended to the callee. Privileges in this context refer to
signers and writable accounts. For example, if the instruction the caller is
processing contains a signer or writable account, then the caller can invoke an
instruction that also contains that signer and/or writable account.
This privilege extension relies on the fact that programs are immutable. In the
case of the `acme` program, the runtime can safely treat the transaction's
signature as a signature of a `token` instruction. When the runtime sees the
`token` instruction references `alice_pubkey`, it looks up the key in the `acme`
instruction to see if that key corresponds to a signed account. In this case, it
does and thereby authorizes the `token` program to modify Alice's account.
### Program signed accounts
Programs can issue instructions that contain signed accounts that were not
signed in the original transaction by using [Program derived
addresses](program-derived-addresses.md).
To sign an account with program derived addresses, a program may
`invoke_signed()`.
```rust,ignore
invoke_signed(
&instruction,
accounts,
&[&["First addresses seed"],
&["Second addresses first seed", "Second addresses second seed"]],
)?;
```
### Call Depth
Cross-program invocations allow programs to invoke other programs directly but
the depth is constrained currently to 4.
### Reentrancy
Reentrancy is currently limited to direct self recursion capped at a fixed
depth. This restriction prevents situations where a program might invoke another
from an intermediary state without the knowledge that it might later be called
back into. Direct recursion gives the program full control of its state at the
point that it gets called back.

12
docs/src/developing/programming-model/overview.md
View File

@@ -2,11 +2,13 @@
title: "Overview"
---
An _app_ interacts with a Solana cluster by sending it _transactions_ with one
or more _instructions_. The Solana _runtime_ passes those instructions to
_programs_ deployed by app developers beforehand. An instruction might, for
example, tell a program to transfer _lamports_ from one _account_ to another or
create an interactive contract that governs how lamports are transferred.
An [app](terminology.md#app) interacts with a Solana cluster by sending it
[transactions](transactions.md) with one or more
[instructions](transactions.md#instructions). The Solana [runtime](runtime.md)
passes those instructions to [programs](terminology.md#program) deployed by app developers
beforehand. An instruction might, for example, tell a program to transfer
[lamports](terminology.md#lamports) from one [account](accounts.md) to another
or create an interactive contract that governs how lamports are transferred.
Instructions are executed sequentially and atomically for each transaction. If
any instruction is invalid, all account changes in the transaction are
discarded.

159
docs/src/developing/programming-model/program-derived-addresses.md
View File

@@ -1,159 +0,0 @@
---
title: Program Derived Addresses
---
## Problem
Programs cannot generate signatures when issuing instructions to other programs
as defined in the [Cross-Program Invocations](cpi.md)
design.
The lack of programmatic signature generation limits the kinds of programs that
can be implemented in Solana. A program may be given the authority over an
account and later want to transfer that authority to another. This is impossible
today because the program cannot act as the signer in the transaction that gives
authority.
For example, if two users want to make a wager on the outcome of a game in
Solana, they must each transfer their wager's assets to some intermediary that
will honor their agreement. Currently, there is no way to implement this
intermediary as a program in Solana because the intermediary program cannot
transfer the assets to the winner.
This capability is necessary for many DeFi applications since they require
assets to be transferred to an escrow agent until some event occurs that
determines the new owner.
- Decentralized Exchanges that transfer assets between matching bid and ask
orders.
- Auctions that transfer assets to the winner.
- Games or prediction markets that collect and redistribute prizes to the
winners.
## Solution
The key to the design is two-fold:
1. Allow programs to control specific addresses, called program addresses, in
such a way that no external user can generate valid transactions with
signatures for those addresses.
2. Allow programs to programmatically sign for programa addresses that are
present in instructions invoked via [Cross-Program
Invocations](cpi.md).
Given the two conditions, users can securely transfer or assign the authority of
on-chain assets to program addresses and the program can then assign that
authority elsewhere at its discretion.
### Private keys for program addresses
A Program address does not lie on the ed25519 curve and therefore has no valid
private key associated with it, and thus generating a signature for it is
impossible. While it has no private key of its own, it can be used by a program
to issue an instruction that includes the Program address as a signer.
### Hash-based generated program addresses
Program addresses are deterministically derived from a collection of seeds and a
program id using a 256-bit pre-image resistant hash function. Program address
must not lie on the ed25519 curve to ensure there is no associated private key.
During generation an error will be returned if the address is found to lie on
the curve. There is about a 50/50 change of this happening for a given
collection of seeds and program id. If this occurs a different set of seeds or
a seed bump (additional 8 bit seed) can be used to find a valid program address
off the curve.
Deterministic program addresses for programs follow a similar derivation path as
Accounts created with `SystemInstruction::CreateAccountWithSeed` which is
implemented with `system_instruction::create_address_with_seed`.
For reference that implementation is as follows:
```rust,ignore
pub fn create_address_with_seed(
base: &Pubkey,
seed: &str,
program_id: &Pubkey,
) -> Result<Pubkey, SystemError> {
if seed.len() > MAX_ADDRESS_SEED_LEN {
return Err(SystemError::MaxSeedLengthExceeded);
}
Ok(Pubkey::new(
hashv(&[base.as_ref(), seed.as_ref(), program_id.as_ref()]).as_ref(),
))
}
```
Programs can deterministically derive any number of addresses by using seeds.
These seeds can symbolically identify how the addresses are used.
From `Pubkey`::
```rust,ignore
/// Generate a derived program address
/// * seeds, symbolic keywords used to derive the key
/// * program_id, program that the address is derived for
pub fn create_program_address(
seeds: &[&[u8]],
program_id: &Pubkey,
) -> Result<Pubkey, PubkeyError>
```
### Using program addresses
Clients can use the `create_program_address` function to generate a destination
address.
```rust,ignore
// deterministically derive the escrow key
let escrow_pubkey = create_program_address(&[&["escrow"]], &escrow_program_id);
// construct a transfer message using that key
let message = Message::new(vec![
token_instruction::transfer(&alice_pubkey, &escrow_pubkey, 1),
]);
// process the message which transfer one 1 token to the escrow
client.send_and_confirm_message(&[&alice_keypair], &message);
```
Programs can use the same function to generate the same address. In the function
below the program issues a `token_instruction::transfer` from a program address
as if it had the private key to sign the transaction.
```rust,ignore
fn transfer_one_token_from_escrow(
program_id: &Pubkey,
keyed_accounts: &[KeyedAccount]
) -> Result<()> {
// User supplies the destination
let alice_pubkey = keyed_accounts[1].unsigned_key();
// Deterministically derive the escrow pubkey.
let escrow_pubkey = create_program_address(&[&["escrow"]], program_id);
// Create the transfer instruction
let instruction = token_instruction::transfer(&escrow_pubkey, &alice_pubkey, 1);
// The runtime deterministically derives the key from the currently
// executing program ID and the supplied keywords.
// If the derived address matches a key marked as signed in the instruction
// then that key is accepted as signed.
invoke_signed(&instruction, &[&["escrow"]])?
}
```
### Instructions that require signers
The addresses generated with `create_program_address` are indistinguishable from
any other public key. The only way for the runtime to verify that the address
belongs to a program is for the program to supply the seeds used to generate the
address.
The runtime will internally call `create_program_address`, and compare the
result against the addresses supplied in the instruction.

26
docs/src/developing/programming-model/runtime-features.md
View File

@@ -1,26 +0,0 @@
---
title: "Runtime Features"
---
As Solana evolves, new features or patches may be introduced that changes the
behavior of the cluster and how programs run. Changes in behavior must be
coordinated between the various nodes of the cluster, if nodes do not coordinate
then these changes can result in a break-down of consensus. Solana supports a
mechanism called runtime features to facilitate the smooth adoption of changes.
Runtime features are epoch coordinated events where one or more behavior changes
to the cluster will occur. New changes to Solana that will change behavior are
wrapped with feature gates and disabled by default. The Solana tools are then
used to activate a feature, which marks it pending, once marked pending the
feature will be activated at the next epoch.
To determine which features are activated use the [Solana command-line
tools](cli/install-solana-cli-tools.md):
```bash
solana feature status
```
If you encounter problems first ensure that the Solana tools version you are
using match the version returned by `solana cluster-version`. If they do not
match [install the correct tool suite](cli/install-solana-cli-tools.md).

127
docs/src/developing/programming-model/runtime.md Normal file
View File

@@ -0,0 +1,127 @@
---
title: "Runtime"
---
## Capability of Programs
The runtime only permits the owner program to debit the account or modify its
data. The program then defines additional rules for whether the client can
modify accounts it owns. In the case of the System program, it allows users to
transfer lamports by recognizing transaction signatures. If it sees the client
signed the transaction using the keypair's _private key_, it knows the client
authorized the token transfer.
In other words, the entire set of accounts owned by a given program can be
regarded as a key-value store where a key is the account address and value is
program-specific arbitrary binary data. A program author can decide how to
manage the program's whole state as possibly many accounts.
After the runtime executes each of the transaction's instructions, it uses the
account metadata to verify that the access policy was not violated. If a program
violates the policy, the runtime discards all account changes made by all
instructions in the transaction and marks the transaction as failed.
### Policy
After a program has processed an instruction the runtime verifies that the
program only performed operations it was permitted to, and that the results
adhere to the runtime policy.
The policy is as follows:
- Only the owner of the account may change owner.
- And only if the account is writable.
- And only if the data is zero-initialized or empty.
- An account not assigned to the program cannot have its balance decrease.
- The balance of read-only and executable accounts may not change.
- Only the system program can change the size of the data and only if the system
program owns the account.
- Only the owner may change account data.
- And if the account is writable.
- And if the account is not executable.
- Executable is one-way (false->true) and only the account owner may set it.
- No one modification to the rent_epoch associated with this account.
## Compute Budget
To prevent a program from abusing computation resources each instruction in a
transaction is given a compute budget. The budget consists of computation units
that are consumed as the program performs various operations and bounds that the
program may not exceed. When the program consumes its entire budget or exceeds
a bound then the runtime halts the program and returns an error.
The following operations incur a compute cost:
- Executing BPF instructions
- Calling system calls
- logging
- creating program addresses
- cross-program invocations
- ...
For cross-program invocations the programs invoked inherit the budget of their
parent. If an invoked program consume the budget or exceeds a bound the entire
invocation chain and the parent are halted.
The current [compute
budget](https://github.com/solana-labs/solana/blob/d3a3a7548c857f26ec2cb10e270da72d373020ec/sdk/src/process_instruction.rs#L65)
can be found in the Solana SDK.
For example, if the current budget is:
```rust
max_units: 200,000,
log_units: 100,
log_u64_units: 100,
create_program address units: 1500,
invoke_units: 1000,
max_invoke_depth: 4,
max_call_depth: 64,
stack_frame_size: 4096,
log_pubkey_units: 100,
```
Then the program
- Could execute 200,000 BPF instructions if it does nothing else
- Could log 2,000 log messages
- Can not exceed 4k of stack usage
- Can not exceed a BPF call depth of 64
- Cannot exceed 4 levels of cross-program invocations.
Since the compute budget is consumed incrementally as the program executes the
total budget consumption will be a combination of the various costs of the
operations it performs.
At runtime a program may log how much of the compute budget remains. See
[debugging](developing/deployed-programs/debugging.md#monitoring-compute-budget-consumption)
for more information.
The budget values are conditional on feature enablement, take a look the compute
budget's
[new](https://github.com/solana-labs/solana/blob/d3a3a7548c857f26ec2cb10e270da72d373020ec/sdk/src/process_instruction.rs#L97)
function to find out how the budget is constructed. An understanding of how
[features](runtime.md#features) work and what features are enabled on the
cluster being used are required to determine the current budget's values.
## New Features
As Solana evolves, new features or patches may be introduced that changes the
behavior of the cluster and how programs run. Changes in behavior must be
coordinated between the various nodes of the cluster, if nodes do not coordinate
then these changes can result in a break-down of consensus. Solana supports a
mechanism called runtime features to facilitate the smooth adoption of changes.
Runtime features are epoch coordinated events where one or more behavior changes
to the cluster will occur. New changes to Solana that will change behavior are
wrapped with feature gates and disabled by default. The Solana tools are then
used to activate a feature, which marks it pending, once marked pending the
feature will be activated at the next epoch.
To determine which features are activated use the [Solana command-line
tools](cli/install-solana-cli-tools.md):
```bash
solana feature status
```
If you encounter problems first ensure that the Solana tools version you are
using match the version returned by `solana cluster-version`. If they do not
match [install the correct tool suite](cli/install-solana-cli-tools.md).

92
docs/src/developing/programming-model/secpk1-instructions.md
View File

@@ -1,92 +0,0 @@
---
title: secp256k1 builtin instructions
---
## Problem
Performing multiple secp256k1 pubkey recovery operations (ecrecover) in BPF
would exceed the transction bpf instruction limit and even if the limit is
increased it would take a long time to process. ecrecover is an ethereum
instruction which takes a signature and message and recovers a publickey, a
comparison to that public key can thus verify that the signature is valid.
Since there needs to be 10-20 signatures in the transaction as well as the
signing data which is on the order of 500 bytes, transaction space is a concern.
But also having more concentrated similar work should provide for easier
optimization.
## Solution
Add a new builtin instruction which takes in as the first byte a count of the
following struct serialized in the instruction data:
```
struct Secp256k1SignatureOffsets {
secp_signature_key_offset: u16, // offset to [signature,recovery_id,etherum_address] of 64+1+20 bytes
secp_signature_instruction_index: u8, // instruction index to find data
secp_pubkey_offset: u16, // offset to [signature,recovery_id] of 64+1 bytes
secp_signature_instruction_index: u8, // instruction index to find data
secp_message_data_offset: u16, // offset to start of message data
secp_message_data_size: u16, // size of message data
secp_message_instruction_index: u8, // index of instruction data to get message data
}
```
Pseudo code of the operation:
```
process_instruction() {
for i in 0..count {
// i'th index values referenced:
instructions = &transaction.message().instructions
signature = instructions[secp_signature_instruction_index].data[secp_signature_offset..secp_signature_offset + 64]
recovery_id = instructions[secp_signature_instruction_index].data[secp_signature_offset + 64]
ref_eth_pubkey = instructions[secp_pubkey_instruction_index].data[secp_pubkey_offset..secp_pubkey_offset + 32]
message_hash = keccak256(instructions[secp_message_instruction_index].data[secp_message_data_offset..secp_message_data_offset + secp_message_data_size])
pubkey = ecrecover(signature, recovery_id, message_hash)
eth_pubkey = keccak256(pubkey[1..])[12..]
if eth_pubkey != ref_eth_pubkey {
return Error
}
}
return Success
}
```
This allows the user to specify any instruction data in the transaction for
signature and message data. By specifying a special instructions sysvar, one can
also receive data from the transaction itself.
Cost of the transaction will count the number of signatures to verify multiplied
by the signature cost verify multiplier.
## Optimization notes
The operation will have to take place after (at least partial) deserialization,
but all inputs come from the transaction data itself, this allows it to be
relatively easy to execute in parallel to transaction processing and PoH
verification.
## Other solutions
* Instruction available as CPI such that the program can call as desired or a
syscall which can operate on the instruction inline.
- Could be harder to optimize given that it generally either requires bpf
program scan to determine the inputs to the operation, or the
implementation needs to just wait until the program hits the operation in
bpf processing to evaluate it.
- Vector version of the operation could allow for somewhat efficient simd/gpu
execution. For most efficient though, batching with other instructions in
the pipeline would be ideal.
- Pros - Nicer interface for the user.
* Async execution environment inside bpf
- Might be hard to optimize for devices like gpus which cannot queue work for
itself easily
- Might be easier to optimize on cpu since ordering can be more explicit
* All inputs have to come from the instruction
- Pros - easier to optimize, data is already sent to the GPU for instance for
regular sigverify. Probably still need to wait for deserialize though.
- Cons - ask for pubkeys outside the transaction data itself since they would
not be stored on the transaction sending client, and larger transaction
size.

121
docs/src/developing/programming-model/sysvars.md
View File

@@ -1,121 +0,0 @@
---
title: Sysvar Cluster Data
---
Solana exposes a variety of cluster state data to programs via
[`sysvar`](terminology.md#sysvar) accounts. These accounts are populated at
known addresses published along with the account layouts in the
[`solana-program`
crate](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/sysvar/index.html),
and outlined below.
To include sysvar data in program operations, pass the sysvar account address in
the list of accounts in a transaction. The account can be read in your
instruction processor like any other account. Access to sysvars accounts ßis
always *readonly*.
## Clock
The Clock sysvar contains data on cluster time, including the current slot,
epoch, and estimated wall-clock Unix timestamp. It is updated every slot.
- Address: `SysvarC1ock11111111111111111111111111111111`
- Layout: [Clock](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/clock/struct.Clock.html)
- Fields:
- `slot`: the current slot
- `epoch_start_timestamp`: the Unix timestamp of the first slot in this epoch. In the first slot of an epoch, this timestamp is identical to the `unix_timestamp` (below).
- `epoch`: the current epoch
- `leader_schedule_epoch`: the most recent epoch for which the leader schedule has already been generated
- `unix_timestamp`: the Unix timestamp of this slot.
Each slot has an estimated duration based on Proof of History. But in reality,
slots may elapse faster and slower than this estimate. As a result, the Unix
timestamp of a slot is generated based on oracle input from voting validators.
This timestamp is calculated as the stake-weighted median of timestamp
estimates provided by votes, bounded by the expected time elapsed since the
start of the epoch.
More explicitly: for each slot, the most recent vote timestamp provided by
each validator is used to generate a timestamp estimate for the current slot
(the elapsed slots since the vote timestamp are assumed to be
Bank::ns_per_slot). Each timestamp estimate is associated with the stake
delegated to that vote account to create a distribution of timestamps by
stake. The median timestamp is used as the `unix_timestamp`, unless the
elapsed time since the `epoch_start_timestamp` has deviated from the expected
elapsed time by more than 25%.
## EpochSchedule
The EpochSchedule sysvar contains epoch scheduling constants that are set in
genesis, and enables calculating the number of slots in a given epoch, the epoch
for a given slot, etc. (Note: the epoch schedule is distinct from the [`leader
schedule`](terminology.md#leader-schedule))
- Address: `SysvarEpochSchedu1e111111111111111111111111`
- Layout:
[EpochSchedule](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/epoch_schedule/struct.EpochSchedule.html)
## Fees
The Fees sysvar contains the fee calculator for the current slot. It is updated
every slot, based on the fee-rate governor.
- Address: `SysvarFees111111111111111111111111111111111`
- Layout:
[Fees](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/sysvar/fees/struct.Fees.html)
## Instructions
The Instructions sysvar contains the serialized instructions in a Message while
that Message is being processed. This allows program instructions to reference
other instructions in the same transaction. Read more information on
[instruction introspection](implemented-proposals/instruction_introspection.md).
- Address: `Sysvar1nstructions1111111111111111111111111`
- Layout:
[Instructions](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/sysvar/instructions/type.Instructions.html)
## RecentBlockhashes
The RecentBlockhashes sysvar contains the active recent blockhashes as well as
their associated fee calculators. It is updated every slot.
- Address: `SysvarRecentB1ockHashes11111111111111111111`
- Layout:
[RecentBlockhashes](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/sysvar/recent_blockhashes/struct.RecentBlockhashes.html)
## Rent
The Rent sysvar contains the rental rate. Currently, the rate is static and set
in genesis. The Rent burn percentage is modified by manual feature activation.
- Address: `SysvarRent111111111111111111111111111111111`
- Layout:
[Rent](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/rent/struct.Rent.html)
## SlotHashes
The SlotHashes sysvar contains the most recent hashes of the slot's parent
banks. It is updated every slot.
- Address: `SysvarS1otHashes111111111111111111111111111`
- Layout:
[SlotHashes](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/slot_hashes/struct.SlotHashes.html)
## SlotHistory
The SlotHistory sysvar contains a bitvector of slots present over the last
epoch. It is updated every slot.
- Address: `SysvarS1otHistory11111111111111111111111111`
- Layout:
[SlotHistory](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/slot_history/struct.SlotHistory.html)
## StakeHistory
The StakeHistory sysvar contains the history of cluster-wide stake activations
and de-activations per epoch. It is updated at the start of every epoch.
- Address: `SysvarStakeHistory1111111111111111111111111`
- Layout:
[StakeHistory](https://docs.rs/solana-program/VERSION_FOR_DOCS_RS/solana_program/stake_history/struct.StakeHistory.html)

97
docs/src/developing/programming-model/transactions.md
View File

@@ -7,8 +7,81 @@ submitted to the cluster. The Solana runtime will execute a program to process
each of the [instructions](terminology.md#instruction) contained in the
transaction, in order, and atomically.
See [Anatomy of a Transaction](transaction.md) for more information about how a
transaction is encoded.
## Anatomy of a Transaction
This section covers the binary format of a transaction.
### Transaction Format
A transaction contains a [compact-array](#compact-array-format) of signatures,
followed by a [message](#message-format). Each item in the signatures array is
a [digital signature](#signature-format) of the given message. The Solana
runtime verifies that the number of signatures matches the number in the first
8 bits of the [message header](#message-header-format). It also verifies that
each signature was signed by the private key corresponding to the public key at
the same index in the message's account addresses array.
#### Signature Format
Each digital signature is in the ed25519 binary format and consumes 64 bytes.
### Message Format
A message contains a [header](#message-header-format), followed by a
compact-array of [account addresses](#account-addresses-format), followed by a
recent [blockhash](#blockhash-format), followed by a compact-array of
[instructions](#instruction-format).
#### Message Header Format
The message header contains three unsigned 8-bit values. The first value is the
number of required signatures in the containing transaction. The second value
is the number of those corresponding account addresses that are read-only. The
third value in the message header is the number of read-only account addresses
not requiring signatures.
#### Account Addresses Format
The addresses that require signatures appear at the beginning of the account
address array, with addresses requesting write access first and read-only
accounts following. The addresses that do not require signatures follow the
addresses that do, again with read-write accounts first and read-only accounts
following.
#### Blockhash Format
A blockhash contains a 32-byte SHA-256 hash. It is used to indicate when a
client last observed the ledger. Validators will reject transactions when the
blockhash is too old.
### Instruction Format
An instruction contains a program id index, followed by a compact-array of
account address indexes, followed by a compact-array of opaque 8-bit data. The
program id index is used to identify an on-chain program that can interpret the
opaque data. The program id index is an unsigned 8-bit index to an account
address in the message's array of account addresses. The account address
indexes are each an unsigned 8-bit index into that same array.
### Compact-Array Format
A compact-array is serialized as the array length, followed by each array item.
The array length is a special multi-byte encoding called compact-u16.
#### Compact-u16 Format
A compact-u16 is a multi-byte encoding of 16 bits. The first byte contains the
lower 7 bits of the value in its lower 7 bits. If the value is above 0x7f, the
high bit is set and the next 7 bits of the value are placed into the lower 7
bits of a second byte. If the value is above 0x3fff, the high bit is set and
the remaining 2 bits of the value are placed into the lower 2 bits of a third
byte.
### Account Address Format
An account address is 32-bytes of arbitrary data. When the address requires a
digital signature, the runtime interprets it as the public key of an ed25519
keypair.
## Instructions
@@ -53,14 +126,22 @@ Which can be found here:
https://github.com/solana-labs/solana/blob/6606590b8132e56dab9e60b3f7d20ba7412a736c/sdk/program/src/system_instruction.rs#L220
### Program ID
### Program Id
The instruction's [program id](terminology.md#program-id) specifies which
program will process this instruction. The program's account data contains
information about how the runtime should execute the program, in the case of BPF
programs, the account data holds the BPF bytecode. Program accounts are marked
as executable once they are successfully deployed. The runtime will reject
transactions that specify programs that are not executable.
program will process this instruction. The program's account's owner specifies
which loader should be used to load and execute the program and the data
contains information about how the runtime should execute the program.
In the case of [deployed BPF
programs](developing/deployed-programs/overview.md), the owner is the BPF Loader
and the account data holds the BPF bytecode. Program accounts are permanently
marked as executable by the loader once they are successfully deployed. The
runtime will reject transactions that specify programs that are not executable.
Unlike deployed programs, [builtins](developing/builtins/programs.md) are handled
differently in that they are built directly into the Solana runtime.
### Accounts