core/rawdb: freezer batch write (#23462)
This change is a rewrite of the freezer code. When writing ancient chain data to the freezer, the previous version first encoded each individual item to a temporary buffer, then wrote the buffer. For small item sizes (for example, in the block hash freezer table), this strategy causes a lot of system calls for writing tiny chunks of data. It also allocated a lot of temporary []byte buffers. In the new version, we instead encode multiple items into a re-useable batch buffer, which is then written to the file all at once. This avoids performing a system call for every inserted item. To make the internal batching work, the ancient database API had to be changed. While integrating this new API in BlockChain.InsertReceiptChain, additional optimizations were also added there. Co-authored-by: Felix Lange <fjl@twurst.com>
This commit is contained in:
committed by
GitHub
parent
9a0df80bbc
commit
794c6133ef
@ -92,9 +92,10 @@ type AncientReader interface {
|
||||
|
||||
// AncientWriter contains the methods required to write to immutable ancient data.
|
||||
type AncientWriter interface {
|
||||
// AppendAncient injects all binary blobs belong to block at the end of the
|
||||
// append-only immutable table files.
|
||||
AppendAncient(number uint64, hash, header, body, receipt, td []byte) error
|
||||
// ModifyAncients runs a write operation on the ancient store.
|
||||
// If the function returns an error, any changes to the underlying store are reverted.
|
||||
// The integer return value is the total size of the written data.
|
||||
ModifyAncients(func(AncientWriteOp) error) (int64, error)
|
||||
|
||||
// TruncateAncients discards all but the first n ancient data from the ancient store.
|
||||
TruncateAncients(n uint64) error
|
||||
@ -103,6 +104,15 @@ type AncientWriter interface {
|
||||
Sync() error
|
||||
}
|
||||
|
||||
// AncientWriteOp is given to the function argument of ModifyAncients.
|
||||
type AncientWriteOp interface {
|
||||
// Append adds an RLP-encoded item.
|
||||
Append(kind string, number uint64, item interface{}) error
|
||||
|
||||
// AppendRaw adds an item without RLP-encoding it.
|
||||
AppendRaw(kind string, number uint64, item []byte) error
|
||||
}
|
||||
|
||||
// Reader contains the methods required to read data from both key-value as well as
|
||||
// immutable ancient data.
|
||||
type Reader interface {
|
||||
|
Reference in New Issue
Block a user