core, eth/downloader: commit block data using batches (#15115)
* ethdb: add Putter interface and Has method * ethdb: improve docs and add IdealBatchSize * ethdb: remove memory batch lock Batches are not safe for concurrent use. * core: use ethdb.Putter for Write* functions This covers the easy cases. * core/state: simplify StateSync * trie: optimize local node check * ethdb: add ValueSize to Batch * core: optimize HasHeader check This avoids one random database read get the block number. For many uses of HasHeader, the expectation is that it's actually there. Using Has avoids a load + decode of the value. * core: write fast sync block data in batches Collect writes into batches up to the ideal size instead of issuing many small, concurrent writes. * eth/downloader: commit larger state batches Collect nodes into a batch up to the ideal size instead of committing whenever a node is received. * core: optimize HasBlock check This avoids a random database read to get the number. * core: use numberCache in HasHeader numberCache has higher capacity, increasing the odds of finding the header without a database lookup. * core: write imported block data using a batch Restore batch writes of state and add blocks, tx entries, receipts to the same batch. The change also simplifies the miner. This commit also removes posting of logs when a forked block is imported. * core: fix DB write error handling * ethdb: use RLock for Has * core: fix HasBlock comment
This commit is contained in:
		
				
					committed by
					
						 Péter Szilágyi
						Péter Szilágyi
					
				
			
			
				
	
			
			
			
						parent
						
							ac193e36ce
						
					
				
				
					commit
					10181b57a9
				
			| @@ -16,15 +16,29 @@ | ||||
|  | ||||
| package ethdb | ||||
|  | ||||
| type Database interface { | ||||
| // Code using batches should try to add this much data to the batch. | ||||
| // The value was determined empirically. | ||||
| const IdealBatchSize = 100 * 1024 | ||||
|  | ||||
| // Putter wraps the database write operation supported by both batches and regular databases. | ||||
| type Putter interface { | ||||
| 	Put(key []byte, value []byte) error | ||||
| } | ||||
|  | ||||
| // Database wraps all database operations. All methods are safe for concurrent use. | ||||
| type Database interface { | ||||
| 	Putter | ||||
| 	Get(key []byte) ([]byte, error) | ||||
| 	Has(key []byte) (bool, error) | ||||
| 	Delete(key []byte) error | ||||
| 	Close() | ||||
| 	NewBatch() Batch | ||||
| } | ||||
|  | ||||
| // Batch is a write-only database that commits changes to its host database | ||||
| // when Write is called. Batch cannot be used concurrently. | ||||
| type Batch interface { | ||||
| 	Put(key, value []byte) error | ||||
| 	Putter | ||||
| 	ValueSize() int // amount of data in the batch | ||||
| 	Write() error | ||||
| } | ||||
|   | ||||
		Reference in New Issue
	
	Block a user