investigate system performance test degradation (#17919)

* Add stats and counter around cost model ops, mainly:
- calculate transaction cost
- check transaction can fit in a block
- update block cost tracker after transactions are added to block
- replay_stage to update/insert execution cost to table

* Change mutex on cost_tracker to RwLock

* removed cloning cost_tracker for local use, as the metrics show clone is very expensive.

* acquire and hold locks for block of TXs, instead of acquire and release per transaction;

* remove redundant would_fit check from cost_tracker update execution path

* refactor cost checking with less frequent lock acquiring

* avoid many Transaction_cost heap allocation when calculate cost, which
is in the hot path - executed per transaction.

* create hashmap with new_capacity to reduce runtime heap realloc.

* code review changes: categorize stats, replace explicit drop calls, concisely initiate to default

* address potential deadlock by acquiring locks one at time

core/src/execute_cost_table.rs

@@ -33,8 +33,8 @@ impl ExecuteCostTable {
     pub fn new(cap: usize) -> Self {
         Self {
             capacity: cap,
-            table: HashMap::new(),
-            occurrences: HashMap::new(),
+            table: HashMap::with_capacity(cap),
+            occurrences: HashMap::with_capacity(cap),
         }
     }