9 Commits

Author SHA1 Message Date
mergify[bot]
77558c315d
Fixed issue #22124 -- missing historical data if slot updated later. (#22193) (#22259)
* Fixed issue #22124 -- missing historical data if slot updated later.

* Fixed a couple of comments

(cherry picked from commit 5b6027bef0430047a4b2b7ab26b1366a798abd32)

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
2022-01-04 07:18:58 +00:00
mergify[bot]
6a556c5adb
Stream additional block metadata via plugin (#22023) (#22179)
* Stream additional block metadata through plugin
blockhash, block_height, block_time, rewards are streamed

(cherry picked from commit f14928a970b03334a18450e43e5b750f23a7c474)

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
2021-12-30 05:44:12 +00:00
Lijun Wang
f5b0764795
Accountsdb plugin transaction part 4 -- postgres plugin implementations -- DB models (#21407)
AccountsDb plugin for transactions -- part 4 -- persisting to the Db.
1. DB models for transactions
2. Rust models for transactions
3. Transform from SDK models to rust db models
4. Unit tests
2021-12-01 09:23:26 -08:00
Lijun Wang
696501500f
Accountsdb plugin postgres improvement (#21034)
Summary of Changes

Added the reference postgresql.conf
Prepare slot update statement to reduce overhead in updating slot
Support custom connection string
Allow the plugin to panic on replication issues to ensure consistency
2021-10-30 20:18:11 -07:00
Lijun Wang
bbe3ce3db5
Accountsdb plugin write ordering (#20948)
Use the write_version in the Accounts's meta data so that account write with lower write_version would not overwrite the higher ones.
2021-10-25 14:07:56 -07:00
Lijun Wang
f14365f4b4
Accountsdb plugin postgres -- bulk insertion at startup (#20763)
Use bulk insertion to Postgres at startup to reduce time taken for initial snapshot restore for postgres plugin. Avoid duplicate writes of accounts at startup. Doing account plugin notification and indexing in parallel.

Improved error handling for postgres plugin to show the real db issues for debug purpose
Added more metrics for postgres plugin.
Refactored plugin centric code out to a sub module from accounts_db and added unit tests
2021-10-24 12:43:33 -07:00
Lijun Wang
ad0a88f1f2
Support port number in postgres connection (#20662)
* Support port number in postgres connection

* Addressed a few comments from Trent
2021-10-14 11:55:10 -07:00
Lijun Wang
08e40bf32e
Accountsdb plugin metrics (#20606)
Added metrics for accountsdb plugin
Handle and log postgres db errors
Print account pubkeys nicely in logging
2021-10-13 08:36:39 -07:00
Lijun Wang
d621994fee
Accountsdb stream plugin improvement (#20419)
Support using connection pooling and use multiple threads to do Postgres db operations. The performance is improved from 1500 RPS to 40,000 RPS measured during validator start.

Support multiple plugins at the same time.
2021-10-08 20:06:58 -07:00