Page MenuHomePhabricator

[Chronik] Optimize `GroupHistoryWriter` using batching and RocksDB's merge operator
ClosedPublic

Authored by tobias_ruck on Aug 21 2023, 19:44.

Details

Summary

Employ the following strategies:

  1. Add a new column family, script_history_num_txs, which tracks the number of txs for each script. This tells us directly which page we need to append new txs to and how many without looking up anything else.
  2. Use a "merge operator" (available in RocksDB, see https://github.com/facebook/rocksdb/wiki/Merge-Operator), which simply concats bytes to pages (for insert) or trims them (for delete).
  3. Store pages not as serialized Vec<TxNum>, but as concatenated byte stream of serialized TxNum. This allows us to simply byte-append serialized tx nums, which simplifies the merge operator.
  4. Use multi_get when querying the num of existing txs for scripts, which allows RocksDB to fetch data more efficiently than when querying one-by-one.

Data that has been gathered + methodology for the optimization can be found here:
https://marble-timimus-187.notion.site/Bitcoin-ABC-Chronik-082e0ebbb02343198c79d2c58d2e9b45?pvs=4

Test Plan

ninja check-crates && ninja check-functional

Diff Detail

Repository
rABC Bitcoin ABC
Branch
chronik-optimize-group-history
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 24839
Build 49268: Build Diffbuild-chronik
Build 49267: arc lint + arc unit

Event Timeline

Fabien requested changes to this revision.Aug 21 2023, 21:30
Fabien added inline comments.
chronik/chronik-indexer/src/indexer.rs
148 ↗(On Diff #41878)

You should at least hide this behind a flag, it's not useful for the end user

172 ↗(On Diff #41878)

Did you just ignore and override the -chronikscriptnumtxscachesize which is in the params ?

245 ↗(On Diff #41878)

.ron ?

This revision now requires changes to proceed.Aug 21 2023, 21:30

actually use -chronikscriptnumtxscache

Fabien requested changes to this revision.Aug 22 2023, 08:54

One done, 2 items remaining. Per our telegram discussion:

  • Update the .ron to something else
  • Add a flag to gate the perf data generation, move this to its own diff and rebase this one on top
This revision now requires changes to proceed.Aug 22 2023, 08:54
This revision is now accepted and ready to land.Aug 22 2023, 16:38

Remove LRU cache, empirically, a disabled cache is faster, and the bigger the cache, the longer insert takes:

0 cache size: 1463s, 0% cache utilization
10k cache size: 1500s, 30% cache utilization
100k cache size: 1590s, 51% cache utilization
1M cache size: 1654s, 65% cache utilization

Remove DEFAULT_SCRIPT_NUM_TXS_CACHE_SIZE const

tobias_ruck retitled this revision from [Chronik] Optimize `GroupHistoryWriter` using batching, merging and LRU caching to [Chronik] Optimize `GroupHistoryWriter` using batching and RocksDB's merge operator.Aug 24 2023, 12:15
tobias_ruck edited the summary of this revision. (Show Details)