Details

Reviewers

Fabien

Group Reviewers

Restricted Project

Commits

rABC4b031fd0029c: [Chronik] Optimize `prepare_indexed_txs` by using batch queries and removing…

Summary

Currently, prepare_indexed_txs is doubly inefficient:

If two inputs spend two outputs from the same tx, it'll query the tx num of this tx twice
It queries tx nums one-by-one, instead of utilizing the optimized batch query gets that RocksDB provides

We fix this by adding batch query versions to LookupColumn and TxReader, and then utilize those in prepare_indexed_txs.

Note: We can't use sorted_inputs anywhere in this function when querying input_tx_nums, since those can come in a random order.

Benchmarks show there's a speedup of about 1.6 when syncing the first 300000ish blocks:

bench	total time [s]	time prepare [s]
no prepare	1066.85	0
master	4375.39	3308.54
this diff	3148.51	2081.66

Test Plan

ninja check-crates && ninja check-functional

Diff Detail

Repository

rABC Bitcoin ABC

Branch

chronik-optimize-prepare_indexed_txs

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 27506
Build 54575: Build Diff	build-chronik-plugins · build-chronik · chronik-client-integration-tests
Build 54574: arc lint + arc unit

Event Timeline

tobias_ruck created this revision.Feb 28 2024, 11:37

Herald added a reviewer: Restricted Project. · View Herald TranscriptFeb 28 2024, 11:37

tobias_ruck edited the summary of this revision. (Show Details)Feb 28 2024, 11:38

Harbormaster completed remote builds in B27489: Diff 45700.Feb 28 2024, 11:58

add a bit of test coverage

Harbormaster completed remote builds in B27498: Diff 45709.Feb 28 2024, 13:32

remove wrong usage of sorted_inputs

Harbormaster completed remote builds in B27506: Diff 45720.Feb 28 2024, 18:44

Build Bitcoin ABC Diffs / Diff Testing (chronik-client-integration-tests) failed.

Failed tests logs:

====== Get blocktxs, txs, and history for SLP NFT1 token txs: "before each" hook for "Gets an SLP NFT1 child genesis tx from the mempool".Get blocktxs, txs, and history for SLP NFT1 token txs "before each" hook for "Gets an SLP NFT1 child genesis tx from the mempool" ======
Error: Timeout of 60000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/work/modules/chronik-client/test/integration/token_slp_nft1.ts)
    at listOnTimeout (node:internal/timers:573:17)
    at processTimers (node:internal/timers:514:7)

Each failure log is accessible here:
Get blocktxs, txs, and history for SLP NFT1 token txs: "before each" hook for "Gets an SLP NFT1 child genesis tx from the mempool".Get blocktxs, txs, and history for SLP NFT1 token txs "before each" hook for "Gets an SLP NFT1 child genesis tx from the mempool"