Page MenuHomePhabricator

[Chronik] Non-allocating `ScriptGroupIter` for `ScriptGroup`
ClosedPublic

Authored by tobias_ruck on Feb 26 2024, 08:53.

Details

Summary

Removing the need to allocate here and instead iterate over the tx directly improves performance slightly.

In D15552, in the WS loop, applying this diff sped it up to 850.29s from 969.39s when resyncing the first 300000ish blocks. This loop is now removed when empty, but we can predict a similar improvement of 150s-300s based on that benchmark for GroupHistoryWriter and GroupUtxoWriter.

Test Plan

ninja check-crates && ninja check-functional

Diff Detail

Repository
rABC Bitcoin ABC
Branch
chronik-non-allocating-group-iter
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 27427
Build 54417: Build Diffchronik-client-integration-tests · build-chronik-plugins · build-chronik
Build 54416: arc lint + arc unit

Event Timeline

Tail of the build log:

   Compiling libc v0.2.147
   Compiling bytes v1.5.0
   Compiling memchr v2.5.0
   Compiling abc-rust-lint v0.1.0 (/work/chronik/abc-rust-lint)
   Compiling object v0.31.1
   Compiling http v1.0.0
   Compiling cc v1.0.83
   Compiling backtrace v0.3.68
   Compiling stable-eyre v0.2.2
   Compiling abc-rust-error v0.1.0 (/work/chronik/abc-rust-error)
    Finished test [unoptimized + debuginfo] target(s) in 4m 26s
     Running unittests src/lib.rs (abc-ci-builds/build-chronik/cargo/build/debug/deps/abc_rust_error-fbb1aab080abd8f8)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/test_error.rs (abc-ci-builds/build-chronik/cargo/build/debug/deps/test_error-5ef5f197f15f6364)

running 1 test
test test_error ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (abc-ci-builds/build-chronik/cargo/build/debug/deps/abc_rust_lint-dba48c2d41985cd4)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests abc-rust-error

running 1 test
test chronik/abc-rust-error/src/http_status.rs - http_status::parse_error_status (line 10) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.20s

   Doc-tests abc-rust-lint

running 1 test
test chronik/abc-rust-lint/src/lib.rs - lint (line 13) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s

[6/6] cd /work && /usr/bin/cmake -E env CARGO_TARGET_DIR="/work/abc-ci-builds/build-chronik/cargo/build" CARGO_BUILD_RUSTC="/root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/rustc" CARGO_BUILD_RUSTDOC="/root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/rustdoc" /root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/cargo --locked clippy --package bitcoinsuite-* -- -D warnings
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on build directory
    Checking either v1.9.0
    Checking serde v1.0.185
    Checking thiserror v1.0.47
    Checking itertools v0.10.5
    Checking bytes v1.5.0
    Checking bitcoinsuite-core v0.1.0 (/work/chronik/bitcoinsuite-core)
    Checking bitcoinsuite-slp v0.1.0 (/work/chronik/bitcoinsuite-slp)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 29s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Tail of the build log:

test io::token::tests::test_batch_nft::test_batch_slp_nft1 ... ok
test io::token::tests::test_batch_unknown::test_batch_unknown ... ok
test io::blocks::tests::test_blocks ... ok
test ser::tests::test_deserialize_err ... ok
test ser::tests::test_deserialize_leftover_err ... ok
test ser::tests::test_err_display_deserialize ... ok
test ser::tests::test_err_display_deserialize_leftover ... ok
test ser::tests::test_err_display_serialize ... ok
test ser::tests::test_roundtrip ... ok
test ser::tests::test_roundtrip_vec ... ok
test ser::tests::test_serialize_err ... ok
test io::token::tests::test_batch_vault::test_batch_vault ... ok
test io::group_utxos::tests::test_value_group_utxos ... ok
test mem::tokens::tests::test_mempool_tokens ... ok
test io::block_stats::tests::test_block_stats ... ok
test reverse_lookup::tests::test_reverse_lookup ... ok
test index_tx::tests::test_prepare_indexed_txs ... ok
test reverse_lookup::tests::test_reverse_lookup_rng ... ok
test io::group_history::tests::test_value_group_history ... ok
test io::txs::tests::test_insert_txs ... ok
Error details: Test init merge error

MERGE ERROR: Test init merge error

test io::merge::tests::test_catch_merge ... ok
test io::spent_by::tests::test_spent_by ... ok

failures:

---- groups::script::tests::test_script_group stdout ----
thread 'groups::script::tests::test_script_group' panicked at 'assertion failed: `(left == right)`
  left: `[]`,
 right: `[Script(b"S"), Script(b"Q")]`', chronik/chronik-db/src/groups/script.rs:202:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    groups::script::tests::test_script_group

test result: FAILED. 33 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.14s

error: test failed, to rerun pass `-p chronik-db --lib`
[6/6] cd /work && /usr/bin/cmake -E env CARGO_TARGET_DIR="/work/abc-ci-builds/build-chronik-plugins/cargo/build" CARGO_BUILD_RUSTC="/root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/rustc" CARGO_BUILD_RUSTDOC="/root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/rustdoc" /root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/cargo --locked clippy --package bitcoinsuite-* -- -D warnings
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on build directory
    Checking either v1.9.0
    Checking serde v1.0.185
    Checking thiserror v1.0.47
    Checking itertools v0.10.5
    Checking bytes v1.5.0
    Checking bitcoinsuite-core v0.1.0 (/work/chronik/bitcoinsuite-core)
    Checking bitcoinsuite-slp v0.1.0 (/work/chronik/bitcoinsuite-slp)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 37s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
tobias_ruck edited the summary of this revision. (Show Details)

build-chronik and -plugins succeeded

This revision is now accepted and ready to land.Feb 26 2024, 19:27