Page MenuHomePhabricator

[chronik] add scripthash option to the various script endpoints
Needs RevisionPublic

Authored by PiRK on Sat, Oct 19, 15:48.

Details

Reviewers
tobias_ruck
Group Reviewers
Restricted Project
Summary

After this diff, all the existing script endpoints have a new script_type=scripthash option, except for the utxo scripthash endpoint will be dealt with in a separate diff, as it requires decompression of the script which is not trivial (decompressing p2pk scripts may require secp256k1)

A user can now fetch a script's history and utxo's by querying it's sha256 hash.

Depends on D16827

Test Plan

ninja check-functional

Event Timeline

PiRK requested review of this revision.Sat, Oct 19, 15:48
PiRK planned changes to this revision.Sat, Oct 19, 15:49

that was supposed to be published as a draft

Tail of the build log:

   Doc-tests chronik-bridge

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-http

running 1 test
test chronik/chronik-http/src/protobuf.rs - protobuf::Protobuf (line 29) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.35s

   Doc-tests chronik-indexer

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-common

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-impl

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-proto

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-util

running 2 tests
test chronik/chronik-util/src/log.rs - log::log (line 65) ... ignored
test chronik/chronik-util/src/log.rs - log::log_chronik (line 87) ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Tail of the build log:

   Doc-tests chronik-bridge

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-http

running 1 test
test chronik/chronik-http/src/protobuf.rs - protobuf::Protobuf (line 29) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s

   Doc-tests chronik-indexer

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-common

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-impl

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-proto

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-util

running 2 tests
test chronik/chronik-util/src/log.rs - log::log (line 65) ... ignored
test chronik/chronik-util/src/log.rs - log::log_chronik (line 87) ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
tobias_ruck added a subscriber: tobias_ruck.
tobias_ruck added inline comments.
chronik/chronik-db/src/io/group_history.rs
217 ↗(On Diff #50203)

I usually use this syntax

260 ↗(On Diff #50203)

what's going on here? why switch to storing the reverse hash?

589 ↗(On Diff #50203)

same here, makes the codebase more consistent

chronik/chronik-indexer/src/query/group_history.rs
109 ↗(On Diff #50203)

and then match that here, and when it's MemberHash, look up the script hash first (and also return the 404 here if there's none)

111 ↗(On Diff #50203)

Instead of splitting the function, it's probably best add an enum (with either MemberSer or Sha256), and the caller specifies which, and this function then looks up first the member_ser if needed. Then we don't have to add anytihng to ChronikIndexer, too

something like

pub enum GroupMember<M> {
    MemberSer(M),
    MemberHash(Sha256),
}
113 ↗(On Diff #50203)
chronik/chronik-db/src/io/group_history.rs
260 ↗(On Diff #50203)

We could store the regular hash and reverse it later when doing the API call. We could also use the regular little-endian hash for chronik and only reverse it for the Electrum API. I don't really have an opinion on what's best, here. For now I just went for the solution of storing everything in the retarded Electrum protocol format, because it is the only use case I'm aware of.

Screenshot from 2024-10-19 17-24-41.png (754×710 px, 165 KB)

use an enum, move code to query/group_history

Tail of the build log:

   Compiling pyo3 v0.22.2
    Checking winnow v0.6.13
    Checking sync_wrapper v0.1.2
    Checking topo_sort v0.4.0
    Checking mime v0.3.17
    Checking seahash v4.1.0
    Checking hyper v1.2.0
    Checking tokio-tungstenite v0.21.0
    Checking tower v0.4.13
    Checking axum-core v0.4.3
    Checking memoffset v0.9.1
    Checking serde_json v1.0.115
    Checking serde_urlencoded v0.7.1
    Checking futures-executor v0.3.30
    Checking serde_path_to_error v0.1.16
    Checking hyper-util v0.1.3
    Checking unicode-segmentation v1.11.0
    Checking matchit v0.7.3
    Checking sync_wrapper v1.0.1
    Checking unindent v0.2.3
    Checking base64 v0.21.7
    Checking tower-http v0.5.2
    Checking futures v0.3.30
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking convert_case v0.6.0
    Checking toml_edit v0.22.14
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking axum v0.7.5
   Compiling pyo3-macros v0.22.2
    Checking toml v0.8.14
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
error[E0308]: mismatched types
   --> chronik/chronik-db/src/io/group_history.rs:593:9
    |
592 |       ) -> Option<G::MemberSer> {
    |            -------------------- expected `std::option::Option<<G as group::Group>::MemberSer>` because of return type
593 | /         self.col
594 | |             .get_member_ser_by_member_hash(member_hash)
595 | |             .expect("Error while accessing scripthash db")
    | |__________________________________________________________^ expected `Option<<G as Group>::MemberSer>`, found `Option<&[u8]>`
    |
    = note: expected enum `std::option::Option<<G as group::Group>::MemberSer>`
               found enum `std::option::Option<&[u8]>`
help: a method is available that returns `<G as group::Group>::MemberSer`
   --> chronik/chronik-db/src/group.rs:100:5
    |
100 |     fn ser_member(&self, member: &Self::Member<'_>) -> Self::MemberSer;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ consider calling `group::Group::ser_member`
help: consider constraining the associated type `<G as group::Group>::MemberSer` to `&[u8]`
    |
545 | impl<'a, G: Group<MemberSer = &[u8]>> GroupHistoryReader<'a, G> {
    |                  +++++++++++++++++++

For more information about this error, try `rustc --explain E0308`.
error: could not compile `chronik-db` (lib) due to 1 previous error
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Tail of the build log:

[243/587] Building CXX object src/CMakeFiles/script.dir/script/interpreter.cpp.o
[244/587] Building CXX object src/CMakeFiles/common.dir/config.cpp.o
[245/587] Building CXX object src/CMakeFiles/common.dir/cashaddrenc.cpp.o
[246/587] Building CXX object src/CMakeFiles/common.dir/chainparams.cpp.o
[247/587] Building CXX object src/CMakeFiles/common.dir/eventloop.cpp.o
[248/587] Building CXX object src/CMakeFiles/script.dir/script/sign.cpp.o
[249/587] Building CXX object src/CMakeFiles/script.dir/script/signingprovider.cpp.o
[250/587] Building CXX object src/CMakeFiles/common.dir/feerate.cpp.o
[251/587] Building CXX object src/CMakeFiles/common.dir/compressor.cpp.o
[252/587] Building CXX object src/CMakeFiles/common.dir/common/configfile.cpp.o
[253/587] Building CXX object src/CMakeFiles/common.dir/merkleblock.cpp.o
[254/587] Building CXX object src/CMakeFiles/common.dir/coins.cpp.o
[255/587] Building CXX object src/CMakeFiles/common.dir/key.cpp.o
[256/587] Building CXX object src/CMakeFiles/common.dir/key_io.cpp.o
[257/587] Building CXX object src/CMakeFiles/common.dir/net_permissions.cpp.o
[258/587] Building CXX object src/CMakeFiles/common.dir/kernel/chainparams.cpp.o
[259/587] Building CXX object src/CMakeFiles/common.dir/outputtype.cpp.o
[260/587] Building CXX object src/CMakeFiles/common.dir/common/args.cpp.o
[261/587] Building CXX object src/CMakeFiles/script.dir/script/descriptor.cpp.o
[262/587] Building CXX object src/CMakeFiles/common.dir/primitives/block.cpp.o
[263/587] Building CXX object src/CMakeFiles/common.dir/policy/policy.cpp.o
[264/587] Building C object src/secp256k1/CMakeFiles/recover-bench.dir/src/bench_recover.c.o
[265/587] Building CXX object src/CMakeFiles/common.dir/networks/abc/chainparamsconstants.cpp.o
[266/587] Building CXX object src/CMakeFiles/common.dir/netaddress.cpp.o
[267/587] Building CXX object src/CMakeFiles/common.dir/protocol.cpp.o
[268/587] Building CXX object src/CMakeFiles/common.dir/warnings.cpp.o
[269/587] Building C object src/secp256k1/CMakeFiles/verify-bench.dir/src/bench_verify.c.o
[270/587] Building CXX object src/CMakeFiles/common.dir/scheduler.cpp.o
[271/587] Building C object src/secp256k1/CMakeFiles/sign-bench.dir/src/bench_sign.c.o
[272/587] Building CXX object src/CMakeFiles/common.dir/core_read.cpp.o
[273/587] Building CXX object src/CMakeFiles/common.dir/networks/abc/checkpoints.cpp.o
[274/587] Building CXX object src/CMakeFiles/common.dir/netbase.cpp.o
[275/587] Building CXX object src/CMakeFiles/common.dir/core_write.cpp.o
[276/587] Building C object src/secp256k1/CMakeFiles/secp256k1.dir/src/secp256k1.c.o
[277/587] Linking C static library src/secp256k1/libsecp256k1.a
[278/587] Linking C executable src/secp256k1/recover-bench
[279/587] Linking C executable src/secp256k1/verify-bench
[280/587] Linking C executable src/secp256k1/sign-bench
[281/587] Building C object src/secp256k1/CMakeFiles/ecmult-bench.dir/src/bench_ecmult.c.o
[282/587] Linking C executable src/secp256k1/ecmult-bench
[283/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana_formatter.cpp.o
[284/587] Building C object src/secp256k1/CMakeFiles/internal-bench.dir/src/bench_internal.c.o
[285/587] Linking C executable src/secp256k1/internal-bench
[286/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana_interpreter.cpp.o
[287/587] Building CXX object src/CMakeFiles/common.dir/psbt.cpp.o
[288/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana.cpp.o
[289/587] Building CXX object src/CMakeFiles/common.dir/rpc/rawtransaction_util.cpp.o
[290/587] Building CXX object src/CMakeFiles/bitcoin-cli.dir/bitcoin-cli.cpp.o
[291/587] Building CXX object src/CMakeFiles/bitcoin-tx.dir/bitcoin-tx.cpp.o
[292/587] Building CXX object src/CMakeFiles/common.dir/rpc/util.cpp.o
[293/587] Linking CXX static library src/libcommon.a
[294/587] Linking CXX static library src/libscript.a
[295/587] Linking CXX static library src/libbitcoinconsensus.a
[296/587] Linking CXX shared library src/libbitcoinconsensus.so.0.30.3
[297/587] Creating library symlink src/libbitcoinconsensus.so.0 src/libbitcoinconsensus.so
[298/587] Linking CXX executable src/iguana/iguana
[299/587] Linking CXX executable src/bitcoin-cli
[300/587] Linking CXX executable src/bitcoin-tx
ninja: build stopped: cannot make progress due to previous errors.
Build ecash-agora-integration-tests failed with exit code 1

Tail of the build log:

[243/587] Building CXX object src/CMakeFiles/common.dir/common/bloom.cpp.o
[244/587] Building CXX object src/CMakeFiles/common.dir/config.cpp.o
[245/587] Building CXX object src/CMakeFiles/common.dir/cashaddrenc.cpp.o
[246/587] Building CXX object src/CMakeFiles/common.dir/chainparams.cpp.o
[247/587] Building CXX object src/CMakeFiles/common.dir/eventloop.cpp.o
[248/587] Building CXX object src/CMakeFiles/common.dir/feerate.cpp.o
[249/587] Building CXX object src/CMakeFiles/script.dir/script/sign.cpp.o
[250/587] Building CXX object src/CMakeFiles/script.dir/script/signingprovider.cpp.o
[251/587] Building CXX object src/CMakeFiles/common.dir/compressor.cpp.o
[252/587] Building CXX object src/CMakeFiles/common.dir/common/configfile.cpp.o
[253/587] Building CXX object src/CMakeFiles/common.dir/merkleblock.cpp.o
[254/587] Building CXX object src/CMakeFiles/common.dir/coins.cpp.o
[255/587] Building CXX object src/CMakeFiles/common.dir/key.cpp.o
[256/587] Building CXX object src/CMakeFiles/common.dir/kernel/chainparams.cpp.o
[257/587] Building CXX object src/CMakeFiles/common.dir/net_permissions.cpp.o
[258/587] Building CXX object src/CMakeFiles/common.dir/common/args.cpp.o
[259/587] Building CXX object src/CMakeFiles/common.dir/key_io.cpp.o
[260/587] Building CXX object src/CMakeFiles/common.dir/policy/policy.cpp.o
[261/587] Building CXX object src/CMakeFiles/script.dir/script/descriptor.cpp.o
[262/587] Building CXX object src/CMakeFiles/common.dir/outputtype.cpp.o
[263/587] Building C object src/secp256k1/CMakeFiles/recover-bench.dir/src/bench_recover.c.o
[264/587] Building CXX object src/CMakeFiles/common.dir/primitives/block.cpp.o
[265/587] Building CXX object src/CMakeFiles/common.dir/networks/abc/chainparamsconstants.cpp.o
[266/587] Building CXX object src/CMakeFiles/common.dir/netaddress.cpp.o
[267/587] Building CXX object src/CMakeFiles/common.dir/warnings.cpp.o
[268/587] Building CXX object src/CMakeFiles/common.dir/scheduler.cpp.o
[269/587] Building C object src/secp256k1/CMakeFiles/verify-bench.dir/src/bench_verify.c.o
[270/587] Building C object src/secp256k1/CMakeFiles/sign-bench.dir/src/bench_sign.c.o
[271/587] Building CXX object src/CMakeFiles/common.dir/protocol.cpp.o
[272/587] Building CXX object src/CMakeFiles/common.dir/netbase.cpp.o
[273/587] Building CXX object src/CMakeFiles/common.dir/core_read.cpp.o
[274/587] Building CXX object src/CMakeFiles/common.dir/core_write.cpp.o
[275/587] Building CXX object src/CMakeFiles/common.dir/networks/abc/checkpoints.cpp.o
[276/587] Building C object src/secp256k1/CMakeFiles/secp256k1.dir/src/secp256k1.c.o
[277/587] Linking C static library src/secp256k1/libsecp256k1.a
[278/587] Linking C executable src/secp256k1/recover-bench
[279/587] Linking C executable src/secp256k1/sign-bench
[280/587] Linking C executable src/secp256k1/verify-bench
[281/587] Building C object src/secp256k1/CMakeFiles/ecmult-bench.dir/src/bench_ecmult.c.o
[282/587] Linking C executable src/secp256k1/ecmult-bench
[283/587] Building C object src/secp256k1/CMakeFiles/internal-bench.dir/src/bench_internal.c.o
[284/587] Linking C executable src/secp256k1/internal-bench
[285/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana_formatter.cpp.o
[286/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana_interpreter.cpp.o
[287/587] Building CXX object src/CMakeFiles/common.dir/psbt.cpp.o
[288/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana.cpp.o
[289/587] Building CXX object src/CMakeFiles/common.dir/rpc/rawtransaction_util.cpp.o
[290/587] Building CXX object src/CMakeFiles/common.dir/rpc/util.cpp.o
[291/587] Building CXX object src/CMakeFiles/bitcoin-cli.dir/bitcoin-cli.cpp.o
[292/587] Linking CXX static library src/libcommon.a
[293/587] Linking CXX static library src/libscript.a
[294/587] Linking CXX static library src/libbitcoinconsensus.a
[295/587] Linking CXX shared library src/libbitcoinconsensus.so.0.30.3
[296/587] Creating library symlink src/libbitcoinconsensus.so.0 src/libbitcoinconsensus.so
[297/587] Linking CXX executable src/iguana/iguana
[298/587] Linking CXX executable src/bitcoin-cli
[299/587] Building CXX object src/CMakeFiles/bitcoin-tx.dir/bitcoin-tx.cpp.o
[300/587] Linking CXX executable src/bitcoin-tx
ninja: build stopped: cannot make progress due to previous errors.
Build ecash-lib-integration-tests failed with exit code 1

Tail of the build log:

[243/587] Building CXX object src/CMakeFiles/script.dir/script/interpreter.cpp.o
[244/587] Building CXX object src/CMakeFiles/common.dir/config.cpp.o
[245/587] Building CXX object src/CMakeFiles/common.dir/cashaddrenc.cpp.o
[246/587] Building CXX object src/CMakeFiles/common.dir/chainparams.cpp.o
[247/587] Building CXX object src/CMakeFiles/common.dir/eventloop.cpp.o
[248/587] Building CXX object src/CMakeFiles/script.dir/script/sign.cpp.o
[249/587] Building CXX object src/CMakeFiles/common.dir/feerate.cpp.o
[250/587] Building CXX object src/CMakeFiles/script.dir/script/signingprovider.cpp.o
[251/587] Building CXX object src/CMakeFiles/common.dir/compressor.cpp.o
[252/587] Building CXX object src/CMakeFiles/common.dir/common/configfile.cpp.o
[253/587] Building CXX object src/CMakeFiles/common.dir/merkleblock.cpp.o
[254/587] Building CXX object src/CMakeFiles/common.dir/coins.cpp.o
[255/587] Building CXX object src/CMakeFiles/common.dir/key.cpp.o
[256/587] Building CXX object src/CMakeFiles/common.dir/common/args.cpp.o
[257/587] Building CXX object src/CMakeFiles/common.dir/key_io.cpp.o
[258/587] Building CXX object src/CMakeFiles/common.dir/kernel/chainparams.cpp.o
[259/587] Building CXX object src/CMakeFiles/common.dir/net_permissions.cpp.o
[260/587] Building CXX object src/CMakeFiles/common.dir/policy/policy.cpp.o
[261/587] Building CXX object src/CMakeFiles/common.dir/outputtype.cpp.o
[262/587] Building CXX object src/CMakeFiles/script.dir/script/descriptor.cpp.o
[263/587] Building CXX object src/CMakeFiles/common.dir/primitives/block.cpp.o
[264/587] Building C object src/secp256k1/CMakeFiles/recover-bench.dir/src/bench_recover.c.o
[265/587] Building CXX object src/CMakeFiles/common.dir/netaddress.cpp.o
[266/587] Building CXX object src/CMakeFiles/common.dir/networks/abc/chainparamsconstants.cpp.o
[267/587] Building CXX object src/CMakeFiles/common.dir/scheduler.cpp.o
[268/587] Building CXX object src/CMakeFiles/common.dir/warnings.cpp.o
[269/587] Building C object src/secp256k1/CMakeFiles/verify-bench.dir/src/bench_verify.c.o
[270/587] Building C object src/secp256k1/CMakeFiles/sign-bench.dir/src/bench_sign.c.o
[271/587] Building CXX object src/CMakeFiles/common.dir/protocol.cpp.o
[272/587] Building CXX object src/CMakeFiles/common.dir/core_read.cpp.o
[273/587] Building CXX object src/CMakeFiles/common.dir/networks/abc/checkpoints.cpp.o
[274/587] Building CXX object src/CMakeFiles/common.dir/netbase.cpp.o
[275/587] Building CXX object src/CMakeFiles/common.dir/core_write.cpp.o
[276/587] Building C object src/secp256k1/CMakeFiles/secp256k1.dir/src/secp256k1.c.o
[277/587] Linking C static library src/secp256k1/libsecp256k1.a
[278/587] Linking C executable src/secp256k1/recover-bench
[279/587] Linking C executable src/secp256k1/verify-bench
[280/587] Linking C executable src/secp256k1/sign-bench
[281/587] Building C object src/secp256k1/CMakeFiles/ecmult-bench.dir/src/bench_ecmult.c.o
[282/587] Linking C executable src/secp256k1/ecmult-bench
[283/587] Building C object src/secp256k1/CMakeFiles/internal-bench.dir/src/bench_internal.c.o
[284/587] Linking C executable src/secp256k1/internal-bench
[285/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana_formatter.cpp.o
[286/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana_interpreter.cpp.o
[287/587] Building CXX object src/CMakeFiles/common.dir/rpc/rawtransaction_util.cpp.o
[288/587] Building CXX object src/iguana/CMakeFiles/iguana.dir/iguana.cpp.o
[289/587] Building CXX object src/CMakeFiles/common.dir/psbt.cpp.o
[290/587] Building CXX object src/CMakeFiles/common.dir/rpc/util.cpp.o
[291/587] Building CXX object src/CMakeFiles/bitcoin-cli.dir/bitcoin-cli.cpp.o
[292/587] Linking CXX static library src/libcommon.a
[293/587] Linking CXX static library src/libscript.a
[294/587] Building CXX object src/CMakeFiles/bitcoin-tx.dir/bitcoin-tx.cpp.o
[295/587] Linking CXX static library src/libbitcoinconsensus.a
[296/587] Linking CXX shared library src/libbitcoinconsensus.so.0.30.3
[297/587] Creating library symlink src/libbitcoinconsensus.so.0 src/libbitcoinconsensus.so
[298/587] Linking CXX executable src/iguana/iguana
[299/587] Linking CXX executable src/bitcoin-cli
[300/587] Linking CXX executable src/bitcoin-tx
ninja: build stopped: cannot make progress due to previous errors.
Build chronik-client-integration-tests failed with exit code 1

Tail of the build log:

   Compiling tokio-tungstenite v0.21.0
   Compiling toml v0.8.14
   Compiling tower v0.4.13
   Compiling tower-http v0.5.2
   Compiling hyper-util v0.1.3
   Compiling chronik-util v0.1.0 (/work/chronik/chronik-util)
   Compiling axum v0.7.5
   Compiling librocksdb-sys v0.11.0+8.1.1
   Compiling pyo3-macros v0.22.2
   Compiling pyo3 v0.22.2
   Compiling chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
   Compiling rocksdb v0.21.0
   Compiling chronik-db v0.1.0 (/work/chronik/chronik-db)
error[E0308]: mismatched types
   --> chronik/chronik-db/src/io/group_history.rs:593:9
    |
592 |       ) -> Option<G::MemberSer> {
    |            -------------------- expected `std::option::Option<<G as group::Group>::MemberSer>` because of return type
593 | /         self.col
594 | |             .get_member_ser_by_member_hash(member_hash)
595 | |             .expect("Error while accessing scripthash db")
    | |__________________________________________________________^ expected `Option<<G as Group>::MemberSer>`, found `Option<&[u8]>`
    |
    = note: expected enum `std::option::Option<<G as group::Group>::MemberSer>`
               found enum `std::option::Option<&[u8]>`
help: a method is available that returns `<G as group::Group>::MemberSer`
   --> chronik/chronik-db/src/group.rs:100:5
    |
100 |     fn ser_member(&self, member: &Self::Member<'_>) -> Self::MemberSer;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ consider calling `group::Group::ser_member`
help: consider constraining the associated type `<G as group::Group>::MemberSer` to `&[u8]`
    |
545 | impl<'a, G: Group<MemberSer = &[u8]>> GroupHistoryReader<'a, G> {
    |                  +++++++++++++++++++

For more information about this error, try `rustc --explain E0308`.
error: could not compile `chronik-db` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `chronik-db` (lib test) due to 1 previous error
[7/7] cd /work && /usr/bin/cmake -E env CARGO_TARGET_DIR="/work/abc-ci-builds/build-chronik-plugins/cargo/build" CARGO_BUILD_RUSTC="/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/bin/rustc" CARGO_BUILD_RUSTDOC="/root/.cargo/bin/rustdoc" /root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/bin/cargo --locked clippy --package 'abc-rust-*' -- -D warnings
warning: virtual workspace defaulting to `resolver = "1"` despite one or more workspace members being on edition 2021 which implies `resolver = "2"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
note: for more details see https://doc.rust-lang.org/cargo/reference/resolver.html#resolver-versions
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on build directory
    Checking memchr v2.7.2
    Checking libc v0.2.153
    Checking bytes v1.6.0
    Checking http v1.1.0
    Checking object v0.32.2
    Checking backtrace v0.3.71
    Checking stable-eyre v0.2.2
    Checking abc-rust-error v0.1.0 (/work/chronik/abc-rust-error)
    Finished dev [unoptimized + debuginfo] target(s) in 6m 03s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1

Tail of the build log:

   Doc-tests chronik-bridge

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-http

running 1 test
test chronik/chronik-http/src/protobuf.rs - protobuf::Protobuf (line 29) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.35s

   Doc-tests chronik-indexer

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-common

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-impl

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-proto

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-util

running 2 tests
test chronik/chronik-util/src/log.rs - log::log (line 65) ... ignored
test chronik/chronik-util/src/log.rs - log::log_chronik (line 87) ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Tail of the build log:

   Doc-tests chronik-bridge

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-http

running 1 test
test chronik/chronik-http/src/protobuf.rs - protobuf::Protobuf (line 29) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.32s

   Doc-tests chronik-indexer

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-common

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-impl

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-proto

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-util

running 2 tests
test chronik/chronik-util/src/log.rs - log::log (line 65) ... ignored
test chronik/chronik-util/src/log.rs - log::log_chronik (line 87) ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
PiRK planned changes to this revision.Mon, Oct 21, 19:09

still a WIP

fix style warnings, better handle invalid scripthash payloads

PiRK planned changes to this revision.Tue, Oct 22, 08:33

Tail of the build log:

   Doc-tests chronik-bridge

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-http

running 1 test
test chronik/chronik-http/src/protobuf.rs - protobuf::Protobuf (line 29) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.35s

   Doc-tests chronik-indexer

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-common

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-impl

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-proto

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-util

running 2 tests
test chronik/chronik-util/src/log.rs - log::log (line 65) ... ignored
test chronik/chronik-util/src/log.rs - log::log_chronik (line 87) ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Tail of the build log:

   Doc-tests chronik-bridge

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-http

running 1 test
test chronik/chronik-http/src/protobuf.rs - protobuf::Protobuf (line 29) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s

   Doc-tests chronik-indexer

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-common

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-impl

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-proto

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-util

running 2 tests
test chronik/chronik-util/src/log.rs - log::log (line 65) ... ignored
test chronik/chronik-util/src/log.rs - log::log_chronik (line 87) ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
PiRK edited the summary of this revision. (Show Details)

add more tests, more endpoints (unconfirmed_txs, history)

the bytes dependency for chronik-http is no longer needed

Fabien added inline comments.
test/functional/chronik_script_confirmed_txs.py
31

Just make it the default, if not on mainnet at least on regtest (add it to the regtest config file in util.py)

tobias_ruck added inline comments.
chronik/chronik-db/src/io/group_history.rs
224–225

doesn't this work? If not, try Ok(Some(value.as_ref().to_vec()))

260

make sure to harmonize this with the other diff, this is a bit confusing atm

592–595

No panics for normal DB failures

chronik/chronik-http/src/handlers.rs
111–118

this would be more idiomatic I suppose, but your solution is good too (in case my doesn't compile)

136

same here

156

I'm starting to think you should factor this into it's own function, maybe next to parse_script_hash. Return GroupMember<Script>.

You then probably need to add a function as_member_ref turning GroupMember<M> into GroupMember<&M> (to make the borrow checker happy):

something like this next to enum GroupMember

impl<M> GroupMember<M> {
    pub fn as_member_ref(&self) -> GroupMember<&M> {
       match self {
           Member(member) => Member(member),
           MemberHash(hash) => MemberHash(*hash),
       }
    }
}

or maybe even implement AsRef for this, but might be difficult.

164

and then use it here (ideally using AsRef ofc)

This revision now requires changes to proceed.Thu, Oct 24, 09:28
PiRK marked an inline comment as done.

rebase, feedback

chronik/chronik-http/src/handlers.rs
156

Yeah I tried factoring the code into a function, but ran into this kind of script lifetime issue and GroupMember<M> vs GroupMember<&M> issue.
Will try the suggested method.

test/functional/chronik_script_confirmed_txs.py
31

That would be the only optional index whose default enabled state is controlled by the framework's bitcoin.conf
It feels weird.

tobias_ruck added inline comments.
chronik/chronik-db/src/group.rs
45 ↗(On Diff #50373)

Btw, it seems ideomatic to just name this as_ref, see this for example: https://doc.rust-lang.org/src/core/result.rs.html#707

Does the same thing but no special name

chronik/chronik-indexer/src/query/group_history.rs
117 ↗(On Diff #50373)

Pretty sure you can factor this into a method, something like

fn member_ser_from_member(&self, member: &GroupMember<G::Member<'_>>) -> Result<Bytes> {
    ...
}
This revision now requires changes to proceed.Thu, Oct 24, 12:24

as_member_ref -> as_ref, factor repeated code into a member_ser_from_member method

Tail of the build log:

   Compiling prost-build v0.11.9
    Checking chronik-plugin-common v0.1.0 (/work/chronik/chronik-plugin-common)
    Checking utf-8 v0.7.6
    Checking embedded-io v0.4.0
    Checking ryu v1.0.17
    Checking smallvec v1.13.2
    Checking postcard v1.0.8
    Checking chronik-plugin v0.1.0 (/work/chronik/chronik-plugin)
    Checking tungstenite v0.21.0
    Checking hyper v1.2.0
    Checking pin-project v1.1.5
    Checking serde_spanned v0.6.6
    Checking toml_datetime v0.6.6
   Compiling pyo3 v0.22.2
    Checking mime v0.3.17
    Checking winnow v0.6.13
    Checking topo_sort v0.4.0
    Checking sync_wrapper v0.1.2
    Checking seahash v4.1.0
    Checking axum-core v0.4.3
    Checking tokio-tungstenite v0.21.0
    Checking tower v0.4.13
    Checking hyper-util v0.1.3
    Checking memoffset v0.9.1
    Checking serde_json v1.0.115
    Checking serde_urlencoded v0.7.1
    Checking futures-executor v0.3.30
    Checking serde_path_to_error v0.1.16
    Checking base64 v0.21.7
    Checking unicode-segmentation v1.11.0
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking matchit v0.7.3
    Checking unindent v0.2.3
    Checking sync_wrapper v1.0.1
    Checking tower-http v0.5.2
    Checking futures v0.3.30
    Checking convert_case v0.6.0
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
    Checking toml_edit v0.22.14
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking axum v0.7.5
   Compiling pyo3-macros v0.22.2
    Checking toml v0.8.14
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
    Checking chronik-indexer v0.1.0 (/work/chronik/chronik-indexer)
error: this expression creates a reference which is immediately dereferenced by the compiler
   --> chronik/chronik-indexer/src/query/group_history.rs:107:39
    |
107 |                 self.group.ser_member(&member).as_ref(),
    |                                       ^^^^^^^ help: change this to: `member`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
    = note: `-D clippy::needless-borrow` implied by `-D warnings`
    = help: to override `-D warnings` add `#[allow(clippy::needless_borrow)]`

error: could not compile `chronik-indexer` (lib) due to 1 previous error
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Tail of the build log:

...  |
19 | |     mod test;
20 | | }
   | |_^
   = note: `#[warn(unused_imports)]` implied by `#[warn(unused)]`
   = note: this warning originates in the macro `abc_rust_lint::lint` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: `chronik-db` (lib test) generated 1 warning (run `cargo fix --lib -p chronik-db --tests` to apply 1 suggestion)
    Finished test [unoptimized + debuginfo] target(s) in 6m 32s
     Running unittests src/lib.rs (abc-ci-builds/build-chronik-plugins/cargo/build/debug/deps/chronik_db-a3c26c153718321c)

running 37 tests
test groups::script::tests::test_script_group ... ok
test index_tx::tests::test_tx_num_cache ... ok
test io::block_stats::tests::test_block_stats ... ok
test io::group_utxos::tests::test_value_group_utxos ... ok
test io::token::tests::test_batch_common::test_batch_skip_validation ... ok
test io::group_history::tests::test_value_group_history ... ok
test io::token::tests::test_batch_common::test_batch_cycle ... ok
test io::token::tests::test_batch_common::test_batch_topological_sort ... ok
test io::token::tests::test_batch_burn::test_batch_burn ... ok
test io::token::tests::test_batch_alp::test_batch_alp ... ok
test mem::group_history::tests::test_mempool_group_history ... ok
test mem::group_utxos::tests::test_mempool_group_utxos ... ok
test mem::spent_by::tests::test_mempool_spent_by ... ok
test io::token::tests::test_batch_genesis::test_batch_genesis_alp ... ok
test io::token::tests::test_batch_disconnect_block::test_batch_disconnect ... ok
test io::spent_by::tests::test_spent_by ... ok
test io::token::tests::test_batch_genesis::test_batch_genesis_slp_fungible ... ok
test io::token::tests::test_batch_vault::test_batch_vault ... ok
test ser::tests::test_deserialize_err ... ok
test ser::tests::test_deserialize_leftover_err ... ok
test ser::tests::test_err_display_deserialize ... ok
test ser::tests::test_err_display_deserialize_leftover ... ok
test ser::tests::test_err_display_serialize ... ok
test ser::tests::test_roundtrip ... ok
test ser::tests::test_roundtrip_vec ... ok
test ser::tests::test_serialize_err ... ok
test io::token::tests::test_batch_unknown::test_batch_unknown ... ok
test io::token::tests::test_batch_nft::test_batch_slp_nft1 ... ok
test mem::tokens::tests::test_mempool_tokens ... ok
test io::merge::tests::test_catch_merge ... ok
test reverse_lookup::tests::test_reverse_lookup ... ok
test plugins::io::tests::test_plugin_metas ... ok
test plugins::io::tests::test_plugin_writer ... ok
test io::blocks::tests::test_blocks ... ok
test index_tx::tests::test_prepare_indexed_txs ... ok
test io::txs::tests::test_insert_txs ... ok
test reverse_lookup::tests::test_reverse_lookup_rng ... ok

test result: ok. 37 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.20s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
PiRK planned changes to this revision.Fri, Oct 25, 06:56
tobias_ruck added inline comments.
test/functional/chronik_script_confirmed_txs.py
1 ↗(On Diff #50426)

Tbh I think it makes more sense to test this in its own test file chronik_script_hash_index

Then leave these tests untouched; especially the script history test is really complicated already

Also test enabling / disabling (wiping) the index, like I did for the lokad ID index

We don’t need to test pagination with the script hash index

31 ↗(On Diff #50426)

Then we can also remove this

81 ↗(On Diff #50426)

Maybe add some more cases, eg empty string, odd length, off by one, non-hex etc

This revision now requires changes to proceed.Sat, Oct 26, 11:59