Page MenuHomePhabricator
Differential D16525

[Chronik] Add `plugin` field to `TxInput` in `tx.py`
ClosedPublic
Actions

Authored by tobias_ruck on Jul 23 2024, 13:14.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Commits
rABCf9ae1740cb21: [Chronik] Add `plugin` field to `TxInput` in `tx.py`
Summary

Allows plugins to access the plugin data of the spent inputs from previously ran plugins.

We can get this data cheaply and it makes plugins vastly more powerful as information can flow from transaction to transaction.

Depends on D16523 and D16530.

Test Plan

cargo test -p chronik-plugin-impl

Diff Detail

Event Timeline

tobias_ruck created this revision.Jul 23 2024, 13:14
Herald added a reviewer: Restricted Project. · View Herald TranscriptJul 23 2024, 13:14
Harbormaster completed remote builds in B29674: Diff 48792.Jul 23 2024, 13:48
tobias_ruck added a child revision: D16527: [Chronik] Add `PluginContext::run_on_tx`.Jul 23 2024, 19:32
tobias_ruck updated this revision to Diff 48801.Jul 23 2024, 19:51

remove PluginTxOutputs

tobias_ruck published this revision for review.Jul 23 2024, 19:56
Harbormaster completed remote builds in B29681: Diff 48801.Jul 23 2024, 20:12
tobias_ruck planned changes to this revision.Jul 23 2024, 22:28
tobias_ruck updated this revision to Diff 48804.Jul 23 2024, 23:06

use plugin idx instead of string to identify plugins

Harbormaster completed remote builds in B29684: Diff 48804.Jul 23 2024, 23:15
teamcity added a comment.Jul 23 2024, 23:15
Build Bitcoin ABC Diffs / Diff Testing (build-chronik-plugins) failed.

Tail of the build log:

[239/582] Building CXX object src/CMakeFiles/script.dir/script/standard.cpp.o
[240/582] Building CXX object src/CMakeFiles/common.dir/cashaddr.cpp.o
[241/582] Building CXX object src/CMakeFiles/script.dir/script/sign.cpp.o
[242/582] Building CXX object src/CMakeFiles/script.dir/script/descriptor.cpp.o
[243/582] Building CXX object src/CMakeFiles/common.dir/common/bloom.cpp.o
[244/582] Building CXX object src/CMakeFiles/common.dir/common/args.cpp.o
[245/582] Building CXX object src/CMakeFiles/common.dir/config.cpp.o
[246/582] Building CXX object src/CMakeFiles/common.dir/common/configfile.cpp.o
[247/582] Building CXX object src/CMakeFiles/common.dir/common/system.cpp.o
[248/582] Building CXX object src/CMakeFiles/common.dir/consensus/merkle.cpp.o
[249/582] Building CXX object src/CMakeFiles/common.dir/eventloop.cpp.o
[250/582] Building CXX object src/CMakeFiles/script.dir/script/signingprovider.cpp.o
[251/582] Building CXX object src/CMakeFiles/common.dir/feerate.cpp.o
[252/582] Building CXX object src/CMakeFiles/common.dir/cashaddrenc.cpp.o
[253/582] Building CXX object src/CMakeFiles/common.dir/compressor.cpp.o
[254/582] Building CXX object src/CMakeFiles/common.dir/chainparams.cpp.o
[255/582] Building CXX object src/CMakeFiles/common.dir/merkleblock.cpp.o
[256/582] Building CXX object src/CMakeFiles/common.dir/kernel/chainparams.cpp.o
[257/582] Building CXX object src/CMakeFiles/common.dir/key.cpp.o
[258/582] Building CXX object src/CMakeFiles/common.dir/net_permissions.cpp.o
[259/582] Building CXX object src/CMakeFiles/common.dir/coins.cpp.o
[260/582] Building CXX object src/CMakeFiles/common.dir/core_read.cpp.o
[261/582] Building CXX object src/CMakeFiles/common.dir/key_io.cpp.o
[262/582] Building CXX object src/CMakeFiles/common.dir/netaddress.cpp.o
[263/582] Building CXX object src/CMakeFiles/common.dir/primitives/block.cpp.o
[264/582] Building CXX object src/CMakeFiles/common.dir/outputtype.cpp.o
[265/582] Building CXX object src/CMakeFiles/common.dir/protocol.cpp.o
[266/582] Building CXX object src/CMakeFiles/common.dir/core_write.cpp.o
[267/582] Building C object src/secp256k1/CMakeFiles/recover-bench.dir/src/bench_recover.c.o
[268/582] Building CXX object src/CMakeFiles/common.dir/netbase.cpp.o
[269/582] Building CXX object src/CMakeFiles/common.dir/scheduler.cpp.o
[270/582] Building CXX object src/CMakeFiles/common.dir/warnings.cpp.o
[271/582] Building CXX object src/CMakeFiles/common.dir/policy/policy.cpp.o
[272/582] Building CXX object src/CMakeFiles/common.dir/networks/abc/checkpoints.cpp.o
[273/582] Building CXX object src/CMakeFiles/common.dir/networks/abc/chainparamsconstants.cpp.o
[274/582] Building C object src/secp256k1/CMakeFiles/ecmult-bench.dir/src/bench_ecmult.c.o
[275/582] Building C object src/secp256k1/CMakeFiles/verify-bench.dir/src/bench_verify.c.o
[276/582] Building C object src/secp256k1/CMakeFiles/sign-bench.dir/src/bench_sign.c.o
[277/582] Building C object src/secp256k1/CMakeFiles/secp256k1.dir/src/secp256k1.c.o
[278/582] Building C object src/secp256k1/CMakeFiles/internal-bench.dir/src/bench_internal.c.o
[279/582] Building CXX object src/CMakeFiles/common.dir/psbt.cpp.o
[280/582] Building CXX object src/CMakeFiles/bitcoin-cli.dir/bitcoin-cli.cpp.o
[281/582] Linking C static library src/secp256k1/libsecp256k1.a
[282/582] Building CXX object src/CMakeFiles/common.dir/rpc/rawtransaction_util.cpp.o
[283/582] Building CXX object src/CMakeFiles/bitcoin-tx.dir/bitcoin-tx.cpp.o
[284/582] Building CXX object src/CMakeFiles/common.dir/rpc/util.cpp.o
[285/582] Linking C executable src/secp256k1/ecmult-bench
[286/582] Linking C executable src/secp256k1/internal-bench
[287/582] Linking C executable src/secp256k1/recover-bench
[288/582] Linking C executable src/secp256k1/verify-bench
[289/582] Linking C executable src/secp256k1/sign-bench
[290/582] Linking CXX static library src/libcommon.a
[291/582] Linking CXX static library src/libscript.a
[292/582] Linking CXX static library src/libbitcoinconsensus.a
[293/582] Linking CXX shared library src/libbitcoinconsensus.so.0.29.9
[294/582] Creating library symlink src/libbitcoinconsensus.so.0 src/libbitcoinconsensus.so
[295/582] Linking CXX executable src/bitcoin-cli
[296/582] Linking CXX executable src/bitcoin-tx
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
tobias_ruck updated this revision to Diff 48805.Jul 23 2024, 23:16

switch to BiMap

tobias_ruck planned changes to this revision.Jul 23 2024, 23:20
Harbormaster completed remote builds in B29685: Diff 48805.Jul 23 2024, 23:27
teamcity added a comment.Jul 23 2024, 23:27
Build Bitcoin ABC Diffs / Diff Testing (chronik-client-integration-tests) failed.

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/util.py", line 296, in wait_until_helper
    raise AssertionError(
AssertionError: Predicate ''''
            self.wait_until(lambda: is_finalblock(next_blockhash))
''' not true after 60.0 seconds
2024-07-23T23:26:11.700000Z TestFramework (INFO): Stopping nodes
2024-07-23T23:26:11.955000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240723_232458/setup_scripts/chronik-client_websocket_0
2024-07-23T23:26:11.955000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240723_232458/setup_scripts/chronik-client_websocket_0/test_framework.log
2024-07-23T23:26:11.955000Z TestFramework (ERROR): 
2024-07-23T23:26:11.956000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240723_232458/setup_scripts/chronik-client_websocket_0' to consolidate all logs
2024-07-23T23:26:11.956000Z TestFramework (ERROR): 
2024-07-23T23:26:11.957000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-07-23T23:26:11.957000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-07-23T23:26:11.957000Z TestFramework (ERROR): 
Running Unit Tests for Test Framework Modules
setup_scripts/chronik-client_websocket.py started
setup_scripts/chronik-client_websocket.py failed, Duration: 73 s

stdout:

stderr:


TEST                                      | STATUS    | DURATION

setup_scripts/chronik-client_websocket.py | ✖ Failed  | 73 s

ALL                                       | ✖ Failed  | 73 s (accumulated) 
Runtime: 73 s

Test runner for chronik-client_websocket completed with code 1
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |   28.35 |     8.64 |   24.57 |   28.35 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |    21.6 |     6.07 |    13.9 |   21.74 |                                   
  chronik.ts           |     6.1 |     0.83 |    2.54 |    6.09 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |   33.03 |    10.56 |    23.6 |   33.28 | ...,4991-5030,5038-5111,5146-5151 
 chronik-client/src    |   65.77 |    42.39 |   63.41 |   65.38 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 33-163,178-222,290-692            
  ChronikClientNode.ts |   90.84 |     72.8 |   96.38 |   90.84 | ...,1068,1078,1103,1115,1121,1127 
  failoverProxy.ts     |   75.22 |    51.61 |   62.06 |   74.52 | ...67,275-285,294,301,305,310,314 
  hex.ts               |   89.47 |       50 |      75 |   87.87 | 58,66-68                          
  validation.ts        |   93.33 |    81.81 |     100 |   92.59 | 33,39                             
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='1188']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='4189']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='337']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='3897']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='187']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='761']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='1176']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='4148']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1
teamcity added a comment.Jul 23 2024, 23:29
Build Bitcoin ABC Diffs / Diff Testing (build-chronik-plugins) failed.

Tail of the build log:

[239/582] Building CXX object src/CMakeFiles/script.dir/script/interpreter.cpp.o
[240/582] Building CXX object src/CMakeFiles/script.dir/script/standard.cpp.o
[241/582] Building CXX object src/CMakeFiles/common.dir/common/system.cpp.o
[242/582] Building CXX object src/CMakeFiles/script.dir/script/descriptor.cpp.o
[243/582] Building CXX object src/CMakeFiles/script.dir/script/sign.cpp.o
[244/582] Building CXX object src/CMakeFiles/common.dir/common/bloom.cpp.o
[245/582] Building CXX object src/CMakeFiles/script.dir/script/signingprovider.cpp.o
[246/582] Building CXX object src/CMakeFiles/common.dir/common/configfile.cpp.o
[247/582] Building CXX object src/CMakeFiles/common.dir/consensus/merkle.cpp.o
[248/582] Building CXX object src/CMakeFiles/common.dir/common/args.cpp.o
[249/582] Building CXX object src/CMakeFiles/common.dir/config.cpp.o
[250/582] Building CXX object src/CMakeFiles/common.dir/eventloop.cpp.o
[251/582] Building CXX object src/CMakeFiles/common.dir/cashaddrenc.cpp.o
[252/582] Building CXX object src/CMakeFiles/common.dir/chainparams.cpp.o
[253/582] Building CXX object src/CMakeFiles/common.dir/coins.cpp.o
[254/582] Building CXX object src/CMakeFiles/common.dir/compressor.cpp.o
[255/582] Building CXX object src/CMakeFiles/common.dir/feerate.cpp.o
[256/582] Building CXX object src/CMakeFiles/common.dir/kernel/chainparams.cpp.o
[257/582] Building CXX object src/CMakeFiles/common.dir/key_io.cpp.o
[258/582] Building CXX object src/CMakeFiles/common.dir/netaddress.cpp.o
[259/582] Building CXX object src/CMakeFiles/common.dir/core_read.cpp.o
[260/582] Building CXX object src/CMakeFiles/common.dir/key.cpp.o
[261/582] Building CXX object src/CMakeFiles/common.dir/core_write.cpp.o
[262/582] Building CXX object src/CMakeFiles/common.dir/merkleblock.cpp.o
[263/582] Building CXX object src/CMakeFiles/common.dir/primitives/block.cpp.o
[264/582] Building CXX object src/CMakeFiles/common.dir/outputtype.cpp.o
[265/582] Building CXX object src/CMakeFiles/common.dir/policy/policy.cpp.o
[266/582] Building CXX object src/CMakeFiles/common.dir/net_permissions.cpp.o
[267/582] Building C object src/secp256k1/CMakeFiles/recover-bench.dir/src/bench_recover.c.o
[268/582] Building CXX object src/CMakeFiles/common.dir/netbase.cpp.o
[269/582] Building CXX object src/CMakeFiles/common.dir/warnings.cpp.o
[270/582] Building CXX object src/CMakeFiles/common.dir/networks/abc/chainparamsconstants.cpp.o
[271/582] Building C object src/secp256k1/CMakeFiles/verify-bench.dir/src/bench_verify.c.o
[272/582] Building C object src/secp256k1/CMakeFiles/ecmult-bench.dir/src/bench_ecmult.c.o
[273/582] Building CXX object src/CMakeFiles/common.dir/psbt.cpp.o
[274/582] Building C object src/secp256k1/CMakeFiles/secp256k1.dir/src/secp256k1.c.o
[275/582] Building CXX object src/CMakeFiles/common.dir/rpc/rawtransaction_util.cpp.o
[276/582] Building CXX object src/CMakeFiles/common.dir/scheduler.cpp.o
[277/582] Building CXX object src/CMakeFiles/common.dir/networks/abc/checkpoints.cpp.o
[278/582] Building C object src/secp256k1/CMakeFiles/internal-bench.dir/src/bench_internal.c.o
[279/582] Building C object src/secp256k1/CMakeFiles/sign-bench.dir/src/bench_sign.c.o
[280/582] Building CXX object src/CMakeFiles/common.dir/protocol.cpp.o
[281/582] Building CXX object src/CMakeFiles/common.dir/rpc/util.cpp.o
[282/582] Building CXX object src/CMakeFiles/bitcoin-cli.dir/bitcoin-cli.cpp.o
[283/582] Linking C static library src/secp256k1/libsecp256k1.a
[284/582] Building CXX object src/CMakeFiles/bitcoin-tx.dir/bitcoin-tx.cpp.o
[285/582] Linking C executable src/secp256k1/ecmult-bench
[286/582] Linking C executable src/secp256k1/recover-bench
[287/582] Linking C executable src/secp256k1/sign-bench
[288/582] Linking C executable src/secp256k1/verify-bench
[289/582] Linking C executable src/secp256k1/internal-bench
[290/582] Linking CXX static library src/libcommon.a
[291/582] Linking CXX static library src/libscript.a
[292/582] Linking CXX static library src/libbitcoinconsensus.a
[293/582] Linking CXX shared library src/libbitcoinconsensus.so.0.29.9
[294/582] Creating library symlink src/libbitcoinconsensus.so.0 src/libbitcoinconsensus.so
[295/582] Linking CXX executable src/bitcoin-cli
[296/582] Linking CXX executable src/bitcoin-tx
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
tobias_ruck updated this revision to Diff 48808.Jul 24 2024, 00:29

rebase on as_ref fix

tobias_ruck edited the summary of this revision. (Show Details)Jul 24 2024, 00:36
tobias_ruck added a parent revision: D16530: [Chronik] Remove unneeded `.as_ref()` call.
Harbormaster completed remote builds in B29688: Diff 48808.Jul 24 2024, 00:47
Fabien added inline comments.Jul 24 2024, 10:19
chronik/chronik-plugin-common/src/data.rs
35

I suggest you use a couple lookup functions instead of passing this to all methods.
There are 2 reasons:

  1. This avoid exposing the internals
  2. This allow for further optimization. The use of a BiMap is very likely overkill in 99% of the use cases. While this is an easy way to to get started, having functions to perform the lookup will allow for changing this easily if we feel like this is necessary.
tobias_ruck updated this revision to Diff 48833.Jul 24 2024, 19:44

use encapsulation in PluginNameMap

tobias_ruck updated this revision to Diff 48834.Jul 24 2024, 19:45

fix lint

Harbormaster completed remote builds in B29706: Diff 48833.Jul 24 2024, 20:40
Harbormaster completed remote builds in B29707: Diff 48834.Jul 24 2024, 20:53
Fabien accepted this revision.Jul 25 2024, 14:13
Fabien added inline comments.
chronik/chronik-plugin-impl/src/tx.rs
258 ↗(On Diff #48834)
This revision is now accepted and ready to land.Jul 25 2024, 14:13
tobias_ruck updated this revision to Diff 48851.Jul 26 2024, 15:33

skip unloaded plugins

Harbormaster completed remote builds in B29723: Diff 48851.Jul 26 2024, 15:52
Closed by commit rABCf9ae1740cb21: [Chronik] Add `plugin` field to `TxInput` in `tx.py` (authored by tobias_ruck). · Explain WhyJul 26 2024, 16:19
This revision was automatically updated to reflect the committed changes.
tobias_ruck added a commit: rABCf9ae1740cb21: [Chronik] Add `plugin` field to `TxInput` in `tx.py`.
igibjanci1106 awarded a token.Jul 28 2024, 21:01

Revision Contents
Changeset List

PathSize
chronik/
chronik-db/
src/
plugins/
4 lines
chronik-plugin-common/
3 lines
src/
35 lines
1 line
chronik-plugin-impl/
src/
13 lines
57 lines
tests/
17 lines
443 lines

Diff 48808

View Options

Cargo.lock

Loading...
View Options

chronik/chronik-db/src/plugins/io.rs

Loading...
View Options

chronik/chronik-plugin-common/Cargo.toml

Loading...
View Options

chronik/chronik-plugin-common/src/data.rs

Loading...
View Options

chronik/chronik-plugin-common/src/lib.rs

Loading...
View Options

chronik/chronik-plugin-impl/src/tx.py

Loading...
View Options

chronik/chronik-plugin-impl/src/tx.rs

Loading...
View Options

chronik/chronik-plugin-impl/tests/test_tx_to_py.py

Loading...
View Options

chronik/chronik-plugin-impl/tests/test_tx_to_py.rs

Loading...