Page MenuHomePhabricator

[chronik-client] Support for script endpoints
ClosedPublic

Authored by bytesofman on Feb 6 2024, 00:06.

Details

Reviewers
PiRK
Group Reviewers
Restricted Project
Commits
rABC5b3f957504d2: [chronik-client] Support for script endpoints
Summary

Add support for script endpoints for ChronikClientNode in chronik-client module

Test Plan

CI or see D14915

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Tail of the build log:

    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1028:15)
    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/work/modules/chronik-client/node_modules/@cspotcode/source-map-support/source-map-support.js:811:30)
    at Function.Module._load (node:internal/modules/cjs/loader:873:27)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/work/modules/ecashaddrjs/src/cashaddr.js:13:14)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Module.replacementCompile (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at /usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1608:43)
    at Object.<anonymous> (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/work/modules/chronik-client/test/integration/script_endpoints.ts:3:1)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Module.replacementCompile (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:60:13)
    at Module.m._compile (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1618:23)
    at module.exports (/usr/lib/node_modules/nyc/node_modules/default-require-extensions/js.js:7:9)
    at /usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1621:12)
    at Object.<anonymous> (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.exports.requireOrImport (/work/modules/chronik-client/node_modules/mocha/lib/nodejs/esm-utils.js:60:20)
    at async Object.exports.loadFilesAsync (/work/modules/chronik-client/node_modules/mocha/lib/nodejs/esm-utils.js:103:20)
    at async singleRun (/work/modules/chronik-client/node_modules/mocha/lib/cli/run-helpers.js:125:3)
    at async Object.exports.handler (/work/modules/chronik-client/node_modules/mocha/lib/cli/run.js:374:5)
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |    5.33 |     0.63 |    1.43 |    5.31 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |     4.9 |     0.67 |    1.72 |    4.89 |                                   
  chronik.ts           |    5.44 |     0.84 |    1.81 |    5.42 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |    4.15 |     0.45 |    1.59 |    4.17 | ...,2682-2729,2740-2804,2808-2813 
 chronik-client/src    |    7.95 |        0 |       0 |    7.89 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 29-159,174-218,286-686            
  ChronikClientNode.ts |    8.47 |        0 |       0 |    8.47 | 23-99,118-281                     
  failoverProxy.ts     |    4.44 |        0 |       0 |    4.59 | 27-276                            
  hex.ts               |   31.57 |        0 |       0 |   33.33 | 30-34,38-42,46-59,63-65           
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='157']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='2944']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='21']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='3293']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='8']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='559']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='155']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='2916']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1

tests working for avalanche finalization, p2pkh, and p2sh

Tail of the build log:

    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1028:15)
    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/work/modules/chronik-client/node_modules/@cspotcode/source-map-support/source-map-support.js:811:30)
    at Function.Module._load (node:internal/modules/cjs/loader:873:27)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/work/modules/ecashaddrjs/src/cashaddr.js:13:14)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Module.replacementCompile (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at /usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1608:43)
    at Object.<anonymous> (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/work/modules/chronik-client/test/integration/script_endpoints.ts:3:1)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Module.replacementCompile (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:60:13)
    at Module.m._compile (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1618:23)
    at module.exports (/usr/lib/node_modules/nyc/node_modules/default-require-extensions/js.js:7:9)
    at /usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1621:12)
    at Object.<anonymous> (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.exports.requireOrImport (/work/modules/chronik-client/node_modules/mocha/lib/nodejs/esm-utils.js:60:20)
    at async Object.exports.loadFilesAsync (/work/modules/chronik-client/node_modules/mocha/lib/nodejs/esm-utils.js:103:20)
    at async singleRun (/work/modules/chronik-client/node_modules/mocha/lib/cli/run-helpers.js:125:3)
    at async Object.exports.handler (/work/modules/chronik-client/node_modules/mocha/lib/cli/run.js:374:5)
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |    5.33 |     0.63 |    1.43 |    5.31 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |     4.9 |     0.67 |    1.72 |    4.89 |                                   
  chronik.ts           |    5.44 |     0.84 |    1.81 |    5.42 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |    4.15 |     0.45 |    1.59 |    4.17 | ...,2682-2729,2740-2804,2808-2813 
 chronik-client/src    |    7.95 |        0 |       0 |    7.89 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 29-159,174-218,286-686            
  ChronikClientNode.ts |    8.47 |        0 |       0 |    8.47 | 23-99,118-281                     
  failoverProxy.ts     |    4.44 |        0 |       0 |    4.59 | 27-276                            
  hex.ts               |   31.57 |        0 |       0 |   33.33 | 30-34,38-42,46-59,63-65           
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='157']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='2944']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='21']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='3293']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='8']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='559']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='155']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='2916']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1

Tail of the build log:

    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1028:15)
    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/work/modules/chronik-client/node_modules/@cspotcode/source-map-support/source-map-support.js:811:30)
    at Function.Module._load (node:internal/modules/cjs/loader:873:27)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/work/modules/ecashaddrjs/src/cashaddr.js:13:14)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Module.replacementCompile (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at /usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1608:43)
    at Object.<anonymous> (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/work/modules/chronik-client/test/integration/script_endpoints.ts:3:1)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Module.replacementCompile (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:60:13)
    at Module.m._compile (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1618:23)
    at module.exports (/usr/lib/node_modules/nyc/node_modules/default-require-extensions/js.js:7:9)
    at /usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/work/modules/chronik-client/node_modules/ts-node/src/index.ts:1621:12)
    at Object.<anonymous> (/usr/lib/node_modules/nyc/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.exports.requireOrImport (/work/modules/chronik-client/node_modules/mocha/lib/nodejs/esm-utils.js:60:20)
    at async Object.exports.loadFilesAsync (/work/modules/chronik-client/node_modules/mocha/lib/nodejs/esm-utils.js:103:20)
    at async singleRun (/work/modules/chronik-client/node_modules/mocha/lib/cli/run-helpers.js:125:3)
    at async Object.exports.handler (/work/modules/chronik-client/node_modules/mocha/lib/cli/run.js:374:5)
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |    5.33 |     0.63 |    1.43 |    5.31 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |     4.9 |     0.67 |    1.72 |    4.89 |                                   
  chronik.ts           |    5.44 |     0.84 |    1.81 |    5.42 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |    4.15 |     0.45 |    1.59 |    4.17 | ...,2682-2729,2740-2804,2808-2813 
 chronik-client/src    |    7.95 |        0 |       0 |    7.89 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 29-159,174-218,286-686            
  ChronikClientNode.ts |    8.47 |        0 |       0 |    8.47 | 23-99,118-281                     
  failoverProxy.ts     |    4.44 |        0 |       0 |    4.59 | 27-276                            
  hex.ts               |   31.57 |        0 |       0 |   33.33 | 30-34,38-42,46-59,63-65           
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='157']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='2944']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='21']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='3293']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='8']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='559']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='155']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='2916']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1

Installing ecashaddrjs at latest version

add pagination tests, tests for p2pk type, tests for other type

doxygen fixes, comment cleanup, remove unrelated changes

add tests for other scrips, organize code

create functions to organize tests

refactor to handle txs_broadcast of arbitrary amount, tested to 100, bump timeout

proofreading, better comments

bytesofman published this revision for review.Feb 8 2024, 00:30
bytesofman added inline comments.
modules/chronik-client/.mocharc.js
3 ↗(On Diff #45026)

test usually takes 10s now with the avalanche quorum

modules/chronik-client/test/integration/script_endpoints.ts
157 ↗(On Diff #45049)

Need to do this check to pass typescript checks, as cashaddr.decode can return UInt8Array for hash

test/functional/setup_scripts/chronik-client_script_utxos_and_history.py
54 ↗(On Diff #45050)

I wasn't sure how to get this output script from the address here. So, hardcoded.

61 ↗(On Diff #45050)

ditto. got it from ecashaddrjs in bash.

tobias_ruck added inline comments.
test/functional/setup_scripts/chronik-client_script_utxos_and_history.py
54 ↗(On Diff #45050)

Maybe it's time to add a decode_script function to test/functional/test_framework/cashaddr.py?

then it would be p2pkh_output_script = cashaddr.decode_script(p2pkh_address).hex()

assertion move to end of step

do not test rpc calls in the setup script, copypasta removal

Nits for the setup_script. I haven't reviewed the rest yet.

test/functional/setup_scripts/chronik-client_script_utxos_and_history.py
13 ↗(On Diff #45082)

Additional import needed for my suggestion below:

from test_framework.script import OP_CHECKSIG, CScript
37 ↗(On Diff #45082)

This probably also need self.skip_if_no_wallet() for fundrawtransaction and sendtoaddress RPCs.
We currently don't have any CI tests that build chronik without the wallet ( -DBUILD_BITCOIN_CHRONIK=ON -DBUILD_BITCOIN_WALLET=OFF), so nothing is going to fail, but better to be future proof.

Tested locally:

$ ./test/functional/test_runner.py setup_scripts/chronik-client_script_utxos_and_history.py
Starting Bitcoin ABC functional tests
Temporary test directory at /home/pierre/dev/bitcoin-abc/buildChronik/test/tmp/test_runner_₿₵_🏃_20240209_123903
Running Unit Tests for Test Framework Modules
.........
----------------------------------------------------------------------
Ran 9 tests in 0.039s

OK
setup_scripts/chronik-client_script_utxos_and_history.py started
2024-02-09T11:39:03.151000Z TestFramework (INFO): Initializing test directory /home/pierre/dev/bitcoin-abc/buildChronik/test/tmp/test_runner_₿₵_🏃_20240209_123903/setup_scripts/chronik-client_script_utxos_and_history_0
{"chronik": "http://127.0.0.1:30001"}
{"p2pkh_address": "ecregtest:qrfhcnyqnl5cgrnmlfmms675w93ld7mvvqjh9pgptw"}
{"p2sh_address": "ecregtest:prfhcnyqnl5cgrnmlfmms675w93ld7mvvq9jcw0zsn"}
{"p2pk_script": "047fa64f6874fb7213776b24c40bc915451b57ef7f17ad7b982561f99f7cdc7010d141b856a092ee169c5405323895e1962c6b0d7c101120d360164c9e4b3997bd"}
{"other_script": "deadbeef"}
{"status": "ready"}
next
2024-02-09T11:39:05.562000Z TestFramework (INFO): Step 1: Broadcast txs to a p2pk, p2pkh, and p2sh address
{"txs_broadcast": 25}
2024-02-09T11:39:05.563000Z TestFramework (ERROR): JSONRPC error
Traceback (most recent call last):
  File "/home/pierre/dev/bitcoin-abc/test/functional/setup_scripts/../test_framework/test_framework.py", line 148, in main
    self._run_test_internal()
  File "/home/pierre/dev/bitcoin-abc/test/functional/setup_scripts/setup_framework.py", line 46, in _run_test_internal
    next(setup_steps)
  File "/home/pierre/dev/bitcoin-abc/test/functional/setup_scripts/chronik-client_script_utxos_and_history.py", line 91, in run_test
    p2pkh_txid = node.sendtoaddress(p2pkh_address, (x + 1) * 1000)
  File "/home/pierre/dev/bitcoin-abc/test/functional/setup_scripts/../test_framework/coverage.py", line 47, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
  File "/home/pierre/dev/bitcoin-abc/test/functional/setup_scripts/../test_framework/authproxy.py", line 179, in __call__
    raise JSONRPCException(response["error"], status)
test_framework.authproxy.JSONRPCException: Method not found (-32601)
2024-02-09T11:39:05.614000Z TestFramework (INFO): Stopping nodes
54–55 ↗(On Diff #45082)

The intermediate variable is not used anywhere else.
Maybe also drop the _bytes suffix in the variable name.

61–62 ↗(On Diff #45082)

same suggestion as above

67–71 ↗(On Diff #45082)

This could be simplified and made more readable as suggested. CScript is a subclass of bytes, so the rest of the code should keep working without change.
p2pk_script_for_tx_building would be a better name, more consistent with the other names used in this test.

bytesofman marked 5 inline comments as done.

Improving py script by following suggested comments

PiRK added inline comments.
modules/chronik-client/src/ChronikClientNode.ts
124 ↗(On Diff #45096)

The next line sounds like it is actually the anti-anti-chronological order (first seen first).

This revision is now accepted and ready to land.Feb 9 2024, 16:11
This revision was automatically updated to reflect the committed changes.
bytesofman marked an inline comment as done.
modules/chronik-client/src/ChronikClientNode.ts
124 ↗(On Diff #45096)

updated to reduce ambiguity

/**
  * Fetches the tx history of this script, in anti-chronological order.
  * This means it's ordered by first-seen first, i.e. TxHistoryPage_InNode.txs[0]
  * will be the most recent tx. If the tx hasn't been seen
  * by the indexer before, it's ordered by the block timestamp.
  * @param page Page index of the tx history.
  * @param pageSize Number of txs per page.
  */