Page MenuHomePhabricator
Differential D16544

[ecash-agora] Add `agora.py` plugin to index ONESHOT offers
ClosedPublic
Actions

Authored by tobias_ruck on Jul 29 2024, 16:03.

Details

Reviewers
Fabien
bytesofman
Group Reviewers
Restricted Project
Commits
rABC4df038a7cffb: [ecash-agora] Add `agora.py` plugin to index ONESHOT offers
Summary

Now that we have plugin support for Chronik, we can build a plugin to index Agora Oneshot offers efficiently.

Using the /utxos endpoint for the Agora plugin, users can basically fetch the equivalent of an orderbook from the blockchain.

D16551 allows us to use the test_framework's CScript class and all the opcodes.

We have to add a hash160 function; the current impl requires it to be enabled in OpenSSL, but we might add it from ripemd160.py from the test_framework too at some point, or add a function from Rust.

We also add Agora and AgoraOffer to ecash-agora, allowing easy fetching, accepting and cancelling of offers via the plugin, and to have a precise estimation of the required sats.

Depends on D16674.

Test Plan

npm run integration-tests

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
tobias_ruck added a comment.Jul 30 2024, 18:35

rebase onto D16551

Harbormaster completed remote builds in B29792: Diff 48938.Jul 30 2024, 18:56
teamcity added a comment.Jul 30 2024, 18:56
Build Bitcoin ABC Diffs / Diff Testing (chronik-client-integration-tests) failed.

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/util.py", line 296, in wait_until_helper
    raise AssertionError(
AssertionError: Predicate ''''
            self.wait_until(lambda: is_finalblock(next_blockhash))
''' not true after 60.0 seconds
2024-07-30T18:55:35.444000Z TestFramework (INFO): Stopping nodes
2024-07-30T18:55:35.698000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240730_185422/setup_scripts/chronik-client_websocket_0
2024-07-30T18:55:35.698000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240730_185422/setup_scripts/chronik-client_websocket_0/test_framework.log
2024-07-30T18:55:35.698000Z TestFramework (ERROR): 
2024-07-30T18:55:35.699000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240730_185422/setup_scripts/chronik-client_websocket_0' to consolidate all logs
2024-07-30T18:55:35.700000Z TestFramework (ERROR): 
2024-07-30T18:55:35.700000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-07-30T18:55:35.700000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-07-30T18:55:35.701000Z TestFramework (ERROR): 
Running Unit Tests for Test Framework Modules
setup_scripts/chronik-client_websocket.py started
setup_scripts/chronik-client_websocket.py failed, Duration: 73 s

stdout:

stderr:


TEST                                      | STATUS    | DURATION

setup_scripts/chronik-client_websocket.py | ✖ Failed  | 73 s

ALL                                       | ✖ Failed  | 73 s (accumulated) 
Runtime: 73 s

Test runner for chronik-client_websocket completed with code 1
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |   26.72 |     8.16 |   22.74 |   26.72 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |   20.19 |     5.71 |   12.67 |   20.34 |                                   
  chronik.ts           |     6.1 |     0.83 |    2.54 |    6.09 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |   29.37 |     9.45 |      20 |   29.61 | ...,5662-5701,5709-5782,5817-5826 
 chronik-client/src    |   65.21 |    42.39 |   62.27 |   64.81 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 33-163,178-222,290-692            
  ChronikClientNode.ts |   89.07 |     72.8 |   93.02 |   89.07 | ...,1103,1113,1138,1150,1156,1162 
  failoverProxy.ts     |   75.22 |    51.61 |   62.06 |   74.52 | ...67,275-285,294,301,305,310,314 
  hex.ts               |   89.47 |       50 |      75 |   87.87 | 58,66-68                          
  validation.ts        |   93.33 |    81.81 |     100 |   92.59 | 33,39                             
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='1194']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='4468']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='337']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='4125']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='187']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='822']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='1182']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='4423']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1
tobias_ruck updated this revision to Diff 48939.Jul 30 2024, 19:03

rerun ci

Harbormaster completed remote builds in B29793: Diff 48939.Jul 30 2024, 19:28
tobias_ruck mentioned this in rABC3d759543a0a9: [Chronik] Use the test framework's CScript in plugins instead of our own version.Jul 31 2024, 13:30
tobias_ruck updated this revision to Diff 49129.Aug 9 2024, 23:14

rebase, add group prefixes

Harbormaster completed remote builds in B29924: Diff 49129.Aug 9 2024, 23:41
tobias_ruck updated this revision to Diff 49162.Aug 12 2024, 21:32

Add Agora and AgoraOffer TS classes to simplify accepting Agora offers

Harbormaster completed remote builds in B29949: Diff 49162.Aug 12 2024, 22:14
tobias_ruck published this revision for review.Aug 12 2024, 22:30
bytesofman requested changes to this revision.Aug 12 2024, 22:58
bytesofman added a subscriber: bytesofman.
bytesofman added inline comments.
chronik/chronik-plugin-impl/src/module.rs
28 ↗(On Diff #49162)

I don't understand what's happening here

chronik/plugins/agora.py
181 ↗(On Diff #49162)

Do these specific opcode checks contradict this diff description?

There's currently no nice framework for plugins, so for now we e.g. just include all opcodes, and a hash160 function.

or are these still blanket included in the plugin, but just not in some parsed specifics?

modules/ecash-agora/src/agora.ts
51 ↗(On Diff #49162)

Is individual here distinct from oneshot ?

97 ↗(On Diff #49162)

it's a bit awkward here since we want these to be editable, and it is probably the right idea to accept an object here instead of a long list of params -- but really I don't think we can ask app devs to always specify something like dustAmount. It might always be 546.

99 ↗(On Diff #49162)

same here -- should have some way of keeping 1000 as default, allowing dev to specify if they really want to

138 ↗(On Diff #49162)

see above

140 ↗(On Diff #49162)

see above

225 ↗(On Diff #49162)

didn't even realize JS had this syntax

why this (do {...} while () and not

while (nextStart !== '') {
// do stuff
}
modules/ecash-agora/src/oneshot.ts
175 ↗(On Diff #49162)

is this a typo fix or an expected behavior change?

179 ↗(On Diff #49162)

is this a typo fix or an expected behavior change?

modules/ecash-agora/tests/oneshot.test.ts
316 ↗(On Diff #49162)
325 ↗(On Diff #49162)
488 ↗(On Diff #49162)

while this is possible and consistent with the existing test setup -- recommended approach would be to use the plugin utxos() endpoint to do this, right?

If so, we should model that here, or at least comment to that effect

This revision now requires changes to proceed.Aug 12 2024, 22:58
tobias_ruck edited the summary of this revision. (Show Details)Aug 13 2024, 07:21
tobias_ruck added inline comments.Aug 13 2024, 07:28
chronik/chronik-plugin-impl/src/module.rs
28 ↗(On Diff #49162)

We need functions like slp_send, which the test_framework already provides in slp.py.
slp.py imports CScript, but using test_framework as import path. Just replacing test_framework -> chronik_plugins allows us to import it in the Plugin system.

chronik/plugins/agora.py
181 ↗(On Diff #49162)

fixed the description, we indeed use the test_framework now (previously pasted all opcodes here https://reviews.bitcoinabc.org/D16544?id=48898).

modules/ecash-agora/src/agora.ts
51 ↗(On Diff #49162)

Yes, it's one UTXO offering tokens, but in the future I'd like to add other kinds of offers, e.g. allowing partially accepting offers.

97 ↗(On Diff #49162)

Good idea—I can make it dustAmount?: number and then default to 546.

99 ↗(On Diff #49162)

Good idea

225 ↗(On Diff #49162)

well there was a reason but since then I changed the code again and the reason disappeared

modules/ecash-agora/src/oneshot.ts
175 ↗(On Diff #49162)

Typo fix (rather, copy paste fix). If wanted I can split it out

179 ↗(On Diff #49162)

same here

modules/ecash-agora/tests/oneshot.test.ts
488 ↗(On Diff #49162)

The recommended approach is to use the Agora class. I just keep this to have parseAgoraTx tested.

tobias_ruck updated this revision to Diff 49175.Aug 13 2024, 07:43

address review

Harbormaster completed remote builds in B29961: Diff 49175.Aug 13 2024, 08:39
bytesofman added inline comments.Aug 13 2024, 17:40
modules/ecash-agora/src/agora.ts
51 ↗(On Diff #49162)
modules/ecash-agora/src/oneshot.ts
175 ↗(On Diff #49162)

nbd just making sure I'm understanding correctly

bytesofman requested changes to this revision.Aug 14 2024, 16:44
bytesofman added inline comments.
modules/ecash-agora/src/agora.ts
95 ↗(On Diff #49175)

what happens if fuel inputs are insufficient to cover the tx?

does it fail on broadcast or on creation?

would be a good test case to see. At the moment, all utxo selection is on the app developer. ecash-coinselect used to handle this with utxo-lib.

imo we need to get some kind of utxo selection integrated into ecash-lib. But, while it is absent, need some clarity on how an app dev would go about picking the right inputs here.

This revision now requires changes to proceed.Aug 14 2024, 16:44
bytesofman added inline comments.Aug 15 2024, 23:00
modules/ecash-agora/src/agora.ts
8 ↗(On Diff #49175)

will need to be updated after rebasing now that D16627 is landed

tobias_ruck updated this revision to Diff 49317.Aug 22 2024, 02:59

rebase, update using a dataclass, add sats calculation as requested

tobias_ruck updated this revision to Diff 49318.Aug 22 2024, 02:59

remove unrelated chronik-db change

tobias_ruck edited the summary of this revision. (Show Details)Aug 22 2024, 03:03
tobias_ruck added a parent revision: D16673: [ecash-lib] Fix fee estimation for signatories that depend on tx outputs.
Harbormaster completed remote builds in B30058: Diff 49317.Aug 22 2024, 03:28
Harbormaster completed remote builds in B30059: Diff 49318.Aug 22 2024, 03:51
tobias_ruck updated this revision to Diff 49322.Aug 22 2024, 12:25

minor fixes/cleanup, rebase on D16674

tobias_ruck edited the summary of this revision. (Show Details)Aug 22 2024, 12:30
tobias_ruck added a parent revision: D16674: [Chronik] Rename `group` in `PluginOutput` to `groups`.
tobias_ruck removed parent revisions: D16673: [ecash-lib] Fix fee estimation for signatories that depend on tx outputs, D16543: [chronik-client] Add `plugins` field to outputs/inputs, add `PluginEndpoint`.
Harbormaster completed remote builds in B30063: Diff 49322.Aug 22 2024, 12:45
teamcity added a comment.Aug 22 2024, 12:45
Build Bitcoin ABC Diffs / Diff Testing (build-chronik) failed.

Tail of the build log:

    Checking url v2.5.0
    Checking rand v0.8.5
    Checking tracing v0.1.40
    Checking http-body-util v0.1.1
    Checking chronik-util v0.1.0 (/work/chronik/chronik-util)
    Checking sha1 v0.10.6
    Checking utf-8 v0.7.6
    Checking data-encoding v2.5.0
    Checking httpdate v1.0.3
    Checking ryu v1.0.17
    Checking embedded-io v0.4.0
    Checking cobs v0.2.3
    Checking smallvec v1.13.2
    Checking postcard v1.0.8
    Checking tungstenite v0.21.0
    Checking pin-project v1.1.5
    Checking chronik-plugin-common v0.1.0 (/work/chronik/chronik-plugin-common)
   Compiling prost-build v0.11.9
    Checking chronik-plugin v0.1.0 (/work/chronik/chronik-plugin)
    Checking serde_spanned v0.6.6
    Checking toml_datetime v0.6.6
   Compiling pyo3 v0.22.2
    Checking sync_wrapper v0.1.2
    Checking topo_sort v0.4.0
    Checking seahash v4.1.0
    Checking winnow v0.6.13
    Checking mime v0.3.17
    Checking memoffset v0.9.1
    Checking serde_json v1.0.115
    Checking hyper v1.2.0
    Checking axum-core v0.4.3
    Checking tokio-tungstenite v0.21.0
    Checking tower v0.4.13
    Checking serde_urlencoded v0.7.1
    Checking futures-executor v0.3.30
    Checking hyper-util v0.1.3
    Checking serde_path_to_error v0.1.16
    Checking unicode-segmentation v1.11.0
    Checking base64 v0.21.7
    Checking matchit v0.7.3
    Checking sync_wrapper v1.0.1
    Checking unindent v0.2.3
    Checking futures v0.3.30
    Checking convert_case v0.6.0
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking tower-http v0.5.2
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
    Checking toml_edit v0.22.14
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking axum v0.7.5
   Compiling pyo3-macros v0.22.2
    Checking toml v0.8.14
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
    Checking chronik-indexer v0.1.0 (/work/chronik/chronik-indexer)
    Checking chronik-http v0.1.0 (/work/chronik/chronik-http)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 25s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1
teamcity added a comment.Aug 22 2024, 12:54
Build Bitcoin ABC Diffs / Diff Testing (build-chronik-plugins) failed.

Tail of the build log:

    Checking heapless v0.7.17
    Checking rand v0.8.5
    Checking http-body-util v0.1.1
    Checking tracing v0.1.40
    Checking chronik-util v0.1.0 (/work/chronik/chronik-util)
    Checking sha1 v0.10.6
    Checking smallvec v1.13.2
    Checking cobs v0.2.3
    Checking embedded-io v0.4.0
    Checking ryu v1.0.17
    Checking httpdate v1.0.3
    Checking utf-8 v0.7.6
    Checking data-encoding v2.5.0
    Checking postcard v1.0.8
    Checking pin-project v1.1.5
    Checking chronik-plugin-common v0.1.0 (/work/chronik/chronik-plugin-common)
   Compiling prost-build v0.11.9
    Checking chronik-plugin v0.1.0 (/work/chronik/chronik-plugin)
    Checking tungstenite v0.21.0
    Checking serde_spanned v0.6.6
    Checking toml_datetime v0.6.6
   Compiling pyo3 v0.22.2
    Checking seahash v4.1.0
    Checking winnow v0.6.13
    Checking mime v0.3.17
    Checking hyper v1.2.0
    Checking topo_sort v0.4.0
    Checking sync_wrapper v0.1.2
    Checking tokio-tungstenite v0.21.0
    Checking axum-core v0.4.3
    Checking tower v0.4.13
    Checking memoffset v0.9.1
    Checking serde_json v1.0.115
    Checking serde_urlencoded v0.7.1
    Checking futures-executor v0.3.30
    Checking hyper-util v0.1.3
    Checking serde_path_to_error v0.1.16
    Checking unindent v0.2.3
    Checking unicode-segmentation v1.11.0
    Checking matchit v0.7.3
    Checking base64 v0.21.7
    Checking sync_wrapper v1.0.1
    Checking tower-http v0.5.2
    Checking futures v0.3.30
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking convert_case v0.6.0
    Checking toml_edit v0.22.14
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking axum v0.7.5
   Compiling pyo3-macros v0.22.2
    Checking toml v0.8.14
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
    Checking chronik-indexer v0.1.0 (/work/chronik/chronik-indexer)
    Checking chronik-http v0.1.0 (/work/chronik/chronik-http)
    Finished dev [unoptimized + debuginfo] target(s) in 6m 24s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
tobias_ruck updated this revision to Diff 49325.Aug 22 2024, 12:58

move agora.py into ecash-agora (so it becomes part of the npm package), rebase to fix tests, update + fix comments

tobias_ruck updated this revision to Diff 49326.Aug 22 2024, 13:08

fix groups for the error case

teamcity added a comment.Aug 22 2024, 13:12
Build Bitcoin ABC Diffs / Diff Testing (chronik-client-integration-tests) failed.

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
  File "/work/test/functional/setup_scripts/../test_framework/authproxy.py", line 176, in __call__
    response, status = self._request(
  File "/work/test/functional/setup_scripts/../test_framework/authproxy.py", line 124, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1010, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/lib/python3.9/http/client.py", line 921, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.9/socket.py", line 843, in create_connection
    raise err
  File "/usr/lib/python3.9/socket.py", line 831, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused


TEST                                          | STATUS    | DURATION

setup_scripts/chronik-client_plugin_groups.py | ✖ Failed  | 2 s

ALL                                           | ✖ Failed  | 2 s (accumulated) 
Runtime: 2 s

Test runner for chronik-client_plugin_groups completed with code 1
----------------------|---------|----------|---------|---------|------------------------------------
File                  | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                  
----------------------|---------|----------|---------|---------|------------------------------------
All files             |   27.67 |     8.46 |   24.95 |   27.59 |                                    
 chronik-client       |     100 |      100 |     100 |     100 |                                    
  index.ts            |     100 |      100 |     100 |     100 |                                    
 chronik-client/proto |   20.51 |     6.19 |   14.49 |   20.69 |                                    
  chronik.ts          |   20.51 |     6.19 |   14.49 |   20.69 | ...3,6059-6062,6068-6110,6145-6154 
 chronik-client/src   |   62.05 |    38.06 |   60.16 |   61.21 |                                    
  ChronikClient.ts    |   52.87 |    35.33 |   56.66 |   52.87 | ...2,1164,1174,1195-1203,1211-1276 
  failoverProxy.ts    |   79.43 |    56.66 |   69.23 |   78.84 | ...282-285,288,297,304,308,313,317 
  hex.ts              |   89.47 |       50 |      75 |   87.87 | 58,66-68                           
  validation.ts       |   66.66 |    18.18 |   66.66 |   62.96 | 17,21,33,38-49,62-63               
----------------------|---------|----------|---------|---------|------------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='816']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='2949']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='209']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='2470']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='134']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='537']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='805']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='2917']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1
Harbormaster completed remote builds in B30065: Diff 49325.Aug 22 2024, 14:02
teamcity added a comment.Aug 22 2024, 14:02
Build Bitcoin ABC Diffs / Diff Testing (build-chronik-plugins) failed.

Tail of the build log:

    Checking tracing v0.1.40
    Checking rand v0.8.5
    Checking url v2.5.0
    Checking http-body-util v0.1.1
    Checking chronik-util v0.1.0 (/work/chronik/chronik-util)
    Checking sha1 v0.10.6
    Checking data-encoding v2.5.0
    Checking smallvec v1.13.2
    Checking utf-8 v0.7.6
    Checking embedded-io v0.4.0
    Checking cobs v0.2.3
    Checking httpdate v1.0.3
    Checking ryu v1.0.17
    Checking tungstenite v0.21.0
    Checking postcard v1.0.8
    Checking chronik-plugin-common v0.1.0 (/work/chronik/chronik-plugin-common)
    Checking pin-project v1.1.5
   Compiling prost-build v0.11.9
    Checking chronik-plugin v0.1.0 (/work/chronik/chronik-plugin)
    Checking toml_datetime v0.6.6
    Checking serde_spanned v0.6.6
    Checking hyper v1.2.0
   Compiling pyo3 v0.22.2
    Checking mime v0.3.17
    Checking sync_wrapper v0.1.2
    Checking winnow v0.6.13
    Checking topo_sort v0.4.0
    Checking seahash v4.1.0
    Checking axum-core v0.4.3
    Checking tokio-tungstenite v0.21.0
    Checking tower v0.4.13
    Checking memoffset v0.9.1
    Checking serde_json v1.0.115
    Checking serde_urlencoded v0.7.1
    Checking hyper-util v0.1.3
    Checking futures-executor v0.3.30
    Checking serde_path_to_error v0.1.16
    Checking unindent v0.2.3
    Checking unicode-segmentation v1.11.0
    Checking base64 v0.21.7
    Checking sync_wrapper v1.0.1
    Checking matchit v0.7.3
    Checking tower-http v0.5.2
    Checking futures v0.3.30
    Checking convert_case v0.6.0
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking toml_edit v0.22.14
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking axum v0.7.5
   Compiling pyo3-macros v0.22.2
    Checking toml v0.8.14
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
    Checking chronik-indexer v0.1.0 (/work/chronik/chronik-indexer)
    Checking chronik-http v0.1.0 (/work/chronik/chronik-http)
    Finished dev [unoptimized + debuginfo] target(s) in 6m 24s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
teamcity added a comment.Aug 22 2024, 14:16
Build Bitcoin ABC Diffs / Diff Testing (chronik-client-integration-tests) failed.

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
  File "/work/test/functional/setup_scripts/../test_framework/authproxy.py", line 176, in __call__
    response, status = self._request(
  File "/work/test/functional/setup_scripts/../test_framework/authproxy.py", line 124, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1010, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/lib/python3.9/http/client.py", line 921, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.9/socket.py", line 843, in create_connection
    raise err
  File "/usr/lib/python3.9/socket.py", line 831, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused


TEST                                          | STATUS    | DURATION

setup_scripts/chronik-client_plugin_groups.py | ✖ Failed  | 4 s

ALL                                           | ✖ Failed  | 4 s (accumulated) 
Runtime: 4 s

Test runner for chronik-client_plugin_groups completed with code 1
----------------------|---------|----------|---------|---------|------------------------------------
File                  | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                  
----------------------|---------|----------|---------|---------|------------------------------------
All files             |   27.67 |     8.46 |   24.95 |   27.59 |                                    
 chronik-client       |     100 |      100 |     100 |     100 |                                    
  index.ts            |     100 |      100 |     100 |     100 |                                    
 chronik-client/proto |   20.51 |     6.19 |   14.49 |   20.69 |                                    
  chronik.ts          |   20.51 |     6.19 |   14.49 |   20.69 | ...3,6059-6062,6068-6110,6145-6154 
 chronik-client/src   |   62.05 |    38.06 |   60.16 |   61.21 |                                    
  ChronikClient.ts    |   52.87 |    35.33 |   56.66 |   52.87 | ...2,1164,1174,1195-1203,1211-1276 
  failoverProxy.ts    |   79.43 |    56.66 |   69.23 |   78.84 | ...282-285,288,297,304,308,313,317 
  hex.ts              |   89.47 |       50 |      75 |   87.87 | 58,66-68                           
  validation.ts       |   66.66 |    18.18 |   66.66 |   62.96 | 17,21,33,38-49,62-63               
----------------------|---------|----------|---------|---------|------------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='816']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='2949']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='209']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='2470']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='134']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='537']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='805']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='2917']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1
Harbormaster completed remote builds in B30066: Diff 49326.Aug 22 2024, 14:17
teamcity added a comment.Aug 22 2024, 14:17
Build Bitcoin ABC Diffs / Diff Testing (build-chronik-plugins) failed.

Tail of the build log:

    Checking url v2.5.0
    Checking heapless v0.7.17
    Checking rand v0.8.5
    Checking http-body-util v0.1.1
    Checking chronik-util v0.1.0 (/work/chronik/chronik-util)
    Checking sha1 v0.10.6
    Checking embedded-io v0.4.0
    Checking smallvec v1.13.2
    Checking httpdate v1.0.3
    Checking cobs v0.2.3
    Checking ryu v1.0.17
    Checking data-encoding v2.5.0
    Checking utf-8 v0.7.6
    Checking postcard v1.0.8
    Checking pin-project v1.1.5
    Checking chronik-plugin-common v0.1.0 (/work/chronik/chronik-plugin-common)
   Compiling prost-build v0.11.9
    Checking tungstenite v0.21.0
    Checking serde_spanned v0.6.6
    Checking chronik-plugin v0.1.0 (/work/chronik/chronik-plugin)
    Checking toml_datetime v0.6.6
   Compiling pyo3 v0.22.2
    Checking sync_wrapper v0.1.2
    Checking mime v0.3.17
    Checking topo_sort v0.4.0
    Checking seahash v4.1.0
    Checking winnow v0.6.13
    Checking axum-core v0.4.3
    Checking hyper v1.2.0
    Checking tokio-tungstenite v0.21.0
    Checking tower v0.4.13
    Checking memoffset v0.9.1
    Checking serde_json v1.0.115
    Checking serde_urlencoded v0.7.1
    Checking futures-executor v0.3.30
    Checking serde_path_to_error v0.1.16
    Checking unindent v0.2.3
    Checking hyper-util v0.1.3
    Checking matchit v0.7.3
    Checking base64 v0.21.7
    Checking sync_wrapper v1.0.1
    Checking unicode-segmentation v1.11.0
    Checking futures v0.3.30
    Checking tower-http v0.5.2
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking convert_case v0.6.0
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
    Checking toml_edit v0.22.14
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking axum v0.7.5
   Compiling pyo3-macros v0.22.2
    Checking toml v0.8.14
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
    Checking chronik-indexer v0.1.0 (/work/chronik/chronik-indexer)
    Checking chronik-http v0.1.0 (/work/chronik/chronik-http)
    Finished dev [unoptimized + debuginfo] target(s) in 6m 32s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1
teamcity added a comment.Aug 22 2024, 14:28
Build Bitcoin ABC Diffs / Diff Testing (chronik-client-integration-tests) failed.

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
  File "/work/test/functional/setup_scripts/../test_framework/authproxy.py", line 176, in __call__
    response, status = self._request(
  File "/work/test/functional/setup_scripts/../test_framework/authproxy.py", line 124, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1010, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/lib/python3.9/http/client.py", line 921, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.9/socket.py", line 843, in create_connection
    raise err
  File "/usr/lib/python3.9/socket.py", line 831, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused


TEST                                          | STATUS    | DURATION

setup_scripts/chronik-client_plugin_groups.py | ✖ Failed  | 4 s

ALL                                           | ✖ Failed  | 4 s (accumulated) 
Runtime: 4 s

Test runner for chronik-client_plugin_groups completed with code 1
----------------------|---------|----------|---------|---------|------------------------------------
File                  | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                  
----------------------|---------|----------|---------|---------|------------------------------------
All files             |   27.67 |     8.46 |   24.95 |   27.59 |                                    
 chronik-client       |     100 |      100 |     100 |     100 |                                    
  index.ts            |     100 |      100 |     100 |     100 |                                    
 chronik-client/proto |   20.51 |     6.19 |   14.49 |   20.69 |                                    
  chronik.ts          |   20.51 |     6.19 |   14.49 |   20.69 | ...3,6059-6062,6068-6110,6145-6154 
 chronik-client/src   |   62.05 |    38.06 |   60.16 |   61.21 |                                    
  ChronikClient.ts    |   52.87 |    35.33 |   56.66 |   52.87 | ...2,1164,1174,1195-1203,1211-1276 
  failoverProxy.ts    |   79.43 |    56.66 |   69.23 |   78.84 | ...282-285,288,297,304,308,313,317 
  hex.ts              |   89.47 |       50 |      75 |   87.87 | 58,66-68                           
  validation.ts       |   66.66 |    18.18 |   66.66 |   62.96 | 17,21,33,38-49,62-63               
----------------------|---------|----------|---------|---------|------------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='816']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='2949']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='209']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='2470']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='134']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='537']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='805']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='2917']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1
tobias_ruck updated this revision to Diff 49332.Aug 22 2024, 17:02

rebase to fix build

Harbormaster completed remote builds in B30070: Diff 49332.Aug 22 2024, 17:52
bytesofman accepted this revision.Aug 26 2024, 19:31

update using a dataclass

could you summarize what this does and why this became preferable? I can see the organization change, is it just easier to read and maintain as a class?

why was this not used before adding the fee calculation methods?

modules/ecash-agora/src/agora.ts
245 ↗(On Diff #49332)

no blocker but something to consider in general with app dev and tools

Cashtab does all satoshis as number and not bigint because js Number.MAX_SAFE_INTEGER is 9007199254740991, about 9 quadrillion, more than the total number of existing ecash satoshis

9,007,199,254,740,991 max safe integer in JS
2,100,000,000,000,000 ecash satoshis

the use of bigint in the app libraries can end up coercing app devs into using them, since sometimes you are working with utxos that have a value key that is a bigint, which perhaps your are adding or comparing to your apps value keys which may be number.

mb this is a good thing...seems like bigint is safer than number, and does at least confirm we are always working with an int. however there will always be a complication if we also support number, and of course token values have to be bigint since these can exceed https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER

I'm leaning toward just using bigint all the time. Not sure there is any meaningful benefit in using number vs bigint, and always remembering the possible distinction when working with satoshis vs when working with token satoshis introduces a possible bug / ambiguity vector.

This revision is now accepted and ready to land.Aug 26 2024, 19:31
tobias_ruck added a comment.Aug 26 2024, 21:11
In D16544#378062, @bytesofman wrote:

update using a dataclass

could you summarize what this does and why this became preferable? I can see the organization change, is it just easier to read and maintain as a class?

This now nicely encapsulates all the parameters of the covenant in a dataclass, and the functions building the script are now nice methods.

For this covenant, the advantages aren’t too obvious, but for the partial offers, it’s quite obvious that encapsulating the parameters makes it much more readable.

why was this not used before adding the fee calculation methods?

Not sure what you’re asking there

tobias_ruck added inline comments.Aug 26 2024, 21:20
modules/ecash-agora/src/agora.ts
245 ↗(On Diff #49332)

We should use bigint for satoshis in general; this library might be used by other chains in the future too, and some don’t have this same limit.

Dogecoin for example has a limit of 10’000’000’000 DOGE, or 1’000’000’000’000’000’000 dogetoshi. Which far exceeds the integer limit of Number of 9,007,199,254,740,991.

While I don’t think we should build our system around other chains like Dogecoin, if there’s two options we have to pick from (in this case Number and BigInt), we should err on picking the one that has better compatibility with other chains.

bytesofman added a comment.Aug 26 2024, 21:39

Not sure what you’re asking there

as in, why no dataclass earlier in the diff -- but refactored for dataclass in the plugin here when fee estimating methods were added to the JS lib --

from your feedback I am guessing it is just coincidence, was easier to write it out without the dataclass first, upon reflection the dataclass was a more extensible improvement -- i..e this change is a general improvement and not necessarily related to the fee estimation stuff?

modules/ecash-agora/src/agora.ts
245 ↗(On Diff #49332)

good point, will keep this in mind going forward and on other libs

tobias_ruck added a comment.Aug 26 2024, 21:48
In D16544#378081, @bytesofman wrote:

Not sure what you’re asking there

from your feedback I am guessing it is just coincidence, was easier to write it out without the dataclass first, upon reflection the dataclass was a more extensible improvement -- i..e this change is a general improvement and not necessarily related to the fee estimation stuff?

Yeah it’s a coincidence, it’s completely unrelated.

tobias_ruck updated this revision to Diff 49377.Aug 27 2024, 21:04

rebase on master

Harbormaster completed remote builds in B30106: Diff 49377.Aug 27 2024, 21:36
Closed by commit rABC4df038a7cffb: [ecash-agora] Add `agora.py` plugin to index ONESHOT offers (authored by tobias_ruck). · Explain WhyAug 28 2024, 07:37
This revision was automatically updated to reflect the committed changes.
tobias_ruck added a commit: rABC4df038a7cffb: [ecash-agora] Add `agora.py` plugin to index ONESHOT offers.

Revision Contents
Changeset List

PathSize
chronik/
chronik-plugin-impl/
src/
8 lines
contrib/
teamcity/
1 line
modules/
ecash-agora/
271 lines
src/
446 lines
1 line
11 lines
tests/
312 lines
ecash-lib/
src/
ecash-agora/
src/
consts.ts
index.ts
7 lines
ecash-lib/
src/
1 line
test/
6 lines
test/
functional/
setup_scripts/
65 lines

Diff 49388

View Options

chronik/chronik-plugin-impl/src/module.rs

Loading...
View Options

contrib/teamcity/build-configurations.yml

Loading...
View Options

modules/ecash-agora/agora.py

Loading...
View Options

modules/ecash-agora/src/agora.ts

Loading...
View Options

modules/ecash-agora/src/index.ts

Loading...
View Options

modules/ecash-agora/src/oneshot.ts

Loading...
View Options

modules/ecash-agora/tests/oneshot.test.ts

Loading...
View Options

modules/ecash-lib/src/consts.ts

Loading...
View Options

modules/ecash-lib/src/index.ts

Loading...
View Options

modules/ecash-lib/src/test/testRunner.ts

Loading...
View Options

test/functional/setup_scripts/ecash-agora_base.py

Loading...