Paths

Table of Contentst

Differential D11385

[Cashtab] Commence implementation of chronik
AbandonedPublic
Actions

Authored by bytesofman on Apr 25 2022, 21:52.

Details

Reviewers

None

Group Reviewers

Restricted Project

Summary

T2419

Chronik is a full featured indexer capable of replacing the entire bch-api stack in Cashtab. The plan is to completley migrate away from bch-api to streamline infrastructure and dependency tree.

This diff (incomplete) is an early experimental approach toward recreating the utxo refresh engine using chronik instead of bch-api

Test Plan

n/a; changes planned. For now, this diff is up for review to help in planning next steps.

Diff Detail

Repository

rABC Bitcoin ABC

Branch

wb5-chronik

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 19048
Build 37848: Build Diff	cashtab-tests
Build 37847: arc lint + arc unit

Event Timeline

bytesofman created this revision.Apr 25 2022, 21:52

Herald added a reviewer: Restricted Project. · View Herald TranscriptApr 25 2022, 21:52

bytesofman requested review of this revision.Apr 25 2022, 21:52

bytesofman planned changes to this revision.Apr 25 2022, 21:54

Harbormaster completed remote builds in B18883: Diff 33305.Apr 25 2022, 21:56

Some thoughts on next steps here

It would be nice to have a function that updates the wallet info based only on which utxos have changed. This will be particularly important for maximizing the speed of the app by using websockets.

hash160 instead of addresses -- this complication should be handled server side, not app side. chronik should accept ecash addresses.

May need to rework how Cashtab deals with token balances. Constantly translating between long and BigNumber.js can't be the final approach.

Loading chronik wallet from local storage, correctly using chronik utxos in sendXEC()

Harbormaster completed remote builds in B19048: Diff 33544.May 16 2022, 17:46

Build Bitcoin ABC Diffs / Diff Testing (cashtab-tests) failed.

Failed tests logs:

====== CashTab Unit Tests: useBCH hook sends XEC correctly ======
TypeError: Cannot read property 'outIdx' of undefined
    at sendXec (/work/web/cashtab/src/hooks/useBCH.js:1669:44)
    at Object.<anonymous> (/work/web/cashtab/src/hooks/__tests__/useBCH.test.js:133:19)
    at Promise.then.completed (/work/web/cashtab/node_modules/jest-circus/build/utils.js:391:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/work/web/cashtab/node_modules/jest-circus/build/utils.js:316:10)
    at _callCircusTest (/work/web/cashtab/node_modules/jest-circus/build/run.js:218:40)
    at processTicksAndRejections (node:internal/process/task_queues:94:5)
    at _runTest (/work/web/cashtab/node_modules/jest-circus/build/run.js:155:3)
    at _runTestsForDescribeBlock (/work/web/cashtab/node_modules/jest-circus/build/run.js:66:9)
    at _runTestsForDescribeBlock (/work/web/cashtab/node_modules/jest-circus/build/run.js:60:9)
    at run (/work/web/cashtab/node_modules/jest-circus/build/run.js:25:3)
    at runAndTransformResultsToJestFormat (/work/web/cashtab/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:170:21)
    at jestAdapter (/work/web/cashtab/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:82:19)
    at runTestInternal (/work/web/cashtab/node_modules/jest-runner/build/runTest.js:389:16)
    at runTest (/work/web/cashtab/node_modules/jest-runner/build/runTest.js:475:34)
    at Object.worker (/work/web/cashtab/node_modules/jest-runner/build/testWorker.js:133:12)
====== CashTab Unit Tests: useBCH hook sends XEC correctly with an encrypted OP_RETURN message ======
TypeError: Cannot read property 'outIdx' of undefined
    at sendXec (/work/web/cashtab/src/hooks/useBCH.js:1669:44)
    at processTicksAndRejections (node:internal/process/task_queues:94:5)
    at Object.<anonymous> (/work/web/cashtab/src/hooks/__tests__/useBCH.test.js:169:13)
====== CashTab Unit Tests: useBCH hook sends one to many XEC correctly ======
TypeError: Cannot read property 'outIdx' of undefined
    at sendXec (/work/web/cashtab/src/hooks/useBCH.js:1669:44)
    at Object.<anonymous> (/work/web/cashtab/src/hooks/__tests__/useBCH.test.js:207:19)
    at Promise.then.completed (/work/web/cashtab/node_modules/jest-circus/build/utils.js:391:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/work/web/cashtab/node_modules/jest-circus/build/utils.js:316:10)
    at _callCircusTest (/work/web/cashtab/node_modules/jest-circus/build/run.js:218:40)
    at processTicksAndRejections (node:internal/process/task_queues:94:5)
    at _runTest (/work/web/cashtab/node_modules/jest-circus/build/run.js:155:3)
    at _runTestsForDescribeBlock (/work/web/cashtab/node_modules/jest-circus/build/run.js:66:9)
    at _runTestsForDescribeBlock (/work/web/cashtab/node_modules/jest-circus/build/run.js:60:9)
    at run (/work/web/cashtab/node_modules/jest-circus/build/run.js:25:3)
    at runAndTransformResultsToJestFormat (/work/web/cashtab/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:170:21)
    at jestAdapter (/work/web/cashtab/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:82:19)
    at runTestInternal (/work/web/cashtab/node_modules/jest-runner/build/runTest.js:389:16)
    at runTest (/work/web/cashtab/node_modules/jest-runner/build/runTest.js:475:34)
    at Object.worker (/work/web/cashtab/node_modules/jest-runner/build/testWorker.js:133:12)
====== CashTab Unit Tests: useBCH hook receives errors from the network and parses it ======
TypeError: process.cwd is not a function
    at formatStack (/work/web/cashtab/node_modules/expect/build/toThrowMatchers.js:455:28)
    at message (/work/web/cashtab/node_modules/expect/build/toThrowMatchers.js:251:13)
    at getMessage (/work/web/cashtab/node_modules/expect/build/index.js:217:15)
    at processResult (/work/web/cashtab/node_modules/expect/build/index.js:333:25)
    at throwingMatcher (/work/web/cashtab/node_modules/expect/build/index.js:402:16)
    at /work/web/cashtab/node_modules/expect/build/index.js:296:74
    at processTicksAndRejections (node:internal/process/task_queues:94:5)
    at Object.<anonymous> (/work/web/cashtab/src/hooks/__tests__/useBCH.test.js:339:9)
====== CashTab Unit Tests: useBCH hook Correctly parses a "receive XEC" transaction ======
TypeError: process.cwd is not a function
    at formatStack (/work/web/cashtab/node_modules/expect/build/toThrowMatchers.js:455:28)
    at message (/work/web/cashtab/node_modules/expect/build/toThrowMatchers.js:251:13)
    at getMessage (/work/web/cashtab/node_modules/expect/build/index.js:217:15)
    at processResult (/work/web/cashtab/node_modules/expect/build/index.js:333:25)
    at throwingMatcher (/work/web/cashtab/node_modules/expect/build/index.js:402:16)
    at /work/web/cashtab/node_modules/expect/build/index.js:296:74
    at processTicksAndRejections (node:internal/process/task_queues:94:5)

Each failure log is accessible here:
CashTab Unit Tests: useBCH hook sends XEC correctly
CashTab Unit Tests: useBCH hook sends XEC correctly with an encrypted OP_RETURN message
CashTab Unit Tests: useBCH hook sends one to many XEC correctly
CashTab Unit Tests: useBCH hook receives errors from the network and parses it
CashTab Unit Tests: useBCH hook Correctly parses a "receive XEC" transaction

This is a proof of concept to explore and test what will need to be changed in order to implement chronik and deprecate bch-api

While it is relatively straightforward to simply "make it work," there are a number of considerations to address in migrating the utxo battery of the app while in production.

This proof of concept:

Fetches and stores utxo and eToken information using chronik instead of bch-api
Successfully stores and retrieves utxo info in this state from local storage
Successfully broadcasts sendXEC() transactions using chronik utxo information

Some gaps

Other tx creation functions have not been adjusted to work with chronik utxo format
Websocket support is not addressed
The bignumber to long conversion is sloppy, additional thought needs to be put into implementing the prod approach here

Consequently -- this diff will be abandoned and replaced by a stacked diff.

hash160 instead of addresses -- this complication should be handled server side, not app side. chronik should accept ecash addresses.

This was a deliberate design decision to make the indexer not have to deal with addresses whatsoever.

For example, what should happen if an etoken: address is provided? Or what about a bitcoincash: address? All of those are valid according to the cashaddr spec. It's not obvious that those should be rejected.

And if an ecash: address is provided, should only XEC-only txs/utxos be returned? Or also txs/utxos that contain etokens?

By limiting it to the pubkeyhash, it's clear that everything that matches a certain Script will be returned. And those will be unique, by design there won't be any other ways to specify the query.

In D11385#256863, @tobias_ruck wrote:

hash160 instead of addresses -- this complication should be handled server side, not app side. chronik should accept ecash addresses.

This was a deliberate design decision to make the indexer not have to deal with addresses whatsoever.

For example, what should happen if an etoken: address is provided? Or what about a bitcoincash: address? All of those are valid according to the cashaddr spec. It's not obvious that those should be rejected.

And if an ecash: address is provided, should only XEC-only txs/utxos be returned? Or also txs/utxos that contain etokens?

By limiting it to the pubkeyhash, it's clear that everything that matches a certain Script will be returned. And those will be unique, by design there won't be any other ways to specify the query.

I put up a task to log this here: T2427

While I think it is more elegant to only deal with the hash160...I'm not sure that's great for dev tools. My preference would be for chronik to accept an ecash or an etoken or even a bitcoincash: address as the same thing, since it's just different encoding of the same thing.

Most wallets and apps will be built around an address. It's a weird complication for app devs to back-translate the address (already complicated enough) into something that most people new to crypto will not understand -- in order to get utxo information for the address.

In this case, it can be handled client-side. A library can provide a wrapper around chronik-client to accept addresses.

bytesofman added inline comments.May 17 2022, 15:14

web/cashtab/src/hooks/useWallet.js
182	NB: this approach is not robust. It just happens to work for a wallet only receiving transactions at 1 or 2 addresses. Prod implementation will need to be more sophisticated.