Page MenuHomePhabricator

[Cashtab] fetchTxDataForNullUtxos array threshold fix
ClosedPublic

Authored by emack on Dec 14 2021, 13:33.

Details

Reviewers
bytesofman
Group Reviewers
Restricted Project
Commits
rABC794777b766af: [Cashtab] fetchTxDataForNullUtxos array threshold fix
Summary
  • The fetchTxDataForNullUtxos() function in useBCH.js builds an array of tx IDs and then passes them to BCH.Electrumx.txData() to retrieve the corresponding transaction data.
  • The problem is BCH.Electrumx.txData() is limited to 20 TXIDs per request according to the spec, and wallets with > 20 null utxos will cause this api call to throw an 'array too large' error and fetchTxDataForNullUtxos()'s catch logic ignores the entire utxo set.
  • This diff batches up the array into 20 elements each before making the API call.
Test Plan

Diff Detail

Repository
rABC Bitcoin ABC
Branch
utxoFix
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 17625
Build 35076: Build Diffcashtab-tests
Build 35075: arc lint + arc unit

Event Timeline

emack requested review of this revision.Dec 14 2021, 13:33
emack edited the summary of this revision. (Show Details)
bytesofman added inline comments.
web/cashtab/src/components/Common/Ticker.js
29

Let's just have one item in Ticker.js and call it xecApiBatchSize

Right now this is 20. it's the same for all batch functions in xec-api so no need for multiple params.

web/cashtab/src/hooks/useBCH.js
541

Build an array of promises and then use Promise.all to make the API request.

This revision now requires changes to proceed.Dec 14 2021, 18:44
emack marked an inline comment as done.

Refactored fetchTxDataForNullUtxos() to build an array of promises and then executed via a subsequent promise.all call.

Per our Telegram discussion, here is some code that handles some of the issues here. It includes a new function that returns a Promise, so that Promise.all can work correctly, and also some adjustments to the fetchTxDataForNullUtxos function

const fetchTxDataPromise = (BCH, txidBatch) => {
        return new Promise((resolve, reject) => {
            BCH.Electrumx.txData(txidBatch).then(
                result => {
                    resolve(result);
                },
                err => {
                    return reject(err);
                },
            );
        });
    };

    const fetchTxDataForNullUtxos = async (BCH, nullUtxos) => {
        // Check nullUtxos. If they aren't eToken txs, count them
        console.log(
            `Null utxos found, checking OP_RETURN fields to confirm they are not eToken txs.`,
        );
        const txids = [];
        for (let i = 0; i < nullUtxos.length; i += 1) {
            // Batch API call to get their OP_RETURN asm info
            txids.push(nullUtxos[i].tx_hash);
        }

        // segment the txids array into chunks under the api limit
        const batchedTxids = batchArray(txids, currency.xecApiBatchSize);

        // build an array of promises
        let txDataPromises = [];
        // loop through each batch of 20 txids
        for (let j = 0; j < batchedTxids.length; j += 1) {
            const txidsForThisFetch = batchedTxids[j];
            //console.log(`txidsForThisPromise`, txidsForThisFetch);
            // build the promise for the api call with the 20 txids in current batch
            const thisTxDataPromise = fetchTxDataPromise(
                BCH,
                txidsForThisFetch,
            );
            txDataPromises.push(thisTxDataPromise);
        }

        try {
            const nullUtxoTxData = await Promise.all(txDataPromises);
            // Scan tx data for each utxo to confirm they are not eToken txs
            let thisTxDataResult;
            let nonEtokenUtxos = [];
            for (let k = 0; k < nullUtxoTxData.length; k += 1) {
                thisTxDataResult = nullUtxoTxData[k].transactions;
                nonEtokenUtxos = nonEtokenUtxos.concat(
                    checkNullUtxosForTokenStatus(thisTxDataResult),
                );
            }
            return nonEtokenUtxos;
        } catch (err) {
            console.log(
                `Error in checkNullUtxosForTokenStatus(nullUtxos)` + err,
            );
            console.log(`nullUtxos`, nullUtxos);
            // If error, ignore these utxos, will be updated next utxo set refresh
            return [];
        }
    };
This revision now requires changes to proceed.Dec 17 2021, 22:54

Revised promise array logic and reverted checkNullUtxosForTokenStatus() changes in cashMethods.js

bytesofman added inline comments.
web/cashtab/src/hooks/useBCH.js
524 ↗(On Diff #31453)

delete return

This revision now requires changes to proceed.Dec 17 2021, 23:28
emack marked an inline comment as done.

Removed redundant return statement

This revision is now accepted and ready to land.Dec 17 2021, 23:36