Page MenuHomePhabricator

[eCash library][ecash-coinselect] Initial npm module for basic utxo selection
ClosedPublic

Authored by emack on Jul 7 2023, 09:16.

Details

Summary

T3225

A UTXO selection module to address some of the feedback from D14198 that interactions with the eCash network should be simplified for app devs.

There are numerous technical nuances when interacting with the XEC chain via the Chronik indexer.
This includes Chronik's:

  • .script().utxos() api returning both XEC and eToken utxos, which requires separation
  • .script().utxos() api's response returning an array of one single object (with utxo array embedded), which requires trimming
  • method of differentiation of eToken utxos via the existance of the utxo.slpToken param, which differs from other SLP indexers
  • output for utxo.value is a string which needs convert to Number

... amongst others.

This module abstracts away the above nuances by providing app developers a higher level API to interact with the network. It hides the underlying implementation details when devs want to carry out simple operations such as sending an XEC transaction.

Included in this initial version:

  • parseChronikUtxos: parsing the utxo response returned from Chronik's .script().utxos() api and separate into XEC/eToken utxos
  • getInputUtxos: collecting enough XEC utxos to pay for a send amount and tx fee
  • calcByteCount: byte count calculation for p2pkh txs

Planned in future iterations:

  • collecting eToken utxos to pay for a send token amount and tx fees
  • collecting XEC/eToken utxos by finding the biggest utxos first
  • collecting XEC/eToken utxos by consolidating dust and using the smallest utxos first
  • collecting XEC/eToken utxos by finding the most appropriately sized for the given tx
  • utxo selection for OP_RETURN message txs
  • utxo selection for one to many txs
  • utxo selection for XEC airdrop txs
  • utxo selection for alias registrations
  • byte count calculation for p2sh, multisigs
Test Plan

npm test
plonk README into https://stackedit.io/app
compare byte size calculations with the bitcoin tx size calculator

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
modules/ecash-coinselect/src/utxo.js
119 ↗(On Diff #41313)

imo using eToken in code presents a bunch of complications, doesn't fit well with camel case conventions. Just use token or slp

This revision now requires changes to proceed.Jul 7 2023, 16:27
emack marked 9 inline comments as done.
  • updated utxo collection approach to assumes 2 outputs initially and then if change output is not required update the bytecount/txFee using 1 output count
  • updated getInputUtxos to return inputs, outputCount and txFee
  • updated terminology and naming throughout module
  • updated .gitignore
  • implemented eslint module to standardize headers
emack planned changes to this revision.Jul 8 2023, 10:42
  • bytecount formula is under review

Refactored p2pkh byte count calculation breakdowns and brought it back to fundamentals

emack edited the test plan for this revision. (Show Details)
emack edited the summary of this revision. (Show Details)
bytesofman added inline comments.
modules/ecash-coinselect/package.json
3 ↗(On Diff #41330)

to be consistent with the other libraries initialized in the monorepo. this is what ecash-script published at.

modules/ecash-coinselect/src/utxo.js
24 ↗(On Diff #41330)

This constant, together with SATOSHIS_PER_BYTE, could be used in other functions. define at the top of the file for now.

32 ↗(On Diff #41330)

I get this this approach probably makes it more difficult for a new-ish dev to burn SLP utxos. Still, seems a bit wasteful, as the only wallet example we are familiar with -- Cashtab -- already stores these separately.

The right solution here would be to patch chronik-client utxos() output to return utxos separated by xec / slp

If that is impractical, imo best to leave parseChronikUtxos as a standalone exported function in this lib, to be used by devs as they see fit

So, this function should require xecUtxos as a param, not chronikUtxos

50 ↗(On Diff #41330)

we don't want to be defining this at every i in the for loop, see comment above re: DUST

58 ↗(On Diff #41330)

imo the complications around prefix and postfix discrepancies with the decrement operator are not worth the character savings

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Decrement

76 ↗(On Diff #41330)

why not return the outputs here?

82 ↗(On Diff #41330)
102 ↗(On Diff #41330)

this won't exist if the user is passing chronik response of an address with no balance. in this case, the function should still return {xecUtxos: [], slpUtxos: []}

105 ↗(On Diff #41330)
126 ↗(On Diff #41330)
128 ↗(On Diff #41330)
129 ↗(On Diff #41330)
142 ↗(On Diff #41330)
modules/ecash-coinselect/test/utxo.test.js
20 ↗(On Diff #41330)

add unit test for chronik response of an address with on balance

41 ↗(On Diff #41330)

instead of 818, find a tx in the wild and get the rawtx. Then rawTx.length / 2

47 ↗(On Diff #41330)
This revision now requires changes to proceed.Jul 10 2023, 17:33
emack planned changes to this revision.Jul 13 2023, 05:14

Subject to D14257 landing

And further discussion / alignment with Pierre on coin selection algos

emack marked 16 inline comments as done.

Responding to feedback

modules/ecash-coinselect/package.json
3 ↗(On Diff #41330)

Updated

modules/ecash-coinselect/src/utxo.js
24 ↗(On Diff #41330)

Added DUST_AMOUNT_SATOSHIS and SATOSHIS_PER_BYTE constants to top of file

32 ↗(On Diff #41330)

As per latest position from D14257, we will proceed with this diff without the chronik-client utxos() patch.
Also parseChronikUtxos is already export via the modules.exports statement at the bottom of the file.

50 ↗(On Diff #41330)

Updated

58 ↗(On Diff #41330)

Updated

76 ↗(On Diff #41330)

Updated to return output values based on the intended sending sats plus whether remainder exists. Corresponding unit tests updated.

82 ↗(On Diff #41330)

Updated

102 ↗(On Diff #41330)

Updated

105 ↗(On Diff #41330)

Updated

126 ↗(On Diff #41330)

Added

128 ↗(On Diff #41330)

Updated

129 ↗(On Diff #41330)

Updated

142 ↗(On Diff #41330)

Updated

modules/ecash-coinselect/test/utxo.test.js
20 ↗(On Diff #41330)

Added unit test for an address with no balance/utxo

41 ↗(On Diff #41330)

Added rawTxHex sample from explorer

47 ↗(On Diff #41330)

Updated

modules/ecash-coinselect/test/utxo.test.js
86 ↗(On Diff #41638)

shouldn't the second output be more than 900 here, since it is change?

this seems to imply a fee of 38052 - 900 sats, which is much bigger than 374

Am I reading something wrong here?

bytesofman added inline comments.
modules/ecash-coinselect/package.json
32 ↗(On Diff #41639)

Add and configure mocha-suppress-logs, which is installed for alias-server, ecash-herald, and ecash-script

requires a .mocharc.js file, contents like

'use strict';

module.exports = {
    require: 'mocha-suppress-logs',
    timeout: 15000,
};
modules/ecash-coinselect/src/utxo.js
25 ↗(On Diff #41639)

update for outputValues array

82 ↗(On Diff #41639)

this should be changeAmountInSats

87 ↗(On Diff #41639)

ok, based on the current scope of the function, let's not return outputValues

instead, return

{inputUtxos, changeAmount, txFee}

this is the info that will be needed for the tx builder in the next step (well, not txFee, but that is still interesting to know and could be used for some applications)

modules/ecash-coinselect/test/utxo.test.js
43 ↗(On Diff #41639)

Add a couple of additional tests for this using different input and output counts

This revision now requires changes to proceed.Jul 31 2023, 21:20
emack marked 5 inline comments as done.
  • Installed mocha-suppress-logs and added .mocharc.js
  • Replaced outputValues with changeAmount in the return object
  • Added additional calcByteCount unit tests
  • Updated README
bytesofman requested changes to this revision.Aug 1 2023, 17:10
bytesofman added inline comments.
modules/ecash-coinselect/src/utxo.js
58 ↗(On Diff #41663)

In this case, your change amount is 0 -- even though remainder may be, say 500 sats. Note this also impacts the fee of the tx: we are returning the ideal tx fee, but the actual fee will be higher, since we aren't going to bother returning change < dust (so actual fee in this case would be ideal fee + 500 sats).

tricky to draw the line here in terms of what a dev using this tool expects as output vs what should just come straight out of this tool. If we return 0 as a change amount, the app dev still has to handle that when creating outputs, i.e. "if change amount is 0, do not create an output for change." If we return 500, as this code would do now, the app dev has to check "if change amount is less than dust, don't add change"

imo we want this to be a simple as possible. App dev shouldn't need to know "546". So, if there is no change, return 0 for change amount.

This revision now requires changes to proceed.Aug 1 2023, 17:10
emack marked an inline comment as done.

Updated to return changeAmount as 0 if it's below dust

bytesofman requested changes to this revision.Aug 2 2023, 17:06
bytesofman added inline comments.
modules/ecash-coinselect/src/utxo.js
57–65 ↗(On Diff #41673)
This revision now requires changes to proceed.Aug 2 2023, 17:06
emack marked an inline comment as done.

Update contextual comments

Note we need to publish this to npm as well

modules/ecash-coinselect/package.json
5 ↗(On Diff #41699)

this is ok and follows the way it works in ecashaddrjs

Note tho that if we add other files to src , this will need to be changed (which can be done, no compatibility issues, just something to be aware of)

This revision is now accepted and ready to land.Aug 3 2023, 17:35