Page MenuHomePhabricator

[ecash-coinselect] Simplify to match functionality of coinselect library
ClosedPublic

Authored by bytesofman on Sep 22 2023, 22:28.

Details

Summary

Remove chronik utxo mgmt from ecash-coinselect. Focus on functionality of original coinselect library to support transaction building for app developers. Implement accumulative utxo selection algorithm.

Summary of changes from coinselect

  • Use eCash dust threshold
  • Convert input utxos value key from string (given by chronik) to number (used by TransactionBuilder)
  • Remove integer and infinity checks from coinselect (assume input utxos from chronik are not corrupted)
  • Replace var declarations with let
  • Ignore chronik slp utxos
  • Throw errors if inputs and outputs cannot be determined (vs behavior of coinselect, which returns {fee} instead of {inputs, outputs, fee} when this happens
Test Plan

npm test

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

emack added inline comments.
modules/ecash-coinselect/src/byteCount.js
44 ↗(On Diff #42351)

The previous calcP2pkhByteCount function is significantly more granula and accurate, with detailed documentation that makes it easier to extend to p2sh, uncompressed pubkeys...etc. I think it's a net positive to use it instead of coinselect's version of estimates.

modules/ecash-coinselect/src/coinSelect.js
13 ↗(On Diff #42351)

Is BLANK_OUTPUT representing the unspendable OP_RETURN output at index 0?

modules/ecash-coinselect/test/coinSelect.test.js
62 ↗(On Diff #42351)

Previous feedback from Fabien on usage of Buffer here:

What is the reason for using a Buffer here ? IMO we should not enforce the callsite to add an external dependency to use our library.
What about passing the hex string, and the doing the buffer conversion internally if that is required ?

Afaict Buffer is a node feature. Are we expecting all calling apps to use node ?
Fabien requested changes to this revision.Sep 25 2023, 08:56
Fabien added a subscriber: Fabien.

Can we get rid of the references to chronik everywhere ? The fact that the utxo format matches chronik is convenient but doesn't mean the lib can't be used with another backend. In fact it's not a dependency of this lib.

modules/ecash-coinselect/src/byteCount.js
1 ↗(On Diff #42351)

You should probably credit the original author in all files where you are using his code as well (you can have several copyright lines with various dates)

44 ↗(On Diff #42351)

Yes and no... It is more accurate for P2PKH but this one also supports P2SH.
Though I agree we should not add regressions. It's trivial to add support for the size as a varint, and some comments on the magic numbers will make this easier to maintain.

modules/ecash-coinselect/src/coinSelect.js
13 ↗(On Diff #42351)

@emack What is that OP_RETURN you're talking about ?

45 ↗(On Diff #42351)

How does that work ? targetOutputs is a json like array

57 ↗(On Diff #42351)

Please make sure the linter formatting is applied here, I think we use brackets everywhere else

modules/ecash-coinselect/test/coinSelect.test.js
62 ↗(On Diff #42351)

In this case it's not a requirement, (it's not even a good test imo).
The idea is that whatever you put to the output is returned by the library untouched. I'm not sure it's worth testing, but if we want to test that it certainly doesn't need a Buffer.

This revision now requires changes to proceed.Sep 25 2023, 08:56
bytesofman marked 8 inline comments as done.

Add support for hex strings as script input, remove chronik references, add validation for script types

modules/ecash-coinselect/src/byteCount.js
53 ↗(On Diff #42377)

This function is almost identical to inputBytes, only difference being the constants used and the name.

I looked into combining them but imo requiring inputOrOutput to be specified as a param to gate different constants is clumsier than keeping distinct functions.

44 ↗(On Diff #42351)

i think it's good to use the existing baseline here with coinselect values so that we can have a diff later focused on improving them. Don't necessarily need to publish this library after this diff.

modules/ecash-coinselect/src/coinSelect.js
13 ↗(On Diff #42351)

In this case, it's testing how the fee would change if a change output is added.

If a change output is less than dust, don't have one, even though this does make the fee higher.

45 ↗(On Diff #42351)

The sum function is defined in utils.js -- it's adding up all the value keys of the`targetOutputs`.

sumValues is probably a better name here, replaced.

Note: the previous sum function from coinselect also performed validation checks to make sure the value key was an integer and not infinite. I have dropped these checks, since parseInt is required to convert string types, and this will also eliminate decimals.

This means the function could create unexpected results if called with inputs and outputs that have decimal places at the value key.

imo it is ok to assume uncorrupted responses from the indexer. failure mode of rounding to the nearest satoshi is acceptable. failure mode if user supplying target outputs with decimal places and these in turn being rounded to the nearest satoshi also acceptable.

57 ↗(On Diff #42351)

This is correct in the linter ... but yes, it is a weird JS exception that this is valid only for one-line if statements, which are used nowhere else in the monorepo. Changed.

modules/ecash-coinselect/test/coinSelect.test.js
62 ↗(On Diff #42351)

The way inputBytes and outputBytes functions are written -- it does need to be a Buffer (or, at least, some kind of byte array, where script.length is equal to byteCount -- vs hex string where byteCount would be script.length / 2.

Buffer is what's used by TransactionBuilder, so it is convenient for this library to support this type. That said, easy enough to add support for hex strings as well.

62 ↗(On Diff #42351)

Are we expecting all calling apps to use node ?

This is a recurring issue in JS apps that work with cryptocurrency -- bitcoin, BCH, ETH, etc.

Buffer is often used in encryption libraries and to build transactions. So, dependencies support Buffer. But your React app might not support this. So it needs tools like webpack to be configured so that these types are supported.

It's possible to use webpack in the lib itself to "build" the dependency before you publish it -- i.e., the dependency will be a converted version of what is in the source code that, say, works on an earlier standard of JS or one without node-specific types. But since amost all app devs are using webpack or some equivalent, and it's not possible to predict their preferred settings, this is usually not worth it.

Fabien requested changes to this revision.Sep 26 2023, 07:37
Fabien added inline comments.
modules/ecash-coinselect/src/byteCount.js
53 ↗(On Diff #42377)

The general rule is to not write function that have an bool to change their behavior. It's a much better design to have 2 functions, it makes the code more readable and helps in avoiding mistakes

44 ↗(On Diff #42351)

Ok but don't bump the version then.

44 ↗(On Diff #42351)

Note: I was wrong and this supports OP_RETURN but not P2SH for now

modules/ecash-coinselect/src/coinSelect.js
45 ↗(On Diff #42351)

if it's not expected to happen, then you can assert it. The failure case will be a crash but it will only occur if there is an immediate need for a code change

modules/ecash-coinselect/test/coinSelect.test.js
62 ↗(On Diff #42351)

From reading https://stackoverflow.com/questions/51511307/nodejs-conversion-from-buffer-data-to-byte-array and checking the docs it appears that we could support Uint8Array which is JS native, and still have seemless Buffer integration

This revision now requires changes to proceed.Sep 26 2023, 07:37
bytesofman marked 5 inline comments as done.

assert check in sumValues, use native js array type checking with unit tested buffer support

remove line break, use determined variable in sumValues

modules/ecash-coinselect/src/byteCount.js
44 ↗(On Diff #42351)

Version number in this case is relative to existing ecash-coinselect and not the inspiring coinselect lib. Needs to bump to 2.0.0 as this diff breaks the previous functionality of ecash-coinselect

Will actually need to bump it to 2.0.1 or similar, since I didn't realize you can only have one version number published even across tags -- and 2.0.0 tag rc is now already occupied by a past version of this diff -- this though should only be done after the diff is approved.

Fabien added inline comments.
modules/ecash-coinselect/src/byteCount.js
44 ↗(On Diff #42351)

At some point the release of the libraries will be automated, and the obvious solution is to check is version has changed to decide whether to publish the lib or not.
When this feature is ready (and really it's something to look into soonish, we have a lot of libs already) then bumping the version will be equal to deployment.
Let's leave it for now but please keep this in mind for the future diffs.

modules/ecash-coinselect/src/utils.js
25 ↗(On Diff #42408)

Nit: can you improve on the message a bit ? Like:

Ill-formed input, should be an object with 'value' as a key and an integer representing the amount in satoshis as a value
This revision is now accepted and ready to land.Sep 27 2023, 08:39