Page MenuHomePhabricator

[ecash-wallet] Automatically consolidate utxos and retry on max sersize errors
ClosedPublic

Authored by bytesofman on Wed, Apr 8, 19:56.

Details

Summary

This is a common enough edge case that is sufficiently terrible to explain to users that the lib should just handle it.

It's possible to hit a tx size error because the user must consolidate utxos. In this case, the user probably does not know what UTXOs are or why they should be consolidated. We just want to fulfill the action.

Adding auto utxo consolidation at the lib level is the best way to handle this.

Going forward, we can also look at alternative utxo selection strategies, ways to make this event less likely. But for now this solves most of the problem. We are already configured to build an array of txs that fulfill a given action; in this case, this array simply needs some additional txs before the action tx(s) to consolidate utxos.

Test Plan

npm test

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

also support (and test) token utxo consolidation for chained multi-output token txs

add a test showing combined consolidation

bytesofman published this revision for review.Wed, Apr 8, 21:18
bytesofman edited the summary of this revision. (Show Details)
Fabien requested changes to this revision.Thu, Apr 9, 15:02
Fabien added a subscriber: Fabien.
Fabien added inline comments.
modules/ecash-wallet/src/wallet.ts
172 ↗(On Diff #58947)

IMO this should:

  • not be used internally only, but exposed to the callsite
  • be false by default (so enableAutoConsolidationOnMaxTxSize instead of skipXXXX).

The reason for that is that this is going to be a big transaction, which implies high fee. For now the price is low so this might not be a big deal but it can't be relied upon. I would rather have a wallet catch the error, ask for user confirmation then retry with auto consolidation enabled so you don't spend money without notifying the user first. If the price goes up this can make a huge difference.

2497 ↗(On Diff #58947)

If you make the auto consolidation opt-in, you can even make the number of round a parameter so it's enabled if autoConsolidationRounds > 0

2539 ↗(On Diff #58947)

This could be a wallet function instead, so you don't have to keep track of all the wallet kinds out there.
I would suggest using a flag list in a function like wallet.getCapabilities() so it's easily extensible.
Can be another diff, non blocking

This revision now requires changes to proceed.Thu, Apr 9, 15:02
bytesofman marked 3 inline comments as done.
bytesofman added inline comments.
modules/ecash-wallet/src/wallet.ts
172 ↗(On Diff #58947)

I don't think it should default to false. I don't currently have any wallets or apps where I would want "prompt user about utxo consolidation" to be the default behavior.

As it stands,

  • a lib user can call build(ALL_BIP143, {skipAutoConsolidationOnMaxTxSize: true}) to get the behavior of "do not try to do the action if you have to consolidate." It's not internal-only.
  • a lib user could also call inspect() and get the maxTxSerSize() error; so any wallet using inspect() to estimate fees could continue this behavior and offer this kind of specific feedback to the user.

Cashtab gets this complaint maybe every other month (tx failing bc of maxTxSersize dev wtf is happening). Users have no idea what is going on, and I do not think they could be expected to get this. This kind of confirmation prompt imo would be just as confusing for the avg user

wallet: Are you sure you want to complete this tx? You need to consolidate UTXOs, so the fee is higher than normal.

user: wtf does that mean. sounds scary. no I don't want the high fee. reject.

now the tx doesn't work and the confirmation simply keeps returning if they try again. I don't think a typical user of a wallet app would think something like "well screw that fee, if we only partially consolidated utxos right now the action would be possible for a lower fee, I'm going to craft that" ... even the prompt of what's going on is confusing to non-experts.

Going forward, ecash-wallet should introduce better utxo selection strategies to prevent this kind of thing from happening (it's particularly annoying in multi-sig wallets, where now you have to do multisig consolidation first). For example, the only strategy now is none, purely accumulative. We could introduce "smallest first" which would keep dust and token dust utxos lower most of the time, or "largest first" which could make txs possible without consolidation if the wallet has lots of older small UTXOs but a handful of newer larger UTXOs.

For now, "just do the thing" is imo the best default approach. In all cases, I think the end user should never see anything about UTXO consolidation or UTXO selection algorithms. The lib user should have some control here, but I think the default should be geared toward the end user.

2497 ↗(On Diff #58947)

imo this is too technical for the intended user of this lib (even though, in this case, we are talking more about a dev working with the lib and not a wallet user).

"25" as a default is arguably a dumb choice, but it's hard to make the case that there are relative benefits of 17 vs 25 vs 3 depending on the user's intended Action.

2539 ↗(On Diff #58947)

This is probably better, but this can reasonably be expected to be "not multisig" behavior -- since multisig can never auto reshape any tx. Will extend this way if another flag / situation would benefit in the same way.

modules/ecash-wallet/src/wallet.ts
172 ↗(On Diff #58947)

I'm not comfortable with sending up to 25 bigs txs (actually can even be more) without the user being told anything about this. This might be a tradeoff a wallet would want to do for the sake of simplicity but it being the default behavior is unexpected for a wallet library. It's nice to have the feature, it's not to have any vibe coded app potentially send lots of txs without the dev being aware this might happen. If you feel like prompting the user is not necessary for your app it's fine (fees are low, and Cashtab doesn't show the fee anyway) but it's the exception and not the norm, most wallet will show this info and ask for confirmation.

I would draw the line here: don't assume what user/dev will accept to happen with their money.

I agree that the right move in the end is to have more utxo selection strategies, and I also agree that this is a useful feature (especially for cashtab), just not a sane default.

2497 ↗(On Diff #58947)

Ok it's likely too technical and hard to determine a value

bytesofman marked 3 inline comments as done.

remove arbitrary 25 retries, consolidateUtxo is already a complete method

bytesofman added inline comments.
modules/ecash-wallet/src/wallet.ts
172 ↗(On Diff #58947)

I removed the 25 thing, which is actually misleading. It's an "AI trying to provide some kind of abundance of caution" bug.

the consolidateUtxo method is already a while loop; if it is called at all, it will consolidate all the utxos.

It should always take only one pass. In some kind of edge case where we are still getting maxTxSersize error now after a consolidate, build() would throw.

I would draw the line here: don't assume what user/dev will accept to happen with their money.

I think this is a reasonable line. What build() does is fulfill the specified action. Perhaps it fulfills it in an unexpected way, but it is some edge case where this type of fulfillment is undesired. The user would want total control over their utxo set, would want to fulfill the original action (which would require some level of consolidation no matter what).

I lean toward this as default since

  • I've gotten the complain dozens of times, and it has never been from someone who wanted more granular control of the UTXO set, always from someone who just wants it to work and doesn't understand the error
  • If it were an option, I would need to enable it in all my implementing apps
  • Outside of Electrum-ABC or mb some other future power-user app, I don't think any user-facing wallet app should ever communicate to the user anything about the UTXO set. A big advantage of XEC over account based chains is that stuff "just works." The byzantine attempt to explain fee failures to users of ETH has never led to happy users. The fee rate is a configurable param in ecash-wallet, tho it defaults to the minimum.
  • Practically speaking, I do not think there is a single user who would require more than say 3 rounds of consolidation so send a given action their wallet can afford. Huge UTXO sets are today coming from:
  1. XECX payouts (which are daily, so usually takes years of inactivity for this to surface a maxTxSersize error)
  2. Firma payouts (same)
  3. Technical users who are creating machine gun txs for various purposes (if they want better UTXO control, then ecash-wallet's accumulative default is already not an ideal default)

... that said I don't mind too much either way, i.e. adding it as a setting isn't an insurmountable objection. I could do something like, update the maxTxSersize error msgs to recommend calling consolidateUtxos or to call build() with autoConsolidate: true

2497 ↗(On Diff #58947)

The original const and implementation was confusing; the consolidateUtxos method already uses a while loop and already makes as many txs as necessary.

One thing to keep in mind is that the wallet will indeed throw if it does not have enough XEC to cover the required fee of all the txs.

I was hesitant to request a major version, but it's not breaking existing API, only updating the behavior so I think minor bump is OK

This revision is now accepted and ready to land.Fri, Apr 10, 19:50