Avoid unsigned integer overflow in bitcoin-tx

Summary:
This is a backport of core#24139

Depends on D12814

Test Plan:
With UBSAN
ninja && ninja check check-functional

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Differential Revision: https://reviews.bitcoinabc.org/D12815