Page MenuHomePhabricator

add loadtxoutset & getchainstates RPCs, add functional tests
AbandonedPublic

Authored by PiRK on Tue, Apr 15, 07:02.

Details

Reviewers
None
Group Reviewers
Restricted Project
Summary

This is a partial backport of core#27596, core#28652, core#29345, core#28589, core#28590, core#28625, core#28639, core#28647, core#28562, core#28666, core#28669, core#28685, core#29215, core#29726, core#29354, core#29394, core#29478, core#29370, core#29617, core#28685, core#30053, core#29973, core#28838, core#29354, core#29478, core#30678, core#31556 and core#30909

Depends on D17933


rpc: add loadtxoutset

Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>

https://github.com/bitcoin/bitcoin/pull/27596/commits/ce585a9a158476b0ad3296477b922e79f308e795
core#28652
core#29345


rpc: add getchainstates

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>

https://github.com/bitcoin/bitcoin/pull/27596/commits/0f64bac6030334d798ae205cd7af4bf248feddd9
https://github.com/bitcoin/bitcoin/pull/28590/commits/a9ef702a877a964bac724a56e2c0b5bee4ea7586


test: add feature_assumeutxo functional test

Initial commit: https://github.com/bitcoin/bitcoin/pull/27596/commits/42cae39356fd20d521aaf99aff1ed85856f3c9f3

core#28589 (race fixes)
core#28590 (getchainstates return a list of chainstates)
core#28625 ( check that loading snapshot not matching AssumeUTXO parameters fails)
https://github.com/bitcoin/bitcoin/pull/28639/commits/fafde92f84fb7c245bc3c1cd946a32c891861e5e
core#28647 (Add assumeutxo test for wrong hash)
core#28652 (fail early if snapshot block hash doesn't match AssumeUTXO parameters)
core#28562 (self.no_op, self.wait_until)
core#28666 (assumeutxo file with unknown block hash)
core#28669 (check au file with changed outpoint index)
core#28685 (add tests for coin maleation)
core#29215 (spend coin from snapshot chainstate after loading)
https://github.com/bitcoin/bitcoin/pull/29726/commits/b7ba60f81a33db876f88b5f9af1e5025d679b5be (add coverage for -reindex and assumeutxo)
core#29354 (Assumeutxo with more than just coinbase transactions)
core#29394 (Add test to ensure failure when mempool not empty)
https://github.com/bitcoin/bitcoin/pull/29478/commits/2bc1ecfaa9b69a20388e913ec64967de2f506cd3 (Remove unnecessary sync_blocks in assumeutxo tests)
core#29478 (Add test for loadtxoutset when headers are not synced)
core#29370 (RPC test for fake nTx and nChainTX values, stale block CheckBlockIndex crash test & assumeutxo snapshot block CheckBlockIndex crash test)
core#29617 (test for coin_height > base_height & amount > money_supply)
https://github.com/bitcoin/bitcoin/pull/28685/commits/f6213929c519d0e615cacd3d6f479f1517be1662 (Check deserialized coins for out of range values)
core#30053 (coverage for "Couldn't open file..." error)
core#29973 (ensure failure when importing a snapshot twice)


test: add assumeutxo wallet test

initial commit: core#28838
https://github.com/bitcoin/bitcoin/pull/29354/commits/fa5cd66f0a47d1b759c93d01524ee4558432c0cc (Assumeutxo with more than just coinbase transactions)
https://github.com/bitcoin/bitcoin/pull/29478/commits/2bc1ecfaa9b69a20388e913ec64967de2f506cd3 (Remove unnecessary sync_blocks in assumeutxo tests)
https://github.com/bitcoin/bitcoin/pull/30678/commits/7e3dbe4180cbeb65e59b53d9fa98509e9189549d (functional test only)
https://github.com/bitcoin/bitcoin/pull/30678/commits/f20fe33e94c6752e5d2ed92511c0bf51a10716ee (Add basic balance coverage)
https://github.com/bitcoin/bitcoin/pull/31556/commits/bc43ecaf6dc0830a27296d3a29428814fed07bb1 (test for balance after snapshot completion)
https://github.com/bitcoin/bitcoin/pull/30909/commits/595edee169045b6735b76ff9721677f0e43f13e5 (import descriptors during background sync)

Test Plan

ninja all check-all

Event Timeline

PiRK requested review of this revision.Tue, Apr 15, 07:02

arcanist created a new diff for some reason

Failed tests logs:

====== Bitcoin ABC functional tests: feature_assumeutxo.py ======

------- Stdout: -------
2025-04-15T07:11:57.265000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20250415_070641/feature_assumeutxo_285
2025-04-15T07:11:58.984000Z TestFramework (INFO): -- Testing assumeutxo + some indexes + pruning
2025-04-15T07:11:58.985000Z TestFramework (INFO): Creating a UTXO snapshot at height 299
2025-04-15T07:11:59.030000Z TestFramework (INFO): Test loading snapshot when headers are not synced
2025-04-15T07:12:00.711000Z TestFramework (INFO): Test bitcoind should fail when mempool not empty.
2025-04-15T07:12:01.974000Z TestFramework (INFO): Test different scenarios of loading invalid snapshot files
2025-04-15T07:12:01.974000Z TestFramework (INFO):   - snapshot file refering to a block that is not in the assumeutxo parameters
2025-04-15T07:12:01.976000Z TestFramework (INFO):   - snapshot file with wrong number of coins
2025-04-15T07:12:02.252000Z TestFramework (INFO):   - snapshot file with alternated UTXO data
2025-04-15T07:12:03.036000Z TestFramework (INFO): Test bitcoind should fail when file path is invalid.
2025-04-15T07:12:03.037000Z TestFramework (INFO): Loading snapshot into second node from /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20250415_070641/feature_assumeutxo_285/node0/regtest/utxos.dat
2025-04-15T07:12:03.309000Z TestFramework (INFO): Submit a stale block that forked off the chain before the snapshot
2025-04-15T07:12:03.318000Z TestFramework (INFO): Submit a spending transaction for a snapshot chainstate coin to the mempool
2025-04-15T07:12:03.328000Z TestFramework (INFO): Restarting node to stop at height 359
2025-04-15T07:12:05.436000Z TestFramework (INFO): Checking that blocks are segmented on disk
2025-04-15T07:12:05.436000Z TestFramework (INFO): Restarted node before snapshot validation completed, reloading...
2025-04-15T07:12:06.587000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 152, in main
    self._run_test_internal()
  File "/work/test/functional/test_framework/test_framework.py", line 142, in _run_test_internal
    self.run_test()
  File "/work/test/functional/feature_assumeutxo.py", line 412, in run_test
    n1.submitblock(snapshot_block)
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 115, in _request
    return self._get_response()
           ^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 187, in _get_response
    http_response = self.__conn.getresponse()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.11/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 287, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
2025-04-15T07:12:06.639000Z TestFramework (INFO): Stopping nodes
[node 2] Cleaning up leftover process
[node 1] Cleaning up leftover process
[node 0] Cleaning up leftover process
------- Stderr: -------
Traceback (most recent call last):
  File "/work/test/functional/feature_assumeutxo.py", line 549, in <module>
    AssumeutxoTest().main()
  File "/work/test/functional/test_framework/test_framework.py", line 175, in main
    exit_code = self.shutdown()
                ^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/test_framework.py", line 404, in shutdown
    self.stop_nodes()
  File "/work/test/functional/test_framework/test_framework.py", line 659, in stop_nodes
    node.stop_node(wait=wait, wait_until_stopped=False)
  File "/work/test/functional/test_framework/test_node.py", line 532, in stop_node
    self.stop(wait=wait)
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 114, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.11/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib/python3.11/http/client.py", line 975, in send
    self.connect()
  File "/usr/lib/python3.11/http/client.py", line 941, in connect
    self.sock = self._create_connection(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 851, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.11/socket.py", line 836, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

Each failure log is accessible here:
Bitcoin ABC functional tests: feature_assumeutxo.py