Improve Indices on pruned nodes via prune blockers
Needs ReviewPublic
Actions

Authored by Fabien on Mon, Dec 23, 10:58.

Details

Reviewers

None

Group Reviewers

Restricted Project

Summary

Completes backport of core#21726:
https://github.com/bitcoin/bitcoin/pull/21726/commits/825d19839bf71245306d4c8edde040e5941caa46
https://github.com/bitcoin/bitcoin/pull/21726/commits/de08932efa953e9a237cbf879460488ad8947411
https://github.com/bitcoin/bitcoin/pull/21726/commits/71c3f0356c01521a95c64fba1e7375aea6286bb0

Depends on D17389.

Backport of core#25123.

Test Plan

./test/functional/test_runner.py feature_index_prune

Diff Detail

Repository

rABC Bitcoin ABC

Branch

PR21726

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 31745
Build 62984: Build Diff	build-clang · build-diff · build-debug · build-clang-tidy · lint-circular-dependencies · build-without-wallet
Build 62983: arc lint + arc unit

Event Timeline

Fabien created this revision.Mon, Dec 23, 10:58

Herald added a reviewer: Restricted Project. · View Herald TranscriptMon, Dec 23, 10:58

Fabien requested review of this revision.Mon, Dec 23, 10:58

teamcity edited the summary of this revision. (Show Details)Mon, Dec 23, 10:58

Harbormaster completed remote builds in B31745: Diff 51722.Mon, Dec 23, 11:10

Build Bitcoin ABC Diffs / Diff Testing (build-debug) failed.

Failed tests logs:

====== Bitcoin ABC functional tests: feature_index_prune.py ======

------- Stdout: -------
2024-12-23T11:05:21.850000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_110033/feature_index_prune_292
2024-12-23T11:05:23.098000Z TestFramework (INFO): check if we can access blockfilters and coinstats when pruning is enabled but no blocks are actually pruned
2024-12-23T11:07:19.110000Z TestFramework (INFO): prune some blocks
2024-12-23T11:07:19.154000Z TestFramework (INFO): check if we can access the tips blockfilter and coinstats when we have pruned some blocks
2024-12-23T11:07:19.177000Z TestFramework (INFO): check if we can access the blockfilter and coinstats of a pruned block
2024-12-23T11:08:17.580000Z TestFramework (INFO): make sure trying to access the indices throws errors
2024-12-23T11:08:26.120000Z TestFramework (INFO): prune exactly up to the indices best blocks while the indices are disabled
2024-12-23T11:08:28.881000Z TestFramework (INFO): make sure that we can continue with the partially synced indices after having pruned up to the index height
2024-12-23T11:09:28.920000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
        self.wait_until(lambda: self.nodes[1].getindexinfo() == expected_stats)
'''
2024-12-23T11:09:28.920000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 149, in main
    self._run_test_internal()
  File "/work/test/functional/test_framework/test_framework.py", line 139, in _run_test_internal
    self.run_test()
  File "/work/test/functional/feature_index_prune.py", line 146, in run_test
    self.sync_index(height=1500)
  File "/work/test/functional/feature_index_prune.py", line 39, in sync_index
    self.wait_until(lambda: self.nodes[1].getindexinfo() == expected_stats)
  File "/work/test/functional/test_framework/test_framework.py", line 849, in wait_until
    return wait_until_helper(
           ^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/util.py", line 296, in wait_until_helper
    raise AssertionError(
AssertionError: Predicate ''''
        self.wait_until(lambda: self.nodes[1].getindexinfo() == expected_stats)
''' not true after 60.0 seconds
2024-12-23T11:09:28.971000Z TestFramework (INFO): Stopping nodes
2024-12-23T11:09:29.125000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_110033/feature_index_prune_292
2024-12-23T11:09:29.125000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_110033/feature_index_prune_292/test_framework.log
2024-12-23T11:09:29.125000Z TestFramework (ERROR): 
2024-12-23T11:09:29.125000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_110033/feature_index_prune_292' to consolidate all logs
2024-12-23T11:09:29.125000Z TestFramework (ERROR): 
2024-12-23T11:09:29.125000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-12-23T11:09:29.125000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-12-23T11:09:29.125000Z TestFramework (ERROR):

Each failure log is accessible here:
Bitcoin ABC functional tests: feature_index_prune.py

Fabien planned changes to this revision.Mon, Dec 23, 15:47

Bump the coinstatindex syncing timeout

Harbormaster completed remote builds in B31747: Diff 51727.Mon, Dec 23, 16:31

Build Bitcoin ABC Diffs / Diff Testing (build-debug) failed.

Failed tests logs:

====== Bitcoin ABC functional tests: feature_index_prune.py ======

------- Stdout: -------
2024-12-23T16:20:14.482000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_161544/feature_index_prune_292
2024-12-23T16:20:15.727000Z TestFramework (INFO): check if we can access blockfilters and coinstats when pruning is enabled but no blocks are actually pruned
2024-12-23T16:22:03.722000Z TestFramework (INFO): prune some blocks
2024-12-23T16:22:03.759000Z TestFramework (INFO): check if we can access the tips blockfilter and coinstats when we have pruned some blocks
2024-12-23T16:22:03.779000Z TestFramework (INFO): check if we can access the blockfilter and coinstats of a pruned block
2024-12-23T16:22:57.986000Z TestFramework (INFO): make sure trying to access the indices throws errors
2024-12-23T16:23:06.079000Z TestFramework (INFO): prune exactly up to the indices best blocks while the indices are disabled
2024-12-23T16:23:08.830000Z TestFramework (INFO): make sure that we can continue with the partially synced indices after having pruned up to the index height
2024-12-23T16:24:18.631000Z TestFramework (INFO): prune further than the indices best blocks while the indices are disabled
2024-12-23T16:24:45.334000Z TestFramework (INFO): make sure we get an init error when starting the nodes again with the indices
2024-12-23T16:24:46.933000Z TestFramework (INFO): make sure the nodes start again with the indices and an additional -reindex arg
2024-12-23T16:29:52.706000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 149, in main
    self._run_test_internal()
  File "/work/test/functional/test_framework/test_framework.py", line 139, in _run_test_internal
    self.run_test()
  File "/work/test/functional/feature_index_prune.py", line 180, in run_test
    self.sync_blocks(timeout=300)
  File "/work/test/functional/test_framework/test_framework.py", line 788, in sync_blocks
    raise AssertionError(f"Block sync timed out after {timeout}s:{best_hashes}")
AssertionError: Block sync timed out after 300s:
  '1614c329bfe78852e01556859f70e5185ff4ee27610502e7c0e853af59b47729'
  '202724d707732d951a76d1af2b42941e8df00a80422a844695091a2d873aa3c4'
  '3bda0fb3ce24cdd5de9791ad99df9102d2463dbd6be2997fd2f24e448bd0b7e1'
  '1614c329bfe78852e01556859f70e5185ff4ee27610502e7c0e853af59b47729'
2024-12-23T16:29:52.757000Z TestFramework (INFO): Stopping nodes
2024-12-23T16:29:53.763000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_161544/feature_index_prune_292
2024-12-23T16:29:53.763000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_161544/feature_index_prune_292/test_framework.log
2024-12-23T16:29:53.763000Z TestFramework (ERROR): 
2024-12-23T16:29:53.763000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_161544/feature_index_prune_292' to consolidate all logs
2024-12-23T16:29:53.763000Z TestFramework (ERROR): 
2024-12-23T16:29:53.763000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-12-23T16:29:53.763000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-12-23T16:29:53.763000Z TestFramework (ERROR):

Each failure log is accessible here:
Bitcoin ABC functional tests: feature_index_prune.py

Fabien planned changes to this revision.Mon, Dec 23, 17:32

Fabien edited the summary of this revision. (Show Details)Mon, Dec 23, 17:37

Add core#25123

Harbormaster completed remote builds in B31748: Diff 51728.Mon, Dec 23, 17:56

Build Bitcoin ABC Diffs / Diff Testing (build-debug) failed.

Failed tests logs:

====== Bitcoin ABC functional tests: feature_index_prune.py ======

------- Stdout: -------
2024-12-23T17:45:34.759000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_173951/feature_index_prune_292
2024-12-23T17:45:37.468000Z TestFramework (INFO): check if we can access blockfilters and coinstats when pruning is enabled but no blocks are actually pruned
2024-12-23T17:47:25.067000Z TestFramework (INFO): prune some blocks
2024-12-23T17:47:25.225000Z TestFramework (INFO): check if we can access the tips blockfilter and coinstats when we have pruned some blocks
2024-12-23T17:47:25.351000Z TestFramework (INFO): check if we can access the blockfilter and coinstats of a pruned block
2024-12-23T17:48:22.110000Z TestFramework (INFO): make sure trying to access the indices throws errors
2024-12-23T17:48:30.357000Z TestFramework (INFO): prune exactly up to the indices best blocks while the indices are disabled
2024-12-23T17:48:33.949000Z TestFramework (INFO): make sure that we can continue with the partially synced indices after having pruned up to the index height
2024-12-23T17:49:43.635000Z TestFramework (INFO): prune further than the indices best blocks while the indices are disabled
2024-12-23T17:50:12.052000Z TestFramework (INFO): make sure we get an init error when starting the nodes again with the indices
2024-12-23T17:50:14.307000Z TestFramework (INFO): make sure the nodes start again with the indices and an additional -reindex arg
2024-12-23T17:55:21.010000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 149, in main
    self._run_test_internal()
  File "/work/test/functional/test_framework/test_framework.py", line 139, in _run_test_internal
    self.run_test()
  File "/work/test/functional/feature_index_prune.py", line 179, in run_test
    self.sync_blocks(timeout=300)
  File "/work/test/functional/test_framework/test_framework.py", line 788, in sync_blocks
    raise AssertionError(f"Block sync timed out after {timeout}s:{best_hashes}")
AssertionError: Block sync timed out after 300s:
  '6829a00b902a3f19c9543e8443f5fabd64eda00c4bc619887a500a693be23237'
  '572510f48202038e65194e513294e717c94409b97bc0e9a7a78d0d4e42c1a881'
  '3cdd4ec9f4ae8303041a7162057c7cb52181b38b7dd44c2a70c0d90fb943581c'
  '6829a00b902a3f19c9543e8443f5fabd64eda00c4bc619887a500a693be23237'
2024-12-23T17:55:21.061000Z TestFramework (INFO): Stopping nodes
2024-12-23T17:55:22.117000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_173951/feature_index_prune_292
2024-12-23T17:55:22.117000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_173951/feature_index_prune_292/test_framework.log
2024-12-23T17:55:22.117000Z TestFramework (ERROR): 
2024-12-23T17:55:22.117000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20241223_173951/feature_index_prune_292' to consolidate all logs
2024-12-23T17:55:22.117000Z TestFramework (ERROR): 
2024-12-23T17:55:22.117000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-12-23T17:55:22.117000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-12-23T17:55:22.117000Z TestFramework (ERROR):

Each failure log is accessible here:
Bitcoin ABC functional tests: feature_index_prune.py