Improve the p2p_inv_download test by using a mocked time
ClosedPublic
Actions

Authored by Fabien on Jul 22 2021, 10:17.

Details

Reviewers

PiRK

Group Reviewers

Restricted Project

Commits

rABC3cca01825bf8: Improve the p2p_inv_download test by using a mocked time

Summary

The test_inv_block from p2p_inv_download.py test case has a few flaws:

The name does not reflect the content (it does not test any block inventory)
The test relies on some event to occur within some timeout, which is unlikely to fail but is not guaranteed, and in case of failure makes it not obvious what the problem is
It takes an absurd amount of time

This diff addresses all these points. It uses mocktime to both improve the robustness and make the test more easy to debug in case a failure occurs. The test duration is reduced by x2 on my machine (from ~120s to ~60s).

Test Plan

./test/functional/test_runnner.py p2p_inv_download

Diff Detail

Repository

rABC Bitcoin ABC

Branch

p2p_inv_download_mocktime

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 16245
Build 32357: Build Diff	build-debug · build-clang-tidy · build-without-wallet · build-diff · build-clang
Build 32356: arc lint + arc unit

Event Timeline

Fabien created this revision.Jul 22 2021, 10:17

Herald added a reviewer: Restricted Project. · View Herald TranscriptJul 22 2021, 10:17

Fabien requested review of this revision.Jul 22 2021, 10:17

Harbormaster completed remote builds in B16245: Diff 29295.Jul 22 2021, 10:31

PiRK accepted this revision.Jul 22 2021, 13:40

This revision is now accepted and ready to land.Jul 22 2021, 13:40

@bot build-master

teamcity awarded a token.Jul 22 2021, 13:45

Build Bitcoin ABC Diffs / Diff Testing (build-master) failed.

Failed tests logs:

====== Bitcoin ABC extended functional tests: feature_block.py ======

------- Stdout: -------
2021-07-22T13:52:36.006000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-master/test/tmp/test_runner_₿₵_  _20210722_135114/feature_block_5
2021-07-22T13:52:37.118000Z TestFramework (INFO): Reject block with invalid tx: OutputMissing
2021-07-22T13:52:39.132000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
        def test_function():
            if check_connected:
                assert self.is_connected
            return test_function_in()
'''
2021-07-22T13:52:42.838000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 127, in main
    self.run_test()
  File "/work/test/functional/feature_block.py", line 164, in run_test
    reconnect=True, timeout=2)
  File "/work/test/functional/feature_block.py", line 1297, in send_blocks
    reject_reason=reject_reason, force_send=force_send, timeout=timeout, expect_disconnect=reconnect)
  File "/work/test/functional/test_framework/p2p.py", line 722, in send_blocks_and_test
    test()
  File "/work/test/functional/test_framework/p2p.py", line 710, in test
    self.wait_for_disconnect(timeout=timeout)
  File "/work/test/functional/test_framework/p2p.py", line 485, in wait_for_disconnect
    self.wait_until(test_function, timeout=timeout, check_connected=False)
  File "/work/test/functional/test_framework/p2p.py", line 481, in wait_until
    timeout_factor=self.timeout_factor)
  File "/work/test/functional/test_framework/util.py", line 278, in wait_until
    "Predicate {} not true after {} seconds".format(predicate_source, timeout))
AssertionError: Predicate ''''
        def test_function():
            if check_connected:
                assert self.is_connected
            return test_function_in()
''' not true after 2.0 seconds
2021-07-22T13:52:42.889000Z TestFramework (INFO): Stopping nodes
2021-07-22T13:52:43.191000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/build-master/test/tmp/test_runner_₿₵_  _20210722_135114/feature_block_5
2021-07-22T13:52:43.191000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/build-master/test/tmp/test_runner_₿₵_  _20210722_135114/feature_block_5/test_framework.log
2021-07-22T13:52:43.191000Z TestFramework (ERROR): 
2021-07-22T13:52:43.191000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/build-master/test/tmp/test_runner_₿₵_  _20210722_135114/feature_block_5' to consolidate all logs
2021-07-22T13:52:43.191000Z TestFramework (ERROR): 
2021-07-22T13:52:43.191000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2021-07-22T13:52:43.191000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2021-07-22T13:52:43.191000Z TestFramework (ERROR):

Each failure log is accessible here:
Bitcoin ABC extended functional tests: feature_block.py

Closed by commit rABC3cca01825bf8: Improve the p2p_inv_download test by using a mocked time (authored by Fabien). · Explain WhyJul 22 2021, 20:21

This revision was automatically updated to reflect the committed changes.

Fabien added a commit: rABC3cca01825bf8: Improve the p2p_inv_download test by using a mocked time.