Merge #11468: [tests] Make comp test framework more debuggable
Summary:
PR11468 backport https://github.com/bitcoin/bitcoin/pull/11468/files
748157913 [tests] Make comp test framework more debuggable (John Newbery)
Pull request description:
We should remove the comparison test framework entirely (see #10603). Until we do that, let's make it a bit more debuggable. Currently, if there's an assert in the framework, it's very difficult to track down where we are in the test generator. Make the logging a bit better to help with debugging. Before this PR: ``` → ./p2p-fullblocktest.py 2017-10-09 14:05:11.302000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testzdnax_yr 2017-10-09 14:05:11.557000 TestFramework.mininode (INFO): Connecting to Bitcoin Node: 127.0.0.1:11975 2017-10-09 14:05:11.712000 TestFramework.comptool (INFO): Test 1: PASS 2017-10-09 14:05:11.947000 TestFramework.comptool (INFO): Test 2: PASS 2017-10-09 14:05:12.057000 TestFramework.comptool (INFO): Test 3: PASS 2017-10-09 14:05:12.058000 TestFramework (ERROR): Assertion failed Traceback (most recent call last): File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 117, in main self.run_test() File "./p2p-fullblocktest.py", line 72, in run_test self.test.run() File "/home/ubuntu/bitcoin/test/functional/test_framework/comptool.py", line 306, in run assert test_number != 4 AssertionError 2017-10-09 14:05:12.059000 TestFramework (INFO): Stopping nodes 2017-10-09 14:05:14.203000 TestFramework (WARNING): Not cleaning up dir /tmp/user/1000/testzdnax_yr 2017-10-09 14:05:14.204000 TestFramework (ERROR): Test failed. Test logging available at /tmp/user/1000/testzdnax_yr/test_framework.log ``` With this PR: ``` → ./p2p-fullblocktest.py 2017-10-09 14:03:54.069000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testuey7t3tf 2017-10-09 14:03:54.329000 TestFramework.mininode (INFO): Connecting to Bitcoin Node: 127.0.0.1:11783 2017-10-09 14:03:54.383000 TestFramework.comptool (INFO): Running test 1: ./p2p-fullblocktest.py line 184 2017-10-09 14:03:54.496000 TestFramework.comptool (INFO): Running test 2: ./p2p-fullblocktest.py line 193 2017-10-09 14:03:54.758000 TestFramework.comptool (INFO): Running test 3: ./p2p-fullblocktest.py line 205 2017-10-09 14:03:54.867000 TestFramework.comptool (INFO): Running test 4: ./p2p-fullblocktest.py line 208 2017-10-09 14:03:54.867000 TestFramework (ERROR): Assertion failed Traceback (most recent call last): File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 117, in main self.run_test() File "./p2p-fullblocktest.py", line 72, in run_test self.test.run() File "/home/ubuntu/bitcoin/test/functional/test_framework/comptool.py", line 309, in run assert test_number != 4 AssertionError 2017-10-09 14:03:54.868000 TestFramework (INFO): Stopping nodes 2017-10-09 14:03:56.950000 TestFramework (WARNING): Not cleaning up dir /tmp/user/1000/testuey7t3tf 2017-10-09 14:03:56.950000 TestFramework (ERROR): Test failed. Test logging available at /tmp/user/1000/testuey7t3tf/test_framework.log ```
Test Plan:
test_runner.py
read test_framework.log for such a test (like abc-schnorr-activation.py)
Reviewers: jasonbcox, Fabien, #bitcoin_abc, deadalnix
Reviewed By: Fabien, #bitcoin_abc
Differential Revision: https://reviews.bitcoinabc.org/D3108