Search the filesystem to discover tests. Use the timing info from build and src directory to chose which ones are extended tests and which are regular ones
Summary:
This is the second commit for T163. The hardcoded list of test has been removed, timings are now use to identify extended tests.
Depends on D1116
Test Plan:
Preconditions: make sure, that you do not build from src tree.
--> Test1: In an absence of timing.json, all tests should be run
--> Make sure that timing.json does not exists in any directory
find -name timing.json
build/test/functional/test_runner.py
cat build/timing.json
--> Check if all test are present in timing.json including both 'txn_doublespend.py --mineblock' and 'txn_doublespend.py'
--> Test2: Run with timings.json present:
--> This time timings.json will be used and only test shorter than 40 seconds will be run
build/test/functional/test_runner.py
cat build/timing.json
--> First few test that should be executed are wallet.py,abc-p2p-fullblocktest.py, wallet-hd.py,fundrawtransaction.py
--> dbcrash.py should not be executed
--> Test3: Let's run again, specifying --extended. All test should be executed, regardless of their timings
build/test/functional/test_runner.py --extended
cat build/timing.json
--> Test4: timings from src directory should be used if not found in build
mv build/timing.json test/functional/timing.json
build/test/functional/test_runner.py
--> long running test (dbcrash.py...) should not be executed. They should not be present in build/timing.json
cat build/timing.json
--> Test5: Timings from build should override timings from src
--> Edit build/timing.json, so that it only contains single line:
--> [{ "time": 0, "name": "dbcrash.py" }]
--> run without --extended flag, the test dbcrash should run,
--> because it is classified as not extended in build and extended in src
build/test/functional/test_runner.py
Reviewers: deadalnix, #bitcoin_abc
Reviewed By: deadalnix, #bitcoin_abc
Subscribers: teamcity
Maniphest Tasks: T163
Differential Revision: https://reviews.bitcoinabc.org/D1135