Page MenuHomePhabricator

Use functional tests timings from the source directory
ClosedPublic

Authored by Fabien on Mon, Jul 8, 19:08.

Details

Summary

The current behavior is to read timing from the timing.json file
location in the build directory is it exists.
But this file is updated every time the functional test suite is run,
which makes it impossible to predict which test will run as part of the
--extended call and which one will run as part as a standard call.
This diffs change this behavior so that the timing.json file from the
source directory is always used as the timings source to determine which
test to run.

Thanks @markblundeberg for reporting the issue.

Test Plan
./test/functional/test_runner.py

Change the timing for a previously ran test to a duration > 40s in the
test/functional/timing.json file.

./test/functional/test_runner.py

The test should not run anymore.

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Fabien created this revision.Mon, Jul 8, 19:08
Herald added a reviewer: Restricted Project. · View Herald TranscriptMon, Jul 8, 19:08

The inspiration for this change was an issue I was seeing: if a test once took longer than 40s to run, then it would never run again until I ran make clean since the last memory of running that test would be >40s. So the number of tests that would actually run was smaller than TeamCity, and gradually reducing over time. Not exactly the desired situation!

markblundeberg accepted this revision.Mon, Jul 8, 20:11
This revision is now accepted and ready to land.Mon, Jul 8, 20:11
Fabien edited the summary of this revision. (Show Details)Mon, Jul 8, 20:20
This revision was automatically updated to reflect the committed changes.