HomePhabricator

Add an option to set the functional test suite name

Description

Add an option to set the functional test suite name

Summary:
Teamcity is relying on the test suite name extracted from the JUnit XML report to display display the suite and its tests.
This test suite name is hardcoded, so Teamcity treats all the functional test runs as a single one, preventing from accessing the test details from the web UI.
This diff adds an option to test_runner.py to allow for setting a name for the test suite, and use it to reflect the intent of the runs.

Teamcity UI before this diff (the test_runner.py script is invoked twice on this build) Link to the example build

AllTestsOverviewBefore.PNG (213×548 px, 10 KB)

The UI show 110 tests instead of 220.

TestDetailsBefore.PNG (282×1 px, 20 KB)

The tests appear only once, despite they are run twice.

Teamcity UI after this diff Link to the example build

AllTestsOverviewAfter.PNG (244×555 px, 11 KB)

The UI shows 220 tests as expected.

TestDetailsAfter.PNG (313×1 px, 25 KB)

The tests appear twice, one for each run. Each test for each run can be accessed directly by clicking the next to the test name and select Show in Build Log

Test Plan:
Look at the test list in the CI UI and check all the tests are present (220 at the moment).
Use the "Tests" tab to navigate through the tests and check the Show in Build Log link jumps to the related part of the build log.

Reviewers: #bitcoin_abc, deadalnix, jasonbcox

Reviewed By: #bitcoin_abc, deadalnix, jasonbcox

Subscribers: jasonbcox

Differential Revision: https://reviews.bitcoinabc.org/D4943

Details

Provenance
FabienAuthored on Jan 14 2020, 21:12
FabienPushed on Jan 27 2020, 11:08
Reviewer
Restricted Project
Differential Revision
D4943: Add an option to set the functional test suite name
Parents
rSTAGING3284c802aae0: move sigops counting from CheckBlock to ContextualCheckBlock
Branches
Unknown
Tags
Unknown