HomePhabricator

[CMAKE] Generate junit output for boost unit tests

Description

[CMAKE] Generate junit output for boost unit tests

Summary:
This diff makes the individual boost tests output a junit report XML,
and merge them in order to produce a test suite junit report. The final
report is located under a test/junit dir under the build directory.

This requires a few behavior changes and implies some limitations:

  • A new option ENABLE_JUNIT_REPORT (disabled by default) let the boost tests generate a Junit report, and proceed incremental merge of the results to a test suite report.
  • A test failure will stop the build process, and the result won't appear in the Junit report by default. One need to run ninja with the -k0 flag to change this behavior and allow the test suite to run to completion.
  • The test suite duration reported is not the actual duration, because it can't account for the tests running parallel. The reported duration is equivalent to running the tests serially.
  • The incremental nature of the report makes it impossible to clear the content by running the tests. Each test will override its previous run but not impact the other tests, independently of what is being run. To clear the report, ninja clean should be used.

Test Plan:

cmake -GNinja ..
ninja check

Check no report is generated.

cmake -GNinja .. -DENABLE_JUNIT_REPORT=ON
ninja check

Check the report are generated.

ninja clean

Check the test/junit directory no longer exist.

Edit a test from the bitcoin test suite to make it fail:

ninja -k0 check

Ensure all the tests including the failure are correctly reported.
Check the return code is non zero.

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Subscribers: deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7002

Details

Provenance
FabienAuthored on Jul 28 2020, 10:33
abc-botPushed on Jul 28 2020, 10:39
Reviewer
Restricted Project
Differential Revision
D7002: [CMAKE] Generate junit output for boost unit tests
Parents
rABCb1728a65f1c9: [backport#16952] gui: make sure to update the UI when deleting a transaction
Branches
Unknown
Tags
Unknown