Page MenuHomePhabricator

make check in parallel - turbo mode
AbandonedPublic

Authored by markblundeberg on Jul 3 2019, 19:03.

Details

Reviewers
deadalnix
Fabien
Group Reviewers
Restricted Project
Summary

In D3535, the C++ test suite parts are now run in parallel with each other,
however they all must complete before the (rather long) secp256k1 tests
get started. Thus, little speed benefit was observed.

This queues up the long-running secp256k1 tests in parallel with the start
of the C++ test suite. Some miscellaneous fast tests are left for the end.
This gives a decent speed benefit (~33% less time).

make -j4 check before:

real	2m57.705s
user	4m39.254s
sys	1m23.918s

make -j4 check after:

real	1m55.976s
user	4m38.210s
sys	1m23.296s

(old tests pre-D3535: real 3m0.421s)

Test Plan

make check and observe output

Diff Detail

Repository
rABC Bitcoin ABC
Branch
turbotests
Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 6656
Build 11359: Bitcoin ABC Buildbot (legacy)
Build 11358: arc lint + arc unit

Event Timeline

markblundeberg created this revision.Jul 3 2019, 19:03
Herald added a reviewer: Restricted Project. · View Herald TranscriptJul 3 2019, 19:03

This is mostly an experiment to see what kind of optimization is possible with just a little work. Would be better if we could use TESTS= but my automake-fu is not good enough for that.

deadalnix requested changes to this revision.Jul 3 2019, 23:39

Core has soem code doing this. cmake needs to be kept in sync with automake.

This revision now requires changes to proceed.Jul 3 2019, 23:39
markblundeberg added a comment.EditedJul 5 2019, 02:49

Core has soem code doing this. cmake needs to be kept in sync with automake.

Our master is now perfectly in sync with what Core does for test parallelization, and this Diff makes us faster than Core's approach. (though overall Core's test suite is much faster than us, apparently because they have fewer tests and also their transaction_tests and coinselector_tests don't take so long). @Fabien perhaps after the cmake is matching we can pick this up again, what do you think?

markblundeberg abandoned this revision.Jul 14 2019, 00:49