test_runner.py
ClosedPublic
Actions

Authored by Fabien on Dec 4 2019, 11:51.

Details

Reviewers

deadalnix

Group Reviewers

Restricted Project

Commits

rSTAGING6aa446205750: qa: Add test/fuzz/test_runner.py
rABC6aa446205750: qa: Add test/fuzz/test_runner.py

Summary

Can be run with ./test/fuzz/test_runner.py after building as described
in doc/fuzzing.md

Backport of core PR15295:
https://github.com/bitcoin/bitcoin/pull/15295/files

Depends on D4615.

Test Plan

export CC=clang CXX=clang++
../configure --enable-fuzz --with-sanitizers=fuzzer,address \
  --disable-wallet \
  --disable-bench \
  --with-utils=no \
  --with-daemon=no \
  --with-libs=no \
  --with-gui=no \
  --with-seeder=no
make
mkdir -p test/fuzz
cp ../test/fuzz/test_runner.py test/fuzz/
./test/fuzz/test_runner.py -l DEBUG <path_to_corpus>

cmake -GNinja .. -DENABLE_SANITIZERS="address;fuzzer" \
  -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
ninja bitcoin-fuzzers link-fuzz-test_runner.py
./test/fuzz/test_runner.py -l DEBUG <path_to_corpus>

Check the corpus is run exactly once for all the tests.

Diff Detail

Repository

rABC Bitcoin ABC

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

Fabien created this revision.Dec 4 2019, 11:51

Herald added a reviewer: Restricted Project. · View Herald TranscriptDec 4 2019, 11:51

Herald added a reviewer: deadalnix. · View Herald Transcript

Build Bitcoin-ABC / Diffs / Diff Testing started.

Build Bitcoin-ABC / Diffs / Diff Testing failed.

Harbormaster failed remote builds in B8396: Diff 14606!Dec 4 2019, 12:06

Fabien added a child revision: D4628: fuzz: Move deserialize tests to test/fuzz/deserialize.cpp.Dec 4 2019, 12:25

deadalnix requested changes to this revision.Dec 6 2019, 11:54

deadalnix added inline comments.

doc/fuzzing.md
20 ↗	(On Diff #14606)	Clearly, this isn't going to cut it.
test/CMakeLists.txt
21 ↗	(On Diff #14606)	What does this do?

This revision now requires changes to proceed.Dec 6 2019, 11:54

Fabien added inline comments.Dec 10 2019, 22:21

test/CMakeLists.txt
21 ↗	(On Diff #14606)	The var is used to fill the config.ini used by the `test_runner.py` scripts to determine which feature has been built.

Rebase, update link.

Build Bitcoin-ABC / Diffs / Diff Testing (build-diff) started.

Waiting for the link to become active.

Build Bitcoin-ABC / Diffs / Diff Testing (build-diff) passed.

Harbormaster completed remote builds in B9822: Diff 16943.Mar 16 2020, 16:06

Link is active.

deadalnix requested changes to this revision.Mar 19 2020, 00:38

deadalnix added inline comments.

src/Makefile.test.include
31 ↗	(On Diff #14606)	Why isn't test_bitcoin built when the fuzzer is on? Why isn't that behavior matched with cmake?
test/CMakeLists.txt
53 ↗	(On Diff #14606)	Just always link. Also create a directory.

This revision now requires changes to proceed.Mar 19 2020, 00:38

Fabien edited the test plan for this revision. (Show Details)Mar 19 2020, 08:32

Fabien added inline comments.Mar 19 2020, 08:45

src/Makefile.test.include
31 ↗	(On Diff #14606)	No other executable will link with `-fsanitize=fuzzer` because it defines a main and expect some functions to be defined. CMake has a different target to build all the fuzzer executables so you don't need to disable everything to build them. I updated the test plan so you get a better idea of the difference.