Paths

Table of Contentst

Differential D4588

[tests] Add libFuzzer support.
ClosedPublic
Actions

Authored by Fabien on Nov 29 2019, 17:28.

Details

Reviewers

deadalnix

Group Reviewers

Restricted Project

Commits

rSTAGINGe83e7cb51c7e: [tests] Add libFuzzer support.
rABCe83e7cb51c7e: [tests] Add libFuzzer support.

Summary

This is a backport of core PR10440:
https://github.com/bitcoin/bitcoin/pull/10440/files

I added some modifications to the CMake files to support enabling the
libFuzzer support easily (it requires manual compilation otherwise).
This is natively supported by ECM, but will try to apply to every build
with a main() which is not the desired behavior as it should only
apply to the test_bitcoin_fuzzy target.

Depends on D5182 (for the test plan).

Test Plan

Build with Clang >= 6.0:

cmake -GNinja .. -DENABLE_SANITIZERS=fuzzer
ninja test_bitcoin_fuzzy
./src/test/test_bitcoin_fuzzy

Stop it with CTRL+C when you're bored, as it will run indefinitely.

Diff Detail

Repository

rABC Bitcoin ABC

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

Fabien created this revision.Nov 29 2019, 17:28

Herald added a reviewer: Restricted Project. · View Herald TranscriptNov 29 2019, 17:28

Herald added a reviewer: deadalnix. · View Herald Transcript

Build Bitcoin-ABC / Diffs / Diff Testing started.

Build Bitcoin-ABC / Diffs / Diff Testing passed.

Harbormaster completed remote builds in B8340: Diff 14511.Nov 29 2019, 17:38

Add missing static.

Build Bitcoin-ABC / Diffs / Diff Testing started.

Build Bitcoin-ABC / Diffs / Diff Testing passed.

Harbormaster completed remote builds in B8342: Diff 14516.Nov 29 2019, 20:51

deadalnix requested changes to this revision.Dec 1 2019, 22:35

deadalnix added inline comments.

src/test/CMakeLists.txt
193 ↗	(On Diff #14516)	Does building that target without the sanitizer makes any sense at all? Also, are you sure that it doesn't have all to be build with the sanitizer?

This revision now requires changes to proceed.Dec 1 2019, 22:35

Fabien requested review of this revision.Dec 2 2019, 07:35

Fabien added inline comments.

src/test/CMakeLists.txt
193 ↗	(On Diff #14516)	The reason for not setting the option by default is that there are 2 possible ways to use the executable for fuzzing: You can build against an AFL compiler, such as `afl-clang-fast` then use `afl-fuzz` to manage the instrumented binary (this is the method described in the `fuzzing.md` documentation) Or you can use clang's integrated `libFuzzer` which is what is done by passing the `-fsanitize=fuzzer` option. I don't think you can do both at the same time, as I don't expect the afl-* enabled compilers to support `libFuzzer`. Regarding your second question, there is no need for building anything else with the fuzzer sanitizer (see https://github.com/bitcoin/bitcoin/pull/16338). But you can (should ?) build with other sanitizers such as ASAN for example, then run with an abort on error option (don't remember the exact syntax). That would allow you to catch the vectors that trigger ASAN faults.

Fabien added a child revision: D4611: Bugfix: Include <memory> for std::unique_ptr.Dec 2 2019, 12:59

Fabien added a child revision: D4612: tests: Use MakeUnique to construct objects owned by unique_ptrs.Dec 2 2019, 13:36

Fabien removed a child revision: D4611: Bugfix: Include <memory> for std::unique_ptr.Dec 3 2019, 08:09

Fabien mentioned this in D4611: Bugfix: Include <memory> for std::unique_ptr.Dec 3 2019, 08:10

Fabien planned changes to this revision.Dec 9 2019, 15:06

Fabien mentioned this in D5182: [CMAKE] Get rid of ECM for running the sanitizers.Feb 6 2020, 17:07

Rebase on top of D5182.

The -fsanitize=fuzzer flag is now passed to all the targets when
built with cmake. Note that this causes the build to fail on any
target other than test_bitcoin_fuzzy (this will be fixed in another
diff, as it is another backport).

Build Bitcoin-ABC / Diffs / Diff Testing started.

Fabien edited the summary of this revision. (Show Details)Feb 6 2020, 17:14

Fabien added a parent revision: D5182: [CMAKE] Get rid of ECM for running the sanitizers.

Fabien edited the summary of this revision. (Show Details)

Build Bitcoin-ABC / Diffs / Diff Testing failed.

Snippet of first build failure:

[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_users.py                            | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_zmq.py                              | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_abandonconflict.py               | ✓ Passed  | 5 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_basic.py                         | ✓ Passed  | 25 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py                  | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py --usecli         | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_disable.py                       | ✓ Passed  | 0 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_dump.py                          | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_encryption.py                    | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_groups.py                        | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_hd.py                            | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_import_rescan.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importmulti.py                   | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importprunedfunds.py             | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool.py                       | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool_topup.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_labels.py                        | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listreceivedby.py                | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listsinceblock.py                | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listtransactions.py              | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py                   | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py --usecli          | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_resendwallettransactions.py      | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py                     | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py --mineblock         | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py               | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py --mineblock   | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_zapwallettxes.py                 | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;31mp2p_feefilter.py                        | ✖ Failed  | 81 s
[17:16:32] :	 [Step 1/1]  [0m [0;31m [1m
[17:16:32] :	 [Step 1/1] ALL                                     | ✖ Failed  | 540 s (accumulated) 
[17:16:32] :	 [Step 1/1]  [0m [0mRuntime: 122 s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] [219/357] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/contrib/devtools/chainparams && /usr/bin/python3 ./test_make_chainparams.py
[17:16:32] :	 [Step 1/1] .....
[17:16:32] :	 [Step 1/1] ----------------------------------------------------------------------
[17:16:32] :	 [Step 1/1] Ran 5 tests in 0.001s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] OK
[17:16:32] :	 [Step 1/1] [222/357] Running bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] [343/357] Running bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] [351/357] Running bitcoin test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin test suite
[17:16:32] :	 [Step 1/1] [354/357] Running secp256k1 test suite
[17:16:32] :	 [Step 1/1] PASSED: secp256k1 test suite
[17:16:32] :	 [Step 1/1] [356/357] Running leveldb test suite
[17:16:32] :	 [Step 1/1] PASSED: leveldb test suite
[17:16:32] :	 [Step 1/1] FAILED: test/CMakeFiles/check-functional 
[17:16:32] :	 [Step 1/1] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/build/test && /usr/bin/python3 ./functional/test_runner.py
[17:16:32] :	 [Step 1/1] ninja: build stopped: subcommand failed.
[17:16:32] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/*.log.* ***
[17:16:32]W:	 [Step 1/1] ++ print_sanitizers_log
[17:16:32]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[17:16:32]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/*.log.* ***'
[17:16:32]W:	 [Step 1/1] ++ cat '/tmp/sanitizer_logs/*.log.*'
[17:16:32]W:	 [Step 1/1] cat: '/tmp/sanitizer_logs/*.log.*': No such file or directory
[17:16:32]W:	 [Step 1/1] Process exited with code 1
[17:16:32]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Each failure log is accessible here:
Bitcoin ABC functional tests: p2p_feefilter.py

Harbormaster failed remote builds in B9306: Diff 16081!Feb 6 2020, 17:16

Build Bitcoin-ABC / Diffs / Diff Testing started.

Build Bitcoin-ABC / Diffs / Diff Testing passed.

Harbormaster completed remote builds in B9306: Diff 16081.Feb 6 2020, 18:44

Fabien mentioned this in rABCda4599790500: [CMAKE] Get rid of ECM for running the sanitizers.Feb 7 2020, 09:55

Fabien added a child revision: D4613: build: Allow to configure --with-sanitizers=fuzzer.Feb 10 2020, 11:39

deadalnix accepted this revision.Feb 10 2020, 13:00

This revision is now accepted and ready to land.Feb 10 2020, 13:00

Closed by commit rABCe83e7cb51c7e: [tests] Add libFuzzer support. (authored by practicalswift <practicalswift@users.noreply.github.com>, committed by Fabien). · Explain WhyFeb 10 2020, 13:17

This revision was automatically updated to reflect the committed changes.

Fabien added a commit: rABCe83e7cb51c7e: [tests] Add libFuzzer support..