Page MenuHomePhabricator

[tests] Add libFuzzer support.
ClosedPublic

Authored by Fabien on Nov 29 2019, 17:28.

Details

Reviewers
deadalnix
Group Reviewers
Restricted Project
Commits
rABCe83e7cb51c7e: [tests] Add libFuzzer support.
Summary

This is a backport of core PR10440:
https://github.com/bitcoin/bitcoin/pull/10440/files

I added some modifications to the CMake files to support enabling the
libFuzzer support easily (it requires manual compilation otherwise).
This is natively supported by ECM, but will try to apply to every build
with a main() which is not the desired behavior as it should only
apply to the test_bitcoin_fuzzy target.

Depends on D5182 (for the test plan).

Test Plan

Build with Clang >= 6.0:

cmake -GNinja .. -DENABLE_SANITIZERS=fuzzer
ninja test_bitcoin_fuzzy
./src/test/test_bitcoin_fuzzy

Stop it with CTRL+C when you're bored, as it will run indefinitely.

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Fabien created this revision.Nov 29 2019, 17:28
Herald added a reviewer: Restricted Project. · View Herald TranscriptNov 29 2019, 17:28
Fabien updated this revision to Diff 14516.Nov 29 2019, 20:38

Add missing static.

deadalnix requested changes to this revision.Dec 1 2019, 22:35
deadalnix added inline comments.
src/test/CMakeLists.txt
193 ↗(On Diff #14516)

Does building that target without the sanitizer makes any sense at all? Also, are you sure that it doesn't have all to be build with the sanitizer?

This revision now requires changes to proceed.Dec 1 2019, 22:35
Fabien requested review of this revision.Dec 2 2019, 07:35
Fabien added inline comments.
src/test/CMakeLists.txt
193 ↗(On Diff #14516)

The reason for not setting the option by default is that there are 2 possible ways to use the executable for fuzzing:

    • You can build against an AFL compiler, such as afl-clang-fast then use afl-fuzz to manage the instrumented binary (this is the method described in the fuzzing.md documentation)
  • Or you can use clang's integrated libFuzzer which is what is done by passing the -fsanitize=fuzzer option. I don't think you can do both at the same time, as I don't expect the afl-* enabled compilers to support libFuzzer.

Regarding your second question, there is no need for building anything else with the fuzzer sanitizer (see https://github.com/bitcoin/bitcoin/pull/16338).
But you can (should ?) build with other sanitizers such as ASAN for example, then run with an abort on error option (don't remember the exact syntax). That would allow you to catch the vectors that trigger ASAN faults.

Fabien planned changes to this revision.Dec 9 2019, 15:06
Fabien updated this revision to Diff 16081.Feb 6 2020, 17:13

Rebase on top of D5182.

The -fsanitize=fuzzer flag is now passed to all the targets when
built with cmake. Note that this causes the build to fail on any
target other than test_bitcoin_fuzzy (this will be fixed in another
diff, as it is another backport).

Fabien edited the summary of this revision. (Show Details)Feb 6 2020, 17:14
Fabien edited the summary of this revision. (Show Details)
teamcity added a comment.Feb 6 2020, 17:16

Snippet of first build failure:

[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_users.py                            | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_zmq.py                              | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_abandonconflict.py               | ✓ Passed  | 5 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_basic.py                         | ✓ Passed  | 25 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py                  | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py --usecli         | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_disable.py                       | ✓ Passed  | 0 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_dump.py                          | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_encryption.py                    | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_groups.py                        | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_hd.py                            | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_import_rescan.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importmulti.py                   | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importprunedfunds.py             | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool.py                       | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool_topup.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_labels.py                        | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listreceivedby.py                | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listsinceblock.py                | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listtransactions.py              | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py                   | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py --usecli          | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_resendwallettransactions.py      | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py                     | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py --mineblock         | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py               | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py --mineblock   | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_zapwallettxes.py                 | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;31mp2p_feefilter.py                        | ✖ Failed  | 81 s
[17:16:32] :	 [Step 1/1]  [0m [0;31m [1m
[17:16:32] :	 [Step 1/1] ALL                                     | ✖ Failed  | 540 s (accumulated) 
[17:16:32] :	 [Step 1/1]  [0m [0mRuntime: 122 s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] [219/357] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/contrib/devtools/chainparams && /usr/bin/python3 ./test_make_chainparams.py
[17:16:32] :	 [Step 1/1] .....
[17:16:32] :	 [Step 1/1] ----------------------------------------------------------------------
[17:16:32] :	 [Step 1/1] Ran 5 tests in 0.001s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] OK
[17:16:32] :	 [Step 1/1] [222/357] Running bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] [343/357] Running bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] [351/357] Running bitcoin test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin test suite
[17:16:32] :	 [Step 1/1] [354/357] Running secp256k1 test suite
[17:16:32] :	 [Step 1/1] PASSED: secp256k1 test suite
[17:16:32] :	 [Step 1/1] [356/357] Running leveldb test suite
[17:16:32] :	 [Step 1/1] PASSED: leveldb test suite
[17:16:32] :	 [Step 1/1] FAILED: test/CMakeFiles/check-functional 
[17:16:32] :	 [Step 1/1] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/build/test && /usr/bin/python3 ./functional/test_runner.py
[17:16:32] :	 [Step 1/1] ninja: build stopped: subcommand failed.
[17:16:32] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/*.log.* ***
[17:16:32]W:	 [Step 1/1] ++ print_sanitizers_log
[17:16:32]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[17:16:32]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/*.log.* ***'
[17:16:32]W:	 [Step 1/1] ++ cat '/tmp/sanitizer_logs/*.log.*'
[17:16:32]W:	 [Step 1/1] cat: '/tmp/sanitizer_logs/*.log.*': No such file or directory
[17:16:32]W:	 [Step 1/1] Process exited with code 1
[17:16:32]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Each failure log is accessible here:
Bitcoin ABC functional tests: p2p_feefilter.py

deadalnix accepted this revision.Feb 10 2020, 13:00
This revision is now accepted and ready to land.Feb 10 2020, 13:00
This revision was automatically updated to reflect the committed changes.