Page MenuHomePhabricator

[tests] Add libFuzzer support.
ClosedPublic

Authored by Fabien on Nov 29 2019, 17:28.

Details

Summary

This is a backport of core PR10440:
https://github.com/bitcoin/bitcoin/pull/10440/files

I added some modifications to the CMake files to support enabling the
libFuzzer support easily (it requires manual compilation otherwise).
This is natively supported by ECM, but will try to apply to every build
with a main() which is not the desired behavior as it should only
apply to the test_bitcoin_fuzzy target.

Depends on D5182 (for the test plan).

Test Plan

Build with Clang >= 6.0:

cmake -GNinja .. -DENABLE_SANITIZERS=fuzzer
ninja test_bitcoin_fuzzy
./src/test/test_bitcoin_fuzzy

Stop it with CTRL+C when you're bored, as it will run indefinitely.

Diff Detail

Repository
rABC Bitcoin ABC
Branch
PR10440
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 9306
Build 16549: Default Diff Build & Tests
Build 16548: arc lint + arc unit

Event Timeline

deadalnix requested changes to this revision.Dec 1 2019, 22:35
deadalnix added inline comments.
src/test/CMakeLists.txt
193 ↗(On Diff #14516)

Does building that target without the sanitizer makes any sense at all? Also, are you sure that it doesn't have all to be build with the sanitizer?

This revision now requires changes to proceed.Dec 1 2019, 22:35
Fabien requested review of this revision.Dec 2 2019, 07:35
Fabien added inline comments.
src/test/CMakeLists.txt
193 ↗(On Diff #14516)

The reason for not setting the option by default is that there are 2 possible ways to use the executable for fuzzing:

    • You can build against an AFL compiler, such as afl-clang-fast then use afl-fuzz to manage the instrumented binary (this is the method described in the fuzzing.md documentation)
  • Or you can use clang's integrated libFuzzer which is what is done by passing the -fsanitize=fuzzer option. I don't think you can do both at the same time, as I don't expect the afl-* enabled compilers to support libFuzzer.

Regarding your second question, there is no need for building anything else with the fuzzer sanitizer (see https://github.com/bitcoin/bitcoin/pull/16338).
But you can (should ?) build with other sanitizers such as ASAN for example, then run with an abort on error option (don't remember the exact syntax). That would allow you to catch the vectors that trigger ASAN faults.

Fabien planned changes to this revision.Dec 9 2019, 15:06

Rebase on top of D5182.

The -fsanitize=fuzzer flag is now passed to all the targets when
built with cmake. Note that this causes the build to fail on any
target other than test_bitcoin_fuzzy (this will be fixed in another
diff, as it is another backport).

Fabien edited the summary of this revision. (Show Details)

Snippet of first build failure:

[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_users.py                            | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_zmq.py                              | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_abandonconflict.py               | ✓ Passed  | 5 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_basic.py                         | ✓ Passed  | 25 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py                  | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py --usecli         | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_disable.py                       | ✓ Passed  | 0 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_dump.py                          | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_encryption.py                    | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_groups.py                        | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_hd.py                            | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_import_rescan.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importmulti.py                   | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importprunedfunds.py             | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool.py                       | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool_topup.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_labels.py                        | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listreceivedby.py                | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listsinceblock.py                | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listtransactions.py              | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py                   | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py --usecli          | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_resendwallettransactions.py      | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py                     | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py --mineblock         | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py               | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py --mineblock   | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_zapwallettxes.py                 | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;31mp2p_feefilter.py                        | ✖ Failed  | 81 s
[17:16:32] :	 [Step 1/1]  [0m [0;31m [1m
[17:16:32] :	 [Step 1/1] ALL                                     | ✖ Failed  | 540 s (accumulated) 
[17:16:32] :	 [Step 1/1]  [0m [0mRuntime: 122 s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] [219/357] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/contrib/devtools/chainparams && /usr/bin/python3 ./test_make_chainparams.py
[17:16:32] :	 [Step 1/1] .....
[17:16:32] :	 [Step 1/1] ----------------------------------------------------------------------
[17:16:32] :	 [Step 1/1] Ran 5 tests in 0.001s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] OK
[17:16:32] :	 [Step 1/1] [222/357] Running bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] [343/357] Running bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] [351/357] Running bitcoin test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin test suite
[17:16:32] :	 [Step 1/1] [354/357] Running secp256k1 test suite
[17:16:32] :	 [Step 1/1] PASSED: secp256k1 test suite
[17:16:32] :	 [Step 1/1] [356/357] Running leveldb test suite
[17:16:32] :	 [Step 1/1] PASSED: leveldb test suite
[17:16:32] :	 [Step 1/1] FAILED: test/CMakeFiles/check-functional 
[17:16:32] :	 [Step 1/1] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/build/test && /usr/bin/python3 ./functional/test_runner.py
[17:16:32] :	 [Step 1/1] ninja: build stopped: subcommand failed.
[17:16:32] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/*.log.* ***
[17:16:32]W:	 [Step 1/1] ++ print_sanitizers_log
[17:16:32]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[17:16:32]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/*.log.* ***'
[17:16:32]W:	 [Step 1/1] ++ cat '/tmp/sanitizer_logs/*.log.*'
[17:16:32]W:	 [Step 1/1] cat: '/tmp/sanitizer_logs/*.log.*': No such file or directory
[17:16:32]W:	 [Step 1/1] Process exited with code 1
[17:16:32]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Each failure log is accessible here:
Bitcoin ABC functional tests: p2p_feefilter.py

This revision is now accepted and ready to land.Feb 10 2020, 13:00