Page MenuHomePhabricator
Differential D4588

[tests] Add libFuzzer support.
ClosedPublic
Actions

Authored by Fabien on Nov 29 2019, 17:28.

Details

Reviewers
deadalnix
Group Reviewers
Restricted Project
Commits
rSTAGINGe83e7cb51c7e: [tests] Add libFuzzer support.
rABCe83e7cb51c7e: [tests] Add libFuzzer support.
Summary

This is a backport of core PR10440:
https://github.com/bitcoin/bitcoin/pull/10440/files

I added some modifications to the CMake files to support enabling the
libFuzzer support easily (it requires manual compilation otherwise).
This is natively supported by ECM, but will try to apply to every build
with a main() which is not the desired behavior as it should only
apply to the test_bitcoin_fuzzy target.

Depends on D5182 (for the test plan).

Test Plan

Build with Clang >= 6.0:

cmake -GNinja .. -DENABLE_SANITIZERS=fuzzer
ninja test_bitcoin_fuzzy
./src/test/test_bitcoin_fuzzy

Stop it with CTRL+C when you're bored, as it will run indefinitely.

Diff Detail

Repository
rABC Bitcoin ABC
Branch
PR10440
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 8340
Build 14699: Default Diff Build & Tests
Build 14698: arc lint + arc unit

Event Timeline

Fabien created this revision.Nov 29 2019, 17:28
Herald added a reviewer: Restricted Project. · View Herald TranscriptNov 29 2019, 17:28
Herald added a reviewer: deadalnix. · View Herald Transcript
teamcity added a comment.Nov 29 2019, 17:28

Build Bitcoin-ABC / Diffs / Diff Testing started.

teamcity added a comment.Nov 29 2019, 17:38

Build Bitcoin-ABC / Diffs / Diff Testing passed.

Harbormaster completed remote builds in B8340: Diff 14511.Nov 29 2019, 17:38
Fabien updated this revision to Diff 14516.Nov 29 2019, 20:38

Add missing static.

teamcity added a comment.Nov 29 2019, 20:39

Build Bitcoin-ABC / Diffs / Diff Testing started.

teamcity added a comment.Nov 29 2019, 20:50

Build Bitcoin-ABC / Diffs / Diff Testing passed.

Harbormaster completed remote builds in B8342: Diff 14516.Nov 29 2019, 20:51
deadalnix requested changes to this revision.Dec 1 2019, 22:35
deadalnix added inline comments.
src/test/CMakeLists.txt
193 ↗(On Diff #14516)

Does building that target without the sanitizer makes any sense at all? Also, are you sure that it doesn't have all to be build with the sanitizer?

This revision now requires changes to proceed.Dec 1 2019, 22:35
Fabien requested review of this revision.Dec 2 2019, 07:35
Fabien added inline comments.
src/test/CMakeLists.txt
193 ↗(On Diff #14516)

The reason for not setting the option by default is that there are 2 possible ways to use the executable for fuzzing:

    • You can build against an AFL compiler, such as afl-clang-fast then use afl-fuzz to manage the instrumented binary (this is the method described in the fuzzing.md documentation)
  • Or you can use clang's integrated libFuzzer which is what is done by passing the -fsanitize=fuzzer option. I don't think you can do both at the same time, as I don't expect the afl-* enabled compilers to support libFuzzer.

Regarding your second question, there is no need for building anything else with the fuzzer sanitizer (see https://github.com/bitcoin/bitcoin/pull/16338).
But you can (should ?) build with other sanitizers such as ASAN for example, then run with an abort on error option (don't remember the exact syntax). That would allow you to catch the vectors that trigger ASAN faults.

Fabien added a child revision: D4611: Bugfix: Include <memory> for std::unique_ptr.Dec 2 2019, 12:59
Fabien added a child revision: D4612: tests: Use MakeUnique to construct objects owned by unique_ptrs.Dec 2 2019, 13:36
Fabien removed a child revision: D4611: Bugfix: Include <memory> for std::unique_ptr.Dec 3 2019, 08:09
Fabien mentioned this in D4611: Bugfix: Include <memory> for std::unique_ptr.Dec 3 2019, 08:10
Fabien planned changes to this revision.Dec 9 2019, 15:06
Fabien mentioned this in D5182: [CMAKE] Get rid of ECM for running the sanitizers.Feb 6 2020, 17:07
Fabien updated this revision to Diff 16081.Feb 6 2020, 17:13

Rebase on top of D5182.

The -fsanitize=fuzzer flag is now passed to all the targets when
built with cmake. Note that this causes the build to fail on any
target other than test_bitcoin_fuzzy (this will be fixed in another
diff, as it is another backport).

teamcity added a comment.Feb 6 2020, 17:13

Build Bitcoin-ABC / Diffs / Diff Testing started.

Fabien edited the summary of this revision. (Show Details)Feb 6 2020, 17:14
Fabien added a parent revision: D5182: [CMAKE] Get rid of ECM for running the sanitizers.
Fabien edited the summary of this revision. (Show Details)
teamcity added a comment.Feb 6 2020, 17:16
Build Bitcoin-ABC / Diffs / Diff Testing failed.

Snippet of first build failure:

[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_users.py                            | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mrpc_zmq.py                              | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_abandonconflict.py               | ✓ Passed  | 5 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_basic.py                         | ✓ Passed  | 25 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py                  | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py --usecli         | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_disable.py                       | ✓ Passed  | 0 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_dump.py                          | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_encryption.py                    | ✓ Passed  | 4 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_groups.py                        | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_hd.py                            | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_import_rescan.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importmulti.py                   | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_importprunedfunds.py             | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool.py                       | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_keypool_topup.py                 | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_labels.py                        | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listreceivedby.py                | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listsinceblock.py                | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_listtransactions.py              | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py                   | ✓ Passed  | 8 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py --usecli          | ✓ Passed  | 10 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_resendwallettransactions.py      | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py                     | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py --mineblock         | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py               | ✓ Passed  | 3 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py --mineblock   | ✓ Passed  | 1 s
[17:16:32] :	 [Step 1/1]  [0m [0;34mwallet_zapwallettxes.py                 | ✓ Passed  | 2 s
[17:16:32] :	 [Step 1/1]  [0m [0;31mp2p_feefilter.py                        | ✖ Failed  | 81 s
[17:16:32] :	 [Step 1/1]  [0m [0;31m [1m
[17:16:32] :	 [Step 1/1] ALL                                     | ✖ Failed  | 540 s (accumulated) 
[17:16:32] :	 [Step 1/1]  [0m [0mRuntime: 122 s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] [219/357] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/contrib/devtools/chainparams && /usr/bin/python3 ./test_make_chainparams.py
[17:16:32] :	 [Step 1/1] .....
[17:16:32] :	 [Step 1/1] ----------------------------------------------------------------------
[17:16:32] :	 [Step 1/1] Ran 5 tests in 0.001s
[17:16:32] :	 [Step 1/1] 
[17:16:32] :	 [Step 1/1] OK
[17:16:32] :	 [Step 1/1] [222/357] Running bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-seeder test suite
[17:16:32] :	 [Step 1/1] [343/357] Running bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin-qt test suite
[17:16:32] :	 [Step 1/1] [351/357] Running bitcoin test suite
[17:16:32] :	 [Step 1/1] PASSED: bitcoin test suite
[17:16:32] :	 [Step 1/1] [354/357] Running secp256k1 test suite
[17:16:32] :	 [Step 1/1] PASSED: secp256k1 test suite
[17:16:32] :	 [Step 1/1] [356/357] Running leveldb test suite
[17:16:32] :	 [Step 1/1] PASSED: leveldb test suite
[17:16:32] :	 [Step 1/1] FAILED: test/CMakeFiles/check-functional 
[17:16:32] :	 [Step 1/1] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/build/test && /usr/bin/python3 ./functional/test_runner.py
[17:16:32] :	 [Step 1/1] ninja: build stopped: subcommand failed.
[17:16:32] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/*.log.* ***
[17:16:32]W:	 [Step 1/1] ++ print_sanitizers_log
[17:16:32]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[17:16:32]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/*.log.* ***'
[17:16:32]W:	 [Step 1/1] ++ cat '/tmp/sanitizer_logs/*.log.*'
[17:16:32]W:	 [Step 1/1] cat: '/tmp/sanitizer_logs/*.log.*': No such file or directory
[17:16:32]W:	 [Step 1/1] Process exited with code 1
[17:16:32]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Each failure log is accessible here:
Bitcoin ABC functional tests: p2p_feefilter.py

Harbormaster failed remote builds in B9306: Diff 16081!Feb 6 2020, 17:16
teamcity added a comment.Feb 6 2020, 18:39

Build Bitcoin-ABC / Diffs / Diff Testing started.

teamcity added a comment.Feb 6 2020, 18:44

Build Bitcoin-ABC / Diffs / Diff Testing passed.

Harbormaster completed remote builds in B9306: Diff 16081.Feb 6 2020, 18:44
Fabien mentioned this in rABCda4599790500: [CMAKE] Get rid of ECM for running the sanitizers.Feb 7 2020, 09:55
Fabien added a child revision: D4613: build: Allow to configure --with-sanitizers=fuzzer.Feb 10 2020, 11:39
deadalnix accepted this revision.Feb 10 2020, 13:00
This revision is now accepted and ready to land.Feb 10 2020, 13:00
Closed by commit rABCe83e7cb51c7e: [tests] Add libFuzzer support. (authored by practicalswift <practicalswift@users.noreply.github.com>, committed by Fabien). · Explain WhyFeb 10 2020, 13:17
This revision was automatically updated to reflect the committed changes.
Fabien added a commit: rABCe83e7cb51c7e: [tests] Add libFuzzer support..
Fabien mentioned this in rSTAGINGda4599790500: [CMAKE] Get rid of ECM for running the sanitizers.Apr 7 2020, 22:51
Fabien added a commit: rSTAGINGe83e7cb51c7e: [tests] Add libFuzzer support..Apr 7 2020, 22:56

Revision Contents
Changeset List

PathSize
src/
4 lines
test/
4 lines
50 lines

Diff 14511

View Options

src/CMakeLists.txt

Loading...
View Options

src/test/CMakeLists.txt

Loading...
View Options

src/test/test_bitcoin_fuzzy.cpp

Loading...