Page MenuHomePhabricator

[backport#14193][test] Add test to check mempool consistency in case of reorgs
AcceptedPublic

Authored by majcosta on Mon, Sep 14, 22:08.

Details

Reviewers
deadalnix
Group Reviewers
Restricted Project
Summary
Test Plan
cmake -DCMAKE_BUILD_TYPE=Debug
ninja check check-functional

Diff Detail

Event Timeline

[Bot Message]
One or more PR numbers were detected in the summary.
Links to those PRs have been inserted into the summary for reference.

This revision is now accepted and ready to land.Tue, Sep 15, 22:12

Snippet of first build failure:

../../src/test/mempool_tests.cpp(325): Entering test case "MempoolIndexingTest"
../../src/test/mempool_tests.cpp(325): Leaving test case "MempoolIndexingTest"; testing time: 151750us
../../src/test/mempool_tests.cpp(519): Entering test case "MempoolAncestorIndexingTest"
../../src/test/mempool_tests.cpp(519): Leaving test case "MempoolAncestorIndexingTest"; testing time: 156167us
../../src/test/mempool_tests.cpp(661): Entering test case "MempoolSizeLimitTest"
../../src/test/mempool_tests.cpp(661): Leaving test case "MempoolSizeLimitTest"; testing time: 122476us
../../src/test/mempool_tests.cpp(842): Entering test case "TestImportMempool"
==================
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=7052)
  Cycle in lock order graph: M262865 (0x5617d882e010) => M1574750 (0x7ffe72f0d978) => M262865

  Mutex M1574750 acquired here while holding mutex M262865 in main thread:
    #0 pthread_mutex_lock <null> (test_bitcoin+0xea46e)
    #1 __gthread_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/x86_64-linux-gnu/c++/8/bits/gthr-default.h:748:12 (test_bitcoin+0x4664a0)
    #2 __gthread_recursive_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/x86_64-linux-gnu/c++/8/bits/gthr-default.h:810 (test_bitcoin+0x4664a0)
    #3 std::recursive_mutex::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/mutex:107 (test_bitcoin+0x4664a0)
    #4 std::unique_lock<std::recursive_mutex>::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_mutex.h:267 (test_bitcoin+0x4664a0)
    #5 UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::Enter(char const*, char const*, int) /work/abc-ci-builds/build-tsan/../../src/./sync.h:129 (test_bitcoin+0x4664a0)
    #6 UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::UniqueLock(AnnotatedMixin<std::recursive_mutex>&, char const*, char const*, int, bool) /work/abc-ci-builds/build-tsan/../../src/./sync.h:151 (test_bitcoin+0x4664a0)
    #7 mempool_tests::MempoolSizeLimitTest::test_method() /work/abc-ci-builds/build-tsan/../../src/test/mempool_tests.cpp:663 (test_bitcoin+0x4664a0)
    #8 mempool_tests::MempoolSizeLimitTest_invoker() /work/abc-ci-builds/build-tsan/../../src/test/mempool_tests.cpp:661:1 (test_bitcoin+0x465d13)
    #9 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118:11 (test_bitcoin+0x19a889)
    #10 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.67.0+0x55dbd)
    #11 __libc_start_main <null> (libc.so.6+0x2409a)

    Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

  Mutex M262865 acquired here while holding mutex M1574750 in main thread:
    #0 pthread_mutex_lock <null> (test_bitcoin+0xea46e)
    #1 __gthread_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/x86_64-linux-gnu/c++/8/bits/gthr-default.h:748:12 (test_bitcoin+0x47325b)
    #2 __gthread_recursive_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/x86_64-linux-gnu/c++/8/bits/gthr-default.h:810 (test_bitcoin+0x47325b)
    #3 std::recursive_mutex::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/mutex:107 (test_bitcoin+0x47325b)
    #4 std::unique_lock<std::recursive_mutex>::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_mutex.h:267 (test_bitcoin+0x47325b)
    #5 UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::Enter(char const*, char const*, int) /work/abc-ci-builds/build-tsan/../../src/./sync.h:129 (test_bitcoin+0x47325b)
    #6 UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::UniqueLock(AnnotatedMixin<std::recursive_mutex>&, char const*, char const*, int, bool) /work/abc-ci-builds/build-tsan/../../src/./sync.h:151 (test_bitcoin+0x47325b)
    #7 mempool_tests::TestImportMempool::test_method() /work/abc-ci-builds/build-tsan/../../src/test/mempool_tests.cpp:899 (test_bitcoin+0x47325b)
    #8 mempool_tests::TestImportMempool_invoker() /work/abc-ci-builds/build-tsan/../../src/test/mempool_tests.cpp:842:1 (test_bitcoin+0x471a53)
    #9 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118:11 (test_bitcoin+0x19a889)
    #10 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.67.0+0x55dbd)
    #11 __libc_start_main <null> (libc.so.6+0x2409a)

SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/work/abc-ci-builds/build-tsan/src/test/test_bitcoin+0xea46e) in pthread_mutex_lock
==================
../../src/test/mempool_tests.cpp(842): Leaving test case "TestImportMempool"; testing time: 138832us
../../src/test/mempool_tests.cpp(939): Entering test case "MempoolAncestryTests"
../../src/test/mempool_tests.cpp(939): Leaving test case "MempoolAncestryTests"; testing time: 133025us
../../src/test/mempool_tests.cpp(19): Leaving test suite "mempool_tests"; testing time: 1531359us
Leaving test module "Bitcoin ABC unit tests"; testing time: 1531745us

*** No errors detected
ThreadSanitizer: reported 1 warnings
[375/403] Running pow test suite
PASSED: pow test suite
[377/403] Running bitcoin-qt test suite
PASSED: bitcoin-qt test suite
[400/403] Running utility command for check-bitcoin-coinselector_tests
FAILED: test/CMakeFiles/check-functional 
cd /work/abc-ci-builds/build-tsan/test && /usr/bin/cmake -E make_directory /work/abc-ci-builds/build-tsan/test/junit && /usr/bin/cmake -E make_directory /work/abc-ci-builds/build-tsan/test/log && /usr/bin/cmake -E env TSAN_OPTIONS=suppressions=/work/test/sanitizer_suppressions/tsan:log_path=stdout /usr/bin/python3.7 ./functional/test_runner.py "--testsuitename=Bitcoin ABC functional tests" --junitoutput=/work/abc-ci-builds/build-tsan/test/junit/functional_tests.xml
ninja: build stopped: cannot make progress due to previous errors.
Build build-tsan failed with exit code 1

Each failure log is accessible here:
Bitcoin ABC functional tests: abc_wallet_standardness.py
Bitcoin ABC functional tests: interface_rest.py
Bitcoin ABC functional tests: interface_zmq.py
Bitcoin ABC functional tests: mempool_reorg.py
Bitcoin ABC functional tests: p2p_compactblocks.py
Bitcoin ABC functional tests: rpc_psbt.py
Bitcoin ABC functional tests: rpc_rawtransaction.py
Bitcoin ABC functional tests: wallet_abandonconflict.py
Bitcoin ABC functional tests: wallet_address_types.py
Bitcoin ABC functional tests: wallet_avoidreuse.py
Bitcoin ABC functional tests: wallet_backup.py
Bitcoin ABC functional tests: wallet_balance.py
Bitcoin ABC functional tests: wallet_basic.py
Bitcoin ABC functional tests: wallet_groups.py
Bitcoin ABC functional tests: wallet_listreceivedby.py
Bitcoin ABC functional tests: wallet_listsinceblock.py
Bitcoin ABC functional tests: wallet_listtransactions.py
Bitcoin ABC functional tests: wallet_txn_doublespend.py
Bitcoin ABC functional tests: wallet_txn_doublespend.py --mineblock