clear the validation interface before destructing the WalletTestingSetup
AbandonedPublic
Actions

Authored by PiRK on Sep 24 2021, 13:23.

Details

Reviewers

Fabien
deadalnix

Group Reviewers

Restricted Project

Summary

We see occasional segfaults in the wallet_tests. This seems to happen on destructing the wallet test fixture, and involves the validation interface thread. This is possibly due to the validation interface not being cleared when the fixture is destroyed at the end of the test.

(gdb) thread apply all bt

Thread 2 (Thread 0x7fc25e3bfc00 (LWP 10107)):
...
#7  TestingSetup::~TestingSetup (this=0x7ffdc4d73150, __in_chrg=<optimized out>) at ../src/test/util/setup_common.cpp:226
#8  0x000055a69f4aa8c6 in wallet_tests::WatchOnlyPubKeys::~WatchOnlyPubKeys (this=0x7ffdc4d73150, __in_chrg=<optimized out>)
    at ../src/wallet/test/wallet_tests.cpp:499
...

Thread 1 (Thread 0x7fc257199700 (LWP 10489)):
#0  0x00007fc25eb9b4b3 in std::_Rb_tree_increment(std::_Rb_tree_node_base*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x000055a69f874fa2 in std::_Rb_tree_const_iterator<std::pair<uint256 const, std::unique_ptr<ScriptPubKeyMan, std::default_delete<ScriptPubKeyMan> > > >::operator++ (this=<synthetic pointer>) at /usr/include/c++/8/bits/stl_tree.h:364
#2  CWallet::IsMine (this=this@entry=0x7ffdc4d73358, script=...) at ../src/wallet/wallet.cpp:1355
#3  0x000055a69f88738b in CWallet::IsMine (txout=..., this=0x7ffdc4d73358) at ../src/wallet/wallet.cpp:1411
#4  CWallet::IsMine (tx=..., this=0x7ffdc4d73358) at ../src/wallet/wallet.cpp:1411
#5  CWallet::AddToWalletIfInvolvingMe (this=0x7ffdc4d73358, ptx=..., confirm=..., fUpdate=<optimized out>) at ../src/wallet/wallet.cpp:1041
#6  0x000055a69f88748d in CWallet::SyncTransaction (this=this@entry=0x7ffdc4d73358, 
    ptx=std::shared_ptr<const class CTransaction> (use count 1, weak count 0) = {...}, confirm=..., update_tx=update_tx@entry=true)
    at ../src/wallet/wallet.cpp:1200
#7  0x000055a69f8878d4 in CWallet::blockConnected (this=0x7ffdc4d73358, block=..., height=0) at /usr/include/c++/8/bits/stl_vector.h:948
#8  0x000055a69f75379e in CMainSignals::<lambda()>::<lambda(CValidationInterface&)>::operator() (__closure=<optimized out>, __closure=<optimized out>, 
    callbacks=...) at /usr/include/x86_64-linux-gnu/c++/8/bits/gthr-default.h:778
#9  MainSignalsInstance::Iterate<CMainSignals::BlockConnected(const std::shared_ptr<const CBlock>&, const CBlockIndex*)::<lambda()>::<lambda(CValidationInterface&)> > (f=..., this=0x7fc25d1766c0) at ../src/validationinterface.cpp:91
#10 CMainSignals::<lambda()>::operator() (__closure=0x7fc25d1b95a0) at ../src/validationinterface.cpp:230
#11 CMainSignals::<lambda()>::operator() (__closure=0x7fc25d1b9580) at ../src/validationinterface.cpp:234
#12 std::_Function_handler<void(), CMainSignals::BlockConnected(const std::shared_ptr<const CBlock>&, const CBlockIndex*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/8/bits/std_function.h:297
#13 0x000055a69f949b60 in std::function<void ()>::operator()() const (this=0x7fc257195880) at /usr/include/c++/8/bits/std_function.h:682
#14 SingleThreadedSchedulerClient::ProcessQueue (this=0x7fc25d176738) at ../src/scheduler.cpp:184
#15 0x000055a69f948fbe in std::function<void ()>::operator()() const (this=0x7fc257195930) at /usr/include/c++/8/bits/std_function.h:682
#16 CScheduler::serviceQueue (this=0x7fc25d0c4840) at ../src/scheduler.cpp:61
#17 0x00007fc25ecbc615 in ?? () from /lib/x86_64-linux-gnu/libboost_thread.so.1.67.0
#18 0x00007fc25e93bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#19 0x00007fc25e86a4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

This is an attempt at fixing this.

Test Plan

With TSAN:
ninja all check-all

Diff Detail

Repository

rABC Bitcoin ABC

Branch

destructor_walletTestingSetup

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 16779
Build 33408: Build Diff	build-clang · lint-circular-dependencies · build-without-wallet · build-diff · build-clang-tidy · build-debug
Build 33407: arc lint + arc unit

Event Timeline

PiRK created this revision.Sep 24 2021, 13:23

Herald added a reviewer: Restricted Project. · View Herald TranscriptSep 24 2021, 13:23

PiRK requested review of this revision.Sep 24 2021, 13:23

Fabien requested changes to this revision.Sep 24 2021, 13:26

Fabien added a subscriber: Fabien.

Fabien added inline comments.

src/wallet/test/wallet_test_fixture.cpp
25 ↗	(On Diff #30157)	newline

This revision now requires changes to proceed.Sep 24 2021, 13:26

newline

Harbormaster completed remote builds in B16778: Diff 30157.Sep 24 2021, 13:36

Harbormaster completed remote builds in B16779: Diff 30158.Sep 24 2021, 13:45

I'd like an explanation of what's going on here. If this is presumably something, then there is a way to test if it is that thing.

This revision now requires changes to proceed.Sep 24 2021, 19:57

I have tried to find a more reliable way to reproduce the crash by generating a lot of transactions at the end of a test just before it is destructed, hoping it would cause the validation interface thread to keep processing pointers to freed memory, but so far I did not have any significant success. For now, the only way i know of to cause this is to run the test in a loop for multiple hours. I need to spend more time understanding this stuff.

PiRK mentioned this in D10449: net, refactor: Move AddLocal call one level up.Nov 10 2021, 08:56

Abandoned in favor of D10883

abandon for real