Page MenuHomePhabricator

Fully revert D4803 due to TSAN test failures
AbandonedPublic

Authored by jasonbcox on Jan 15 2020, 19:13.

Details

Reviewers
deadalnix
fpelliccioni
Group Reviewers
Restricted Project
Summary

It seems that the call to function LimitValidationInterfaceQueue() in UnwindBlock() (D4803) broke the tests.
I Fully revert D4803 until the exact cause of the error is determined.

Test Plan
ABC_BUILD_NAME=build-tsan ./build-configurations.sh

Diff Detail

Repository
rABC Bitcoin ABC
Branch
feature-revert-D4803
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 8963
Build 15899: Default Diff Build & Tests
Build 15898: arc lint + arc unit

Event Timeline

deadalnix requested changes to this revision.Jan 15 2020, 22:23

I don't think this is wise to revert half of this patch. What is the problem to begin with?

This revision now requires changes to proceed.Jan 15 2020, 22:23
fpelliccioni retitled this revision from Partially revert D4803 due to TSAN test failures to Fully revert D4803 due to TSAN test failures.Jan 16 2020, 01:05
fpelliccioni edited the summary of this revision. (Show Details)

Snippet of first build failure:

[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py                  | ✓ Passed  | 2 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_createwallet.py --usecli         | ✓ Passed  | 1 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_disable.py                       | ✓ Passed  | 0 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_dump.py                          | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_encryption.py                    | ✓ Passed  | 4 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_groups.py                        | ✓ Passed  | 7 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_hd.py                            | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_import_rescan.py                 | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_importmulti.py                   | ✓ Passed  | 2 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_importprunedfunds.py             | ✓ Passed  | 1 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_keypool.py                       | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_keypool_topup.py                 | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_labels.py                        | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_listreceivedby.py                | ✓ Passed  | 11 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_listsinceblock.py                | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_listtransactions.py              | ✓ Passed  | 11 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py                   | ✓ Passed  | 7 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py --usecli          | ✓ Passed  | 9 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_resendwallettransactions.py      | ✓ Passed  | 1 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py                     | ✓ Passed  | 2 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py --mineblock         | ✓ Passed  | 2 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py               | ✓ Passed  | 2 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py --mineblock   | ✓ Passed  | 3 s
[01:07:38] :	 [Step 1/1]  [0m [0;34mwallet_zapwallettxes.py                 | ✓ Passed  | 2 s
[01:07:38] :	 [Step 1/1]  [0m [1m
[01:07:38] :	 [Step 1/1] ALL                                     | ✓ Passed  | 456 s (accumulated) 
[01:07:38] :	 [Step 1/1]  [0mRuntime: 103 s
[01:07:38] :	 [Step 1/1] 
[01:07:38] :	 [Step 1/1] [211/447] Running bitcoin-seeder-upgrade-activated test suite
[01:07:38] :	 [Step 1/1] PASSED: bitcoin-seeder-upgrade-activated test suite
[01:07:38] :	 [Step 1/1] [212/447] Running bitcoin-seeder test suite
[01:07:38] :	 [Step 1/1] PASSED: bitcoin-seeder test suite
[01:07:38] :	 [Step 1/1] [215/445] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/contrib/devtools/chainparams && /usr/bin/python3 ./test_make_chainparams.py
[01:07:38] :	 [Step 1/1] .....
[01:07:38] :	 [Step 1/1] ----------------------------------------------------------------------
[01:07:38] :	 [Step 1/1] Ran 5 tests in 0.001s
[01:07:38] :	 [Step 1/1] 
[01:07:38] :	 [Step 1/1] OK
[01:07:38] :	 [Step 1/1] [242/445] Running bitcoin-qt test suite
[01:07:38] :	 [Step 1/1] PASSED: bitcoin-qt test suite
[01:07:38] :	 [Step 1/1] [262/445] bitcoin: testing util_tests
[01:07:38] :	 [Step 1/1] FAILED: src/test/CMakeFiles/check-bitcoin-util_tests 
[01:07:38] :	 [Step 1/1] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/build/src/test && /home/teamcity/buildAgent/work/c4a5708f2bae7929/build/src/test/run-util_tests.sh
[01:07:38] :	 [Step 1/1] Running 37 test cases...
[01:07:38] :	 [Step 1/1] Running 37 test cases...
[01:07:38] :	 [Step 1/1] 
[01:07:38] :	 [Step 1/1] *** No errors detected
[01:07:38] :	 [Step 1/1] error: in "util_tests/test_LockDirectory": check LockDirectory(dirname, lockname, true) == true has failed [false != true]
[01:07:38] :	 [Step 1/1] 
[01:07:38] :	 [Step 1/1] *** 1 failure is detected in the test module "Bitcoin Test Suite"
[01:07:38] :	 [Step 1/1] [274/445] leveldb: testing db_test
[01:07:38] :	 [Step 1/1] ninja: build stopped: subcommand failed.
[01:07:38] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/*.log.* ***
[01:07:38]W:	 [Step 1/1] ++ print_sanitizers_log
[01:07:38]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[01:07:38]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/*.log.* ***'
[01:07:38]W:	 [Step 1/1] ++ cat '/tmp/sanitizer_logs/*.log.*'
[01:07:38]W:	 [Step 1/1] cat: '/tmp/sanitizer_logs/*.log.*': No such file or directory
[01:07:38]W:	 [Step 1/1] Process exited with code 1
[01:07:38]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Snippet of first build failure:

[02:01:07] :	 [Step 1/1]  [0m [0;34mwallet_multiwallet.py --usecli          | ✓ Passed  | 30 s
[02:01:07] :	 [Step 1/1]  [0m [0;34mwallet_resendwallettransactions.py      | ✓ Passed  | 3 s
[02:01:07] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py                     | ✓ Passed  | 4 s
[02:01:07] :	 [Step 1/1]  [0m [0;34mwallet_txn_clone.py --mineblock         | ✓ Passed  | 5 s
[02:01:07] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py               | ✓ Passed  | 4 s
[02:01:07] :	 [Step 1/1]  [0m [0;34mwallet_txn_doublespend.py --mineblock   | ✓ Passed  | 5 s
[02:01:07] :	 [Step 1/1]  [0m [0;34mwallet_zapwallettxes.py                 | ✓ Passed  | 10 s
[02:01:07] :	 [Step 1/1]  [0m [0;31mfeature_assumevalid.py                  | ✖ Failed  | 156 s
[02:01:07] :	 [Step 1/1]  [0m [0;31m [1m
[02:01:07] :	 [Step 1/1] ALL                                     | ✖ Failed  | 1853 s (accumulated) 
[02:01:07] :	 [Step 1/1]  [0m [0mRuntime: 384 s
[02:01:07] :	 [Step 1/1] 
[02:01:07] :	 [Step 1/1] [8/244] Test Bitcoin RPC authentication...
[02:01:07] :	 [Step 1/1] ...
[02:01:07] :	 [Step 1/1] ----------------------------------------------------------------------
[02:01:07] :	 [Step 1/1] Ran 3 tests in 0.002s
[02:01:07] :	 [Step 1/1] 
[02:01:07] :	 [Step 1/1] OK
[02:01:07] :	 [Step 1/1] [9/244] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/contrib/devtools/chainparams && /usr/bin/python3 ./test_make_chainparams.py
[02:01:07] :	 [Step 1/1] .....
[02:01:07] :	 [Step 1/1] ----------------------------------------------------------------------
[02:01:07] :	 [Step 1/1] Ran 5 tests in 0.001s
[02:01:07] :	 [Step 1/1] 
[02:01:07] :	 [Step 1/1] OK
[02:01:07] :	 [Step 1/1] [108/244] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/uint256_tests.cpp.o
[02:01:07] :	 [Step 1/1] ../src/test/uint256_tests.cpp:287:7: warning: explicitly assigning value of variable of type 'arith_uint256' to itself [-Wself-assign-overloaded]
[02:01:07] :	 [Step 1/1]     v /= v;
[02:01:07] :	 [Step 1/1]     ~ ^  ~
[02:01:07] :	 [Step 1/1] ../src/test/uint256_tests.cpp:291:7: warning: explicitly assigning value of variable of type 'arith_uint256' to itself [-Wself-assign-overloaded]
[02:01:07] :	 [Step 1/1]     v -= v;
[02:01:07] :	 [Step 1/1]     ~ ^  ~
[02:01:07] :	 [Step 1/1] 2 warnings generated.
[02:01:07] :	 [Step 1/1] [118/244] Running bitcoin-seeder test suite
[02:01:07] :	 [Step 1/1] PASSED: bitcoin-seeder test suite
[02:01:07] :	 [Step 1/1] [238/244] Running bitcoin-qt test suite
[02:01:07] :	 [Step 1/1] PASSED: bitcoin-qt test suite
[02:01:07] :	 [Step 1/1] [243/244] Running bitcoin test suite
[02:01:07] :	 [Step 1/1] PASSED: bitcoin test suite
[02:01:07] :	 [Step 1/1] FAILED: test/CMakeFiles/check-functional 
[02:01:07] :	 [Step 1/1] cd /home/teamcity/buildAgent/work/c4a5708f2bae7929/build/test && /usr/bin/python3 ./functional/test_runner.py
[02:01:07] :	 [Step 1/1] ninja: build stopped: subcommand failed.
[02:01:07] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/ubsan.log.24996 ***
[02:01:07]W:	 [Step 1/1] ++ print_sanitizers_log
[02:01:07]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[02:01:07]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/ubsan.log.24996 ***'
[02:01:07]W:	 [Step 1/1] ++ cat /tmp/sanitizer_logs/ubsan.log.24996
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07] :	 [Step 1/1] WARNING: too long mutex cycle found
[02:01:07]W:	 [Step 1/1] Process exited with code 1
[02:01:07]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Hmm weird that build failures happen with this patch, it smells like one of the prior diffs (which all got landed sort of at once) may actually be responsible. Safest option might be to revert all.

jasonbcox added a subscriber: jasonbcox.

Something appears to be flaky that is unrelated. Hopefully we'll see more info on CI soon, because that last failed build was missing some logs (log issue is now fixed).

jasonbcox edited reviewers, added: fpelliccioni; removed: jasonbcox.

Fabien took care of the revert in this diff: D4959