[electrum] be selective about ignoring errors
d0466eaf73c4
Actions

Description

[electrum] be selective about ignoring errors

Summary:
Only ignore a whitelist of acceptable errors instead of ignoring all errors during test discovery. Ignoring all errors could lead to tests just being skipped and the test suite succeeding, which could go unnoticed for a long time.

This will help us detect some unexpected errors, and provide us with a useful error stack trace to debug it.

For instance, intentionnaly raising an error in the main context of a module that is imported by an __init__.py produces this output:

$ python electrum/test_runner.py
Testing `setup.py --version`: OK

........................................................................................................................................................................................................................s.........s.........s..................................E......
======================================================================
ERROR: electrumabc_gui.qt (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: electrumabc_gui.qt
Traceback (most recent call last):
  File "/usr/lib/python3.10/unittest/loader.py", line 470, in _find_test_path
    package = self._get_module_from_name(name)
  File "/usr/lib/python3.10/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/pierre/dev/bitcoin-abc/electrum/electrumabc_gui/qt/__init__.py", line 86, in <module>
    from .exception_window import ExceptionHook
  File "/home/pierre/dev/bitcoin-abc/electrum/electrumabc_gui/qt/exception_window.py", line 46, in <module>
    from .main_window import ElectrumWindow
  File "/home/pierre/dev/bitcoin-abc/electrum/electrumabc_gui/qt/main_window.py", line 88, in <module>
    from . import address_dialog, external_plugins_window, qrwindow
  File "/home/pierre/dev/bitcoin-abc/electrum/electrumabc_gui/qt/address_dialog.py", line 33, in <module>
    from .history_list import HistoryList
  File "/home/pierre/dev/bitcoin-abc/electrum/electrumabc_gui/qt/history_list.py", line 46, in <module>
    raise ImportError("spam spam spam spam spam")
ImportError: spam spam spam spam spam

----------------------------------------------------------------------
Ran 278 tests in 9.094s

FAILED (errors=1, skipped=3)

Use addTest instead of addTests, because the former can take a TestCase (which _FailedTest is a subclass of) or a TestSuite, whereas the latter expected an iterable object (i.e a TestSuite or a list of TestCase)

Test Plan: python test_runner.py

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Differential Revision: https://reviews.bitcoinabc.org/D14262