[buildbot] Introduce an ignore list for known flaky errors
ClosedPublic
Actions

Authored by jasonbcox on Oct 6 2020, 17:19.

Details

Reviewers

deadalnix

Group Reviewers

Restricted Project

Commits

rABC9e32b347175d: [buildbot] Introduce an ignore list for known flaky errors

Summary

The buildbot currently provides periodic reminders when flakiness
arises. This is a hint for devs to investigate the problem and hopefully fix
the flakiness in a reasonable amount of time. However, we cannot always control
how well third-party software performs. In these cases, we may need to wait for
long release cycles or for bug fixes to become available on our current distrobution.

Instead of bringing noise to the dev channel, we can silence known flakiness.
This feature should be used with caution, as ignoring flakiness in our own software
can lead to issues metastasizing.

This patch introduces the first such case, where we ignore a deadlock that occurs
in Wine during cross builds, but only at a frequency of a few times per week.

Test Plan

pytest

Diff Detail

Repository

rABC Bitcoin ABC

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

jasonbcox created this revision.Oct 6 2020, 17:19

Herald added a reviewer: Restricted Project. · View Herald TranscriptOct 6 2020, 17:19

jasonbcox requested review of this revision.Oct 6 2020, 17:19

Harbormaster completed remote builds in B13217: Diff 24312.Oct 6 2020, 17:34

Build Bitcoin ABC Diffs / Diff Testing (build-diff) failed.

Snippet of first build failure:

[0m[0;32mrpc_getblockstats.py                             | ✓ Passed  | 1 s
[0m[0;32mrpc_getchaintips.py                              | ✓ Passed  | 4 s
[0m[0;32mrpc_help.py                                      | ✓ Passed  | 1 s
[0m[0;32mrpc_invalidateblock.py                           | ✓ Passed  | 6 s
[0m[0;32mrpc_misc.py                                      | ✓ Passed  | 1 s
[0m[0;32mrpc_named_arguments.py                           | ✓ Passed  | 1 s
[0m[0;32mrpc_net.py                                       | ✓ Passed  | 1 s
[0m[0;32mrpc_preciousblock.py                             | ✓ Passed  | 1 s
[0m[0;32mrpc_psbt.py                                      | ✓ Passed  | 15 s
[0m[0;32mrpc_scantxoutset.py                              | ✓ Passed  | 5 s
[0m[0;32mrpc_setban.py                                    | ✓ Passed  | 2 s
[0m[0;32mrpc_signmessage.py                               | ✓ Passed  | 1 s
[0m[0;32mrpc_signrawtransaction.py                        | ✓ Passed  | 1 s
[0m[0;32mrpc_txoutproof.py                                | ✓ Passed  | 2 s
[0m[0;32mrpc_uptime.py                                    | ✓ Passed  | 1 s
[0m[0;32mrpc_users.py                                     | ✓ Passed  | 2 s
[0m[0;32mrpc_whitelist.py                                 | ✓ Passed  | 1 s
[0m[0;32mtool_wallet.py                                   | ✓ Passed  | 5 s
[0m[0;32mwallet_abandonconflict.py                        | ✓ Passed  | 24 s
[0m[0;32mwallet_address_types.py                          | ✓ Passed  | 14 s
[0m[0;32mwallet_avoidreuse.py                             | ✓ Passed  | 4 s
[0m[0;32mwallet_backup.py                                 | ✓ Passed  | 43 s
[0m[0;32mwallet_balance.py                                | ✓ Passed  | 20 s
[0m[0;32mwallet_basic.py                                  | ✓ Passed  | 22 s
[0m[0;32mwallet_coinbase_category.py                      | ✓ Passed  | 1 s
[0m[0;32mwallet_create_tx.py                              | ✓ Passed  | 6 s
[0m[0;32mwallet_createwallet.py                           | ✓ Passed  | 2 s
[0m[0;32mwallet_createwallet.py --usecli                  | ✓ Passed  | 2 s
[0m[0;32mwallet_disable.py                                | ✓ Passed  | 0 s
[0m[0;32mwallet_dump.py                                   | ✓ Passed  | 3 s
[0m[0;32mwallet_encryption.py                             | ✓ Passed  | 5 s
[0m[0;32mwallet_groups.py                                 | ✓ Passed  | 9 s
[0m[0;32mwallet_hd.py                                     | ✓ Passed  | 5 s
[0m[0;32mwallet_import_rescan.py                          | ✓ Passed  | 6 s
[0m[0;32mwallet_import_with_label.py                      | ✓ Passed  | 1 s
[0m[0;32mwallet_importmulti.py                            | ✓ Passed  | 4 s
[0m[0;32mwallet_importprunedfunds.py                      | ✓ Passed  | 2 s
[0m[0;32mwallet_keypool.py                                | ✓ Passed  | 3 s
[0m[0;32mwallet_keypool_topup.py                          | ✓ Passed  | 3 s
[0m[0;32mwallet_labels.py                                 | ✓ Passed  | 2 s
[0m[0;32mwallet_listreceivedby.py                         | ✓ Passed  | 16 s
[0m[0;32mwallet_listsinceblock.py                         | ✓ Passed  | 4 s
[0m[0;32mwallet_listtransactions.py                       | ✓ Passed  | 17 s
[0m[0;32mwallet_multiwallet.py                            | ✓ Passed  | 15 s
[0m[0;32mwallet_multiwallet.py --usecli                   | ✓ Passed  | 16 s
[0m[0;32mwallet_reorgsrestore.py                          | ✓ Passed  | 3 s
[0m[0;32mwallet_resendwallettransactions.py               | ✓ Passed  | 19 s
[0m[0;32mwallet_txn_clone.py                              | ✓ Passed  | 3 s
[0m[0;32mwallet_txn_clone.py --mineblock                  | ✓ Passed  | 3 s
[0m[0;32mwallet_txn_doublespend.py                        | ✓ Passed  | 1 s
[0m[0;32mwallet_txn_doublespend.py --mineblock            | ✓ Passed  | 4 s
[0m[0;32mwallet_watchonly.py                              | ✓ Passed  | 1 s
[0m[0;32mwallet_watchonly.py --usecli                     | ✓ Passed  | 1 s
[0m[0;32mwallet_zapwallettxes.py                          | ✓ Passed  | 4 s
[0m[1m
ALL                                              | ✓ Passed  | 804 s (accumulated) 
[0mRuntime: 162 s

ninja: build stopped: cannot make progress due to previous errors.
Build build-diff failed with exit code 1

deadalnix accepted this revision.Oct 7 2020, 18:34

This revision is now accepted and ready to land.Oct 7 2020, 18:34

Closed by commit rABC9e32b347175d: [buildbot] Introduce an ignore list for known flaky errors (authored by jasonbcox). · Explain WhyOct 7 2020, 20:05

This revision was automatically updated to reflect the committed changes.

jasonbcox added a commit: rABC9e32b347175d: [buildbot] Introduce an ignore list for known flaky errors.

jasonbcox mentioned this in D7824: [buildbot] Fix getIgnoreList dangling arg.Oct 8 2020, 17:05

jasonbcox mentioned this in rABC1fca0f0b7b2e: [buildbot] Fix getIgnoreList dangling arg.Oct 8 2020, 17:59