HomePhabricator

interface_rpc: improve reliability of test_work_queue_exceeded with…

Description

interface_rpc: improve reliability of test_work_queue_exceeded with multiprocessing

Summary:
Threading in python does not work very well, everything runs on a single thread most of the time because of the global interpreter lock]. Using multiple threads only makes sense when doing I/O operations or sleeping, in this case Python can emulate a multi-threading behavior by switching threads when the current one becomes idle.

This causes much variability in the interface_rpc test, after D13182. The test duration varies most of the time between 1 and 30 seconds, but a duration of 282 seconds was recorded at least by one user. The test struggles to spam the work queue faster than the RPC calls are handled by bitcoind.

Fix this by using multiprocessing instead of threading. With this, I consistently get a test duration of 1 or 2 seconds.

The trade-off is that the code is slightly more complicated because processes need to use queue objects to commnunicate, as they don't share their variables.

A minor behavior improvement is that now all errror messages are checked if multiple processes manage to trigger the error at roughly the same time.

Test Plan: seq 10 | while read i;do test/functional/test_runner.py interface_rpc;done

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Differential Revision: https://reviews.bitcoinabc.org/D13216

Details

Provenance
PiRKAuthored on Mar 1 2023, 16:32
PiRKPushed on Mar 1 2023, 18:16
Reviewer
Restricted Project
Differential Revision
D13216: interface_rpc: improve reliability of test_work_queue_exceeded with multiprocessing
Parents
rABC33e9b2623426: [alias-server] Accept overpaid tx fees
Branches
Unknown
Tags
Unknown