Page MenuHomePhabricator
Differential D17832

[avalanche] Poll for the transactions that didn't make it into the last block
ClosedPublic
Actions

Authored by Fabien on Mar 19 2025, 20:50.

Details

Reviewers
roqqit
Group Reviewers
Restricted Project
Commits
rABCa2de9f32d7c5: [avalanche] Poll for the transactions that didn't make it into the last block
Summary

After a block is finalized we need to poll for the remaining mempool transactions.
In order to select which transaction to poll we use the legacy block template constructor.

Depends on D17929.

Test Plan
ninja all check-all

Diff Detail

Repository
rABC Bitcoin ABC
Branch
avalanche_add_back_to_poll_after_block
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 32781
Build 65050: Build Diffbuild-clang-tidy · build-debug · lint-circular-dependencies · build-clang · build-diff · build-without-wallet
Build 65049: arc lint + arc unit

Event Timeline

Fabien created this revision.Mar 19 2025, 20:50
Herald added a reviewer: Restricted Project. · View Herald TranscriptMar 19 2025, 20:50
Fabien requested review of this revision.Mar 19 2025, 20:50
Harbormaster completed remote builds in B32775: Diff 53188.Mar 19 2025, 21:03
teamcity added a comment.Mar 19 2025, 21:03
Build Bitcoin ABC Diffs / Diff Testing (build-debug) failed.

Failed tests logs:

====== Bitcoin ABC functional tests: abc_p2p_avalanche_contender_voting.py ======

------- Stdout: -------
2025-03-19T20:56:48.197000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20250319_205644/abc_p2p_avalanche_contender_voting_5
2025-03-19T20:57:02.246000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20320 due to [Errno 104] Connection reset by peer
2025-03-19T20:57:02.247000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20320 due to [Errno 104] Connection reset by peer
2025-03-19T20:57:02.247000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20320 due to [Errno 104] Connection reset by peer
2025-03-19T20:57:02.248000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 149, in main
    self._run_test_internal()
  File "/work/test/functional/test_framework/test_framework.py", line 139, in _run_test_internal
    self.run_test()
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 86, in run_test
    self.wait_until(lambda: has_finalized_proof(peer.proof.proofid))
  File "/work/test/functional/test_framework/test_framework.py", line 874, in wait_until
    return wait_until_helper(
           ^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/util.py", line 283, in wait_until_helper
    if predicate():
       ^^^^^^^^^^^
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 86, in <lambda>
    self.wait_until(lambda: has_finalized_proof(peer.proof.proofid))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 83, in has_finalized_proof
    return node.getrawavalancheproof(uint256_hex(proofid))["finalized"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 115, in _request
    return self._get_response()
           ^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 187, in _get_response
    http_response = self.__conn.getresponse()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.11/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer
2025-03-19T20:57:02.300000Z TestFramework (INFO): Stopping nodes
[node 0] Cleaning up leftover process
------- Stderr: -------
Traceback (most recent call last):
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 471, in <module>
    AvalancheContenderVotingTest().main()
  File "/work/test/functional/test_framework/test_framework.py", line 172, in main
    exit_code = self.shutdown()
                ^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/test_framework.py", line 398, in shutdown
    self.stop_nodes()
  File "/work/test/functional/test_framework/test_framework.py", line 653, in stop_nodes
    node.stop_node(wait=wait, wait_until_stopped=False)
  File "/work/test/functional/test_framework/test_node.py", line 532, in stop_node
    self.stop(wait=wait)
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 114, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.11/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib/python3.11/http/client.py", line 975, in send
    self.connect()
  File "/usr/lib/python3.11/http/client.py", line 941, in connect
    self.sock = self._create_connection(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 851, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.11/socket.py", line 836, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
====== Bitcoin ABC functional tests: abc_p2p_avalanche_transaction_finalization.py ======

------- Stdout: -------
2025-03-19T20:56:48.179000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20250319_205644/abc_p2p_avalanche_transaction_finalization_3
2025-03-19T20:57:07.203000Z TestFramework (INFO): Check the finalization of simple non-chained txs
2025-03-19T20:57:28.343000Z TestFramework (INFO): Check the finalization of chained txs
2025-03-19T20:57:46.675000Z TestFramework (INFO): Check the finalization of diamond shaped tx chains
2025-03-19T20:58:07.630000Z TestFramework (INFO): Check the finalization of tx when the next block is full
2025-03-19T20:58:52.742000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20192 due to [Errno 104] Connection reset by peer
2025-03-19T20:58:52.743000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20192 due to [Errno 104] Connection reset by peer
2025-03-19T20:58:52.743000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20192 due to [Errno 104] Connection reset by peer
2025-03-19T20:58:52.744000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 149, in main
    self._run_test_internal()
  File "/work/test/functional/test_framework/test_framework.py", line 139, in _run_test_internal
    self.run_test()
  File "/work/test/functional/abc_p2p_avalanche_transaction_finalization.py", line 351, in run_test
    self.test_block_full()
  File "/work/test/functional/abc_p2p_avalanche_transaction_finalization.py", line 309, in test_block_full
    self.finalize_tip()
  File "/work/test/functional/abc_p2p_avalanche_transaction_finalization.py", line 59, in finalize_tip
    self.wait_until(vote_until_final)
  File "/work/test/functional/test_framework/test_framework.py", line 874, in wait_until
    return wait_until_helper(
           ^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/util.py", line 283, in wait_until_helper
    if predicate():
       ^^^^^^^^^^^
  File "/work/test/functional/abc_p2p_avalanche_transaction_finalization.py", line 57, in vote_until_final
    return self.nodes[0].isfinalblock(tip)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 115, in _request
    return self._get_response()
           ^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 187, in _get_response
    http_response = self.__conn.getresponse()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.11/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer
2025-03-19T20:58:52.797000Z TestFramework (INFO): Stopping nodes
[node 0] Cleaning up leftover process
------- Stderr: -------
Traceback (most recent call last):
  File "/work/test/functional/abc_p2p_avalanche_transaction_finalization.py", line 355, in <module>
    AvalancheTransactionFinalizationTest().main()
  File "/work/test/functional/test_framework/test_framework.py", line 172, in main
    exit_code = self.shutdown()
                ^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/test_framework.py", line 398, in shutdown
    self.stop_nodes()
  File "/work/test/functional/test_framework/test_framework.py", line 653, in stop_nodes
    node.stop_node(wait=wait, wait_until_stopped=False)
  File "/work/test/functional/test_framework/test_node.py", line 532, in stop_node
    self.stop(wait=wait)
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 114, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.11/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib/python3.11/http/client.py", line 975, in send
    self.connect()
  File "/usr/lib/python3.11/http/client.py", line 941, in connect
    self.sock = self._create_connection(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 851, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.11/socket.py", line 836, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

Each failure log is accessible here:
Bitcoin ABC functional tests: abc_p2p_avalanche_contender_voting.py
Bitcoin ABC functional tests: abc_p2p_avalanche_transaction_finalization.py

roqqit requested changes to this revision.Mar 19 2025, 22:56
roqqit added a subscriber: roqqit.

back to you

This revision now requires changes to proceed.Mar 19 2025, 22:56
Fabien updated this revision to Diff 53193.Mar 19 2025, 23:22

Fix mempool lock

Harbormaster completed remote builds in B32779: Diff 53193.Mar 19 2025, 23:31
teamcity added a comment.Mar 19 2025, 23:31
Build Bitcoin ABC Diffs / Diff Testing (build-debug) failed.

Failed tests logs:

====== Bitcoin ABC functional tests: abc_p2p_avalanche_contender_voting.py ======

------- Stdout: -------
2025-03-19T23:25:09.005000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-debug/test/tmp/test_runner_₿₵_🏃_20250319_232506/abc_p2p_avalanche_contender_voting_5
2025-03-19T23:25:21.112000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20320 due to [Errno 104] Connection reset by peer
2025-03-19T23:25:21.112000Z TestFramework.p2p (WARNING): Connection lost to 127.0.0.1:20320 due to [Errno 104] Connection reset by peer
2025-03-19T23:25:21.113000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 149, in main
    self._run_test_internal()
  File "/work/test/functional/test_framework/test_framework.py", line 139, in _run_test_internal
    self.run_test()
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 86, in run_test
    self.wait_until(lambda: has_finalized_proof(peer.proof.proofid))
  File "/work/test/functional/test_framework/test_framework.py", line 874, in wait_until
    return wait_until_helper(
           ^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/util.py", line 283, in wait_until_helper
    if predicate():
       ^^^^^^^^^^^
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 86, in <lambda>
    self.wait_until(lambda: has_finalized_proof(peer.proof.proofid))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 83, in has_finalized_proof
    return node.getrawavalancheproof(uint256_hex(proofid))["finalized"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 115, in _request
    return self._get_response()
           ^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 187, in _get_response
    http_response = self.__conn.getresponse()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.11/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer
2025-03-19T23:25:21.166000Z TestFramework (INFO): Stopping nodes
[node 0] Cleaning up leftover process
------- Stderr: -------
Traceback (most recent call last):
  File "/work/test/functional/abc_p2p_avalanche_contender_voting.py", line 471, in <module>
    AvalancheContenderVotingTest().main()
  File "/work/test/functional/test_framework/test_framework.py", line 172, in main
    exit_code = self.shutdown()
                ^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/test_framework.py", line 398, in shutdown
    self.stop_nodes()
  File "/work/test/functional/test_framework/test_framework.py", line 653, in stop_nodes
    node.stop_node(wait=wait, wait_until_stopped=False)
  File "/work/test/functional/test_framework/test_node.py", line 532, in stop_node
    self.stop(wait=wait)
  File "/work/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 146, in __call__
    response, status = self._request(
                       ^^^^^^^^^^^^^^
  File "/work/test/functional/test_framework/authproxy.py", line 114, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.11/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib/python3.11/http/client.py", line 975, in send
    self.connect()
  File "/usr/lib/python3.11/http/client.py", line 941, in connect
    self.sock = self._create_connection(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 851, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.11/socket.py", line 836, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

Each failure log is accessible here:
Bitcoin ABC functional tests: abc_p2p_avalanche_contender_voting.py

Fabien updated this revision to Diff 53196.Mar 20 2025, 08:21

Fix another lock issue (cs_main => cs_peermanager)

Harbormaster completed remote builds in B32781: Diff 53196.Mar 20 2025, 08:34
Fabien planned changes to this revision.Mar 21 2025, 08:31
Fabien updated this revision to Diff 53430.Apr 9 2025, 14:27

Rebase

Harbormaster completed remote builds in B32931: Diff 53430.Apr 9 2025, 14:41
Fabien planned changes to this revision.Apr 10 2025, 19:54
Fabien updated this revision to Diff 53454.Apr 11 2025, 12:43

Add children first to optimize the polls

Harbormaster completed remote builds in B32945: Diff 53454.Apr 11 2025, 12:57
roqqit requested changes to this revision.Apr 11 2025, 15:41
roqqit added inline comments.
test/functional/abc_p2p_avalanche_transaction_finalization.py
256 ↗(On Diff #53454)

here too

316–318 ↗(On Diff #53454)

We should verify the number of txs polled for since this number is less than the number of txs finalized

This revision now requires changes to proceed.Apr 11 2025, 15:41
Fabien added inline comments.Apr 14 2025, 10:31
test/functional/abc_p2p_avalanche_transaction_finalization.py
256 ↗(On Diff #53454)

It doesn't apply here. All the transactions could fit the block so they will all be polled, but only 4 will make it into the radix tree

316–318 ↗(On Diff #53454)

We can even make sure we don't poll for unexpected txs

Fabien edited the summary of this revision. (Show Details)Apr 14 2025, 10:52
Fabien added a parent revision: D17929: [avalanche] Test: Simplify checking for undesired hashes in polls.
Fabien updated this revision to Diff 53468.Apr 14 2025, 10:52

Check against undesired txs being polled

Fabien removed a parent revision: D17831: [avalanche] Limit the finalized txs radix tree size so it always fits in the next block.Apr 14 2025, 10:52
Harbormaster completed remote builds in B32957: Diff 53468.Apr 14 2025, 11:03
roqqit accepted this revision.Apr 22 2025, 16:26
This revision is now accepted and ready to land.Apr 22 2025, 16:26
Closed by commit rABCa2de9f32d7c5: [avalanche] Poll for the transactions that didn't make it into the last block (authored by Fabien). · Explain WhyApr 22 2025, 21:01
This revision was automatically updated to reflect the committed changes.
Fabien added a commit: rABCa2de9f32d7c5: [avalanche] Poll for the transactions that didn't make it into the last block.
Fabien mentioned this in D17971: [avalanche] Fix a lock order inversion.Apr 22 2025, 21:56
Fabien mentioned this in rABC0971f430079f: [avalanche] Fix a lock order inversion.Apr 23 2025, 07:38

Revision Contents
Changeset List

PathSize
src/
59 lines
node/
15 lines
test/
functional/
40 lines

Diff 53196

View Options

src/net_processing.cpp

Loading...
View Options

src/node/miner.h

Loading...
View Options

test/functional/abc_p2p_avalanche_transaction_finalization.py

Loading...