Page MenuHomePhabricator

Improve performance of p2p inv to send queues
Changes PlannedPublic

Authored by PiRK on Oct 10 2024, 08:07.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Summary

txmempool: have CompareTopologically sort missing txs first

We use CompareTopologically to choose an order of txs to inv. Rather
than sorting txs that have been evicted from the mempool at the end
of the list, sort them at the beginning so they are removed from
the queue immediately.

net_processing: Boost inv trickle rate

If transactions are being added to the mempool at a rate faster than
INVENTORY_BROADCAST_MAX_PER_MB * BLOCK_SIZE_MB per second then peers'
inventory_to_send queue can become relatively large. If this happens,
increase the number of txids we include in an INV message by 5 for
each 1000 txids in the queue.

This will tend to clear a temporary excess out reasonably quickly; an
excess of 4000 invs to send will be cleared down to 1000 in about 30
minutes, while an excess of 20000 invs would be cleared down to 1000 in
about 60 minutes.

This is a backport of core#27610

Test Plan

ninja all check-all

Event Timeline

PiRK requested review of this revision.Oct 10 2024, 08:07

no need to recompute INVENTORY_BROADCAST_MAX_PER_MB * config.GetMaxBlockSize() / 1000000 for each iteration of the loop

Fabien requested changes to this revision.Oct 10 2024, 12:14
Fabien added a subscriber: Fabien.
Fabien added inline comments.
src/net_processing.cpp
8608 ↗(On Diff #50021)

No reason to compute this in the loop either

This revision now requires changes to proceed.Oct 10 2024, 12:14
PiRK requested review of this revision.Oct 10 2024, 12:33
PiRK added inline comments.
src/net_processing.cpp
8608 ↗(On Diff #50021)

There is a tx_relay->m_tx_inventory_to_send.erase(it); in the loop's body.

Fabien requested changes to this revision.Oct 10 2024, 20:59
Fabien added inline comments.
src/net_processing.cpp
8608 ↗(On Diff #50021)

Exactly, and you need to run it a number of time dependent of the initial size of the inventory set

This revision now requires changes to proceed.Oct 10 2024, 20:59

fix the while condition, add a ceiling to the max value (1000)

PiRK planned changes to this revision.Oct 11 2024, 08:42

this need some work to raise other limits first. Clipping broadcast_max at 1000 makes the situation actually worse than our current limit (INVENTORY_BROADCAST_MAX_PER_MB * 32 = 1120)