HomePhabricator

Introduce well-defined CAddress disk serialization

Description

Introduce well-defined CAddress disk serialization

Summary:
Before this commit, CAddress disk serialization was messy. It stored
CLIENT_VERSION in the first 4 bytes, optionally OR'ed with ADDRV2_FORMAT.

  • All bits except ADDRV2_FORMAT were ignored, making it hard to use for actual future format changes.
  • ADDRV2_FORMAT determines whether or not nServices is serialized in LE64 format or in CompactSize format.
  • Whether or not the embedded CService is serialized in V1 or V2 format is determined by the stream's version having ADDRV2_FORMAT (as opposed to the nServices encoding, which is determined by the disk version).

To improve the situation, this commit introduces the following disk
serialization format, compatible with earlier versions, but better defined for
future changes:

  • The first 4 bytes store a format version number. Its low 19 bits are ignored (as it historically stored the CLIENT_VERSION), but its high 13 bits specify the serialization exactly:
    • 0x00000000: LE64 encoding for nServices, V1 encoding for CService
    • 0x20000000: CompactSize encoding for nServices, V2 encoding for CService
    • Any other value triggers an unsupported format error on deserialization, and can be used for future format changes.
  • The ADDRV2_FORMAT flag in the stream's version does not impact the actual serialization format; it only determines whether V2 encoding is permitted; whether it's actually enabled depends solely on the disk version number.

Operationally the changes to the deserializer are:

  • Failure when the stored format version number is unexpected.
  • The embedded CService's format is determined by the stored format version number rather than the stream's version number.

These do no introduce incompatibilities, as no code versions exist that write
any value other than 0 or 0x20000000 in the top 13 bits, and no code paths
where the stream's version differs from the stored version.

This is a backport of core#20516

Test Plan:
ninja all check-all bitcoin-fuzzers

run a few hours of IBD, stop and restart the node, check that addresses are loaded successfully from peers.dat

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Subscribers: Fabien

Differential Revision: https://reviews.bitcoinabc.org/D14752

Details

Provenance
Pieter Wuille <pieter@wuille.net>Authored on Nov 27 2020, 00:46
PiRKCommitted on Nov 8 2023, 15:10
PiRKPushed on Nov 8 2023, 15:10
Reviewer
Restricted Project
Differential Revision
D14752: Introduce well-defined CAddress disk serialization
Parents
rABCe4a1f43c41ea: fuzz: Check for addrv1 compatibility before using serializers
Branches
Unknown
Tags
Unknown