Page MenuHomePhabricator

[Backport]bug-fix macos: give free bytes to F_PREALLOCATE

Authored by PiRK on Tue, Sep 22, 18:49.


Group Reviewers
Restricted Owners Package(Owns No Changed Paths)
Restricted Project
rABC862f3806058a: [Backport]bug-fix macos: give free bytes to F_PREALLOCATE

The Initial Block Download (IBD) on macOS Catalina uses a lot more disk space than expected.

The macos manpage for fcntl (for F_PEOFPOSMODE) states:

Allocate from the physical end of file. In this case, fst_length indicates the number of newly allocated bytes desired.

This would result in the rev files being essentially pre-allocating 2x their necessary size (this is the case for block files as well, but these are flushed down to their right sizes every time) as they would pre-allocate pos + length free bytes, rather than allocating length bytes after pos, as expected.

Note: more explanations on the issue from Apple in this bug report

Backport of Bitcoin Core PR17887

Test Plan

ninja check to verify that the patch does not accidentaly break anything else.
But I do not have a MacOS to run the tests on, and try to replicate the issue.

Diff Detail

rABC Bitcoin ABC
Automatic diff as part of commit; lint not applicable.
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Owners added a reviewer: Restricted Owners Package.Tue, Sep 22, 18:49
PiRK requested review of this revision.Tue, Sep 22, 18:49

[Bot Message]
One or more PR numbers were detected in the summary.
Links to those PRs have been inserted into the summary for reference.

PiRK retitled this revision from bug-fix macos: give free bytes to F_PREALLOCATE to [Backport]bug-fix macos: give free bytes to F_PREALLOCATE.Tue, Sep 22, 18:50

Snippet of first build failure:

[350/409] Running utility command for check-bitcoin-streams_tests
[351/409] Running utility command for check-bitcoin-timedata_tests
[352/409] bitcoin: testing uint256_tests
[353/409] Running utility command for check-bitcoin-uint256_tests
[354/409] bitcoin: testing serialize_tests
[355/409] Running utility command for check-bitcoin-serialize_tests
[356/409] bitcoin: testing undo_tests
[357/409] bitcoin: testing util_threadnames_tests
[358/409] bitcoin: testing radix_tests
[359/409] bitcoin: testing blockcheck_tests
[360/409] Running utility command for check-bitcoin-undo_tests
[361/409] bitcoin: testing crypto_tests
[362/409] Running utility command for check-bitcoin-util_threadnames_tests
[363/409] Running utility command for check-bitcoin-radix_tests
[364/409] Running utility command for check-bitcoin-blockcheck_tests
[365/409] Running utility command for check-bitcoin-crypto_tests
[366/409] bitcoin: testing walletdb_tests
[367/409] bitcoin: testing validationinterface_tests
[368/409] Running utility command for check-bitcoin-walletdb_tests
[369/409] bitcoin: testing script_standard_tests
[370/409] Running utility command for check-bitcoin-validationinterface_tests
[371/409] Running utility command for check-bitcoin-script_standard_tests
[372/409] bitcoin: testing validation_tests
[373/409] bitcoin: testing script_tests
[374/409] Running utility command for check-bitcoin-validation_tests
[375/409] Running utility command for check-bitcoin-script_tests
[376/409] bitcoin: testing blockstatus_tests
[377/409] bitcoin: testing ismine_tests
[378/409] Running utility command for check-bitcoin-blockstatus_tests
[379/409] Running utility command for check-bitcoin-ismine_tests
[380/409] bitcoin: testing versionbits_tests
[381/409] Running utility command for check-bitcoin-versionbits_tests
[382/409] bitcoin: testing cashaddr_tests
[383/409] bitcoin: testing getarg_tests
[384/409] bitcoin: testing validation_block_tests
[385/409] Running utility command for check-bitcoin-cashaddr_tests
[386/409] Running utility command for check-bitcoin-getarg_tests
[387/409] bitcoin: testing transaction_tests
[388/409] Running utility command for check-bitcoin-validation_block_tests
[389/409] bitcoin: testing skiplist_tests
[390/409] Running utility command for check-bitcoin-transaction_tests
[391/409] Running utility command for check-bitcoin-skiplist_tests
[392/409] bitcoin: testing bswap_tests
[393/409] Running utility command for check-bitcoin-bswap_tests
[394/409] bitcoin: testing util_tests
[395/409] Running utility command for check-bitcoin-util_tests
[396/409] bitcoin: testing op_reversebytes_tests
[397/409] Running utility command for check-bitcoin-op_reversebytes_tests
[398/409] bitcoin: testing cuckoocache_tests
[399/409] Running utility command for check-bitcoin-cuckoocache_tests
[400/409] bitcoin: testing coins_tests
[401/409] Running utility command for check-bitcoin-coins_tests
[402/409] bitcoin: testing coinselector_tests
[403/409] Running utility command for check-bitcoin-coinselector_tests
[404/409] Running bitcoin test suite
PASSED: bitcoin test suite
[405/409] secp256k1: testing secp256k1-tests
[406/409] Running secp256k1 test suite
PASSED: secp256k1 test suite
Build build-clang-10 timed out after 1200.0s
deadalnix requested changes to this revision.Tue, Sep 22, 19:21
deadalnix added a subscriber: deadalnix.
deadalnix added inline comments.
1128 ↗(On Diff #23690)

Move the comment on the previous line.

This revision now requires changes to proceed.Tue, Sep 22, 19:21

Move a comment to the previous line

This revision is now accepted and ready to land.Tue, Sep 22, 19:59