Page MenuHomePhabricator

[Backport]bug-fix macos: give free bytes to F_PREALLOCATE
ClosedPublic

Authored by PiRK on Tue, Sep 22, 18:49.

Details

Reviewers
deadalnix
Group Reviewers
Restricted Owners Package(Owns No Changed Paths)
Restricted Project
Commits
rABC862f3806058a: [Backport]bug-fix macos: give free bytes to F_PREALLOCATE
Summary

The Initial Block Download (IBD) on macOS Catalina uses a lot more disk space than expected.

The macos manpage for fcntl (for F_PEOFPOSMODE) states:

Allocate from the physical end of file. In this case, fst_length indicates the number of newly allocated bytes desired.

This would result in the rev files being essentially pre-allocating 2x their necessary size (this is the case for block files as well, but these are flushed down to their right sizes every time) as they would pre-allocate pos + length free bytes, rather than allocating length bytes after pos, as expected.

Note: more explanations on the issue from Apple in this bug report
https://openradar.appspot.com/radar?id=4930713610616832

Backport of Bitcoin Core PR17887

Test Plan

ninja check to verify that the patch does not accidentaly break anything else.
But I do not have a MacOS to run the tests on, and try to replicate the issue.

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Owners added a reviewer: Restricted Owners Package.Tue, Sep 22, 18:49
PiRK requested review of this revision.Tue, Sep 22, 18:49

[Bot Message]
One or more PR numbers were detected in the summary.
Links to those PRs have been inserted into the summary for reference.

PiRK retitled this revision from bug-fix macos: give free bytes to F_PREALLOCATE to [Backport]bug-fix macos: give free bytes to F_PREALLOCATE.Tue, Sep 22, 18:50

Snippet of first build failure:

[350/409] Running utility command for check-bitcoin-streams_tests
[351/409] Running utility command for check-bitcoin-timedata_tests
[352/409] bitcoin: testing uint256_tests
[353/409] Running utility command for check-bitcoin-uint256_tests
[354/409] bitcoin: testing serialize_tests
[355/409] Running utility command for check-bitcoin-serialize_tests
[356/409] bitcoin: testing undo_tests
[357/409] bitcoin: testing util_threadnames_tests
[358/409] bitcoin: testing radix_tests
[359/409] bitcoin: testing blockcheck_tests
[360/409] Running utility command for check-bitcoin-undo_tests
[361/409] bitcoin: testing crypto_tests
[362/409] Running utility command for check-bitcoin-util_threadnames_tests
[363/409] Running utility command for check-bitcoin-radix_tests
[364/409] Running utility command for check-bitcoin-blockcheck_tests
[365/409] Running utility command for check-bitcoin-crypto_tests
[366/409] bitcoin: testing walletdb_tests
[367/409] bitcoin: testing validationinterface_tests
[368/409] Running utility command for check-bitcoin-walletdb_tests
[369/409] bitcoin: testing script_standard_tests
[370/409] Running utility command for check-bitcoin-validationinterface_tests
[371/409] Running utility command for check-bitcoin-script_standard_tests
[372/409] bitcoin: testing validation_tests
[373/409] bitcoin: testing script_tests
[374/409] Running utility command for check-bitcoin-validation_tests
[375/409] Running utility command for check-bitcoin-script_tests
[376/409] bitcoin: testing blockstatus_tests
[377/409] bitcoin: testing ismine_tests
[378/409] Running utility command for check-bitcoin-blockstatus_tests
[379/409] Running utility command for check-bitcoin-ismine_tests
[380/409] bitcoin: testing versionbits_tests
[381/409] Running utility command for check-bitcoin-versionbits_tests
[382/409] bitcoin: testing cashaddr_tests
[383/409] bitcoin: testing getarg_tests
[384/409] bitcoin: testing validation_block_tests
[385/409] Running utility command for check-bitcoin-cashaddr_tests
[386/409] Running utility command for check-bitcoin-getarg_tests
[387/409] bitcoin: testing transaction_tests
[388/409] Running utility command for check-bitcoin-validation_block_tests
[389/409] bitcoin: testing skiplist_tests
[390/409] Running utility command for check-bitcoin-transaction_tests
[391/409] Running utility command for check-bitcoin-skiplist_tests
[392/409] bitcoin: testing bswap_tests
[393/409] Running utility command for check-bitcoin-bswap_tests
[394/409] bitcoin: testing util_tests
[395/409] Running utility command for check-bitcoin-util_tests
[396/409] bitcoin: testing op_reversebytes_tests
[397/409] Running utility command for check-bitcoin-op_reversebytes_tests
[398/409] bitcoin: testing cuckoocache_tests
[399/409] Running utility command for check-bitcoin-cuckoocache_tests
[400/409] bitcoin: testing coins_tests
[401/409] Running utility command for check-bitcoin-coins_tests
[402/409] bitcoin: testing coinselector_tests
[403/409] Running utility command for check-bitcoin-coinselector_tests
[404/409] Running bitcoin test suite
PASSED: bitcoin test suite
[405/409] secp256k1: testing secp256k1-tests
[406/409] Running secp256k1 test suite
PASSED: secp256k1 test suite
Build build-clang-10 timed out after 1200.0s
deadalnix requested changes to this revision.Tue, Sep 22, 19:21
deadalnix added a subscriber: deadalnix.
deadalnix added inline comments.
src/util/system.cpp
1128 ↗(On Diff #23690)

Move the comment on the previous line.

This revision now requires changes to proceed.Tue, Sep 22, 19:21

Move a comment to the previous line

This revision is now accepted and ready to land.Tue, Sep 22, 19:59