Page MenuHomePhabricator
Differential D16102

prune: scan and unlink already pruned block files on startup
ClosedPublic
Actions

Authored by PiRK on May 3 2024, 09:32.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Commits
rABC7b4e342de013: prune: scan and unlink already pruned block files on startup
Summary

There are a few cases where we can mark a block and undo file as pruned in our block index, but not actually remove the files from disk.

  • If we call FindFilesToPrune or FindFilesToPruneManual and crash before UnlinkPrunedFiles.
  • If on Windows there is an open file handle to the file somewhere else when calling fs::remove in UnlinkPrunedFiles. This could be from another process, or if we are calling ReadBlockFromDisk/ReadRawBlockFromDisk without having a lock on cs_main (allowed since D2979).

This PR mitigates this by scanning all pruned block files on startup after LoadBlockIndexDB and unlinking them again.

This is a backport of core#26533

Test Plan

ninja all check-all

Diff Detail

Event Timeline

PiRK created this revision.May 3 2024, 09:32
Herald added a reviewer: Restricted Project. · View Herald TranscriptMay 3 2024, 09:32
PiRK requested review of this revision.May 3 2024, 09:32
teamcity edited the summary of this revision. (Show Details)May 3 2024, 09:32
PiRK updated this revision to Diff 47593.May 3 2024, 09:36

nit: make file permission and shebang conistent with other functional tests

Harbormaster completed remote builds in B28871: Diff 47591.May 3 2024, 10:10
Harbormaster completed remote builds in B28872: Diff 47593.May 3 2024, 10:54
Fabien requested changes to this revision.May 3 2024, 11:56
Fabien added a subscriber: Fabien.
Fabien added inline comments.
src/node/blockstorage.cpp
581 ↗(On Diff #47593)

please use a better name here, ec usually stands for elliptic crypto in our codebase

This revision now requires changes to proceed.May 3 2024, 11:56
PiRK updated this revision to Diff 47595.May 3 2024, 12:42

better variable name

Harbormaster completed remote builds in B28874: Diff 47595.May 3 2024, 13:03
Fabien accepted this revision.May 6 2024, 07:27
This revision is now accepted and ready to land.May 6 2024, 07:27
Closed by commit rABC7b4e342de013: prune: scan and unlink already pruned block files on startup (authored by Andrew Toth <andrewstoth@gmail.com>, committed by PiRK). · Explain WhyMay 6 2024, 11:54
This revision was automatically updated to reflect the committed changes.
PiRK added a commit: rABC7b4e342de013: prune: scan and unlink already pruned block files on startup.

Revision Contents
Changeset List

PathSize
src/
node/
7 lines
29 lines
test/
48 lines
2 lines
test/
functional/
68 lines

Diff 47591

View Options

src/node/blockstorage.h

Loading...
View Options

src/node/blockstorage.cpp

Loading...
View Options

src/test/blockmanager_tests.cpp

Loading...
View Options

src/validation.cpp

Loading...
View Options

test/functional/feature_remove_pruned_files_on_startup.py

Loading...