Page MenuHomePhabricator

kernel: De-globalize fReindex
ClosedPublic

Authored by PiRK on Wed, Sep 2, 13:34.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Commits
rABCecbd604076fa: kernel: De-globalize fReindex
Summary

fReindex is one of the last remaining globals exposed by the kernel
library, so move it into the blockstorage class to reduce the amount of
global mutable state and make the kernel library a bit less awkward to
use.

This is a backport of core#29817

Test Plan

ninja all check-all

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

PiRK requested review of this revision.Wed, Sep 2, 13:34
PiRK planned changes to this revision.Wed, Sep 2, 13:38

needs a rebase (commit on top of D20540)

Tail of the build log:

-- Performing Test have_C__Wundef - Success
-- Performing Test have_C__Wreserved_identifier
-- Performing Test have_C__Wreserved_identifier - Success
-- Performing Test USE_ASM_X86_64
-- Performing Test USE_ASM_X86_64 - Success
-- Performing Test have_linker__Wl___no_undefined
-- Performing Test have_linker__Wl___no_undefined - Success
-- Found Event component event: /usr/lib/x86_64-linux-gnu/libevent.so
-- Found Event: /usr/include (found suitable version "2.1.12-stable", minimum required is "2.1.8") found components: event 
-- Found Boost component headers: /usr/include (found suitable version "1.74.0", minimum required is "1.64")
-- Found Event component pthreads: /usr/lib/x86_64-linux-gnu/libevent_pthreads.so
-- Found Event: /usr/include (found suitable version "2.1.12-stable", minimum required is "2.1.8") found components: pthreads 
-- Performing Test fuzz_target_builds_without_main_fuzz
-- Performing Test fuzz_target_builds_without_main_fuzz - Failed
-- Found BerkeleyDB component CXX: /usr/lib/x86_64-linux-gnu/libdb_cxx-5.3.so
-- Found BerkeleyDB: /usr/include (found suitable version "5.3.28", minimum required is "5.3") found components: CXX 
-- Found SQLite3: /usr/include (found suitable version "3.40.1", minimum required is "3.7.17") 
-- Found ZeroMQ component zmq: /usr/lib/x86_64-linux-gnu/libzmq.so
-- Found ZeroMQ: /usr/include (found suitable version "4.3.4", minimum required is "4.1.5")  
-- Could NOT find Protobuf (missing: Protobuf_DIR)
-- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so (found version "3.21.12") 
-- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "3.0.20")  
-- Looking for EVP_MD_CTX_new
-- Looking for EVP_MD_CTX_new - found
-- Found QREncode component qrencode: /usr/lib/x86_64-linux-gnu/libqrencode.so
-- Found QREncode: /usr/include   
[1/9] Creating directories for 'corrosion-populate'
[1/9] Performing download step (git clone) for 'corrosion-populate'
Cloning into 'corrosion-src'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing
Cloning into 'corrosion-src'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing
Cloning into 'corrosion-src'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing
-- Had to git clone more than once: 3 times.
CMake Error at corrosion-subbuild/corrosion-populate-prefix/tmp/corrosion-populate-gitclone.cmake:39 (message):
  Failed to clone repository: 'https://github.com/corrosion-rs/corrosion.git'


FAILED: corrosion-populate-prefix/src/corrosion-populate-stamp/corrosion-populate-download /work/abc-ci-builds/build-clang-tidy/_deps/corrosion-subbuild/corrosion-populate-prefix/src/corrosion-populate-stamp/corrosion-populate-download 
cd /work/abc-ci-builds/build-clang-tidy/_deps && /usr/bin/cmake -P /work/abc-ci-builds/build-clang-tidy/_deps/corrosion-subbuild/corrosion-populate-prefix/tmp/corrosion-populate-gitclone.cmake && /usr/bin/cmake -E touch /work/abc-ci-builds/build-clang-tidy/_deps/corrosion-subbuild/corrosion-populate-prefix/src/corrosion-populate-stamp/corrosion-populate-download
ninja: build stopped: subcommand failed.

CMake Error at /usr/share/cmake-3.25/Modules/FetchContent.cmake:1616 (message):
  Build step for corrosion failed: 1
Call Stack (most recent call first):
  /usr/share/cmake-3.25/Modules/FetchContent.cmake:1756:EVAL:2 (__FetchContent_directPopulate)
  /usr/share/cmake-3.25/Modules/FetchContent.cmake:1756 (cmake_language)
  /usr/share/cmake-3.25/Modules/FetchContent.cmake:1970 (FetchContent_Populate)
  cmake/modules/RustHelper.cmake:9 (FetchContent_MakeAvailable)
  chronik/CMakeLists.txt:6 (include)


-- Configuring incomplete, errors occurred!
See also "/work/abc-ci-builds/build-clang-tidy/CMakeFiles/CMakeOutput.log".
See also "/work/abc-ci-builds/build-clang-tidy/CMakeFiles/CMakeError.log".
Build build-clang-tidy failed with exit code 1

Build Bitcoin ABC Diffs / Diff Testing (ai-review) passed.
CodeRabbit Review

Diff : committed changes only
Compare : HEAD → master
Directory : work
────────────────────────────────────────

(\(\
(• .•) Easter bunny by day, code reviewer by night.

────────────────────────────────────────────────────────────────────────

minor [Functional Correctness]
→ ]8;;vscode://file//work/src/test/util/setup_common.cpp:228src/test/util/setup_common.cpp:228]8;;

Apply block-manager arguments in the test setup.

LoadVerifyActivateChainstate() now reads options.reindex from
chainman.m_blockman.m_reindexing, but ChainTestingSetup never applies
ApplyArgsManOptions to its BlockManager::Options. A test that passes
-reindex therefore constructs the manager with reindex=false and
silently skips the reindex path. Apply the block-manager arguments before
constructing m_node.chainman.

────────────────────────────────────────────────────────────────────────

major [Stability & Availability]
→ ]8;;vscode://file//work/src/init.cpp:2769src/init.cpp:2769]8;;

Carry the recovery request into the next ChainstateManager.

node.chainman is recreated at the start of each loop, but
blockman_opts is created once before that loop. This line updates only
the current manager's m_reindexing; the next manager is still
constructed with blockman_opts.reindex == false, so options.reindex
becomes false again and the accepted recovery is lost. Set
blockman_opts.reindex = true before restarting the loop.


Proposed fix

+                    blockman_opts.reindex = true;
                     chainman.m_blockman.m_reindexing = true;

────────────────────────────────────────
Review complete
2 findings ✔

Major 1
Minor 1

9 files reviewed:

  • src/bitcoin-chainstate.cpp
  • src/init.cpp
  • src/kernel/blockmanager_opts.h
  • src/node/blockmanager_args.cpp
  • src/node/blockstorage.cpp
  • src/node/blockstorage.h
  • src/node/chainstate.cpp
  • src/test/util/setup_common.cpp
  • src/validation.cpp

────────────────────────────────────────

Print all AI prompts: coderabbit review --show-prompts

PiRK planned changes to this revision.Thu, Sep 3, 07:56

needs squashing with a bugfix commit

squash with bugfix: https://github.com/bitcoin/bitcoin/pull/30132/changes/533eab7d67d78f217f74909662133086b79ea808
And also use m_args in setup_common.cpp instead of using the blockman's default (false)

Tail of the build log:

-- Performing Test have_C__Wundef - Success
-- Performing Test have_C__Wreserved_identifier
-- Performing Test have_C__Wreserved_identifier - Success
-- Performing Test USE_ASM_X86_64
-- Performing Test USE_ASM_X86_64 - Success
-- Performing Test have_linker__Wl___no_undefined
-- Performing Test have_linker__Wl___no_undefined - Success
-- Found Event component event: /usr/lib/x86_64-linux-gnu/libevent.so
-- Found Event: /usr/include (found suitable version "2.1.12-stable", minimum required is "2.1.8") found components: event 
-- Found Boost component headers: /usr/include (found suitable version "1.74.0", minimum required is "1.64")
-- Found Event component pthreads: /usr/lib/x86_64-linux-gnu/libevent_pthreads.so
-- Found Event: /usr/include (found suitable version "2.1.12-stable", minimum required is "2.1.8") found components: pthreads 
-- Performing Test fuzz_target_builds_without_main_fuzz
-- Performing Test fuzz_target_builds_without_main_fuzz - Failed
-- Found BerkeleyDB component CXX: /usr/lib/x86_64-linux-gnu/libdb_cxx-5.3.so
-- Found BerkeleyDB: /usr/include (found suitable version "5.3.28", minimum required is "5.3") found components: CXX 
-- Found SQLite3: /usr/include (found suitable version "3.40.1", minimum required is "3.7.17") 
-- Found ZeroMQ component zmq: /usr/lib/x86_64-linux-gnu/libzmq.so
-- Found ZeroMQ: /usr/include (found suitable version "4.3.4", minimum required is "4.1.5")  
-- Could NOT find Protobuf (missing: Protobuf_DIR)
-- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so (found version "3.21.12") 
-- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "3.0.20")  
-- Looking for EVP_MD_CTX_new
-- Looking for EVP_MD_CTX_new - found
-- Found QREncode component qrencode: /usr/lib/x86_64-linux-gnu/libqrencode.so
-- Found QREncode: /usr/include   
[1/9] Creating directories for 'corrosion-populate'
[1/9] Performing download step (git clone) for 'corrosion-populate'
Cloning into 'corrosion-src'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing
Cloning into 'corrosion-src'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing
Cloning into 'corrosion-src'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing
-- Had to git clone more than once: 3 times.
CMake Error at corrosion-subbuild/corrosion-populate-prefix/tmp/corrosion-populate-gitclone.cmake:39 (message):
  Failed to clone repository: 'https://github.com/corrosion-rs/corrosion.git'


FAILED: corrosion-populate-prefix/src/corrosion-populate-stamp/corrosion-populate-download /work/abc-ci-builds/build-clang-tidy/_deps/corrosion-subbuild/corrosion-populate-prefix/src/corrosion-populate-stamp/corrosion-populate-download 
cd /work/abc-ci-builds/build-clang-tidy/_deps && /usr/bin/cmake -P /work/abc-ci-builds/build-clang-tidy/_deps/corrosion-subbuild/corrosion-populate-prefix/tmp/corrosion-populate-gitclone.cmake && /usr/bin/cmake -E touch /work/abc-ci-builds/build-clang-tidy/_deps/corrosion-subbuild/corrosion-populate-prefix/src/corrosion-populate-stamp/corrosion-populate-download
ninja: build stopped: subcommand failed.

CMake Error at /usr/share/cmake-3.25/Modules/FetchContent.cmake:1616 (message):
  Build step for corrosion failed: 1
Call Stack (most recent call first):
  /usr/share/cmake-3.25/Modules/FetchContent.cmake:1756:EVAL:2 (__FetchContent_directPopulate)
  /usr/share/cmake-3.25/Modules/FetchContent.cmake:1756 (cmake_language)
  /usr/share/cmake-3.25/Modules/FetchContent.cmake:1970 (FetchContent_Populate)
  cmake/modules/RustHelper.cmake:9 (FetchContent_MakeAvailable)
  chronik/CMakeLists.txt:6 (include)


-- Configuring incomplete, errors occurred!
See also "/work/abc-ci-builds/build-clang-tidy/CMakeFiles/CMakeOutput.log".
See also "/work/abc-ci-builds/build-clang-tidy/CMakeFiles/CMakeError.log".
Build build-clang-tidy failed with exit code 1

Build Bitcoin ABC Diffs / Diff Testing (ai-review) passed.
CodeRabbit Review

Diff : committed changes only
Compare : HEAD → master
Directory : work
────────────────────────────────────────

(\(\
(• .•) Your favorite GPT-wrapped code reviewer.

────────────────────────────────────────
Review complete
No findings ✔

9 files reviewed:

  • src/bitcoin-chainstate.cpp
  • src/init.cpp
  • src/kernel/blockmanager_opts.h
  • src/node/blockmanager_args.cpp
  • src/node/blockstorage.cpp
  • src/node/blockstorage.h
  • src/node/chainstate.cpp
  • src/test/util/setup_common.cpp
  • src/validation.cpp

────────────────────────────────────────

Build Bitcoin ABC Diffs / Diff Testing (ai-review) passed.
CodeRabbit Review

Diff : committed changes only
Compare : HEAD → master
Directory : work
────────────────────────────────────────

(\(\
(• .•) Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. - Brian Kernighan

────────────────────────────────────────────────────────────────────────

major [Functional Correctness]
→ ]8;;vscode://file//work/src/test/util/setup_common.cpp:228src/test/util/setup_common.cpp:228]8;;

Read -reindex from the parsed test arguments.

extra_args are parsed through m_node.args, but m_args only receives
-datadir in this setup. Therefore m_args.GetBoolArg("-reindex", false)
remains false when a test passes -reindex. The chainstate setup can then
reuse stale state and silently skip the reindex path.

Use m_node.args->GetBoolArg("-reindex", false) here, or populate
m_args from the same argument list.

────────────────────────────────────────
Review complete
1 finding ✔

Major 1

9 files reviewed:

  • src/bitcoin-chainstate.cpp
  • src/init.cpp
  • src/kernel/blockmanager_opts.h
  • src/node/blockmanager_args.cpp
  • src/node/blockstorage.cpp
  • src/node/blockstorage.h
  • src/node/chainstate.cpp
  • src/test/util/setup_common.cpp
  • src/validation.cpp

────────────────────────────────────────

Print all AI prompts: coderabbit review --show-prompts

This revision is now accepted and ready to land.Thu, Sep 3, 19:13
This revision was automatically updated to reflect the committed changes.