Page MenuHomePhabricator
Differential D13277

[Chronik] Resync index
ClosedPublic
Actions

Authored by tobias_ruck on Mar 10 2023, 21:16.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Commits
rABCd5f89bf049ac: [Chronik] Resync index
Summary

Revert the index until the fork block relative to the node and then add blocks from the node until synced

Depends on D13294 and D13293.

Test Plan

./test/functional/test_runner.py chronik_resync

Diff Detail

Event Timeline

tobias_ruck created this revision.Mar 10 2023, 21:16
Herald added a reviewer: Restricted Project. · View Herald TranscriptMar 10 2023, 21:16
Harbormaster completed remote builds in B22332: Diff 38401.Mar 10 2023, 21:16
tobias_ruck updated this revision to Diff 38402.Mar 10 2023, 21:17

fix lint

tobias_ruck edited the summary of this revision. (Show Details)Mar 10 2023, 21:17
tobias_ruck added a parent revision: D13276: [Chronik] Move ffi::Block into chronik-bridge, use CBlockIndex and CBlock.
teamcity added a comment.Mar 10 2023, 21:32
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
Harbormaster completed remote builds in B22333: Diff 38402.Mar 10 2023, 21:35
teamcity added a comment.Mar 10 2023, 21:35
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
tobias_ruck updated this revision to Diff 38431.Mar 11 2023, 13:03

Rebase on the previous diffs

Harbormaster completed remote builds in B22362: Diff 38431.Mar 11 2023, 13:16
teamcity added a comment.Mar 11 2023, 13:16
Build Bitcoin ABC Diffs / Diff Testing (build-chronik) failed.

Tail of the build log:

wallet_listsinceblock.py                  | ✓ Passed  | 6 s
wallet_listsinceblock.py --descriptors    | ✓ Passed  | 6 s
wallet_listtransactions.py                | ✓ Passed  | 4 s
wallet_listtransactions.py --descriptors  | ✓ Passed  | 4 s
wallet_multiwallet.py                     | ✓ Passed  | 55 s
wallet_multiwallet.py --usecli            | ✓ Passed  | 10 s
wallet_reorgsrestore.py                   | ✓ Passed  | 3 s
wallet_resendwallettransactions.py        | ✓ Passed  | 2 s
wallet_send.py                            | ✓ Passed  | 7 s
wallet_startup.py                         | ✓ Passed  | 2 s
wallet_timelock.py                        | ✓ Passed  | 1 s
wallet_txn_clone.py                       | ✓ Passed  | 1 s
wallet_txn_clone.py --mineblock           | ✓ Passed  | 3 s
wallet_txn_doublespend.py                 | ✓ Passed  | 1 s
wallet_txn_doublespend.py --mineblock     | ✓ Passed  | 3 s
wallet_watchonly.py                       | ✓ Passed  | 1 s
wallet_watchonly.py --usecli              | ✓ Passed  | 1 s
interface_usdt_net.py                     | ○ Skipped | 0 s
interface_usdt_utxocache.py               | ○ Skipped | 0 s
interface_usdt_validation.py              | ○ Skipped | 0 s

ALL                                       | ✓ Passed  | 1387 s (accumulated) 
Runtime: 278 s

[135/483] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/script_tests.cpp.o
In file included from /usr/include/boost/test/unit_test.hpp:19,
                 from ../../src/test/script_tests.cpp:30:
../../src/test/script_tests.cpp: In member function ‘void script_tests::script_build::test_method()’:
../../src/test/script_tests.cpp:540:22: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
  540 | BOOST_AUTO_TEST_CASE(script_build) {
      |                      ^~~~~~~~~~~~
[203/483] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o
FAILED: src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o 
/usr/bin/ccache /usr/bin/c++ -DBOOST_AC_USE_STD_ATOMIC -DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_DYN_LINK -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_SP_USE_STD_ATOMIC -DBOOST_TEST_DYN_LINK -DBOOST_THREAD_DYN_LINK -DBOOST_UNIT_TEST_FRAMEWORK_DYN_LINK -DBUILD_BITCOIN_INTERNAL -DENABLE_AVX2 -DENABLE_SHANI -DENABLE_SSE41 -DHAVE_BUILD_INFO -DHAVE_CONFIG_H -DHAVE_CONSENSUS_LIB -DLEVELDB_ATOMIC_PRESENT -DLEVELDB_PLATFORM_POSIX -DOS_LINUX -I../../src/univalue/include -I../../src/. -Isrc -Isrc/crypto/.. -I../../src/secp256k1/include -I../../src/leveldb/include -I../../chronik -Icargo/build/x86_64-unknown-linux-gnu/cxxbridge -isystem /usr/include/jemalloc -isystem /usr/include/miniupnpc -g -O2 -fPIE -fvisibility=hidden -fstack-reuse=none -fstack-protector-all -Wstack-protector -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -Wextra -Wformat -Wvla -Wcast-align -Wunused-parameter -Wmissing-braces -Wredundant-decls -Wsign-compare -Wduplicated-branches -Wduplicated-cond -Wlogical-op -Wformat-security -Wredundant-move -Woverloaded-virtual -Wno-unused-parameter -Wno-implicit-fallthrough -Wno-psabi -pthread -std=gnu++17 -MD -MT src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o -MF src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o.d -o src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o -c ../../chronik/test/chronikbridge_tests.cpp
../../chronik/test/chronikbridge_tests.cpp: In member function ‘void chronikbridge_tests::test_find_fork::test_method()’:
../../chronik/test/chronikbridge_tests.cpp:74:54: error: no matching function for call to ‘chronik_bridge::ChronikBridge::ChronikBridge(node::NodeContext&)’
   74 |     const chronik_bridge::ChronikBridge bridge(m_node);
      |                                                      ^
In file included from cargo/build/x86_64-unknown-linux-gnu/cxxbridge/chronik-bridge/src/ffi.rs.h:3,
                 from ../../chronik/test/chronikbridge_tests.cpp:6:
../../chronik/chronik-cpp/chronik_bridge.h:47:5: note: candidate: ‘chronik_bridge::ChronikBridge::ChronikBridge(const Config&, const node::NodeContext&)’
   47 |     ChronikBridge(const Config &config, const node::NodeContext &node)
      |     ^~~~~~~~~~~~~
../../chronik/chronik-cpp/chronik_bridge.h:47:5: note:   candidate expects 2 arguments, 1 provided
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note: candidate: ‘constexpr chronik_bridge::ChronikBridge::ChronikBridge(const chronik_bridge::ChronikBridge&)’
   42 | class ChronikBridge {
      |       ^~~~~~~~~~~~~
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note:   no known conversion for argument 1 from ‘node::NodeContext’ to ‘const chronik_bridge::ChronikBridge&’
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note: candidate: ‘constexpr chronik_bridge::ChronikBridge::ChronikBridge(chronik_bridge::ChronikBridge&&)’
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note:   no known conversion for argument 1 from ‘node::NodeContext’ to ‘chronik_bridge::ChronikBridge&&’
[207/483] Running pow test suite
PASSED: pow test suite
[211/483] Running seeder test suite
PASSED: seeder test suite
[223/483] Running bitcoin-qt test suite
PASSED: bitcoin-qt test suite
[228/483] Running avalanche test suite
PASSED: avalanche test suite
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1
tobias_ruck updated this revision to Diff 38435.Mar 11 2023, 13:55

Rebase on D13294 to fix flaky tests

tobias_ruck updated this revision to Diff 38436.Mar 11 2023, 14:01

rebase on D13293

tobias_ruck edited the summary of this revision. (Show Details)Mar 11 2023, 14:02
tobias_ruck added parent revisions: D13293: [Chronik] Add `ChronikBridge::load_block`, D13294: [Chronik] Generate Chronik port for functional test.
Harbormaster completed remote builds in B22365: Diff 38435.Mar 11 2023, 14:11
teamcity added a comment.Mar 11 2023, 14:11
Build Bitcoin ABC Diffs / Diff Testing (build-chronik) failed.

Tail of the build log:

wallet_disable.py                         | ✓ Passed  | 1 s
wallet_dump.py                            | ✓ Passed  | 5 s
wallet_encryption.py                      | ✓ Passed  | 5 s
wallet_encryption.py --descriptors        | ✓ Passed  | 5 s
wallet_groups.py                          | ✓ Passed  | 12 s
wallet_hd.py                              | ✓ Passed  | 6 s
wallet_hd.py --descriptors                | ✓ Passed  | 5 s
wallet_import_rescan.py                   | ✓ Passed  | 7 s
wallet_import_with_label.py               | ✓ Passed  | 1 s
wallet_importdescriptors.py               | ✓ Passed  | 6 s
wallet_importmulti.py                     | ✓ Passed  | 2 s
wallet_importprunedfunds.py               | ✓ Passed  | 2 s
wallet_importprunedfunds.py --descriptors | ✓ Passed  | 2 s
wallet_keypool.py                         | ✓ Passed  | 3 s
wallet_keypool_topup.py                   | ✓ Passed  | 4 s
wallet_keypool_topup.py --descriptors     | ✓ Passed  | 4 s
wallet_labels.py                          | ✓ Passed  | 1 s
wallet_labels.py --descriptors            | ✓ Passed  | 1 s
wallet_listreceivedby.py                  | ✓ Passed  | 5 s
wallet_listsinceblock.py                  | ✓ Passed  | 5 s
wallet_listsinceblock.py --descriptors    | ✓ Passed  | 6 s
wallet_listtransactions.py                | ✓ Passed  | 5 s
wallet_listtransactions.py --descriptors  | ✓ Passed  | 4 s
wallet_multiwallet.py                     | ✓ Passed  | 10 s
wallet_multiwallet.py --usecli            | ✓ Passed  | 9 s
wallet_reorgsrestore.py                   | ✓ Passed  | 3 s
wallet_resendwallettransactions.py        | ✓ Passed  | 2 s
wallet_send.py                            | ✓ Passed  | 7 s
wallet_startup.py                         | ✓ Passed  | 2 s
wallet_timelock.py                        | ✓ Passed  | 1 s
wallet_txn_clone.py                       | ✓ Passed  | 1 s
wallet_txn_clone.py --mineblock           | ✓ Passed  | 3 s
wallet_txn_doublespend.py                 | ✓ Passed  | 1 s
wallet_txn_doublespend.py --mineblock     | ✓ Passed  | 3 s
wallet_watchonly.py                       | ✓ Passed  | 1 s
wallet_watchonly.py --usecli              | ✓ Passed  | 1 s
interface_usdt_net.py                     | ○ Skipped | 0 s
interface_usdt_utxocache.py               | ○ Skipped | 0 s
interface_usdt_validation.py              | ○ Skipped | 0 s

ALL                                       | ✓ Passed  | 1308 s (accumulated) 
Runtime: 263 s

[190/483] Running avalanche test suite
PASSED: avalanche test suite
[211/483] Running seeder test suite
PASSED: seeder test suite
[215/483] Running pow test suite
PASSED: pow test suite
[220/483] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/script_tests.cpp.o
In file included from /usr/include/boost/test/unit_test.hpp:19,
                 from ../../src/test/script_tests.cpp:30:
../../src/test/script_tests.cpp: In member function ‘void script_tests::script_build::test_method()’:
../../src/test/script_tests.cpp:540:22: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
  540 | BOOST_AUTO_TEST_CASE(script_build) {
      |                      ^~~~~~~~~~~~
[228/483] Running bitcoin-qt test suite
PASSED: bitcoin-qt test suite
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1
Harbormaster completed remote builds in B22366: Diff 38436.Mar 11 2023, 14:19
tobias_ruck published this revision for review.Mar 11 2023, 14:33
Fabien requested changes to this revision.Mar 13 2023, 09:32

Log nits but otherwise lgtm

chronik/chronik-indexer/src/indexer.rs
108 ↗(On Diff #38436)

Something like {height}/{tip_height} would be more useful

126 ↗(On Diff #38436)

Also print the hash of the fork block so we can compare with the node easily

This revision now requires changes to proceed.Mar 13 2023, 09:32
tobias_ruck updated this revision to Diff 38502.Mar 13 2023, 14:39

Improve logging, also test logging. Add a test for CannotRewindChronik

tobias_ruck updated this revision to Diff 38506.Mar 13 2023, 14:54

Also test whether node without any blocks returns a chronik rewind error

Harbormaster completed remote builds in B22407: Diff 38502.Mar 13 2023, 14:57
Harbormaster completed remote builds in B22409: Diff 38506.Mar 13 2023, 15:23
tobias_ruck updated this revision to Diff 38512.Mar 13 2023, 15:48

remove some overfitted logging tests

Fabien accepted this revision.Mar 13 2023, 15:59
This revision is now accepted and ready to land.Mar 13 2023, 15:59
Harbormaster completed remote builds in B22410: Diff 38512.Mar 13 2023, 16:06
tobias_ruck updated this revision to Diff 38519.Mar 13 2023, 18:58

move expect_unique_ptr to chronik_bridge::util, rebase from master

tobias_ruck added a comment.Mar 13 2023, 19:00

@bot build-linux64 build-linux-aarch64 build-linux-arm build-osx build-win64

teamcity awarded a token.Mar 13 2023, 19:00
Harbormaster completed remote builds in B22415: Diff 38519.Mar 13 2023, 19:23
Closed by commit rABCd5f89bf049ac: [Chronik] Resync index (authored by tobias_ruck). · Explain WhyMar 13 2023, 19:33
This revision was automatically updated to reflect the committed changes.
tobias_ruck added a commit: rABCd5f89bf049ac: [Chronik] Resync index.

Revision Contents
Changeset List

PathSize
chronik/
1 line
chronik-bridge/
src/
22 lines
chronik-cpp/
18 lines
8 lines
41 lines
util/
1 line
6 lines
20 lines
21 lines
chronik-indexer/
1 line
src/
85 lines
chronik-lib/
src/
34 lines
7 lines
test/
functional/
6 lines
71 lines

Diff 38402

View Options

Cargo.lock

Loading...
View Options

chronik/CMakeLists.txt

Loading...
View Options

chronik/chronik-bridge/src/ffi.rs

Loading...
View Options

chronik/chronik-cpp/chronik.cpp

Loading...
View Options

chronik/chronik-cpp/chronik_bridge.h

Loading...
View Options

chronik/chronik-cpp/chronik_bridge.cpp

Loading...
View Options

chronik/chronik-cpp/util/hash.h

Loading...
View Options

chronik/chronik-cpp/util/hash.cpp

Loading...
View Options

chronik/chronik-cpp/util/primitives.h

Loading...
View Options

chronik/chronik-cpp/util/primitives.cpp

Loading...
View Options

chronik/chronik-indexer/Cargo.toml

Loading...
View Options

chronik/chronik-indexer/src/indexer.rs

Loading...
View Options

chronik/chronik-lib/src/bridge.rs

Loading...
View Options

chronik/chronik-lib/src/ffi.rs

Loading...
View Options

test/functional/chronik_block.py

Loading...
View Options

test/functional/chronik_resync.py

Loading...