Page MenuHomePhabricator
Differential D13277

[Chronik] Resync index
ClosedPublic
Actions

Authored by tobias_ruck on Mar 10 2023, 21:16.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Commits
rABCd5f89bf049ac: [Chronik] Resync index
Summary

Revert the index until the fork block relative to the node and then add blocks from the node until synced

Depends on D13294 and D13293.

Test Plan

./test/functional/test_runner.py chronik_resync

Diff Detail

Event Timeline

tobias_ruck created this revision.Mar 10 2023, 21:16
Herald added a reviewer: Restricted Project. · View Herald TranscriptMar 10 2023, 21:16
Harbormaster completed remote builds in B22332: Diff 38401.Mar 10 2023, 21:16
tobias_ruck updated this revision to Diff 38402.Mar 10 2023, 21:17

fix lint

tobias_ruck edited the summary of this revision. (Show Details)Mar 10 2023, 21:17
tobias_ruck added a parent revision: D13276: [Chronik] Move ffi::Block into chronik-bridge, use CBlockIndex and CBlock.
teamcity added a comment.Mar 10 2023, 21:32
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
Harbormaster completed remote builds in B22333: Diff 38402.Mar 10 2023, 21:35
teamcity added a comment.Mar 10 2023, 21:35
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
tobias_ruck updated this revision to Diff 38431.Mar 11 2023, 13:03

Rebase on the previous diffs

Harbormaster completed remote builds in B22362: Diff 38431.Mar 11 2023, 13:16
teamcity added a comment.Mar 11 2023, 13:16
Build Bitcoin ABC Diffs / Diff Testing (build-chronik) failed.

Tail of the build log:

wallet_listsinceblock.py                  | ✓ Passed  | 6 s
wallet_listsinceblock.py --descriptors    | ✓ Passed  | 6 s
wallet_listtransactions.py                | ✓ Passed  | 4 s
wallet_listtransactions.py --descriptors  | ✓ Passed  | 4 s
wallet_multiwallet.py                     | ✓ Passed  | 55 s
wallet_multiwallet.py --usecli            | ✓ Passed  | 10 s
wallet_reorgsrestore.py                   | ✓ Passed  | 3 s
wallet_resendwallettransactions.py        | ✓ Passed  | 2 s
wallet_send.py                            | ✓ Passed  | 7 s
wallet_startup.py                         | ✓ Passed  | 2 s
wallet_timelock.py                        | ✓ Passed  | 1 s
wallet_txn_clone.py                       | ✓ Passed  | 1 s
wallet_txn_clone.py --mineblock           | ✓ Passed  | 3 s
wallet_txn_doublespend.py                 | ✓ Passed  | 1 s
wallet_txn_doublespend.py --mineblock     | ✓ Passed  | 3 s
wallet_watchonly.py                       | ✓ Passed  | 1 s
wallet_watchonly.py --usecli              | ✓ Passed  | 1 s
interface_usdt_net.py                     | ○ Skipped | 0 s
interface_usdt_utxocache.py               | ○ Skipped | 0 s
interface_usdt_validation.py              | ○ Skipped | 0 s

ALL                                       | ✓ Passed  | 1387 s (accumulated) 
Runtime: 278 s

[135/483] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/script_tests.cpp.o
In file included from /usr/include/boost/test/unit_test.hpp:19,
                 from ../../src/test/script_tests.cpp:30:
../../src/test/script_tests.cpp: In member function ‘void script_tests::script_build::test_method()’:
../../src/test/script_tests.cpp:540:22: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
  540 | BOOST_AUTO_TEST_CASE(script_build) {
      |                      ^~~~~~~~~~~~
[203/483] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o
FAILED: src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o 
/usr/bin/ccache /usr/bin/c++ -DBOOST_AC_USE_STD_ATOMIC -DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_DYN_LINK -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_SP_USE_STD_ATOMIC -DBOOST_TEST_DYN_LINK -DBOOST_THREAD_DYN_LINK -DBOOST_UNIT_TEST_FRAMEWORK_DYN_LINK -DBUILD_BITCOIN_INTERNAL -DENABLE_AVX2 -DENABLE_SHANI -DENABLE_SSE41 -DHAVE_BUILD_INFO -DHAVE_CONFIG_H -DHAVE_CONSENSUS_LIB -DLEVELDB_ATOMIC_PRESENT -DLEVELDB_PLATFORM_POSIX -DOS_LINUX -I../../src/univalue/include -I../../src/. -Isrc -Isrc/crypto/.. -I../../src/secp256k1/include -I../../src/leveldb/include -I../../chronik -Icargo/build/x86_64-unknown-linux-gnu/cxxbridge -isystem /usr/include/jemalloc -isystem /usr/include/miniupnpc -g -O2 -fPIE -fvisibility=hidden -fstack-reuse=none -fstack-protector-all -Wstack-protector -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -Wextra -Wformat -Wvla -Wcast-align -Wunused-parameter -Wmissing-braces -Wredundant-decls -Wsign-compare -Wduplicated-branches -Wduplicated-cond -Wlogical-op -Wformat-security -Wredundant-move -Woverloaded-virtual -Wno-unused-parameter -Wno-implicit-fallthrough -Wno-psabi -pthread -std=gnu++17 -MD -MT src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o -MF src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o.d -o src/test/CMakeFiles/test_bitcoin.dir/__/__/chronik/test/chronikbridge_tests.cpp.o -c ../../chronik/test/chronikbridge_tests.cpp
../../chronik/test/chronikbridge_tests.cpp: In member function ‘void chronikbridge_tests::test_find_fork::test_method()’:
../../chronik/test/chronikbridge_tests.cpp:74:54: error: no matching function for call to ‘chronik_bridge::ChronikBridge::ChronikBridge(node::NodeContext&)’
   74 |     const chronik_bridge::ChronikBridge bridge(m_node);
      |                                                      ^
In file included from cargo/build/x86_64-unknown-linux-gnu/cxxbridge/chronik-bridge/src/ffi.rs.h:3,
                 from ../../chronik/test/chronikbridge_tests.cpp:6:
../../chronik/chronik-cpp/chronik_bridge.h:47:5: note: candidate: ‘chronik_bridge::ChronikBridge::ChronikBridge(const Config&, const node::NodeContext&)’
   47 |     ChronikBridge(const Config &config, const node::NodeContext &node)
      |     ^~~~~~~~~~~~~
../../chronik/chronik-cpp/chronik_bridge.h:47:5: note:   candidate expects 2 arguments, 1 provided
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note: candidate: ‘constexpr chronik_bridge::ChronikBridge::ChronikBridge(const chronik_bridge::ChronikBridge&)’
   42 | class ChronikBridge {
      |       ^~~~~~~~~~~~~
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note:   no known conversion for argument 1 from ‘node::NodeContext’ to ‘const chronik_bridge::ChronikBridge&’
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note: candidate: ‘constexpr chronik_bridge::ChronikBridge::ChronikBridge(chronik_bridge::ChronikBridge&&)’
../../chronik/chronik-cpp/chronik_bridge.h:42:7: note:   no known conversion for argument 1 from ‘node::NodeContext’ to ‘chronik_bridge::ChronikBridge&&’
[207/483] Running pow test suite
PASSED: pow test suite
[211/483] Running seeder test suite
PASSED: seeder test suite
[223/483] Running bitcoin-qt test suite
PASSED: bitcoin-qt test suite
[228/483] Running avalanche test suite
PASSED: avalanche test suite
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1
tobias_ruck updated this revision to Diff 38435.Mar 11 2023, 13:55

Rebase on D13294 to fix flaky tests

tobias_ruck updated this revision to Diff 38436.Mar 11 2023, 14:01

rebase on D13293

tobias_ruck edited the summary of this revision. (Show Details)Mar 11 2023, 14:02
tobias_ruck added parent revisions: D13293: [Chronik] Add `ChronikBridge::load_block`, D13294: [Chronik] Generate Chronik port for functional test.
Harbormaster completed remote builds in B22365: Diff 38435.Mar 11 2023, 14:11
teamcity added a comment.Mar 11 2023, 14:11
Build Bitcoin ABC Diffs / Diff Testing (build-chronik) failed.

Tail of the build log:

wallet_disable.py                         | ✓ Passed  | 1 s
wallet_dump.py                            | ✓ Passed  | 5 s
wallet_encryption.py                      | ✓ Passed  | 5 s
wallet_encryption.py --descriptors        | ✓ Passed  | 5 s
wallet_groups.py                          | ✓ Passed  | 12 s
wallet_hd.py                              | ✓ Passed  | 6 s
wallet_hd.py --descriptors                | ✓ Passed  | 5 s
wallet_import_rescan.py                   | ✓ Passed  | 7 s
wallet_import_with_label.py               | ✓ Passed  | 1 s
wallet_importdescriptors.py               | ✓ Passed  | 6 s
wallet_importmulti.py                     | ✓ Passed  | 2 s
wallet_importprunedfunds.py               | ✓ Passed  | 2 s
wallet_importprunedfunds.py --descriptors | ✓ Passed  | 2 s
wallet_keypool.py                         | ✓ Passed  | 3 s
wallet_keypool_topup.py                   | ✓ Passed  | 4 s
wallet_keypool_topup.py --descriptors     | ✓ Passed  | 4 s
wallet_labels.py                          | ✓ Passed  | 1 s
wallet_labels.py --descriptors            | ✓ Passed  | 1 s
wallet_listreceivedby.py                  | ✓ Passed  | 5 s
wallet_listsinceblock.py                  | ✓ Passed  | 5 s
wallet_listsinceblock.py --descriptors    | ✓ Passed  | 6 s
wallet_listtransactions.py                | ✓ Passed  | 5 s
wallet_listtransactions.py --descriptors  | ✓ Passed  | 4 s
wallet_multiwallet.py                     | ✓ Passed  | 10 s
wallet_multiwallet.py --usecli            | ✓ Passed  | 9 s
wallet_reorgsrestore.py                   | ✓ Passed  | 3 s
wallet_resendwallettransactions.py        | ✓ Passed  | 2 s
wallet_send.py                            | ✓ Passed  | 7 s
wallet_startup.py                         | ✓ Passed  | 2 s
wallet_timelock.py                        | ✓ Passed  | 1 s
wallet_txn_clone.py                       | ✓ Passed  | 1 s
wallet_txn_clone.py --mineblock           | ✓ Passed  | 3 s
wallet_txn_doublespend.py                 | ✓ Passed  | 1 s
wallet_txn_doublespend.py --mineblock     | ✓ Passed  | 3 s
wallet_watchonly.py                       | ✓ Passed  | 1 s
wallet_watchonly.py --usecli              | ✓ Passed  | 1 s
interface_usdt_net.py                     | ○ Skipped | 0 s
interface_usdt_utxocache.py               | ○ Skipped | 0 s
interface_usdt_validation.py              | ○ Skipped | 0 s

ALL                                       | ✓ Passed  | 1308 s (accumulated) 
Runtime: 263 s

[190/483] Running avalanche test suite
PASSED: avalanche test suite
[211/483] Running seeder test suite
PASSED: seeder test suite
[215/483] Running pow test suite
PASSED: pow test suite
[220/483] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/script_tests.cpp.o
In file included from /usr/include/boost/test/unit_test.hpp:19,
                 from ../../src/test/script_tests.cpp:30:
../../src/test/script_tests.cpp: In member function ‘void script_tests::script_build::test_method()’:
../../src/test/script_tests.cpp:540:22: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
  540 | BOOST_AUTO_TEST_CASE(script_build) {
      |                      ^~~~~~~~~~~~
[228/483] Running bitcoin-qt test suite
PASSED: bitcoin-qt test suite
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1
Harbormaster completed remote builds in B22366: Diff 38436.Mar 11 2023, 14:19
tobias_ruck published this revision for review.Mar 11 2023, 14:33
Fabien requested changes to this revision.Mar 13 2023, 09:32

Log nits but otherwise lgtm

chronik/chronik-indexer/src/indexer.rs
108

Something like {height}/{tip_height} would be more useful

126

Also print the hash of the fork block so we can compare with the node easily

This revision now requires changes to proceed.Mar 13 2023, 09:32
tobias_ruck updated this revision to Diff 38502.Mar 13 2023, 14:39

Improve logging, also test logging. Add a test for CannotRewindChronik

tobias_ruck updated this revision to Diff 38506.Mar 13 2023, 14:54

Also test whether node without any blocks returns a chronik rewind error

Harbormaster completed remote builds in B22407: Diff 38502.Mar 13 2023, 14:57
Harbormaster completed remote builds in B22409: Diff 38506.Mar 13 2023, 15:23
tobias_ruck updated this revision to Diff 38512.Mar 13 2023, 15:48

remove some overfitted logging tests

Fabien accepted this revision.Mar 13 2023, 15:59
This revision is now accepted and ready to land.Mar 13 2023, 15:59
Harbormaster completed remote builds in B22410: Diff 38512.Mar 13 2023, 16:06
tobias_ruck updated this revision to Diff 38519.Mar 13 2023, 18:58

move expect_unique_ptr to chronik_bridge::util, rebase from master

tobias_ruck added a comment.Mar 13 2023, 19:00

@bot build-linux64 build-linux-aarch64 build-linux-arm build-osx build-win64

teamcity awarded a token.Mar 13 2023, 19:00
Harbormaster completed remote builds in B22415: Diff 38519.Mar 13 2023, 19:23
Closed by commit rABCd5f89bf049ac: [Chronik] Resync index (authored by tobias_ruck). · Explain WhyMar 13 2023, 19:33
This revision was automatically updated to reflect the committed changes.
tobias_ruck added a commit: rABCd5f89bf049ac: [Chronik] Resync index.

Revision Contents
Changeset List

PathSize
chronik/
chronik-cpp/
19 lines
chronik-indexer/
1 line
src/
105 lines
chronik-lib/
src/
41 lines
16 lines
test/
functional/
72 lines

Diff 38436

View Options

Cargo.lock

Loading...
View Options

chronik/chronik-cpp/chronik.cpp

Loading...
View Options

chronik/chronik-indexer/Cargo.toml

Loading...
View Options

chronik/chronik-indexer/src/indexer.rs

Loading...
View Options

chronik/chronik-lib/src/bridge.rs

Loading...
View Options

chronik/chronik-lib/src/ffi.rs

Loading...
View Options

test/functional/chronik_resync.py

Loading...