Details

Reviewers

Group Reviewers

Restricted Project

Commits

rABCe3b90a9aea1b: [Chronik] Relax plugin height checks at startup

Summary

Currently, we require all plugins to be synced for every block since genesis. However, this is overkill for most plugins, as most have the first usage of their LOKAD ID much later.

In this diff, allow plugins to load if there's currently no txs that would apply for it indexed in the DB already.

Depends on D16568.

Test Plan

cargo test -p chronik-indexer && ./test/functional/test_runner.py chronik_plugins_setup

Diff Detail

Repository

rABC Bitcoin ABC

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

tobias_ruck created this revision.Jul 31 2024, 15:23

Herald added a reviewer: Restricted Project. · View Herald TranscriptJul 31 2024, 15:23

Harbormaster completed remote builds in B29806: Diff 48957.Jul 31 2024, 16:10

tobias_ruck published this revision for review.Jul 31 2024, 16:30

Also fix a small bug where can't get the tx details for coinbase txs if the plugin index is enabled.

Could you please split the bugfix in its own diff with test ?

Fabien requested changes to this revision.Aug 1 2024, 09:45

This revision now requires changes to proceed.Aug 1 2024, 09:45

rebase onto D16568

tobias_ruck edited the summary of this revision. (Show Details)Aug 1 2024, 10:23

tobias_ruck added a parent revision: D16568: [Chronik] Fix reading coinbase txs when plugins are loaded.

rebase onto D16568 again

Harbormaster completed remote builds in B29826: Diff 48982.Aug 1 2024, 11:18

Build Bitcoin ABC Diffs / Diff Testing (build-chronik) failed.

Tail of the build log:

    Checking rand v0.8.5
    Checking http-body-util v0.1.1
    Checking url v2.5.0
    Checking tracing v0.1.40
    Checking chronik-util v0.1.0 (/work/chronik/chronik-util)
    Checking sha1 v0.10.6
    Checking embedded-io v0.4.0
    Checking cobs v0.2.3
    Checking data-encoding v2.5.0
    Checking ryu v1.0.17
    Checking smallvec v1.13.2
    Checking httpdate v1.0.3
    Checking utf-8 v0.7.6
    Checking postcard v1.0.8
    Checking pin-project v1.1.5
    Checking tungstenite v0.21.0
    Checking chronik-plugin-common v0.1.0 (/work/chronik/chronik-plugin-common)
   Compiling prost-build v0.11.9
    Checking toml_datetime v0.6.6
    Checking chronik-plugin v0.1.0 (/work/chronik/chronik-plugin)
    Checking serde_spanned v0.6.6
   Compiling pyo3 v0.22.2
    Checking topo_sort v0.4.0
    Checking seahash v4.1.0
    Checking sync_wrapper v0.1.2
    Checking mime v0.3.17
    Checking winnow v0.6.13
    Checking memoffset v0.9.1
    Checking axum-core v0.4.3
    Checking serde_json v1.0.115
    Checking serde_urlencoded v0.7.1
    Checking hyper v1.2.0
    Checking tokio-tungstenite v0.21.0
    Checking tower v0.4.13
    Checking futures-executor v0.3.30
    Checking serde_path_to_error v0.1.16
    Checking hyper-util v0.1.3
    Checking matchit v0.7.3
    Checking unindent v0.2.3
    Checking sync_wrapper v1.0.1
    Checking unicode-segmentation v1.11.0
    Checking base64 v0.21.7
    Checking futures v0.3.30
    Checking tower-http v0.5.2
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking convert_case v0.6.0
    Checking toml_edit v0.22.14
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking axum v0.7.5
   Compiling pyo3-macros v0.22.2
    Checking toml v0.8.14
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
    Checking chronik-indexer v0.1.0 (/work/chronik/chronik-indexer)
    Checking chronik-http v0.1.0 (/work/chronik/chronik-http)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 39s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Build Bitcoin ABC Diffs / Diff Testing (build-chronik-plugins) failed.

Tail of the build log:

   Doc-tests chronik-bridge

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-db

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-http

running 1 test
test chronik/chronik-http/src/protobuf.rs - protobuf::Protobuf (line 29) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.32s

   Doc-tests chronik-indexer

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-common

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-plugin-impl

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-proto

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests chronik-util

running 2 tests
test chronik/chronik-util/src/log.rs - log::log (line 65) ... ignored
test chronik/chronik-util/src/log.rs - log::log_chronik (line 87) ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.00s

ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1

Harbormaster completed remote builds in B29829: Diff 48985.Aug 1 2024, 13:26

Build Bitcoin ABC Diffs / Diff Testing (chronik-client-integration-tests) failed.

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/util.py", line 296, in wait_until_helper
    raise AssertionError(
AssertionError: Predicate ''''
            self.wait_until(lambda: is_finalblock(next_blockhash))
''' not true after 60.0 seconds
2024-08-01T13:25:46.857000Z TestFramework (INFO): Stopping nodes
2024-08-01T13:25:47.011000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240801_132435/setup_scripts/chronik-client_websocket_0
2024-08-01T13:25:47.011000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240801_132435/setup_scripts/chronik-client_websocket_0/test_framework.log
2024-08-01T13:25:47.011000Z TestFramework (ERROR): 
2024-08-01T13:25:47.012000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240801_132435/setup_scripts/chronik-client_websocket_0' to consolidate all logs
2024-08-01T13:25:47.012000Z TestFramework (ERROR): 
2024-08-01T13:25:47.012000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-08-01T13:25:47.012000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-08-01T13:25:47.013000Z TestFramework (ERROR): 
Running Unit Tests for Test Framework Modules
[1msetup_scripts/chronik-client_websocket.py[0m started
[1msetup_scripts/chronik-client_websocket.py[0m failed, Duration: 72 s

[1mstdout:[0m

[1mstderr:[0m

[1m
TEST                                      | STATUS    | DURATION

[0m[0;31msetup_scripts/chronik-client_websocket.py | ✖ Failed  | 72 s
[0m[0;31m[1m
ALL                                       | ✖ Failed  | 72 s (accumulated) 
[0m[0mRuntime: 72 s

Test runner for chronik-client_websocket completed with code 1
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |   28.35 |     8.64 |   24.57 |   28.35 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |    21.6 |     6.07 |    13.9 |   21.74 |                                   
  chronik.ts           |     6.1 |     0.83 |    2.54 |    6.09 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |   33.03 |    10.56 |    23.6 |   33.28 | ...,4991-5030,5038-5111,5146-5151 
 chronik-client/src    |   65.77 |    42.39 |   63.41 |   65.38 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 33-163,178-222,290-692            
  ChronikClientNode.ts |   90.84 |     72.8 |   96.38 |   90.84 | ...,1068,1078,1103,1115,1121,1127 
  failoverProxy.ts     |   75.22 |    51.61 |   62.06 |   74.52 | ...67,275-285,294,301,305,310,314 
  hex.ts               |   89.47 |       50 |      75 |   87.87 | 58,66-68                          
  validation.ts        |   93.33 |    81.81 |     100 |   92.59 | 33,39                             
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='1188']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='4189']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='337']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='3897']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='187']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='761']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='1176']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='4148']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1

rebase onto master

Harbormaster completed remote builds in B29839: Diff 49004.Aug 1 2024, 21:02

Fabien requested changes to this revision.Aug 4 2024, 08:19

Fabien added inline comments.

chronik/chronik-indexer/src/indexer.rs
214 ↗	(On Diff #49004)	if you invalidate the block you will start banning your peers also because your node thinks they are on an invalid chain
238 ↗	(On Diff #49004)
1162 ↗	(On Diff #49004)	Style: it's more a bit more readable imo: you are searching for the minimum so it seems more logical to use <
1190 ↗	(On Diff #49004)	dito
1213 ↗	(On Diff #49004)	Is there a way to improve the message a bit? If several plugins are out of sync you will use the one with the earliest tx num which is good, but also show it as the single unsynched plugin. Can you print the list of plugins instead (but still a single tx) ?

This revision now requires changes to proceed.Aug 4 2024, 08:19

address review

Fabien accepted this revision.Aug 6 2024, 14:39

This revision is now accepted and ready to land.Aug 6 2024, 14:39

Harbormaster completed remote builds in B29871: Diff 49055.Aug 6 2024, 14:45

Closed by commit rABCe3b90a9aea1b: [Chronik] Relax plugin height checks at startup (authored by tobias_ruck). · Explain WhyAug 6 2024, 16:25

This revision was automatically updated to reflect the committed changes.

tobias_ruck added a commit: rABCe3b90a9aea1b: [Chronik] Relax plugin height checks at startup.

[Chronik] Relax plugin height checks at startup
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 49061

chronik/chronik-db/src/io/txs.rs

chronik/chronik-indexer/src/indexer.rs

chronik/chronik-plugin-common/src/plugin.rs

test/functional/chronik_plugins_setup.py

[Chronik] Relax plugin height checks at startupClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 49061

chronik/chronik-db/src/io/txs.rs

chronik/chronik-indexer/src/indexer.rs

chronik/chronik-plugin-common/src/plugin.rs

test/functional/chronik_plugins_setup.py

[Chronik] Relax plugin height checks at startup
ClosedPublic
Actions

Revision Contents
Changeset List