Page MenuHomePhabricator

[Chronik] Add `PluginsWriter::update_sync_height`, enforce plugins are synced
ClosedPublic

Authored by tobias_ruck on Jul 27 2024, 15:05.

Details

Summary

Currently, if a plugin desyncs with the DB, nothing happens, but this could lead to an inconsistent state of Chronik, e.g. where a plugin seems to be up-to-date but actually skipped a few blocks resulting in wrong data being reported to the user.

To avoid this, we previously already added a field sync_height to store the plugin's synced height, but it was unused. In this diff, we enforce that all plugins must be synced at startup, or instruct the user to either reindex Chronik or disable the plugin.

In the future, we can automatically reindex only the transactions that are relevant for the plugin using the LOKAD ID index, but since this is non-trivial this will be implemented later. Prioritizing getting a plugin system with mediocre UX out sooner.

Depends on D16532.

Test Plan

cargo test -p chronik-db && ./test/functional/test_runner.py chronik_plugins_setup

Event Timeline

initialize to -1 by default

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/util.py", line 296, in wait_until_helper
    raise AssertionError(
AssertionError: Predicate ''''
            self.wait_until(lambda: is_finalblock(next_blockhash))
''' not true after 60.0 seconds
2024-07-27T15:44:44.937000Z TestFramework (INFO): Stopping nodes
2024-07-27T15:44:45.191000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240727_154332/setup_scripts/chronik-client_websocket_0
2024-07-27T15:44:45.191000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240727_154332/setup_scripts/chronik-client_websocket_0/test_framework.log
2024-07-27T15:44:45.192000Z TestFramework (ERROR): 
2024-07-27T15:44:45.192000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240727_154332/setup_scripts/chronik-client_websocket_0' to consolidate all logs
2024-07-27T15:44:45.192000Z TestFramework (ERROR): 
2024-07-27T15:44:45.192000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-07-27T15:44:45.192000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-07-27T15:44:45.192000Z TestFramework (ERROR): 
Running Unit Tests for Test Framework Modules
setup_scripts/chronik-client_websocket.py started
setup_scripts/chronik-client_websocket.py failed, Duration: 73 s

stdout:

stderr:


TEST                                      | STATUS    | DURATION

setup_scripts/chronik-client_websocket.py | ✖ Failed  | 73 s

ALL                                       | ✖ Failed  | 73 s (accumulated) 
Runtime: 73 s

Test runner for chronik-client_websocket completed with code 1
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |   28.35 |     8.64 |   24.57 |   28.35 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |    21.6 |     6.07 |    13.9 |   21.74 |                                   
  chronik.ts           |     6.1 |     0.83 |    2.54 |    6.09 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |   33.03 |    10.56 |    23.6 |   33.28 | ...,4991-5030,5038-5111,5146-5151 
 chronik-client/src    |   65.77 |    42.39 |   63.41 |   65.38 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 33-163,178-222,290-692            
  ChronikClientNode.ts |   90.84 |     72.8 |   96.38 |   90.84 | ...,1068,1078,1103,1115,1121,1127 
  failoverProxy.ts     |   75.22 |    51.61 |   62.06 |   74.52 | ...67,275-285,294,301,305,310,314 
  hex.ts               |   89.47 |       50 |      75 |   87.87 | 58,66-68                          
  validation.ts        |   93.33 |    81.81 |     100 |   92.59 | 33,39                             
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='1188']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='4189']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='337']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='3897']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='187']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='761']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='1176']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='4148']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1

plugin_writer -> plugins_writer

This revision is now accepted and ready to land.Jul 29 2024, 07:29