Page MenuHomePhabricator

[chronik.e.cash] Docs site for chronik
ClosedPublic

Authored by bytesofman on Mar 22 2024, 19:40.

Details

Reviewers
tobias_ruck
Fabien
Group Reviewers
Restricted Project
Commits
rABCb4067fdeefd2: [chronik.e.cash] Docs site for chronik
Summary

Docusaurus site for chronik ready to deploy (and preview) at chronik.e.cash

Test Plan

Change line 25 of modules/Dockerfile to read ARG NGINX_CONF=nginx-preview.conf so that header settings specific to https are not present for test build

cd modules/
./build-docker
docker run --rm -p 8080:80 --name chronik-docs chronik-docs

Navigate to localhost:8080 and see working docs

  • Confirm infra and teamcity deployments are configured before landing (ref D15768)

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
This revision now requires changes to proceed.Mar 25 2024, 08:39

broken links, remove unused files

The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
Fabien requested changes to this revision.Mar 26 2024, 09:06
Fabien added a subscriber: Fabien.
Fabien added inline comments.
.arclint
250 ↗(On Diff #46464)

why ?

modules/.dockerignore
1 ↗(On Diff #46464)

this should be excluded in chronik-client/.gitignore and most likely at the upper level as well if not done already.

modules/Dockerfile
17 ↗(On Diff #46464)

why is it different from the previous step where you copied the whole repo at once ?

31 ↗(On Diff #46464)

don't expose in the docker file, you might want to redirect port 80 to 443 using a proxy. Leave that to the caller.

modules/docs/chronik.e.cash/README.md
8 ↗(On Diff #46464)

Do we need another dependency for this ? can't use npm ?

modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
20–38 ↗(On Diff #46464)

This needs to be updated for several reasons:

  1. The files are non longer using the experimental suffix
  2. It's now available for windows as well
  3. The use of the CMakeLists.txt file is wrong here. Because the cmake file contains a version number doesn't mean that the version has been released, so the files might not exist. There are a few options: redirecting to the release page on bitcoinabc.org, using the download.bitcoinabc.org/latest symling, calling the github API to get the latest version, ...

Also not using the CMakeLists.txt for that purpose lets you remove a lot of complexity, scripts and exceptions from this diff.

81 ↗(On Diff #46464)

It is

modules/docs/chronik.e.cash/docusaurus.config.js
112 ↗(On Diff #46464)

I think Bitcoin ABC has a twitter account

modules/docs/chronik.e.cash/package.json
2 ↗(On Diff #46464)

use a more explicit name like chronik-doc. This is NOT chronik

modules/docs/chronik.e.cash/prepbuild.sh
10 ↗(On Diff #46464)

This doesn't work, you can have an old version in place.

You should check if the files are different (e.g. with a sha256sum), see how it's done on CI for the osx sdk: contrib/teamcity/download-apple-sdk.sh

modules/docs/chronik.e.cash/sidebars.js
14 ↗(On Diff #46464)

This looks like a placeholder and can be removed

modules/docs/chronik.e.cash/src/css/custom.css
10 ↗(On Diff #46464)

dito

modules/docs/chronik.e.cash/src/theme/ReactLiveScope/index.tsx
9 ↗(On Diff #46464)

Needs to be updated with our own instance

This revision now requires changes to proceed.Mar 26 2024, 09:06
modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
20–38 ↗(On Diff #46464)

The file names for the latest versions on download.bitcoinabc.org/latest contain the version number, so somewhere we do need the version to build the URL.

I think the best is to have a script that updates a version at file every new release, depending on how the process for a new release works

modules/docs/chronik.e.cash/src/theme/ReactLiveScope/index.tsx
9 ↗(On Diff #46464)

and all other occurences as well

modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
20–38 ↗(On Diff #46464)

The version can be supplied externally, via an environment variable for example. The doc site itself doesn't need to determine the version.
The doc site will be rebuilt/redeployed for every release by the CI that knows what the version is. Another (sexy ?) option: add an endpoint for chronik to return the currently running version, so we can use the client itself to get it.

bytesofman marked 12 inline comments as done.

responding to feedback

.arclint
250 ↗(On Diff #46464)

This throws broken link errors for links that are not actually broken. This is because we have numbered files to maintain order of appearance in sidebar, but the links do not reference the numbers, since we do not want the numbers in our URLs.

I'm not sure how to disable specific rules for this linter.

modules/.dockerignore
1 ↗(On Diff #46464)

My understanding is that Docker does not do anything with the .gitignore file and this needs to be specified separately

https://docs.docker.com/build/building/context/#dockerignore-files

modules/Dockerfile
17 ↗(On Diff #46464)

due to the way the scripts are organized in chronik-client, running npm ci will also run npm run build ... but that will fail if all the files have not yet been copied over

modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
20–38 ↗(On Diff #46464)

As it stands, this is happening at build anyway. I don't think we need to hold up the docs site to wait for a new chronik endpoint.

So -- I can just add the windows link?

modules/docs/chronik.e.cash/docusaurus.config.js
112 ↗(On Diff #46464)

imo better to use the bigger one

modules/docs/chronik.e.cash/prepbuild.sh
10 ↗(On Diff #46464)

this just a convenience method for devs so that npm start and npm run build will work.

We are actually handling the copying in build-docker.sh -- if we run this to build the docker, as we will in prod, then all this script does is verify that the file exists and then do nothing.

If it tries to run in the docker context, it will fail, as there is no ../../../CmakeLists.txs in docker.

since we would like to avoid copying the file at all, imo premature to optimize it. it will be copied fresh every prod deployment by build-docker.sh

modules/docs/chronik.e.cash/sidebars.js
14 ↗(On Diff #46464)

the boilerplate is useful if we ever want to change the sidebars.

the defaults below are being used in docusaurus.config.js

modules/docs/chronik.e.cash/src/css/custom.css
10 ↗(On Diff #46464)

boilerplate is useful for maintaining a generated-ish repo, e.g. you can grep for it while referencing docs or stackoverflow

The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.

rebase for chronik-client ci tests

Fabien requested changes to this revision.Apr 2 2024, 07:51
Fabien added inline comments.
modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
20–38 ↗(On Diff #46464)

You can rename the files, add the windows link and go with the environment variable solution (with some default value for dev and preview like 0.0.0 or 99.99.99).
Parsing the cmake file is not a good solution anyway, it's brittle, complicated and prone to error. We can have a wrapper run ninja print-version and supply the result via the environment to the doc build at release deployment time

modules/docs/chronik.e.cash/prepbuild.sh
10 ↗(On Diff #46464)

OK with the complete removal of the file, so it's no longer an issue

This revision now requires changes to proceed.Apr 2 2024, 07:51

stop using cmakelist to get version

bytesofman added inline comments.
modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
20–38 ↗(On Diff #46464)

For surprising docusaurus related reasons, it is not trivial to get an env variable into a markdown file (at least, I was not able to figure it out easily).

However, it is easy to get this info from a file.

Instead of parsing it from cmakelist, I added a placeholder file .abclatestversion that is 0.0.0 for dev. If we can get the info from the build step in CI, the implementation is the same anyway (similar to how we build secrets.js file for telegram bot in herald).

Fabien requested changes to this revision.Apr 15 2024, 19:23

Minor things, it's getting there

modules/docs/chronik.e.cash/.abclatestversion
1 ↗(On Diff #47148)

Add this to the .gitignore so you don't publish with a dirty worktree

modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
18 ↗(On Diff #47148)

What happens in the file doesn't exist ? I expect it to fail with some error like "file not found".

Also you might want to document this file in README.

modules/docs/chronik.e.cash/static/.nojekyll
1 ↗(On Diff #46561)

What is this ?

This revision now requires changes to proceed.Apr 15 2024, 19:23
bytesofman marked 3 inline comments as done.

document .abclatest version in README, remove .abclatestversion from git repo with gitignore, include sample file

re: nojekyll, this is part of the docusaurus boilerplate:

https://github.com/facebook/docusaurus/issues/2892

modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
18 ↗(On Diff #47148)

build will fail

you can run npm run start but will get an error in console and overlayed on the browser

documented in README

Failed tests logs:

====== Get blocktxs, txs, and history for SLP NFT1 token txs: "before each" hook for "Gets an SLP NFT1 send tx from the mempool".Get blocktxs, txs, and history for SLP NFT1 token txs "before each" hook for "Gets an SLP NFT1 send tx from the mempool" ======
Error: Timeout of 60000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/work/modules/chronik-client/test/integration/token_slp_nft1.ts)
    at listOnTimeout (node:internal/timers:573:17)
    at processTimers (node:internal/timers:514:7)

Each failure log is accessible here:
Get blocktxs, txs, and history for SLP NFT1 token txs: "before each" hook for "Gets an SLP NFT1 send tx from the mempool".Get blocktxs, txs, and history for SLP NFT1 token txs "before each" hook for "Gets an SLP NFT1 send tx from the mempool"

rebase for ci re-run

Please refrain from doing this and actually try to fix the flaky test instead.

tobias_ruck added inline comments.
modules/Dockerfile
12 ↗(On Diff #47172)

the spaces are a bit weird here, I think there should only be one empty line here, and then add empty lines above # comments where it makes sense (you're already mostly doing that)

28 ↗(On Diff #47172)

you're not doing that?

modules/docs/chronik.e.cash/docs/0-chronik-setup/02-host-chronik.md
25 ↗(On Diff #47172)

I think we should also add the CORS stuff here, Nghia already ran into problems

Joey King, [3/18/24, 2:52:07 PM], on TG:

...- Only need add_header 'Access-Control-Allow-Methods' 'GET, POST' always;, not OPTIONS

  • This giant line —- add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With' always; — we could maybe make it shorter. It's just allowing all kinds of requests, which is the general idea

Or better this:

add_header "Access-Control-Allow-Origin"  *;
add_header "Access-Control-Allow-Methods" "GET, POST, OPTIONS, HEAD";
modules/docs/chronik.e.cash/nginx-preview.conf
1 ↗(On Diff #47172)

no copyright?

modules/docs/chronik.e.cash/nginx.conf
1 ↗(On Diff #47172)

copyright here could be added too

This revision now requires changes to proceed.Apr 19 2024, 22:01
modules/docs/chronik.e.cash/docs/0-chronik-setup/00-download.mdx
15 ↗(On Diff #47172)

this extra tab breaks it for me

modules/docs/chronik.e.cash/docs/0-chronik-setup/01-setup-chronik.md
15 ↗(On Diff #47172)

same here

modules/docs/chronik.e.cash/docs/0-chronik-setup/03-congratulations.mdx
44 ↗(On Diff #47172)

on night mode, this button looks like a backend dev designed it

bytesofman marked 4 inline comments as done.

add copyright to nginx, add header setting to hosting chronik documentation, fix dockerfile spaces

remove dockerfile comment about workdir move

Tail of the build log:

  File "/work/test/functional/setup_scripts/../test_framework/util.py", line 297, in wait_until_helper
    raise AssertionError(
AssertionError: Predicate ''''
            self.wait_until(lambda: is_finalblock(next_blockhash))
''' not true after 60.0 seconds
2024-04-19T22:33:05.214000Z TestFramework (INFO): Stopping nodes
2024-04-19T22:33:05.868000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240419_223152/setup_scripts/chronik-client_websocket_0
2024-04-19T22:33:05.868000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240419_223152/setup_scripts/chronik-client_websocket_0/test_framework.log
2024-04-19T22:33:05.868000Z TestFramework (ERROR): 
2024-04-19T22:33:05.868000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/chronik-client-integration-tests/test/tmp/test_runner_₿₵_🏃_20240419_223152/setup_scripts/chronik-client_websocket_0' to consolidate all logs
2024-04-19T22:33:05.868000Z TestFramework (ERROR): 
2024-04-19T22:33:05.868000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-04-19T22:33:05.869000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2024-04-19T22:33:05.869000Z TestFramework (ERROR): 
Running Unit Tests for Test Framework Modules
setup_scripts/chronik-client_websocket.py started
setup_scripts/chronik-client_websocket.py failed, Duration: 74 s

stdout:

stderr:


TEST                                      | STATUS    | DURATION

setup_scripts/chronik-client_websocket.py | ✖ Failed  | 74 s

ALL                                       | ✖ Failed  | 74 s (accumulated) 
Runtime: 74 s

Test runner for chronik-client_websocket completed with code 1
-----------------------|---------|----------|---------|---------|-----------------------------------
File                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                 
-----------------------|---------|----------|---------|---------|-----------------------------------
All files              |   27.47 |    10.47 |   23.06 |   27.49 |                                   
 chronik-client        |     100 |      100 |     100 |     100 |                                   
  index.ts             |     100 |      100 |     100 |     100 |                                   
 chronik-client/proto  |   21.54 |     7.98 |   13.72 |   21.69 |                                   
  chronik.ts           |     6.1 |        1 |    2.54 |    6.09 | ...,3978-3985,3990-4027,4031-4036 
  chronikNode.ts       |   33.15 |    13.34 |   23.49 |    33.4 | ...,4882-4921,4929-5002,5037-5042 
 chronik-client/src    |    63.1 |    42.72 |   60.54 |   62.74 |                                   
  ChronikClient.ts     |    4.24 |        0 |       0 |    4.29 | 33-163,178-222,290-692            
  ChronikClientNode.ts |   90.07 |    70.89 |   95.71 |   90.07 | ...52,876,888,898,923,935,941,947 
  failoverProxy.ts     |   76.19 |    59.09 |   66.66 |   75.49 | ...61-264,267,275-285,294,303,307 
  hex.ts               |   89.47 |       75 |      75 |   87.87 | 58,66-68                          
  validation.ts        |      90 |    84.61 |     100 |   89.47 | 31,37                             
-----------------------|---------|----------|---------|---------|-----------------------------------

##teamcity[blockOpened name='Code Coverage Summary']
##teamcity[buildStatisticValue key='CodeCoverageAbsBCovered' value='1125']
##teamcity[buildStatisticValue key='CodeCoverageAbsBTotal' value='4095']
##teamcity[buildStatisticValue key='CodeCoverageAbsRCovered' value='463']
##teamcity[buildStatisticValue key='CodeCoverageAbsRTotal' value='4422']
##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='170']
##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='737']
##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='1115']
##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='4056']
##teamcity[blockClosed name='Code Coverage Summary']
mv: cannot stat 'test_results/chronik-client-integration-tests-junit.xml': No such file or directory
Build chronik-client-integration-tests failed with exit code 1

touch up the try it out button

tobias_ruck added inline comments.
modules/docs/chronik.e.cash/docs/0-chronik-setup/03-congratulations.scss
26

grafik.png (350×600 px, 283 KB)

This revision is now accepted and ready to land.Apr 20 2024, 09:50
This revision was automatically updated to reflect the committed changes.