Page MenuHomePhabricator

[ecash-herald, mock-chronik-client, alias-server] Migrate ecash-herald to in-node chronik-client and add supporting mock-chronik-client methods
ClosedPublic

Authored by bytesofman on May 15 2024, 22:50.

Details

Summary

Migrate ecash-herald to in-node chronik-client

This diff is intended to maintain existing functionality of existing ecash-herald with as little refactoring as possible. New in-node chronik-client allows for many improvements to ecash-herald that will be handled in other refactors.

ALP txs now appear under "eToken sends" and not EMPP

Currently, we can detect a block that is found but rejected by avalanche bc the herald will send an error msg -- the block connects, but is not indexed, so the herald's chronik.block call fails and we see a default msg. ABC devs get what is going on but this is not clear to users.

I've added caching to preserve this alert feature. When a block is connected, ecash-herald will wait 10s. If no finalized msg comes in, a "block connected but not finalized by avalanche for 10s" is sent.

I added unit tests for this behavior and also got the msg to send testing prod by waiting 10ms instead of 10s:

image.png (201×644 px, 27 KB)

In practical testing, the BLK_FINALIZED msg seems to come in only a few seconds after the BLK_CONNECTED msg.

Some adjustments are made to alias-server unit tests because the mock-chronik-client change breaks them. Imo this removal is justified because

  • the behavior tested in alias-server (ws subscription) is not important to test...really it's testing the dependency.
  • alias-server will need to implement in-node chronik if alias path fwd still requires its use. in this case, it will need the mock-chronik-client changes introduced in this diff, just like the herald did.
  • does not make sense to implement in-node chronik-client in alias-server in this diff just to preserve these tests, the behavior change is expected. implementing in-node chronik-client in alias-server should be its own diff.
Test Plan

npm test, CI for mock-chronik-client

Test block msg from nng:

image.png (769×634 px, 113 KB)

image.png (613×634 px, 141 KB)

Test block msg now:

image.png (740×613 px, 97 KB)

image.png (655×637 px, 157 KB)

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

update mock syntax to match in-node chronik, add token genesisinfo to block mock, remove debug logs

update mock-chronik-client for in-node ws subs, update herald for correct in-node ws subscription

Failed tests logs:

====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',       scriptType: 'p2pkh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',     scriptType: 'p2pkh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
      scriptType: 'p2pkh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
    scriptType: 'p2pkh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:110:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:140:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/main.test.js:117:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff

Each failure log is accessible here:
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address
alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly

Failed tests logs:

====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',       scriptType: 'p2pkh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',     scriptType: 'p2pkh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
      scriptType: 'p2pkh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
    scriptType: 'p2pkh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:110:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:140:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/main.test.js:117:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff

Each failure log is accessible here:
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address
alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly

patch script to also accept cache param

Failed tests logs:

====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',       scriptType: 'p2pkh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',     scriptType: 'p2pkh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
      scriptType: 'p2pkh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
    scriptType: 'p2pkh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:110:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:140:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/main.test.js:117:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff

Each failure log is accessible here:
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address
alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly

call main with correct params

Failed tests logs:

====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',       scriptType: 'p2pkh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',     scriptType: 'p2pkh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
      scriptType: 'p2pkh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
    scriptType: 'p2pkh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:110:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:140:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/main.test.js:117:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff

Each failure log is accessible here:
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address
alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly

add tests for the case of unfinalized blocks

Failed tests logs:

====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',       scriptType: 'p2pkh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',     scriptType: 'p2pkh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
      scriptType: 'p2pkh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: '638568e36d0b5d7d49a6e99854caa27d9772b093',
    scriptType: 'p2pkh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:110:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/chronikWsHandler.test.js:140:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff
====== alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly ======
AssertionError: Expected values to be loosely deep-equal:  {   blocks: false,   lokadIds: [],   scripts: [     {       scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',       scriptType: 'p2sh'     }   ],   tokens: [] }  should loosely deep-equal  [   {     scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',     scriptType: 'p2sh'   } ]
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:

{
  blocks: false,
  lokadIds: [],
  scripts: [
    {
      scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
      scriptType: 'p2sh'
    }
  ],
  tokens: []
}

should loosely deep-equal

[
  {
    scriptPayload: 'd37c4c809fe9840e7bfa77b86bd47163f6fb6c60',
    scriptType: 'p2sh'
  }
]
    at Context.<anonymous> (test/main.test.js:117:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

      + expected - actual:  failed to generate Mocha diff

Each failure log is accessible here:
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2pkh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2pkh address
alias-server chronikWsHandler.js: initializeWebsocket returns expected websocket object for a p2sh address.alias-server chronikWsHandler.js initializeWebsocket returns expected websocket object for a p2sh address
alias-server main.js: main() connects to a websocket, and runs handleAppStartup() correctly.alias-server main.js main() connects to a websocket, and runs handleAppStartup() correctly

remove debug logs, update alias-server tests

bytesofman edited the test plan for this revision. (Show Details)

remove dev logs and comments, do not double version bump mock-chronik-client

bytesofman retitled this revision from [ecash-herald, mock-chronik-client] Migrate to in-node chronik-client to [ecash-herald, mock-chronik-client, alias-server] Migrate ecash-herald to in-node chronik-client and add supporting mock-chronik-client methods.Wed, May 22, 23:57
bytesofman added inline comments.
apps/alias-server/test/chronikWsHandler.test.js
110 ↗(On Diff #47914)

this shape is changed now for in-node chronik-client

However, we do not really need to test this -- it is more testing chronik-client expected behavior than alias-server expected behavior.

see comments in diff description explaining alteration of alias-server tests in this diff.

apps/alias-server/test/main.test.js
117 ↗(On Diff #47914)

ditto

apps/ecash-herald/config.js
7 ↗(On Diff #47914)

settled on this param for amount of time to wait for an avalanche finalization msg.

If we get BLK_CONNECTED, but do not get BLK_FINALIZED within waitForFinalizationMsecs, herald will send a msg explaining this.

This should preserve ecash-herald's current ability to alert for avalanche-rejected blocks.

In practice with debug logging, the finalized msg seems to come only a few seconds after the connected msg. I have not extensively tested for a good value. 30s is definitely too long. 5s may be appropriate. 10s is not really a human-noticeable delay and provides some confidence the block is really not finalized.

apps/ecash-herald/constants/tokens.js
17 ↗(On Diff #47914)

change shape of cache to match chronik.token output, since ALP is indexed here by in-node chronik

apps/ecash-herald/index.js
18 ↗(On Diff #47914)

NNG chronik has no subscribeToBlocks method. Need to subscribe to an address and then you consequently are subscribed to blocks.

We no longer need this shim.

apps/ecash-herald/package.json
28 ↗(On Diff #47914)

not a performance critical thing. used to gauge whether or not a connected block is finalized within expected interval.

https://www.npmjs.com/package/cache-manager

standard JS dep review process...this one has fewer downloads than but was updated 23 days ago and still good download amount

incredibly, this one has more than double the downloads and hasn't been updated in 4 years: https://www.npmjs.com/package/node-cache

also twice the size

JS things

35 ↗(On Diff #47914)

this is used to test the cache / timeout for block connected -> finalized. dev dep only.

PiRK added inline comments.
apps/ecash-herald/src/events.js
136–137 ↗(On Diff #47914)

Is there a guarantee by Chronik that the block finalized message will always come after the block connected message? I imagine there is, but I'm not certain. Otherwise it might be safer to first check if the cache is already set to BLK_FINALIZED and bail out immediately.

apps/ecash-herald/src/events.js
146 ↗(On Diff #47914)

nit: you can flip the logic and return here if the status is FINALIZED, to remove an indentation level for the rest of the function.

This would not print out messages for blocks that are never finalized because another block with the same height is found within a few seconds of the first and the second block ends up being finalized less than 10s after the first one is connected. Not sure if we want to see these messages, and if we do whether this should be handled in this diff or in a follow-up

The fix for this would probably be as simple as indexing the cache by "height+hash" instead of just "height"

Fabien requested changes to this revision.Fri, May 24, 14:45
Fabien added a subscriber: Fabien.

I'm not sure about the finalization thing, being time dependent looks like a bad idea as you will always get false positives/negatives.

I suggest you don't handle finalization at all in this diff, so you get feature parity with the nng version, and add the finalization in follow up diffs.

apps/ecash-herald/config.js
10 ↗(On Diff #47915)

You're missing the only (for now) Bitcoin ABC server...

This revision now requires changes to proceed.Fri, May 24, 14:45
bytesofman marked 2 inline comments as done.

add abc chronik server, improve blk connected logic

bytesofman added inline comments.
apps/ecash-herald/src/events.js
136–137 ↗(On Diff #47914)

Checked with Tobias, who says chronik follows the node here -- I'm not sure but I imagine it's impossible for a block to be finalized before it is connected.

146 ↗(On Diff #47914)

nice, this is much better.

set and get cache by height and hash

In D16171#367284, @PiRK wrote:

This would not print out messages for blocks that are never finalized because another block with the same height is found within a few seconds of the first and the second block ends up being finalized less than 10s after the first one is connected. Not sure if we want to see these messages, and if we do whether this should be handled in this diff or in a follow-up

The fix for this would probably be as simple as indexing the cache by "height+hash" instead of just "height"

good point, updated

I'm not sure about the finalization thing, being time dependent looks like a bad idea as you will always get false positives/negatives.

I suggest you don't handle finalization at all in this diff, so you get feature parity with the nng version, and add the finalization in follow up diffs.

I think the justification for keeping it in is that we would learn more about the blockchain. even with this type of failure, it still improves the herald as an always-on push-notification blockchain monitoring service.

Say we get some blocks that connect and do finalize, but it takes 12s, mb this happens every 500 blocks. Would be useful to learn that, useful to check these blocks out and see if anything interesting caused this.

Mb 1 in 3000 blocks connects and does finalize, but it takes 30s. Would also be interesting to learn this.

Since the failure mode here is bad (but investigatable) msgs, I think it's more a value add than a performance issue.

Fabien requested changes to this revision.Mon, May 27, 07:29

I'm not sure about the finalization thing, being time dependent looks like a bad idea as you will always get false positives/negatives.

I suggest you don't handle finalization at all in this diff, so you get feature parity with the nng version, and add the finalization in follow up diffs.

I think the justification for keeping it in is that we would learn more about the blockchain. even with this type of failure, it still improves the herald as an always-on push-notification blockchain monitoring service.

Say we get some blocks that connect and do finalize, but it takes 12s, mb this happens every 500 blocks. Would be useful to learn that, useful to check these blocks out and see if anything interesting caused this.

Mb 1 in 3000 blocks connects and does finalize, but it takes 30s. Would also be interesting to learn this.

Since the failure mode here is bad (but investigatable) msgs, I think it's more a value add than a performance issue.

I agree that having a way to know if a block is final can be interesting, but that's not my point.
This diff is doing 2 things:

  • Migrate the in-node chronik
  • Add a new feature (finalization)

The first one is hard to revert because it implies a change of server. The second one not so much. So if something is broken or undesirable in practice due to finalization you won't be able to simple revert the diff.

This is why you should split this diff in 2 parts.

This revision now requires changes to proceed.Mon, May 27, 07:29

I'm not sure about the finalization thing, being time dependent looks like a bad idea as you will always get false positives/negatives.

I suggest you don't handle finalization at all in this diff, so you get feature parity with the nng version, and add the finalization in follow up diffs.

I think the justification for keeping it in is that we would learn more about the blockchain. even with this type of failure, it still improves the herald as an always-on push-notification blockchain monitoring service.

Say we get some blocks that connect and do finalize, but it takes 12s, mb this happens every 500 blocks. Would be useful to learn that, useful to check these blocks out and see if anything interesting caused this.

Mb 1 in 3000 blocks connects and does finalize, but it takes 30s. Would also be interesting to learn this.

Since the failure mode here is bad (but investigatable) msgs, I think it's more a value add than a performance issue.

I agree that having a way to know if a block is final can be interesting, but that's not my point.
This diff is doing 2 things:

  • Migrate the in-node chronik
  • Add a new feature (finalization)

The first one is hard to revert because it implies a change of server. The second one not so much. So if something is broken or undesirable in practice due to finalization you won't be able to simple revert the diff.

This is why you should split this diff in 2 parts.

This is true and the diff should have been built this way.

At this point, I do not think we are getting good impact from the exercise (refactoring this diff to remove finalization handling with maintained block connected alerts, with the plan being to immediately add them again). The main value to the herald of switching to in-node is the availability of finalization msgs.

Because ecash-herald is not critical infra and bugs and downtime here would not involve risk beyond temporary annoyance, I would prefer to iterate through any issues here vs revert and redo this (tedious + low impact -- but necessary + one-off) indexer migration.

To mitigate any potential issues from behavior change in this migration, I have restarted the bot on current master in the ecash herald dev tg channel, which will continue to run on NNG chronik until we are confident the prod version is sufficient to meet our needs.

Fabien added inline comments.
apps/ecash-herald/config.js
11 ↗(On Diff #47957)

put it first

apps/ecash-herald/src/events.js
50 ↗(On Diff #47957)

The hash was the useful info because you need it to investigate with a node. Since you have both the height and the hash here you can just print both

This revision is now accepted and ready to land.Tue, May 28, 08:28