Page MenuHomePhabricator

[Chronik] Use `socket2` to enable SO_REUSEADDR on TCP connections
AbandonedPublic

Authored by tobias_ruck on Feb 19 2024, 11:52.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Summary

This may prevent some functional tests from failing.

Depends on D15486.

Test Plan

ninja check-functional

Diff Detail

Repository
rABC Bitcoin ABC
Branch
chronik-tcp-reuseaddress
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 27252
Build 54067: Build Diffchronik-client-integration-tests · build-chronik · build-chronik-plugins
Build 54066: arc lint + arc unit

Event Timeline

Tail of the build log:

    Checking url v2.4.0
   Compiling pyo3-ffi v0.20.2
    Checking stable-eyre v0.2.2
    Checking chronik-util v0.1.0 (/work/chronik/chronik-util)
    Checking abc-rust-error v0.1.0 (/work/chronik/abc-rust-error)
    Checking sha1 v0.10.5
    Checking bitcoinsuite-slp v0.1.0 (/work/chronik/bitcoinsuite-slp)
    Checking data-encoding v2.4.0
   Compiling prost v0.11.9
    Checking tower-service v0.3.2
    Checking tower-layer v0.3.2
    Checking cobs v0.2.3
    Checking ryu v1.0.15
    Checking utf-8 v0.7.6
    Checking httpdate v1.0.3
    Checking postcard v1.0.6
    Checking pin-project v1.1.3
    Checking tungstenite v0.21.0
    Checking http-body-util v0.1.0
    Checking mime v0.3.17
   Compiling prost-types v0.11.9
    Checking topo_sort v0.4.0
    Checking bimap v0.6.3
    Checking sync_wrapper v0.1.2
    Checking seahash v4.1.0
    Checking smallvec v1.13.1
    Checking axum-core v0.4.3
    Checking serde_json v1.0.105
    Checking serde_urlencoded v0.7.1
    Checking futures-executor v0.3.28
    Checking parking_lot_core v0.9.9
    Checking serde_path_to_error v0.1.14
   Compiling pyo3 v0.20.2
    Checking base64 v0.21.2
    Checking matchit v0.7.2
    Checking parking_lot v0.12.1
    Checking futures v0.3.28
    Checking memoffset v0.9.0
    Checking unindent v0.2.3
    Checking chronik-plugin v0.1.0 (/work/chronik/chronik-plugin)
   Compiling prost-build v0.11.9
   Compiling chronik-bridge v0.1.0 (/work/chronik/chronik-bridge)
   Compiling chronik-lib v0.1.0 (/work/chronik/chronik-lib)
    Checking tokio-util v0.7.10
    Checking tower v0.4.13
    Checking tokio-tungstenite v0.21.0
   Compiling chronik-proto v0.1.0 (/work/chronik/chronik-proto)
    Checking h2 v0.4.2
   Compiling librocksdb-sys v0.11.0+8.1.1
    Checking chronik-plugin-impl v0.1.0 (/work/chronik/chronik-plugin-impl)
    Checking hyper v1.1.0
    Checking hyper-util v0.1.3
    Checking axum v0.7.4
    Checking rocksdb v0.21.0
    Checking chronik-db v0.1.0 (/work/chronik/chronik-db)
    Checking chronik-indexer v0.1.0 (/work/chronik/chronik-indexer)
    Checking chronik-http v0.1.0 (/work/chronik/chronik-http)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 20s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik failed with exit code 1

Tail of the build log:

test chronik/bitcoinsuite-core/src/script/pubkey.rs - script::pubkey::PubKey::hex (line 57) ... ok
test chronik/bitcoinsuite-core/src/script/pubkey.rs - script::pubkey::PubKey::as_slice (line 37) ... ok
test chronik/bitcoinsuite-core/src/script/pubkey_variant.rs - script::pubkey_variant::PubKeyVariant (line 14) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::bytecode (line 126) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::is_opreturn (line 159) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::hex (line 148) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::p2pk (line 76) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::iter_ops (line 174) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::p2pk_uncompressed (line 98) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::p2pkh (line 36) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::p2sh (line 56) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::to_vec (line 137) ... ok
test chronik/bitcoinsuite-core/src/script/script_mut.rs - script::script_mut::ScriptMut::freeze (line 135) ... ok
test chronik/bitcoinsuite-core/src/script/script.rs - script::script::Script::variant (line 223) ... ok
test chronik/bitcoinsuite-core/src/script/script_mut.rs - script::script_mut::ScriptMut::put_bytecode (line 44) ... ok
test chronik/bitcoinsuite-core/src/script/script_mut.rs - script::script_mut::ScriptMut::put_opcodes (line 28) ... ok
test chronik/bitcoinsuite-core/src/script/script_mut.rs - script::script_mut::ScriptMut::with_capacity (line 17) ... ok
test chronik/bitcoinsuite-core/src/tx/tx.rs - tx::tx::Tx (line 34) - compile fail ... ok
test chronik/bitcoinsuite-core/src/script/script_mut.rs - script::script_mut::ScriptMut::put_slp_pushdata (line 116) ... ok
test chronik/bitcoinsuite-core/src/script/script_mut.rs - script::script_mut::ScriptMut::put_pushdata (line 58) ... ok
test chronik/bitcoinsuite-core/src/script/uncompressed_pubkey.rs - script::uncompressed_pubkey::UncompressedPubKey (line 13) ... ok
test chronik/bitcoinsuite-core/src/script/uncompressed_pubkey.rs - script::uncompressed_pubkey::UncompressedPubKey::array (line 50) ... ok
test chronik/bitcoinsuite-core/src/script/variant.rs - script::variant::ScriptVariant::from_type_and_payload (line 94) ... ok
test chronik/bitcoinsuite-core/src/script/uncompressed_pubkey.rs - script::uncompressed_pubkey::UncompressedPubKey::as_slice (line 40) ... ok
test chronik/bitcoinsuite-core/src/script/uncompressed_pubkey.rs - script::uncompressed_pubkey::UncompressedPubKey::hex (line 60) ... ok
test chronik/bitcoinsuite-core/src/tx/txid.rs - tx::txid::TxId::new (line 47) ... ok
test chronik/bitcoinsuite-core/src/tx/txid.rs - tx::txid::TxId::as_bytes (line 89) ... ok
test chronik/bitcoinsuite-core/src/script/variant.rs - script::variant::ScriptVariant::to_script (line 108) ... ok
test chronik/bitcoinsuite-core/src/tx/txid.rs - tx::txid::TxId::to_bytes (line 74) ... ok
test chronik/bitcoinsuite-core/src/tx/txid.rs - tx::txid::TxId::to_vec (line 104) ... ok
test chronik/bitcoinsuite-core/src/tx/tx.rs - tx::tx::Tx (line 14) ... ok
test chronik/bitcoinsuite-core/src/tx/txid.rs - tx::txid::TxId::from_tx (line 59) ... ok

test result: ok. 57 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.03s

   Doc-tests bitcoinsuite-slp

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

[6/6] cd /work && /usr/bin/cmake -E env CARGO_TARGET_DIR="/work/abc-ci-builds/build-chronik-plugins/cargo/build" CARGO_BUILD_RUSTC="/root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/rustc" CARGO_BUILD_RUSTDOC="/root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/rustdoc" /root/.rustup/toolchains/1.72.0-x86_64-unknown-linux-gnu/bin/cargo --locked clippy --package bitcoinsuite-* -- -D warnings
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on build directory
    Checking either v1.9.0
    Checking serde v1.0.185
    Checking thiserror v1.0.47
    Checking itertools v0.10.5
    Checking bytes v1.5.0
    Checking bitcoinsuite-core v0.1.0 (/work/chronik/bitcoinsuite-core)
    Checking bitcoinsuite-slp v0.1.0 (/work/chronik/bitcoinsuite-slp)
    Finished dev [unoptimized + debuginfo] target(s) in 4m 57s
ninja: build stopped: cannot make progress due to previous errors.
Build build-chronik-plugins failed with exit code 1

Failed tests logs:

====== /block and /blocks: gives us the block and blocks./block and /blocks gives us the block and blocks ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /block and /blocks: gives us the block at 10 higher./block and /blocks gives us the block at 10 higher ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /block and /blocks: gives us the block after parking the last block and throws expected error attempting to get parked block./block and /blocks gives us the block after parking the last block and throws expected error attempting to get parked block ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /block and /blocks: gives us the block and blocks after unparking the last block./block and /blocks gives us the block and blocks after unparking the last block ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /block and /blocks: gives us the block and blocks after invalidating the last block and throws expected error attempting to get invalidated block./block and /blocks gives us the block and blocks after invalidating the last block and throws expected error attempting to get invalidated block ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /block and /blocks: gives us the block and blocks after reconsiderblock called on the last block./block and /blocks gives us the block and blocks after reconsiderblock called on the last block ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /blockchain-info: gives us the blockchain info + throws expected error on bad connection./blockchain-info gives us the blockchain info + throws expected error on bad connection ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /blockchain-info: gives us the blockchain info with 10 more blocks./blockchain-info gives us the blockchain info with 10 more blocks ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /blockchain-info: gives us the blockchain info with again 10 more blocks./blockchain-info gives us the blockchain info with again 10 more blocks ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /blockchain-info: gives us the blockchain info after parking the last block./blockchain-info gives us the blockchain info after parking the last block ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== /blockchain-info: gives us the blockchain info after unparking the last block./blockchain-info gives us the blockchain info after unparking the last block ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get blocktxs and tx: New regtest chain.Get blocktxs and tx New regtest chain ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get blocktxs and tx: After some txs have been broadcast.Get blocktxs and tx After some txs have been broadcast ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get blocktxs and tx: After these txs are mined.Get blocktxs and tx After these txs are mined ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get blocktxs and tx: After this mined block has been parked.Get blocktxs and tx After this mined block has been parked ======
AssertionError: expected promise to be rejected with an error including 'Failed getting /block-txs/201?page=0&…' but got 'Error connecting to known Chronik ins…'

      + expected - actual

      -Error connecting to known Chronik instances
      +Failed getting /block-txs/201?page=0&page_size=25 (): 404: Block not found: 201
====== Test broadcastTx and broadcastTxs methods from ChronikClientNode: New regtest chain.Test broadcastTx and broadcastTxs methods from ChronikClientNode New regtest chain ======
AssertionError: expected promise to be rejected with an error including 'Failed getting /broadcast-tx (): 400:…' but got 'Error connecting to known Chronik ins…'

      + expected - actual

      -Error connecting to known Chronik instances
      +Failed getting /broadcast-tx (): 400: Broadcast failed: Transaction rejected by mempool: bad-txns-in-belowout, value in (25000000.00) < value out (49999999600.00)
====== Test broadcastTx and broadcastTxs methods from ChronikClientNode: After broadcastTxs are mined.Test broadcastTx and broadcastTxs methods from ChronikClientNode After broadcastTxs are mined ======
AssertionError: expected promise to be rejected with an error including 'Failed getting /broadcast-tx (): 400:…' but got 'Error connecting to known Chronik ins…'

      + expected - actual

      -Error connecting to known Chronik instances
      +Failed getting /broadcast-tx (): 400: Broadcast failed: Transaction already in block chain
====== /chronik-info: gives us the chronik info and throws expected error on bad server connection./chronik-info gives us the chronik info and throws expected error on bad server connection ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get script().history and script().utxos(): New regtest chain.Get script().history and script().utxos() New regtest chain ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get script().history and script().utxos(): After some txs have been broadcast.Get script().history and script().utxos() After some txs have been broadcast ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get script().history and script().utxos(): After these txs are mined.Get script().history and script().utxos() After these txs are mined ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get script().history and script().utxos(): After these txs are avalanche finalized.Get script().history and script().utxos() After these txs are avalanche finalized ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
====== Get script().history and script().utxos(): After a tx is broadcast with outputs of each type.Get script().history and script().utxos() After a tx is broadcast with outputs of each type ======
Error: Error connecting to known Chronik instances
    at FailoverProxy.<anonymous> (src/failoverProxy.ts:34:32)
    at Generator.throw (<anonymous>)
    at rejected (src/failoverProxy.ts:2:3638)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

Each failure log is accessible here:
/block and /blocks: gives us the block and blocks./block and /blocks gives us the block and blocks
/block and /blocks: gives us the block at 10 higher./block and /blocks gives us the block at 10 higher
/block and /blocks: gives us the block after parking the last block and throws expected error attempting to get parked block./block and /blocks gives us the block after parking the last block and throws expected error attempting to get parked block
/block and /blocks: gives us the block and blocks after unparking the last block./block and /blocks gives us the block and blocks after unparking the last block
/block and /blocks: gives us the block and blocks after invalidating the last block and throws expected error attempting to get invalidated block./block and /blocks gives us the block and blocks after invalidating the last block and throws expected error attempting to get invalidated block
/block and /blocks: gives us the block and blocks after reconsiderblock called on the last block./block and /blocks gives us the block and blocks after reconsiderblock called on the last block
/blockchain-info: gives us the blockchain info + throws expected error on bad connection./blockchain-info gives us the blockchain info + throws expected error on bad connection
/blockchain-info: gives us the blockchain info with 10 more blocks./blockchain-info gives us the blockchain info with 10 more blocks
/blockchain-info: gives us the blockchain info with again 10 more blocks./blockchain-info gives us the blockchain info with again 10 more blocks
/blockchain-info: gives us the blockchain info after parking the last block./blockchain-info gives us the blockchain info after parking the last block
/blockchain-info: gives us the blockchain info after unparking the last block./blockchain-info gives us the blockchain info after unparking the last block
Get blocktxs and tx: New regtest chain.Get blocktxs and tx New regtest chain
Get blocktxs and tx: After some txs have been broadcast.Get blocktxs and tx After some txs have been broadcast
Get blocktxs and tx: After these txs are mined.Get blocktxs and tx After these txs are mined
Get blocktxs and tx: After this mined block has been parked.Get blocktxs and tx After this mined block has been parked
Test broadcastTx and broadcastTxs methods from ChronikClientNode: New regtest chain.Test broadcastTx and broadcastTxs methods from ChronikClientNode New regtest chain
Test broadcastTx and broadcastTxs methods from ChronikClientNode: After broadcastTxs are mined.Test broadcastTx and broadcastTxs methods from ChronikClientNode After broadcastTxs are mined
/chronik-info: gives us the chronik info and throws expected error on bad server connection./chronik-info gives us the chronik info and throws expected error on bad server connection
Get script().history and script().utxos(): New regtest chain.Get script().history and script().utxos() New regtest chain
Get script().history and script().utxos(): After some txs have been broadcast.Get script().history and script().utxos() After some txs have been broadcast
Get script().history and script().utxos(): After these txs are mined.Get script().history and script().utxos() After these txs are mined
Get script().history and script().utxos(): After these txs are avalanche finalized.Get script().history and script().utxos() After these txs are avalanche finalized
Get script().history and script().utxos(): After a tx is broadcast with outputs of each type.Get script().history and script().utxos() After a tx is broadcast with outputs of each type

rebase, fix config/bind ordering

The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.

rebase onto master + restart CI

should work now, build failure seems unrelated

The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.
The build failed due to an unexpected infrastructure outage. The administrators have been notified to investigate. Sorry for the inconvenience.

seems like Rust is already enabling SO_REUSEADDR anyway: https://doc.rust-lang.org/nightly/src/std/sys_common/net.rs.html#408, so this doesn't help

chronik/chronik-http/src/server.rs
117

in case this ever gets picked up again, this fixes the test