Page MenuHomePhabricator

[avalanche] Add experimental difficulty penalty for blocks that arrive sooner than 2 minutes from the previous block
Changes PlannedPublic

Authored by sdulfari on Oct 25 2022, 23:57.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Summary

The current fast difficulty adjustment scheme can lead to perverse incentives giving rise short periods of turbo mining by aggressive mining pools. This results in long (>60 min) periods of no blocks when difficulty is high which makes it frustrating for users moving coins to exchanges where confirmations is still the standard. We can disincentivize this behavior with only small impact to steady miners.

This patch introduces an optional Avalanche post-consensus policy called Early Blocks Penalty. The scheme is simple: A difficulty penalty is applied to blocks that arrive within the penalty window from the previous block. If the block hash is not better than this new target, the block will be parked. Avalanche will determine finality. A majority of Avalanche stakers will need to enable this policy for it to be effective on mainnet.

Test Plan
ninja
./test/functional/test_runner.py abc_p2p_avalanche_policy_early_blocks

Remove any old debug log:

rm ~/.bitcoin/debug.log

Build and start:

ninja
bitcoind -daemon

Gather results:

cat ~/.bitcoin/debug.log | grep -B5 "Early block [ar]" ; echo "Early blocks accepted: $(cat ~/.bitcoin/debug.log | grep "Early block accepted" | wc -l)" ; echo "Early blocks rejected: $(cat ~/.bitcoin/debug.log | grep "Early block rejected" | wc -l)" ; ./bitcoin-cli getchaintips

The getchaintips output at the end should indicate that you have not forked yourself off.

Diff Detail

Event Timeline

  • Bump penalty factor to 10
  • Add -avapolicyearlyblockpenalty control flag
  • Minor refactor

Tail of the build log:

[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35952 -> phabricator/diff/35952
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35954 -> phabricator/diff/35954
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35955 -> phabricator/diff/35955
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35956 -> phabricator/diff/35956
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35957 -> phabricator/diff/35957
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35963 -> phabricator/diff/35963
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35969 -> phabricator/diff/35969
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35975 -> phabricator/diff/35975
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35976 -> phabricator/diff/35976
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35977 -> phabricator/diff/35977
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35978 -> phabricator/diff/35978
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35979 -> phabricator/diff/35979
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35980 -> phabricator/diff/35980
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35987 -> phabricator/diff/35987
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35988 -> phabricator/diff/35988
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/35995 -> phabricator/diff/35995
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/36000 -> phabricator/diff/36000
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/36036 -> phabricator/diff/36036
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/36067 -> phabricator/diff/36067
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/36123 -> phabricator/diff/36123
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/8992  -> phabricator/diff/8992
[15:43:45] :	 [Step 1/1]  * [new tag]             phabricator/diff/8993  -> phabricator/diff/8993
[15:43:45] :	 [Step 1/1]  * [new branch]          master                 -> master
[15:43:45] :	 [Step 1/1] --- Building for bullseye amd64 ---
[15:43:45] :	 [Step 1/1] Stopping target if it is up
[15:43:45] :	 [Step 1/1] Error response from daemon: No such container: gitian-target
[15:43:45] :	 [Step 1/1] Error: No such container: gitian-target
[15:43:45] :	 [Step 1/1] Making a new image copy
[15:43:45] :	 [Step 1/1] Starting target
[15:43:45] :	 [Step 1/1] Checking if target is up.
[15:43:45] :	 [Step 1/1] Preparing build environment
[15:43:45] :	 [Step 1/1] Updating apt-get repository (log in var/install.log)
[15:43:45] :	 [Step 1/1] Installing additional packages (log in var/install.log)
[15:43:45] :	 [Step 1/1] ./bin/gbuild:23:in `system!': failed to run on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install autoconf automake binutils-aarch64-linux-gnu binutils-arm-linux-gnueabihf binutils-gold bsdmainutils ca-certificates cmake curl faketime g++-9 g++-9-aarch64-linux-gnu g++-9-arm-linux-gnueabihf gcc-9 gcc-9-aarch64-linux-gnu gcc-9-arm-linux-gnueabihf git gperf libtool ninja-build pkg-config python3 >> var/install.log 2>&1 (RuntimeError)
[15:43:45] :	 [Step 1/1] 	from ./bin/gbuild:120:in `build_one_configuration'
[15:43:45] :	 [Step 1/1] 	from ./bin/gbuild:339:in `block (2 levels) in <main>'
[15:43:45] :	 [Step 1/1] 	from ./bin/gbuild:334:in `each'
[15:43:45] :	 [Step 1/1] 	from ./bin/gbuild:334:in `block in <main>'
[15:43:45] :	 [Step 1/1] 	from ./bin/gbuild:332:in `each'
[15:43:45] :	 [Step 1/1] 	from ./bin/gbuild:332:in `<main>'
[15:43:45] :	 [Step 1/1] mv: cannot stat 'var/build.log': No such file or directory
[15:43:45] :	 [Step 1/1] Build gitian-linux failed with exit code 1
[15:43:45]W:	 [Step 1/1] Process exited with code 1
[15:43:59]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[15:43:59] :	 [Step 1/1] Waiting for 1 service processes to complete
[15:43:59]E:	 [Step 1/1] Ant JUnit report watcher
[15:43:59]E:		 [Ant JUnit report watcher] No reports found for paths:
[15:43:59]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[15:43:59]E:	 [Step 1/1] Step Command Line failed
[15:44:00]E: Ant JUnit report watcher
[15:44:00]E:	 [Ant JUnit report watcher] No reports found for paths:
[15:44:00]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[15:44:00]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[15:44:00] : Publishing internal artifacts (4s)
[15:44:03] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[15:44:04] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[15:44:00]W: Publishing artifacts (4s)
[15:44:00] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[15:44:00]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[15:44:05] : Build finished

Failed tests logs:

====== Bitcoin ABC functional tests: abc_p2p_getavaaddr.py ======

------- Stdout: -------
2022-11-02T15:49:19.588000Z TestFramework (INFO): Initializing test directory /work/abc-ci-builds/build-diff/test/tmp/test_runner_₿₵_  _20221102_154846/abc_p2p_getavaaddr_6
2022-11-02T15:49:32.938000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/work/test/functional/test_framework/test_framework.py", line 132, in main
    self.run_test()
  File "/work/test/functional/abc_p2p_getavaaddr.py", line 505, in run_test
    self.getavaaddr_interval_test()
  File "/work/test/functional/abc_p2p_getavaaddr.py", line 149, in getavaaddr_interval_test
    requester.send_message(msg_getavaaddr())
  File "/usr/lib/python3.9/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "/work/test/functional/test_framework/test_node.py", line 532, in assert_debug_log
    self._raise_assertion_error(
  File "/work/test/functional/test_framework/test_node.py", line 212, in _raise_assertion_error
    raise AssertionError(self._node_msg(msg))
AssertionError: [node 0] Expected messages "['Ignoring repeated getavaaddr from peer']" does not partially match log:

 - 2022-11-02T15:49:30.947238Z (mocktime: 2022-11-02T15:49:19Z) [../../src/net.cpp:3504] [PushMessage] sending avapoll (585 bytes) peer=7


2022-11-02T15:49:34.537000Z TestFramework (INFO): Stopping nodes
2022-11-02T15:49:35.590000Z TestFramework (WARNING): Not cleaning up dir /work/abc-ci-builds/build-diff/test/tmp/test_runner_₿₵_  _20221102_154846/abc_p2p_getavaaddr_6
2022-11-02T15:49:35.590000Z TestFramework (ERROR): Test failed. Test logging available at /work/abc-ci-builds/build-diff/test/tmp/test_runner_₿₵_  _20221102_154846/abc_p2p_getavaaddr_6/test_framework.log
2022-11-02T15:49:35.590000Z TestFramework (ERROR): 
2022-11-02T15:49:35.590000Z TestFramework (ERROR): Hint: Call /work/test/functional/combine_logs.py '/work/abc-ci-builds/build-diff/test/tmp/test_runner_₿₵_  _20221102_154846/abc_p2p_getavaaddr_6' to consolidate all logs
2022-11-02T15:49:35.591000Z TestFramework (ERROR): 
2022-11-02T15:49:35.591000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2022-11-02T15:49:35.591000Z TestFramework (ERROR): https://github.com/Bitcoin-ABC/bitcoin-abc/issues
2022-11-02T15:49:35.591000Z TestFramework (ERROR):

Each failure log is accessible here:
Bitcoin ABC functional tests: abc_p2p_getavaaddr.py

Tail of the build log:

[16:02:50] :	 [Step 1/1] #6 sha256:1821db690b3bd11f767d87599c712d8396183e989a14538d1e8fae57411985cc
[16:02:50] :	 [Step 1/1] #6 DONE 1.7s
[16:02:50] :	 [Step 1/1] 
[16:02:50] :	 [Step 1/1] #7 [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server
[16:02:50] :	 [Step 1/1] #7 sha256:f85376ec1ed015529775771c3532146f46f7c2166e8fe4cf571fa58046943670
[16:02:50] :	 [Step 1/1] #7 0.573 Err:1 http://deb.debian.org/debian bullseye InRelease
[16:02:50] :	 [Step 1/1] #7 0.573   Could not connect to 172.17.0.1:3142 (172.17.0.1). - connect (111: Connection refused)
[16:02:50] :	 [Step 1/1] #7 0.573 Err:2 http://deb.debian.org/debian-security bullseye-security InRelease
[16:02:50] :	 [Step 1/1] #7 0.573   Unable to connect to 172.17.0.1:3142:
[16:02:50] :	 [Step 1/1] #7 0.573 Err:3 http://deb.debian.org/debian bullseye-updates InRelease
[16:02:50] :	 [Step 1/1] #7 0.573   Unable to connect to 172.17.0.1:3142:
[16:02:50] :	 [Step 1/1] #7 0.575 Reading package lists...
[16:02:50] :	 [Step 1/1] #7 0.581 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye/InRelease  Could not connect to 172.17.0.1:3142 (172.17.0.1). - connect (111: Connection refused)
[16:02:50] :	 [Step 1/1] #7 0.581 W: Failed to fetch http://deb.debian.org/debian-security/dists/bullseye-security/InRelease  Unable to connect to 172.17.0.1:3142:
[16:02:50] :	 [Step 1/1] #7 0.581 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye-updates/InRelease  Unable to connect to 172.17.0.1:3142:
[16:02:50] :	 [Step 1/1] #7 0.581 W: Some index files failed to download. They have been ignored, or old ones used instead.
[16:02:50] :	 [Step 1/1] #7 0.587 Reading package lists...
[16:02:50] :	 [Step 1/1] #7 0.595 Building dependency tree...
[16:02:50] :	 [Step 1/1] #7 0.596 Reading state information...
[16:02:50] :	 [Step 1/1] #7 0.598 Package locales is not available, but is referred to by another package.
[16:02:50] :	 [Step 1/1] #7 0.598 This may mean that the package is missing, has been obsoleted, or
[16:02:50] :	 [Step 1/1] #7 0.598 is only available from another source
[16:02:50] :	 [Step 1/1] #7 0.598
[16:02:50] :	 [Step 1/1] #7 0.598 Package openssh-server is not available, but is referred to by another package.
[16:02:50] :	 [Step 1/1] #7 0.598 This may mean that the package is missing, has been obsoleted, or
[16:02:50] :	 [Step 1/1] #7 0.598 is only available from another source
[16:02:50] :	 [Step 1/1] #7 0.598
[16:02:50] :	 [Step 1/1] #7 0.598 E: Unable to locate package pciutils
[16:02:50] :	 [Step 1/1] #7 0.598 E: Unable to locate package build-essential
[16:02:50] :	 [Step 1/1] #7 0.598 E: Unable to locate package git
[16:02:50] :	 [Step 1/1] #7 0.598 E: Unable to locate package subversion
[16:02:50] :	 [Step 1/1] #7 0.598 E: Package 'locales' has no installation candidate
[16:02:50] :	 [Step 1/1] #7 0.598 E: Unable to locate package wget
[16:02:50] :	 [Step 1/1] #7 0.598 E: Unable to locate package lsb-release
[16:02:50] :	 [Step 1/1] #7 0.598 E: Unable to locate package grub
[16:02:50] :	 [Step 1/1] #7 0.598 E: Package 'openssh-server' has no installation candidate
[16:02:50] :	 [Step 1/1] #7 ERROR: executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[16:02:50] :	 [Step 1/1] ------
[16:02:50] :	 [Step 1/1]  > [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server:
[16:02:50] :	 [Step 1/1] ------
[16:02:50] :	 [Step 1/1] executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[16:02:50] :	 [Step 1/1] Build gitian-linux failed with exit code 1
[16:02:50]W:	 [Step 1/1] Process exited with code 1
[16:02:51]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[16:02:51] :	 [Step 1/1] Waiting for 1 service processes to complete
[16:02:51]E:	 [Step 1/1] Ant JUnit report watcher
[16:02:51]E:		 [Ant JUnit report watcher] No reports found for paths:
[16:02:51]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[16:02:51]E:	 [Step 1/1] Step Command Line failed
[16:02:52]E: Ant JUnit report watcher
[16:02:52]E:	 [Ant JUnit report watcher] No reports found for paths:
[16:02:52]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[16:02:52]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[16:02:52] : Publishing internal artifacts (2s)
[16:02:54] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[16:02:54] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[16:02:52]W: Publishing artifacts (2s)
[16:02:52] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[16:02:52]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[16:02:54] : Build finished

Tail of the build log:

[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36079 -> phabricator/diff/36079
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36080 -> phabricator/diff/36080
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36081 -> phabricator/diff/36081
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36082 -> phabricator/diff/36082
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36083 -> phabricator/diff/36083
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36092 -> phabricator/diff/36092
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36093 -> phabricator/diff/36093
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36095 -> phabricator/diff/36095
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36101 -> phabricator/diff/36101
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36103 -> phabricator/diff/36103
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36106 -> phabricator/diff/36106
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36107 -> phabricator/diff/36107
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36108 -> phabricator/diff/36108
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36109 -> phabricator/diff/36109
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36111 -> phabricator/diff/36111
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36112 -> phabricator/diff/36112
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36114 -> phabricator/diff/36114
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36118 -> phabricator/diff/36118
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36122 -> phabricator/diff/36122
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/36123 -> phabricator/diff/36123
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/8992  -> phabricator/diff/8992
[16:08:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/8993  -> phabricator/diff/8993
[16:08:42] :	 [Step 1/1]  * [new branch]          master                 -> master
[16:08:42] :	 [Step 1/1] --- Building for bullseye amd64 ---
[16:08:42] :	 [Step 1/1] Stopping target if it is up
[16:08:42] :	 [Step 1/1] Error response from daemon: No such container: gitian-target
[16:08:42] :	 [Step 1/1] Error: No such container: gitian-target
[16:08:42] :	 [Step 1/1] Making a new image copy
[16:08:42] :	 [Step 1/1] Starting target
[16:08:42] :	 [Step 1/1] Checking if target is up.
[16:08:42] :	 [Step 1/1] Preparing build environment
[16:08:42] :	 [Step 1/1] Updating apt-get repository (log in var/install.log)
[16:08:42] :	 [Step 1/1] Installing additional packages (log in var/install.log)
[16:08:42] :	 [Step 1/1] ./bin/gbuild:23:in `system!': failed to run on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install autoconf automake binutils-aarch64-linux-gnu binutils-arm-linux-gnueabihf binutils-gold bsdmainutils ca-certificates cmake curl faketime g++-9 g++-9-aarch64-linux-gnu g++-9-arm-linux-gnueabihf gcc-9 gcc-9-aarch64-linux-gnu gcc-9-arm-linux-gnueabihf git gperf libtool ninja-build pkg-config python3 >> var/install.log 2>&1 (RuntimeError)
[16:08:42] :	 [Step 1/1] 	from ./bin/gbuild:120:in `build_one_configuration'
[16:08:42] :	 [Step 1/1] 	from ./bin/gbuild:339:in `block (2 levels) in <main>'
[16:08:42] :	 [Step 1/1] 	from ./bin/gbuild:334:in `each'
[16:08:42] :	 [Step 1/1] 	from ./bin/gbuild:334:in `block in <main>'
[16:08:42] :	 [Step 1/1] 	from ./bin/gbuild:332:in `each'
[16:08:42] :	 [Step 1/1] 	from ./bin/gbuild:332:in `<main>'
[16:08:42] :	 [Step 1/1] mv: cannot stat 'var/build.log': No such file or directory
[16:08:42] :	 [Step 1/1] Build gitian-linux failed with exit code 1
[16:08:42]W:	 [Step 1/1] Process exited with code 1
[16:08:42]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[16:08:42] :	 [Step 1/1] Waiting for 1 service processes to complete
[16:08:42]E:	 [Step 1/1] Ant JUnit report watcher
[16:08:42]E:		 [Ant JUnit report watcher] No reports found for paths:
[16:08:42]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[16:08:42]E:	 [Step 1/1] Step Command Line failed
[16:08:43]E: Ant JUnit report watcher
[16:08:43]E:	 [Ant JUnit report watcher] No reports found for paths:
[16:08:43]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[16:08:43]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[16:08:43] : Publishing internal artifacts (2s)
[16:08:45] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[16:08:45] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[16:08:43]W: Publishing artifacts (2s)
[16:08:43] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[16:08:43]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[16:08:45] : Build finished

Tail of the build log:

[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36077 -> phabricator/diff/36077
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36078 -> phabricator/diff/36078
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36079 -> phabricator/diff/36079
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36080 -> phabricator/diff/36080
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36081 -> phabricator/diff/36081
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36082 -> phabricator/diff/36082
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36083 -> phabricator/diff/36083
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36092 -> phabricator/diff/36092
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36093 -> phabricator/diff/36093
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36095 -> phabricator/diff/36095
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36101 -> phabricator/diff/36101
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36103 -> phabricator/diff/36103
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36106 -> phabricator/diff/36106
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36107 -> phabricator/diff/36107
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36108 -> phabricator/diff/36108
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36109 -> phabricator/diff/36109
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36111 -> phabricator/diff/36111
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36112 -> phabricator/diff/36112
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36114 -> phabricator/diff/36114
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36118 -> phabricator/diff/36118
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36122 -> phabricator/diff/36122
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/36123 -> phabricator/diff/36123
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/8992  -> phabricator/diff/8992
[16:10:19] :	 [Step 1/1]  * [new tag]             phabricator/diff/8993  -> phabricator/diff/8993
[16:10:19] :	 [Step 1/1]  * [new branch]          master                 -> master
[16:10:19] :	 [Step 1/1] --- Building for bullseye amd64 ---
[16:10:19] :	 [Step 1/1] Stopping target if it is up
[16:10:19] :	 [Step 1/1] Making a new image copy
[16:10:19] :	 [Step 1/1] Starting target
[16:10:19] :	 [Step 1/1] Checking if target is up.
[16:10:19] :	 [Step 1/1] Preparing build environment
[16:10:19] :	 [Step 1/1] Updating apt-get repository (log in var/install.log)
[16:10:19] :	 [Step 1/1] Installing additional packages (log in var/install.log)
[16:10:19] :	 [Step 1/1] ./bin/gbuild:23:in `system!': failed to run on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install autoconf automake binutils-aarch64-linux-gnu binutils-arm-linux-gnueabihf binutils-gold bsdmainutils ca-certificates cmake curl faketime g++-9 g++-9-aarch64-linux-gnu g++-9-arm-linux-gnueabihf gcc-9 gcc-9-aarch64-linux-gnu gcc-9-arm-linux-gnueabihf git gperf libtool ninja-build pkg-config python3 >> var/install.log 2>&1 (RuntimeError)
[16:10:19] :	 [Step 1/1] 	from ./bin/gbuild:120:in `build_one_configuration'
[16:10:19] :	 [Step 1/1] 	from ./bin/gbuild:339:in `block (2 levels) in <main>'
[16:10:19] :	 [Step 1/1] 	from ./bin/gbuild:334:in `each'
[16:10:19] :	 [Step 1/1] 	from ./bin/gbuild:334:in `block in <main>'
[16:10:19] :	 [Step 1/1] 	from ./bin/gbuild:332:in `each'
[16:10:19] :	 [Step 1/1] 	from ./bin/gbuild:332:in `<main>'
[16:10:19] :	 [Step 1/1] mv: cannot stat 'var/build.log': No such file or directory
[16:10:19] :	 [Step 1/1] Build gitian-linux failed with exit code 1
[16:10:19]W:	 [Step 1/1] Process exited with code 1
[16:10:19]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[16:10:19] :	 [Step 1/1] Waiting for 1 service processes to complete
[16:10:19]E:	 [Step 1/1] Ant JUnit report watcher
[16:10:19]E:		 [Ant JUnit report watcher] No reports found for paths:
[16:10:19]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[16:10:20]E:	 [Step 1/1] Step Command Line failed
[16:10:20]E: Ant JUnit report watcher
[16:10:20]E:	 [Ant JUnit report watcher] No reports found for paths:
[16:10:20]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[16:10:20]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[16:10:20] : Publishing internal artifacts (1s)
[16:10:21] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[16:10:21] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[16:10:20]W: Publishing artifacts (1s)
[16:10:20] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[16:10:20]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[16:10:21] : Build finished

Tail of the build log:

[19:42:07] :	 [Step 1/1] Selecting previously unselected package apt-cacher-ng.
[19:42:07] :	 [Step 1/1] Preparing to unpack .../apt-cacher-ng_3.2.1-1_amd64.deb ...
[19:42:07] :	 [Step 1/1] Unpacking apt-cacher-ng (3.2.1-1) ...
[19:42:07] :	 [Step 1/1] Setting up lsb-base (10.2019051400) ...
[19:42:07] :	 [Step 1/1] Setting up libssl1.1:amd64 (1.1.1n-0+deb10u3) ...
[19:42:07] :	 [Step 1/1] Setting up libwrap0:amd64 (7.6.q-28) ...
[19:42:07] :	 [Step 1/1] Setting up apt-cacher-ng (3.2.1-1) ...
[19:42:08] :	 [Step 1/1] invoke-rc.d: could not determine current runlevel
[19:42:08] :	 [Step 1/1] invoke-rc.d: policy-rc.d denied execution of start.
[19:42:08] :	 [Step 1/1] Processing triggers for libc-bin (2.28-10+deb10u2) ...
[19:42:10] :	 [Step 1/1] Removing intermediate container 3b50ee929e0b
[19:42:10] :	 [Step 1/1]  ---> 4ebc723f3140
[19:42:10] :	 [Step 1/1] Step 4/7 : RUN echo 'Port:3142' >> /etc/apt-cacher-ng/acng.conf
[19:42:10] :	 [Step 1/1]  ---> Running in a891cce8f922
[19:42:11] :	 [Step 1/1] Removing intermediate container a891cce8f922
[19:42:11] :	 [Step 1/1]  ---> 0421203b24d9
[19:42:11] :	 [Step 1/1] Step 5/7 : RUN echo 'BindAddress: 0.0.0.0' >> /etc/apt-cacher-ng/acng.conf
[19:42:11] :	 [Step 1/1]  ---> Running in 17ef35e29b63
[19:42:12] :	 [Step 1/1] Removing intermediate container 17ef35e29b63
[19:42:12] :	 [Step 1/1]  ---> b2a76ce3c0cb
[19:42:12] :	 [Step 1/1] Step 6/7 : RUN echo 'PassThroughPattern: .*' >> /etc/apt-cacher-ng/acng.conf
[19:42:12] :	 [Step 1/1]  ---> Running in 484ca86996dc
[19:42:13] :	 [Step 1/1] Removing intermediate container 484ca86996dc
[19:42:13] :	 [Step 1/1]  ---> d80aa9a93f75
[19:42:13] :	 [Step 1/1] Step 7/7 : CMD chown apt-cacher-ng:apt-cacher-ng /var/cache/apt-cacher-ng &&   service apt-cacher-ng start &&   tail -f /var/log/apt-cacher-ng/*
[19:42:13] :	 [Step 1/1]  ---> Running in 7fc5454582fa
[19:42:13] :	 [Step 1/1] Removing intermediate container 7fc5454582fa
[19:42:13] :	 [Step 1/1]  ---> 4eea0f2922e0
[19:42:13] :	 [Step 1/1] 
[19:42:13] :	 [Step 1/1] Successfully built 4eea0f2922e0
[19:42:13] :	 [Step 1/1] Successfully tagged infra_apt-cache-proxy:latest
[19:42:13]W:	 [Step 1/1] Image for service apt-cache-proxy was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
[19:42:13]W:	 [Step 1/1] Creating abc-apt-cache-proxy ... 
[19:42:14] :	 [Step 1/1] ~/buildAgent/work/jailed-build
[19:42:14] :	 [Step 1/1] ~/buildAgent/work/jailed-build/bitcoin-abc ~/buildAgent/work/jailed-build
[19:42:14]W:	 [Step 1/1]  [1A [2K
[19:42:14]W:	 [Step 1/1] Creating abc-apt-cache-proxy ...  [32mdone [0m
[19:42:14]W:	 [Step 1/1]  [1B
[20:23:00] :	 [Step 1/1] ~/infra ~/buildAgent/work/jailed-build/bitcoin-abc ~/buildAgent/work/jailed-build
[20:23:01]W:	 [Step 1/1] Stopping abc-apt-cache-proxy ... 
[20:23:11] :	 [Step 1/1] ~/buildAgent/work/jailed-build/bitcoin-abc ~/buildAgent/work/jailed-build
[20:23:11]W:	 [Step 1/1]  [1A [2K
[20:23:11]W:	 [Step 1/1] Stopping abc-apt-cache-proxy ...  [32mdone [0m
[20:23:11]W:	 [Step 1/1]  [1B
[20:23:12]W:	 [Step 1/1] /home/teamcity/buildAgent/temp/agentTmp/custom_script2610476195066317973: line 29: exit: 
[20:23:12]W:	 [Step 1/1] ##teamcity[message: numeric argument required
[20:23:12]W:	 [Step 1/1] Process exited with code 2
[20:23:12]E:	 [Step 1/1] Process exited with code 2 (Step: Command Line)
[20:23:12]E:	 [Step 1/1] Step Command Line failed
[20:23:12]E: Ant JUnit report watcher
[20:23:12]E:	 [Ant JUnit report watcher] No reports found for paths:
[20:23:12]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[20:23:12]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[20:23:12] : Publishing internal artifacts (2s)
[20:23:15] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[20:23:15] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[20:23:12]W: Publishing artifacts (2s)
[20:23:12] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[20:23:12]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[20:23:15] : Build finished
  • Make penalty factor and window configurable instead of hard coded in consensus params.
  • Add tests
  • Cleanup/refactor
src/validation.cpp
4316 ↗(On Diff #36250)

What's the point ?

  • Make tests more robust by testing both sides of the boundary for the same block
  • Deduplicate more test code
  • Add some sanity checks to the tests
  • Rebase on master
src/validation.cpp
4316 ↗(On Diff #36250)

You never want to reject a block using an avalanche policy that you will not poll for.

Majority of disruptive turbo mining activity occurs within the < 2 min window so:

  • Shorten penalty window to 2 minutes
  • Simplify penalty to be a step function. This reduces complexity with little to no downside.

Also:

  • Minor refactors
  • Rebase
sdulfari published this revision for review.Nov 17 2022, 19:07
sdulfari retitled this revision from [avalanche] Add experimental difficulty penalty for blocks that arrive sooner than 5 minutes from the previous block to [avalanche] Add experimental difficulty penalty for blocks that arrive sooner than 2 minutes from the previous block.
sdulfari edited the summary of this revision. (Show Details)
sdulfari edited the test plan for this revision. (Show Details)
sdulfari edited the summary of this revision. (Show Details)
Fabien requested changes to this revision.Nov 18 2022, 09:19
Fabien added inline comments.
src/avalanche/postconsensus.cpp
19 ↗(On Diff #36382)

What's the point ? It's already disabled by default

31 ↗(On Diff #36382)

Dito

34 ↗(On Diff #36382)

??

39 ↗(On Diff #36382)

You generally need to make this safer. E.g. -avapolicyearlyblockpenaltyfactor=0 will crash the node with a div by 0.

src/avalanche/postconsensus.h
23 ↗(On Diff #36382)

Should be a double ? Or somehow able to say e.g. 2,5x as a penalty ?

24 ↗(On Diff #36382)

Use chrono

src/init.cpp
1429 ↗(On Diff #36382)

This one is not really needed if you make the penalty default to 1

1435 ↗(On Diff #36382)

ALLOW_BOOL

src/validation.cpp
4395 ↗(On Diff #36382)

Why is !IsBlockAvalancheFinalized() needed ?

4423 ↗(On Diff #36382)

what about newPowLimit = penaltyFactor * GetNextWorkRequired(parent, &header, m_params); ?

4483 ↗(On Diff #36382)

I don't see the point of that bool. You can run the check directly where needed instead of creating a new variable and changing the interface

test/functional/abc_p2p_avalanche_policy_early_blocks.py
44 ↗(On Diff #36382)

Why ?

45 ↗(On Diff #36382)

What RPC requires an extended timeout ?

64 ↗(On Diff #36382)

not needed, there is a single node

117 ↗(On Diff #36382)

You need cases with self.penaltywindow as well as > self.penaltywindow

132 ↗(On Diff #36382)

That's actually an interesting case

143 ↗(On Diff #36382)

The test is missing some key things, like testing that the node will flip the block acceptance is the quorum disagrees with its position

This revision now requires changes to proceed.Nov 18 2022, 09:19
src/init.cpp
1429 ↗(On Diff #36382)

Probably better to make the default penalty 0 (for disabled) and remove the enabled flag. This is better UX.

src/validation.cpp
4395 ↗(On Diff #36382)

Leftovers from when this code block was placed elsewhere. Will clean up.

4483 ↗(On Diff #36382)

isQuorumEstablished() requires cs_main is not held so I call it here and pass it down.

test/functional/abc_p2p_avalanche_policy_early_blocks.py
45 ↗(On Diff #36382)

These are copy paste leftovers.

64 ↗(On Diff #36382)

Good catch. Leftovers from previous version that used 2 nodes.

Tabling this patch for now considering recent improvements in UX on multiple exchanges. The risk now outbalances the benefits.

Added some cleanup and a note to fix getblocktemplate if work on this continues later.