Page MenuHomePhabricator

Migrate gitian builds to Debian Bullseye
ClosedPublic

Authored by Fabien on Sep 30 2022, 11:54.

Details

Reviewers
sdulfari
Group Reviewers
Restricted Project
Commits
rABC5004cf37dcb2: Migrate gitian builds to Debian Bullseye
Summary

This is step toward migrating our build infra from Debian 10 Buster (now
obsolet) to Debian 11 Bullseye. The sensible part here is the use of
gcc/g++ version 9 instead of the default version 10. It prevents from
using the new symbol pthread_cond_clockwait introduced in glibc 2.30
and would make our release incompatible with previous glibc versions.
I didn't manage to prevent using this symbol during the compilation with
gcc/g++ 10, so I'm moving ahead with version 9.

Test Plan

Run each gitian build twice. Check the hashes are deterministic. Check
the binaries run fine on each platform.

Diff Detail

Event Timeline

Fabien requested review of this revision.Sep 30 2022, 11:54

@bot gitian-linux gitian-win gitian-osx

Tail of the build log:

[12:07:30] :	 [Step 1/1] #5 sha256:09850fc0dd4f5e74927c2d4c045f1214904c1cb14ab15faedb60c071ed3b15cd
[12:07:30] :	 [Step 1/1] #5 DONE 0.8s
[12:07:30] :	 [Step 1/1] 
[12:07:30] :	 [Step 1/1] #6 [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server
[12:07:30] :	 [Step 1/1] #6 sha256:487badbdb57cf8497d1c9d1ae8da27bc068adbb020f912a9ab4ab98a0b38dfe5
[12:07:30] :	 [Step 1/1] #6 0.535 Err:1 http://deb.debian.org/debian bullseye InRelease
[12:07:30] :	 [Step 1/1] #6 0.535   Connection failed [IP: 172.17.0.1 3142]
[12:07:30] :	 [Step 1/1] #6 0.536 Err:2 http://deb.debian.org/debian-security bullseye-security InRelease
[12:07:30] :	 [Step 1/1] #6 0.536   Connection failed [IP: 172.17.0.1 3142]
[12:07:30] :	 [Step 1/1] #6 0.536 Err:3 http://deb.debian.org/debian bullseye-updates InRelease
[12:07:30] :	 [Step 1/1] #6 0.536   Connection failed [IP: 172.17.0.1 3142]
[12:07:30] :	 [Step 1/1] #6 0.539 Reading package lists...
[12:07:30] :	 [Step 1/1] #6 0.545 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye/InRelease  Connection failed [IP: 172.17.0.1 3142]
[12:07:30] :	 [Step 1/1] #6 0.545 W: Failed to fetch http://deb.debian.org/debian-security/dists/bullseye-security/InRelease  Connection failed [IP: 172.17.0.1 3142]
[12:07:30] :	 [Step 1/1] #6 0.545 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye-updates/InRelease  Connection failed [IP: 172.17.0.1 3142]
[12:07:30] :	 [Step 1/1] #6 0.545 W: Some index files failed to download. They have been ignored, or old ones used instead.
[12:07:30] :	 [Step 1/1] #6 0.550 Reading package lists...
[12:07:30] :	 [Step 1/1] #6 0.557 Building dependency tree...
[12:07:30] :	 [Step 1/1] #6 0.558 Reading state information...
[12:07:30] :	 [Step 1/1] #6 0.559 Package locales is not available, but is referred to by another package.
[12:07:30] :	 [Step 1/1] #6 0.559 This may mean that the package is missing, has been obsoleted, or
[12:07:30] :	 [Step 1/1] #6 0.559 is only available from another source
[12:07:30] :	 [Step 1/1] #6 0.559
[12:07:30] :	 [Step 1/1] #6 0.559 Package openssh-server is not available, but is referred to by another package.
[12:07:30] :	 [Step 1/1] #6 0.559 This may mean that the package is missing, has been obsoleted, or
[12:07:30] :	 [Step 1/1] #6 0.559 is only available from another source
[12:07:30] :	 [Step 1/1] #6 0.559
[12:07:30] :	 [Step 1/1] #6 0.560 E: Unable to locate package pciutils
[12:07:30] :	 [Step 1/1] #6 0.560 E: Unable to locate package build-essential
[12:07:30] :	 [Step 1/1] #6 0.560 E: Unable to locate package git
[12:07:30] :	 [Step 1/1] #6 0.560 E: Unable to locate package subversion
[12:07:30] :	 [Step 1/1] #6 0.560 E: Package 'locales' has no installation candidate
[12:07:30] :	 [Step 1/1] #6 0.560 E: Unable to locate package wget
[12:07:30] :	 [Step 1/1] #6 0.560 E: Unable to locate package lsb-release
[12:07:30] :	 [Step 1/1] #6 0.560 E: Unable to locate package grub
[12:07:30] :	 [Step 1/1] #6 0.560 E: Package 'openssh-server' has no installation candidate
[12:07:30] :	 [Step 1/1] #6 ERROR: executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[12:07:30] :	 [Step 1/1] ------
[12:07:30] :	 [Step 1/1]  > [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server:
[12:07:30] :	 [Step 1/1] ------
[12:07:30] :	 [Step 1/1] executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[12:07:30] :	 [Step 1/1] Build gitian-osx failed with exit code 1
[12:07:30]W:	 [Step 1/1] Process exited with code 1
[12:07:30]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[12:07:30] :	 [Step 1/1] Waiting for 1 service processes to complete
[12:07:30]E:	 [Step 1/1] Ant JUnit report watcher
[12:07:30]E:		 [Ant JUnit report watcher] No reports found for paths:
[12:07:30]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[12:07:31]E:	 [Step 1/1] Step Command Line failed
[12:07:31]E: Ant JUnit report watcher
[12:07:31]E:	 [Ant JUnit report watcher] No reports found for paths:
[12:07:31]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[12:07:31]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[12:07:31] : Publishing internal artifacts (5s)
[12:07:36] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[12:07:36] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[12:07:31]W: Publishing artifacts (5s)
[12:07:31] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[12:07:31]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[12:07:37] : Build finished

Tail of the build log:

[12:07:58] :	 [Step 1/1] #5 sha256:09850fc0dd4f5e74927c2d4c045f1214904c1cb14ab15faedb60c071ed3b15cd
[12:07:58] :	 [Step 1/1] #5 CACHED
[12:07:58] :	 [Step 1/1] 
[12:07:58] :	 [Step 1/1] #6 [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server
[12:07:58] :	 [Step 1/1] #6 sha256:487badbdb57cf8497d1c9d1ae8da27bc068adbb020f912a9ab4ab98a0b38dfe5
[12:07:58] :	 [Step 1/1] #6 0.444 Err:1 http://deb.debian.org/debian bullseye InRelease
[12:07:58] :	 [Step 1/1] #6 0.444   Connection failed [IP: 172.17.0.1 3142]
[12:07:58] :	 [Step 1/1] #6 0.445 Err:2 http://deb.debian.org/debian-security bullseye-security InRelease
[12:07:58] :	 [Step 1/1] #6 0.445   Connection failed [IP: 172.17.0.1 3142]
[12:07:58] :	 [Step 1/1] #6 0.445 Err:3 http://deb.debian.org/debian bullseye-updates InRelease
[12:07:58] :	 [Step 1/1] #6 0.445   Connection failed [IP: 172.17.0.1 3142]
[12:07:58] :	 [Step 1/1] #6 0.448 Reading package lists...
[12:07:58] :	 [Step 1/1] #6 0.453 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye/InRelease  Connection failed [IP: 172.17.0.1 3142]
[12:07:58] :	 [Step 1/1] #6 0.453 W: Failed to fetch http://deb.debian.org/debian-security/dists/bullseye-security/InRelease  Connection failed [IP: 172.17.0.1 3142]
[12:07:58] :	 [Step 1/1] #6 0.453 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye-updates/InRelease  Connection failed [IP: 172.17.0.1 3142]
[12:07:58] :	 [Step 1/1] #6 0.453 W: Some index files failed to download. They have been ignored, or old ones used instead.
[12:07:58] :	 [Step 1/1] #6 0.460 Reading package lists...
[12:07:58] :	 [Step 1/1] #6 0.467 Building dependency tree...
[12:07:58] :	 [Step 1/1] #6 0.468 Reading state information...
[12:07:58] :	 [Step 1/1] #6 0.469 Package locales is not available, but is referred to by another package.
[12:07:58] :	 [Step 1/1] #6 0.469 This may mean that the package is missing, has been obsoleted, or
[12:07:58] :	 [Step 1/1] #6 0.469 is only available from another source
[12:07:58] :	 [Step 1/1] #6 0.469
[12:07:58] :	 [Step 1/1] #6 0.469 Package openssh-server is not available, but is referred to by another package.
[12:07:58] :	 [Step 1/1] #6 0.469 This may mean that the package is missing, has been obsoleted, or
[12:07:58] :	 [Step 1/1] #6 0.469 is only available from another source
[12:07:58] :	 [Step 1/1] #6 0.469
[12:07:58] :	 [Step 1/1] #6 0.470 E: Unable to locate package pciutils
[12:07:58] :	 [Step 1/1] #6 0.470 E: Unable to locate package build-essential
[12:07:58] :	 [Step 1/1] #6 0.470 E: Unable to locate package git
[12:07:58] :	 [Step 1/1] #6 0.470 E: Unable to locate package subversion
[12:07:58] :	 [Step 1/1] #6 0.470 E: Package 'locales' has no installation candidate
[12:07:58] :	 [Step 1/1] #6 0.470 E: Unable to locate package wget
[12:07:58] :	 [Step 1/1] #6 0.470 E: Unable to locate package lsb-release
[12:07:58] :	 [Step 1/1] #6 0.470 E: Unable to locate package grub
[12:07:58] :	 [Step 1/1] #6 0.470 E: Package 'openssh-server' has no installation candidate
[12:07:58] :	 [Step 1/1] #6 ERROR: executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[12:07:58] :	 [Step 1/1] ------
[12:07:58] :	 [Step 1/1]  > [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server:
[12:07:58] :	 [Step 1/1] ------
[12:07:58] :	 [Step 1/1] executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[12:07:58] :	 [Step 1/1] Build gitian-linux failed with exit code 1
[12:07:59]W:	 [Step 1/1] Process exited with code 1
[12:07:59]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[12:07:59] :	 [Step 1/1] Waiting for 1 service processes to complete
[12:07:59]E:	 [Step 1/1] Ant JUnit report watcher
[12:07:59]E:		 [Ant JUnit report watcher] No reports found for paths:
[12:07:59]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[12:07:59]E:	 [Step 1/1] Step Command Line failed
[12:08:00]E: Ant JUnit report watcher
[12:08:00]E:	 [Ant JUnit report watcher] No reports found for paths:
[12:08:00]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[12:08:00]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[12:08:00] : Publishing internal artifacts (1s)
[12:08:01] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[12:08:01] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[12:08:00]W: Publishing artifacts (2s)
[12:08:00] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[12:08:00]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[12:08:03] : Build finished

@bot gitian-linux gitian-win gitian-osx

Tail of the build log:

[12:36:42] :	 [Step 1/1] #5 sha256:09850fc0dd4f5e74927c2d4c045f1214904c1cb14ab15faedb60c071ed3b15cd
[12:36:42] :	 [Step 1/1] #5 DONE 2.6s
[12:36:42] :	 [Step 1/1] 
[12:36:42] :	 [Step 1/1] #6 [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server
[12:36:42] :	 [Step 1/1] #6 sha256:487badbdb57cf8497d1c9d1ae8da27bc068adbb020f912a9ab4ab98a0b38dfe5
[12:36:42] :	 [Step 1/1] #6 1.537 Err:1 http://deb.debian.org/debian bullseye InRelease
[12:36:42] :	 [Step 1/1] #6 1.537   Could not connect to 172.17.0.1:3142 (172.17.0.1). - connect (113: No route to host)
[12:36:42] :	 [Step 1/1] #6 1.537 Err:2 http://deb.debian.org/debian-security bullseye-security InRelease
[12:36:42] :	 [Step 1/1] #6 1.537   Unable to connect to 172.17.0.1:3142:
[12:36:42] :	 [Step 1/1] #6 1.537 Err:3 http://deb.debian.org/debian bullseye-updates InRelease
[12:36:42] :	 [Step 1/1] #6 1.537   Unable to connect to 172.17.0.1:3142:
[12:36:42] :	 [Step 1/1] #6 1.545 Reading package lists...
[12:36:42] :	 [Step 1/1] #6 1.567 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye/InRelease  Could not connect to 172.17.0.1:3142 (172.17.0.1). - connect (113: No route to host)
[12:36:42] :	 [Step 1/1] #6 1.567 W: Failed to fetch http://deb.debian.org/debian-security/dists/bullseye-security/InRelease  Unable to connect to 172.17.0.1:3142:
[12:36:42] :	 [Step 1/1] #6 1.567 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye-updates/InRelease  Unable to connect to 172.17.0.1:3142:
[12:36:42] :	 [Step 1/1] #6 1.567 W: Some index files failed to download. They have been ignored, or old ones used instead.
[12:36:42] :	 [Step 1/1] #6 1.576 Reading package lists...
[12:36:42] :	 [Step 1/1] #6 1.580 Building dependency tree...
[12:36:42] :	 [Step 1/1] #6 1.581 Reading state information...
[12:36:42] :	 [Step 1/1] #6 1.582 Package locales is not available, but is referred to by another package.
[12:36:42] :	 [Step 1/1] #6 1.582 This may mean that the package is missing, has been obsoleted, or
[12:36:42] :	 [Step 1/1] #6 1.582 is only available from another source
[12:36:42] :	 [Step 1/1] #6 1.582
[12:36:42] :	 [Step 1/1] #6 1.582 Package openssh-server is not available, but is referred to by another package.
[12:36:42] :	 [Step 1/1] #6 1.582 This may mean that the package is missing, has been obsoleted, or
[12:36:42] :	 [Step 1/1] #6 1.582 is only available from another source
[12:36:42] :	 [Step 1/1] #6 1.582
[12:36:42] :	 [Step 1/1] #6 1.583 E: Unable to locate package pciutils
[12:36:42] :	 [Step 1/1] #6 1.583 E: Unable to locate package build-essential
[12:36:42] :	 [Step 1/1] #6 1.583 E: Unable to locate package git
[12:36:42] :	 [Step 1/1] #6 1.583 E: Unable to locate package subversion
[12:36:42] :	 [Step 1/1] #6 1.583 E: Package 'locales' has no installation candidate
[12:36:42] :	 [Step 1/1] #6 1.583 E: Unable to locate package wget
[12:36:42] :	 [Step 1/1] #6 1.583 E: Unable to locate package lsb-release
[12:36:42] :	 [Step 1/1] #6 1.583 E: Unable to locate package grub
[12:36:42] :	 [Step 1/1] #6 1.583 E: Package 'openssh-server' has no installation candidate
[12:36:42] :	 [Step 1/1] #6 ERROR: executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[12:36:42] :	 [Step 1/1] ------
[12:36:42] :	 [Step 1/1]  > [3/5] RUN apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server:
[12:36:42] :	 [Step 1/1] ------
[12:36:42] :	 [Step 1/1] executor failed running [/bin/sh -c apt-get update && apt-get --no-install-recommends -y install pciutils build-essential git subversion locales wget lsb-release grub openssh-server]: exit code: 100
[12:36:42] :	 [Step 1/1] Build gitian-win failed with exit code 1
[12:36:42]W:	 [Step 1/1] Process exited with code 1
[12:36:42]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[12:36:42] :	 [Step 1/1] Waiting for 1 service processes to complete
[12:36:43]E:	 [Step 1/1] Ant JUnit report watcher
[12:36:43]E:		 [Ant JUnit report watcher] No reports found for paths:
[12:36:43]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[12:36:43]E:	 [Step 1/1] Step Command Line failed
[12:36:43]E: Ant JUnit report watcher
[12:36:43]E:	 [Ant JUnit report watcher] No reports found for paths:
[12:36:43]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[12:36:43]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[12:36:43] : Publishing internal artifacts (3s)
[12:36:47] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[12:36:47] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[12:36:43]W: Publishing artifacts (3s)
[12:36:43] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml]
[12:36:43]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[12:36:47] : Build finished
sdulfari added a subscriber: sdulfari.

Looks like a networking issue when building gitian-win. I trust it builds correctly.

contrib/gitian-descriptors/gitian-linux.yml
101

Nit: i is usually used for indices. HOST would be more appropriate.

This revision is now accepted and ready to land.Sep 30 2022, 16:46
This revision was automatically updated to reflect the committed changes.