Page MenuHomePhabricator

scripts: add MACHO PIE check to security-check.py
ClosedPublic

Authored by PiRK on Apr 3 2023, 14:34.

Details

Reviewers
Fabien
Group Reviewers
Restricted Project
Commits
rABCd000384e8f0b: scripts: add MACHO PIE check to security-check.py
Summary

This is a backport of core#17787

Test Plan

gitian builds

Diff Detail

Event Timeline

PiRK requested review of this revision.Apr 3 2023, 14:34
PiRK planned changes to this revision.Apr 3 2023, 14:34

@bot gitian-osx gitian-linux gitian-win

Tail of the build log:

[14:51:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/39154 -> phabricator/diff/39154
[14:51:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/39164 -> phabricator/diff/39164
[14:51:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/8992  -> phabricator/diff/8992
[14:51:42] :	 [Step 1/1]  * [new tag]             phabricator/diff/8993  -> phabricator/diff/8993
[14:51:42] :	 [Step 1/1]  * [new branch]          master                 -> master
[14:51:42] :	 [Step 1/1] --- Building for bullseye amd64 ---
[14:51:42] :	 [Step 1/1] Stopping target if it is up
[14:51:42] :	 [Step 1/1] Error response from daemon: No such container: gitian-target
[14:51:42] :	 [Step 1/1] Error: No such container: gitian-target
[14:51:42] :	 [Step 1/1] Making a new image copy
[14:51:42] :	 [Step 1/1] Starting target
[14:51:42] :	 [Step 1/1] Checking if target is up.
[14:51:42] :	 [Step 1/1] Preparing build environment
[14:51:42] :	 [Step 1/1] Updating apt-get repository (log in var/install.log)
[14:51:42] :	 [Step 1/1] Installing additional packages (log in var/install.log)
[14:51:42] :	 [Step 1/1] Upgrading system, may take a while (log in var/install.log)
[14:51:42] :	 [Step 1/1] Creating package manifest
[14:51:42] :	 [Step 1/1] Creating build script (var/build-script)
[14:51:42] :	 [Step 1/1] Running build script (log in var/build.log)
[14:51:42] :	 [Step 1/1] ./bin/gbuild:23:in `system!': failed to run on-target setarch x86_64 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError)
[14:51:42] :	 [Step 1/1] 	from ./bin/gbuild:185:in `build_one_configuration'
[14:51:42] :	 [Step 1/1] 	from ./bin/gbuild:339:in `block (2 levels) in <main>'
[14:51:42] :	 [Step 1/1] 	from ./bin/gbuild:334:in `each'
[14:51:42] :	 [Step 1/1] 	from ./bin/gbuild:334:in `block in <main>'
[14:51:42] :	 [Step 1/1] 	from ./bin/gbuild:332:in `each'
[14:51:42] :	 [Step 1/1] 	from ./bin/gbuild:332:in `<main>'
[14:51:42] :	 [Step 1/1] Build gitian-osx failed with exit code 1
[14:51:42] :	 [Step 1/1] ~/infra ~/buildAgent/work/jailed-build/bitcoin-abc ~/buildAgent/work/jailed-build
[14:51:42]W:	 [Step 1/1] + RESULT=1
[14:51:42]W:	 [Step 1/1] + pushd /home/teamcity/infra
[14:51:42]W:	 [Step 1/1] + docker-compose stop apt-cache-proxy
[14:51:42]W:	 [Step 1/1] Stopping abc-apt-cache-proxy ... 
[14:51:53] :	 [Step 1/1] ~/buildAgent/work/jailed-build/bitcoin-abc ~/buildAgent/work/jailed-build
[14:51:53]W:	 [Step 1/1]  [1A [2K
[14:51:53]W:	 [Step 1/1] Stopping abc-apt-cache-proxy ...  [32mdone [0m
[14:51:53]W:	 [Step 1/1]  [1B+ popd
[14:51:53]W:	 [Step 1/1] + exit 1
[14:51:53]W:	 [Step 1/1] Process exited with code 1
[14:51:53]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[14:51:53] :	 [Step 1/1] Waiting for 1 service processes to complete
[14:51:53]E:	 [Step 1/1] Ant JUnit report watcher
[14:51:53]E:		 [Ant JUnit report watcher] No reports found for paths:
[14:51:53]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[14:51:53]E:	 [Step 1/1] Step Command Line failed
[14:51:54]E: Ant JUnit report watcher
[14:51:54]E:	 [Ant JUnit report watcher] No reports found for paths:
[14:51:54]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[14:51:54]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[14:51:54] : Publishing internal artifacts (1s)
[14:51:55] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[14:51:55] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[14:51:54]W: Publishing artifacts (2s)
[14:51:54] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml, +:bitcoin-abc/abc-ci-builds/gitian-osx/gitian-results => gitian-osx.tar.gz]
[14:51:54]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[14:51:54] :	 [Publishing artifacts] Creating archive gitian-osx.tar.gz
[14:51:54] :		 [Creating archive gitian-osx.tar.gz] Creating /home/teamcity/buildAgent/temp/buildTmp/TarPreprocessor1640190105790649357/gitian-osx.tar.gz
[14:51:54] :		 [Creating archive gitian-osx.tar.gz] Archive was created, file size 209.76 KB (214796 bytes)
[14:51:56] :	 [Publishing artifacts] Publishing 1 file using [WebPublisher]: bitcoin-abc/abc-ci-builds/gitian-osx/gitian-results => gitian-osx.tar.gz
[14:51:56] :	 [Publishing artifacts] Publishing 1 file using [ArtifactsCachePublisher]: bitcoin-abc/abc-ci-builds/gitian-osx/gitian-results => gitian-osx.tar.gz
[14:51:57] : Build finished

add binutils to gitian-osx for utils needed in security-check.py

Tail of the build log:

[16:11:48]W:		 [Publishing artifacts] Artifacts path 'results/artifacts' not found
[16:11:48] :	 [Step 1/1] Ant JUnit report watcher
[16:11:48]i:	 [Step 1/1] ##teamcity[publishArtifacts '+:results/artifacts=>artifacts.tar.gz']
[16:11:48] :		 [Ant JUnit report watcher] Watching paths:
[16:11:48] :		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[16:11:49]E:	 [Step 1/1] Build gitian-osx failed with exit code 1
[16:11:48]i:	 [Step 1/1] ##teamcity[buildProblem timestamp='2023-04-03T16:11:48.792' description='Build gitian-osx failed with exit code 1']
[16:11:48] :	 [Step 1/1] --- Building for bullseye amd64 ---
[16:11:48] :	 [Step 1/1] Stopping target if it is up
[16:11:48] :	 [Step 1/1] Making a new image copy
[16:11:48] :	 [Step 1/1] Starting target
[16:11:48] :	 [Step 1/1] Checking if target is up.
[16:11:48] :	 [Step 1/1] Preparing build environment
[16:11:48] :	 [Step 1/1] Updating apt-get repository (log in var/install.log)
[16:11:48] :	 [Step 1/1] Installing additional packages (log in var/install.log)
[16:11:48] :	 [Step 1/1] Upgrading system, may take a while (log in var/install.log)
[16:11:48] :	 [Step 1/1] Creating package manifest
[16:11:48] :	 [Step 1/1] Creating build script (var/build-script)
[16:11:48] :	 [Step 1/1] Running build script (log in var/build.log)
[16:11:48] :	 [Step 1/1] ./bin/gbuild:23:in `system!': failed to run on-target setarch x86_64 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError)
[16:11:48] :	 [Step 1/1] 	from ./bin/gbuild:185:in `build_one_configuration'
[16:11:48] :	 [Step 1/1] 	from ./bin/gbuild:339:in `block (2 levels) in <main>'
[16:11:48] :	 [Step 1/1] 	from ./bin/gbuild:334:in `each'
[16:11:48] :	 [Step 1/1] 	from ./bin/gbuild:334:in `block in <main>'
[16:11:48] :	 [Step 1/1] 	from ./bin/gbuild:332:in `each'
[16:11:48] :	 [Step 1/1] 	from ./bin/gbuild:332:in `<main>'
[16:11:48] :	 [Step 1/1] Build gitian-osx failed with exit code 1
[16:11:48] :	 [Step 1/1] ~/infra ~/buildAgent/work/jailed-build/bitcoin-abc ~/buildAgent/work/jailed-build
[16:11:48]W:	 [Step 1/1] + RESULT=1
[16:11:48]W:	 [Step 1/1] + pushd /home/teamcity/infra
[16:11:48]W:	 [Step 1/1] + docker-compose stop apt-cache-proxy
[16:11:49]W:	 [Step 1/1] Stopping abc-apt-cache-proxy ... 
[16:11:59] :	 [Step 1/1] ~/buildAgent/work/jailed-build/bitcoin-abc ~/buildAgent/work/jailed-build
[16:11:59]W:	 [Step 1/1]  [1A [2K
[16:11:59]W:	 [Step 1/1] Stopping abc-apt-cache-proxy ...  [32mdone [0m
[16:11:59]W:	 [Step 1/1]  [1B+ popd
[16:11:59]W:	 [Step 1/1] + exit 1
[16:11:59]W:	 [Step 1/1] Process exited with code 1
[16:11:59]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
[16:11:59] :	 [Step 1/1] Waiting for 1 service processes to complete
[16:12:00]E:	 [Step 1/1] Ant JUnit report watcher
[16:12:00]E:		 [Ant JUnit report watcher] No reports found for paths:
[16:12:00]E:		 [Ant JUnit report watcher] /home/teamcity/buildAgent/work/jailed-build/results/artifacts/junit/*.xml
[16:12:00]E:	 [Step 1/1] Step Command Line failed
[16:12:00]E: Ant JUnit report watcher
[16:12:00]E:	 [Ant JUnit report watcher] No reports found for paths:
[16:12:00]E:	 [Ant JUnit report watcher] +:results/test_bitcoin.xml
[16:12:00]E:	 [Ant JUnit report watcher] +:results/**/junit_results*.xml
[16:12:00] : Publishing internal artifacts (4s)
[16:12:05] :	 [Publishing internal artifacts] Publishing 1 file using [WebPublisher]
[16:12:05] :	 [Publishing internal artifacts] Publishing 1 file using [ArtifactsCachePublisher]
[16:12:00]W: Publishing artifacts (5s)
[16:12:00] :	 [Publishing artifacts] Collecting files to publish: [+:results/**/junit_results*.xml, +:bitcoin-abc/abc-ci-builds/gitian-osx/gitian-results => gitian-osx.tar.gz]
[16:12:00]W:	 [Publishing artifacts] Artifacts path 'results/**/junit_results*.xml' not found
[16:12:00] :	 [Publishing artifacts] Creating archive gitian-osx.tar.gz
[16:12:00] :		 [Creating archive gitian-osx.tar.gz] Creating /home/teamcity/buildAgent/temp/buildTmp/TarPreprocessor7332594038758702536/gitian-osx.tar.gz
[16:12:00] :		 [Creating archive gitian-osx.tar.gz] Archive was created, file size 211.85 KB (216935 bytes)
[16:12:06] :	 [Publishing artifacts] Publishing 1 file using [WebPublisher]: bitcoin-abc/abc-ci-builds/gitian-osx/gitian-results => gitian-osx.tar.gz
[16:12:06] :	 [Publishing artifacts] Publishing 1 file using [ArtifactsCachePublisher]: bitcoin-abc/abc-ci-builds/gitian-osx/gitian-results => gitian-osx.tar.gz
[16:12:07] : Build finished

There is a missing dependency I cannot find, and the trial / error process through arcanist and CI builds is too long to figure it out.

export OTOOL=${CMAKE_OTOOL} in BinaryTest.cmake

Fabien added a subscriber: Fabien.
Fabien added inline comments.
contrib/gitian-descriptors/gitian-osx.yml
13

Remove

This revision is now accepted and ready to land.Apr 4 2023, 09:25