Page MenuHomePhabricator

[CI] Do not re-download the OSX SDK if it's already available
ClosedPublic

Authored by jasonbcox on Mar 19 2020, 17:10.

Details

Summary

This improves build time and bandwidth usage for the OSX gitian build on CI.

Test Plan

OSX Gitian build on CI

Diff Detail

Repository
rABC Bitcoin ABC
Branch
gitian-osx-sdk
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 9895
Build 17649: Default Diff Build & Tests
Build 17648: arc lint + arc unit

Event Timeline

Snippet of first build failure:

[17:38:34] :	 [Step 1/1] (Reading database ... 70%
[17:38:34] :	 [Step 1/1] (Reading database ... 75%
[17:38:34] :	 [Step 1/1] (Reading database ... 80%
[17:38:34] :	 [Step 1/1] (Reading database ... 85%
[17:38:34] :	 [Step 1/1] (Reading database ... 90%
[17:38:34] :	 [Step 1/1] (Reading database ... 95%
[17:38:34] :	 [Step 1/1] (Reading database ... 100%
[17:38:34] :	 [Step 1/1] (Reading database ... 17687 files and directories currently installed.)
[17:38:34] :	 [Step 1/1] Removing rsyslog (8.1901.0-1) ...
[17:38:34] :	 [Step 1/1] invoke-rc.d: could not determine current runlevel
[17:38:34] :	 [Step 1/1] Stopping enhanced syslogd: rsyslogd already stopped.
[17:38:34] :	 [Step 1/1] (Reading database ... 
[17:38:34] :	 [Step 1/1] (Reading database ... 5%
[17:38:34] :	 [Step 1/1] (Reading database ... 10%
[17:38:34] :	 [Step 1/1] (Reading database ... 15%
[17:38:34] :	 [Step 1/1] (Reading database ... 20%
[17:38:34] :	 [Step 1/1] (Reading database ... 25%
[17:38:34] :	 [Step 1/1] (Reading database ... 30%
[17:38:34] :	 [Step 1/1] (Reading database ... 35%
[17:38:34] :	 [Step 1/1] (Reading database ... 40%
[17:38:34] :	 [Step 1/1] (Reading database ... 45%
[17:38:34] :	 [Step 1/1] (Reading database ... 50%
[17:38:34] :	 [Step 1/1] (Reading database ... 55%
[17:38:34] :	 [Step 1/1] (Reading database ... 60%
[17:38:34] :	 [Step 1/1] (Reading database ... 65%
[17:38:34] :	 [Step 1/1] (Reading database ... 70%
[17:38:34] :	 [Step 1/1] (Reading database ... 75%
[17:38:34] :	 [Step 1/1] (Reading database ... 80%
[17:38:34] :	 [Step 1/1] (Reading database ... 85%
[17:38:34] :	 [Step 1/1] (Reading database ... 90%
[17:38:34] :	 [Step 1/1] (Reading database ... 95%
[17:38:34] :	 [Step 1/1] (Reading database ... 100%
[17:38:34] :	 [Step 1/1] (Reading database ... 17629 files and directories currently installed.)
[17:38:34] :	 [Step 1/1] Purging configuration files for rsyslog (8.1901.0-1) ...
[17:38:34] :	 [Step 1/1] Processing triggers for systemd (241-7~deb10u2) ...
[17:38:34] :	 [Step 1/1] Adding 'local diversion of /sbin/initctl to /sbin/initctl.distrib'
[17:38:34] :	 [Step 1/1] Adding 'local diversion of /usr/bin/ischroot to /usr/bin/ischroot.distrib'
[17:38:34]W:	 [Step 1/1] dpkg-divert: warning: diverting file '/usr/bin/ischroot' from an Essential package with rename is dangerous, use --no-rename
[17:38:34] :	 [Step 1/1] Adding 'local diversion of /usr/sbin/policy-rc.d to /usr/sbin/policy-rc.d.distrib'
[17:38:34] :	 [Step 1/1] Starting target
[17:38:35] :	 [Step 1/1] Checking if target is up
[17:38:35] :	 [Step 1/1] Preparing build environment
[17:38:38] :	 [Step 1/1] Updating apt-get repository (log in var/install.log)
[17:38:41] :	 [Step 1/1] Installing additional packages (log in var/install.log)
[17:39:46] :	 [Step 1/1] Upgrading system, may take a while (log in var/install.log)
[17:39:51] :	 [Step 1/1] Creating package manifest
[17:39:55] :	 [Step 1/1] Creating build script (var/build-script)
[17:39:56] :	 [Step 1/1] Running build script (log in var/build.log)
[17:46:31]W:	 [Step 1/1] ./bin/gbuild:23:in `system!': failed to run on-target setarch x86_64 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError)
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:169:in `build_one_configuration'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:320:in `block (2 levels) in <main>'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:315:in `each'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:315:in `block in <main>'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:313:in `each'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:313:in `<main>'
[17:46:31]W:	 [Step 1/1] ++ move_log
[17:46:31]W:	 [Step 1/1] ++ mv var/install.log /home/teamcity/buildAgent/work/c4a5708f2bae7929/gitian-results/
[17:46:31]W:	 [Step 1/1] ++ mv var/build.log /home/teamcity/buildAgent/work/c4a5708f2bae7929/gitian-results/
[17:46:31]W:	 [Step 1/1] Process exited with code 1
[17:46:31]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Snippet of first build failure:

[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-inv_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-blocktransactionsrequest_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-blockmerkleroot
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-bloomfilter_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-coins_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-txoutcompressor_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-messageheader_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/*.log.* ***
[18:10:32]W:	 [Step 1/1] ++ print_sanitizers_log
[18:10:32]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[18:10:32]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/*.log.* ***'
[18:10:32]W:	 [Step 1/1] ++ cat '/tmp/sanitizer_logs/*.log.*'
[18:10:32]W:	 [Step 1/1] cat: '/tmp/sanitizer_logs/*.log.*': No such file or directory
[18:10:32]W:	 [Step 1/1] Process exited with code 1
[18:10:32]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Less convoluted implementation on feedback from Fabien

This revision is now accepted and ready to land.Mar 19 2020, 20:13