Page MenuHomePhabricator

[CI] Do not re-download the OSX SDK if it's already available
ClosedPublic

Authored by jasonbcox on Thu, Mar 19, 17:10.

Details

Summary

This improves build time and bandwidth usage for the OSX gitian build on CI.

Test Plan

OSX Gitian build on CI

Diff Detail

Repository
rABC Bitcoin ABC
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

jasonbcox created this revision.Thu, Mar 19, 17:10
Herald added a reviewer: Restricted Project. · View Herald TranscriptThu, Mar 19, 17:10
jasonbcox planned changes to this revision.Thu, Mar 19, 17:10

Snippet of first build failure:

[17:38:34] :	 [Step 1/1] (Reading database ... 70%
[17:38:34] :	 [Step 1/1] (Reading database ... 75%
[17:38:34] :	 [Step 1/1] (Reading database ... 80%
[17:38:34] :	 [Step 1/1] (Reading database ... 85%
[17:38:34] :	 [Step 1/1] (Reading database ... 90%
[17:38:34] :	 [Step 1/1] (Reading database ... 95%
[17:38:34] :	 [Step 1/1] (Reading database ... 100%
[17:38:34] :	 [Step 1/1] (Reading database ... 17687 files and directories currently installed.)
[17:38:34] :	 [Step 1/1] Removing rsyslog (8.1901.0-1) ...
[17:38:34] :	 [Step 1/1] invoke-rc.d: could not determine current runlevel
[17:38:34] :	 [Step 1/1] Stopping enhanced syslogd: rsyslogd already stopped.
[17:38:34] :	 [Step 1/1] (Reading database ... 
[17:38:34] :	 [Step 1/1] (Reading database ... 5%
[17:38:34] :	 [Step 1/1] (Reading database ... 10%
[17:38:34] :	 [Step 1/1] (Reading database ... 15%
[17:38:34] :	 [Step 1/1] (Reading database ... 20%
[17:38:34] :	 [Step 1/1] (Reading database ... 25%
[17:38:34] :	 [Step 1/1] (Reading database ... 30%
[17:38:34] :	 [Step 1/1] (Reading database ... 35%
[17:38:34] :	 [Step 1/1] (Reading database ... 40%
[17:38:34] :	 [Step 1/1] (Reading database ... 45%
[17:38:34] :	 [Step 1/1] (Reading database ... 50%
[17:38:34] :	 [Step 1/1] (Reading database ... 55%
[17:38:34] :	 [Step 1/1] (Reading database ... 60%
[17:38:34] :	 [Step 1/1] (Reading database ... 65%
[17:38:34] :	 [Step 1/1] (Reading database ... 70%
[17:38:34] :	 [Step 1/1] (Reading database ... 75%
[17:38:34] :	 [Step 1/1] (Reading database ... 80%
[17:38:34] :	 [Step 1/1] (Reading database ... 85%
[17:38:34] :	 [Step 1/1] (Reading database ... 90%
[17:38:34] :	 [Step 1/1] (Reading database ... 95%
[17:38:34] :	 [Step 1/1] (Reading database ... 100%
[17:38:34] :	 [Step 1/1] (Reading database ... 17629 files and directories currently installed.)
[17:38:34] :	 [Step 1/1] Purging configuration files for rsyslog (8.1901.0-1) ...
[17:38:34] :	 [Step 1/1] Processing triggers for systemd (241-7~deb10u2) ...
[17:38:34] :	 [Step 1/1] Adding 'local diversion of /sbin/initctl to /sbin/initctl.distrib'
[17:38:34] :	 [Step 1/1] Adding 'local diversion of /usr/bin/ischroot to /usr/bin/ischroot.distrib'
[17:38:34]W:	 [Step 1/1] dpkg-divert: warning: diverting file '/usr/bin/ischroot' from an Essential package with rename is dangerous, use --no-rename
[17:38:34] :	 [Step 1/1] Adding 'local diversion of /usr/sbin/policy-rc.d to /usr/sbin/policy-rc.d.distrib'
[17:38:34] :	 [Step 1/1] Starting target
[17:38:35] :	 [Step 1/1] Checking if target is up
[17:38:35] :	 [Step 1/1] Preparing build environment
[17:38:38] :	 [Step 1/1] Updating apt-get repository (log in var/install.log)
[17:38:41] :	 [Step 1/1] Installing additional packages (log in var/install.log)
[17:39:46] :	 [Step 1/1] Upgrading system, may take a while (log in var/install.log)
[17:39:51] :	 [Step 1/1] Creating package manifest
[17:39:55] :	 [Step 1/1] Creating build script (var/build-script)
[17:39:56] :	 [Step 1/1] Running build script (log in var/build.log)
[17:46:31]W:	 [Step 1/1] ./bin/gbuild:23:in `system!': failed to run on-target setarch x86_64 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError)
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:169:in `build_one_configuration'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:320:in `block (2 levels) in <main>'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:315:in `each'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:315:in `block in <main>'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:313:in `each'
[17:46:31]W:	 [Step 1/1] 	from ./bin/gbuild:313:in `<main>'
[17:46:31]W:	 [Step 1/1] ++ move_log
[17:46:31]W:	 [Step 1/1] ++ mv var/install.log /home/teamcity/buildAgent/work/c4a5708f2bae7929/gitian-results/
[17:46:31]W:	 [Step 1/1] ++ mv var/build.log /home/teamcity/buildAgent/work/c4a5708f2bae7929/gitian-results/
[17:46:31]W:	 [Step 1/1] Process exited with code 1
[17:46:31]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)

Snippet of first build failure:

[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-inv_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-blocktransactionsrequest_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-blockmerkleroot
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-bloomfilter_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-coins_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-txoutcompressor_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] CMake Error at src/test/fuzz/CMakeLists.txt:5 (add_executable):
[18:10:32]W:	 [Step 1/1]   No SOURCES given to target: fuzz-messageheader_deserialize
[18:10:32]W:	 [Step 1/1] Call Stack (most recent call first):
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:24 (add_fuzz_target)
[18:10:32]W:	 [Step 1/1]   src/test/fuzz/CMakeLists.txt:38 (add_deserialize_fuzz_targets)
[18:10:32]W:	 [Step 1/1] 
[18:10:32]W:	 [Step 1/1] 
[18:10:32] :	 [Step 1/1] *** Output of /tmp/sanitizer_logs/*.log.* ***
[18:10:32]W:	 [Step 1/1] ++ print_sanitizers_log
[18:10:32]W:	 [Step 1/1] ++ for log in "${SAN_LOG_DIR}"/*.log.*
[18:10:32]W:	 [Step 1/1] ++ echo '*** Output of /tmp/sanitizer_logs/*.log.* ***'
[18:10:32]W:	 [Step 1/1] ++ cat '/tmp/sanitizer_logs/*.log.*'
[18:10:32]W:	 [Step 1/1] cat: '/tmp/sanitizer_logs/*.log.*': No such file or directory
[18:10:32]W:	 [Step 1/1] Process exited with code 1
[18:10:32]E:	 [Step 1/1] Process exited with code 1 (Step: Command Line)
jasonbcox planned changes to this revision.Thu, Mar 19, 18:19
jasonbcox requested review of this revision.Thu, Mar 19, 18:41
jasonbcox updated this revision to Diff 17058.Thu, Mar 19, 20:10

Less convoluted implementation on feedback from Fabien

Fabien accepted this revision.Thu, Mar 19, 20:13
This revision is now accepted and ready to land.Thu, Mar 19, 20:13