Page MenuHomePhabricator

Build CPython in depends (Linux only)
ClosedPublic

Authored by Fabien on Oct 22 2024, 12:50.

Details

Reviewers
PiRK
Group Reviewers
Restricted Project
Commits
rABCb68985973b01: Build CPython in depends (Linux only)
Summary

This adds 2 new packages:

  • native_cpython: build cpython for the host. Only the python binary is built, it's required for building the cross cpython
  • cpython: cross build cpython. The (limited) feature set is found in the Setup.local file. Note that it requires sock support from openssl. The hashlib library relies on HACL that we need to copy also so it can later be linked against.

After this diff we end up with 3 different python when cross compiling:

  • The host python, which is the one we rely on to access all the python libraries (including the external ones like lief)
  • The native python, which is only there to boostrap the cross python build
  • The cross python that we want to link against in order to deliver a working binary with chronik plugins support.

The main CMakeLists.txt is updated to reflect this as we should always use the python host interpreter. Because python doesn't support cmake's config mode, we can't use the extended find_package signature to specify which location to search. So we manually disable the cross compilation global variables to work around this limitation.

Note that one can disable openssl in the depends, which will fail the build for cpython. There are other similar issues already (e.g. Qt will fail as well) and this is a very advanced feature, so I think it's fine as-is.

Test Plan
cd depends
make build-linux64
make build-linux-aarch64

Diff Detail

Repository
rABC Bitcoin ABC
Branch
depends_cython
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 30731
Build 60978: Build Diffbuild-without-wallet · build-diff · build-debug · build-clang · build-clang-tidy
Build 60977: arc lint + arc unit

Event Timeline

Fabien requested review of this revision.Oct 22 2024, 12:50

@bot build-linux64 build-linux-aarch64

Fix weird char in a comment

depends/patches/cpython/Setup.local
1 ↗(On Diff #50274)

This file can/should be discussed. It lists the python modules that will be available to the plugin developer, and some might missing or not be needed.

Fabien edited the summary of this revision. (Show Details)

Tail of the build log:

checking whether POSIX semaphores are enabled... yes
checking for broken sem_getvalue... yes
checking whether RTLD_LAZY is declared... yes
checking whether RTLD_NOW is declared... yes
checking whether RTLD_GLOBAL is declared... yes
checking whether RTLD_LOCAL is declared... yes
checking whether RTLD_NODELETE is declared... yes
checking whether RTLD_NOLOAD is declared... yes
checking whether RTLD_DEEPBIND is declared... yes
checking whether RTLD_MEMBER is declared... no
checking digit size for Python's longs... no value specified
checking for wchar.h... (cached) yes
checking size of wchar_t... 4
checking whether wchar_t is signed... yes
checking whether wchar_t is usable... no
checking whether byte ordering is bigendian... no
checking ABIFLAGS... 
checking SOABI... cpython-312-x86_64-linux-gnu
checking LDVERSION... $(VERSION)$(ABIFLAGS)
checking for --with-platlibdir... no
checking for --with-wheel-pkg-dir... no
checking whether right shift extends the sign bit... yes
checking for getc_unlocked() and friends... yes
checking for readline... no
checking for readline/readline.h... no
checking how to link readline... no
checking for broken nice()... no
checking for broken poll()... no
checking for working tzset()... no
checking for tv_nsec in struct stat... yes
checking for tv_nsec2 in struct stat... no
checking for curses.h... yes
checking for ncurses.h... yes
checking for ncursesw... no
checking for initscr in -lncursesw... yes
checking curses module flags... ncursesw (CFLAGS: , LIBS: -lncursesw)
checking for panel.h... yes
checking for panelw... no
checking for update_panels in -lpanelw... yes
checking panel flags... panelw (CFLAGS: , LIBS: -lpanelw)
checking for term.h... yes
checking whether mvwdelch is an expression... yes
checking whether WINDOW has _flags... yes
checking for curses function is_pad... yes
checking for curses function is_term_resized... yes
checking for curses function resize_term... yes
checking for curses function resizeterm... yes
checking for curses function immedok... yes
checking for curses function syncok... yes
checking for curses function wchgat... yes
checking for curses function filter... yes
checking for curses function has_key... yes
checking for curses function typeahead... yes
checking for curses function use_env... yes
configure: checking for device files
checking for /dev/ptmx... not set
configure: error: set ac_cv_file__dev_ptmx to yes/no in your CONFIG_SITE file when cross compiling
make[1]: *** [funcs.mk:297: /work/depends/work/build/x86_64-linux-gnu/cpython/3.12.7-5e49648e90d/./.stamp_configured] Error 1
make: *** [Makefile:242: build-linux64] Error 2
Build build-linux64 failed with exit code 2

Tail of the build log:

checking whether POSIX semaphores are enabled... yes
checking for broken sem_getvalue... yes
checking whether RTLD_LAZY is declared... yes
checking whether RTLD_NOW is declared... yes
checking whether RTLD_GLOBAL is declared... yes
checking whether RTLD_LOCAL is declared... yes
checking whether RTLD_NODELETE is declared... yes
checking whether RTLD_NOLOAD is declared... yes
checking whether RTLD_DEEPBIND is declared... yes
checking whether RTLD_MEMBER is declared... no
checking digit size for Python's longs... no value specified
checking for wchar.h... (cached) yes
checking size of wchar_t... 4
checking whether wchar_t is signed... yes
checking whether wchar_t is usable... no
checking whether byte ordering is bigendian... no
checking ABIFLAGS... 
checking SOABI... cpython-312-aarch64-linux-gnu
checking LDVERSION... $(VERSION)$(ABIFLAGS)
checking for --with-platlibdir... no
checking for --with-wheel-pkg-dir... no
checking whether right shift extends the sign bit... yes
checking for getc_unlocked() and friends... yes
checking for readline... no
checking for readline/readline.h... no
checking how to link readline... no
checking for broken nice()... no
checking for broken poll()... no
checking for working tzset()... no
checking for tv_nsec in struct stat... yes
checking for tv_nsec2 in struct stat... no
checking for curses.h... yes
checking for ncurses.h... yes
checking for ncursesw... no
checking for initscr in -lncursesw... no
checking for ncurses... no
checking for initscr in -lncurses... no
checking curses module flags... no
checking for panel.h... yes
checking panel flags... no
checking for term.h... yes
checking whether mvwdelch is an expression... yes
checking whether WINDOW has _flags... yes
checking for curses function is_pad... yes
checking for curses function is_term_resized... yes
checking for curses function resize_term... yes
checking for curses function resizeterm... yes
checking for curses function immedok... yes
checking for curses function syncok... yes
checking for curses function wchgat... yes
checking for curses function filter... yes
checking for curses function has_key... yes
checking for curses function typeahead... yes
checking for curses function use_env... yes
configure: checking for device files
checking for /dev/ptmx... not set
configure: error: set ac_cv_file__dev_ptmx to yes/no in your CONFIG_SITE file when cross compiling
make[1]: *** [funcs.mk:297: /work/depends/work/build/aarch64-linux-gnu/cpython/3.12.7-c442954f2ec/./.stamp_configured] Error 1
make: *** [Makefile:248: build-linux-aarch64] Error 2
Build build-linux-aarch64 failed with exit code 2

@bot build-linux64 build-linux-aarch64

Fabien retitled this revision from Build CPyhton in depends (Linux only) to Build CPython in depends (Linux only).Oct 22 2024, 13:58

Typo in Diff title

Good catch, fixed

Looks good. I'm not really on top of chronik plugins yet so not sure about the needed packages in Setup.local.

It is always possible to add more libs later if needed

This revision is now accepted and ready to land.Oct 23 2024, 06:53
This revision was automatically updated to reflect the committed changes.