[CI] Remove 32 bits Linux builds

Summary: This has been removed from the released file a long time ago. The build files (e.g. cmake toolchain file) are kept in place so they can still be used for libsecp256k1.

Test Plan: Check the 64 bits build is not affected

Reviewers: #bitcoin_abc, PiRK

Reviewed By: #bitcoin_abc, PiRK

Differential Revision: https://reviews.bitcoinabc.org/D16915