Jorijn van der Graaf
dc239f4257
The Linux host code paths were gated on the x86_64_pc_linux_gnu target define alone, and LoadProject hardcoded --target=x86_64-pc-linux-gnu when compiling project.cpp and the module PCMs. Both break on musl hosts: the glibc release launcher cannot run there at all (no glibc loader; gcompat lacks libgcc_s and the __isoc23_* symbols), and a musl-built launcher would still compile project.cpp for the wrong libc. - The host gates test clang's __linux__ instead of one triple's define: they mean "Linux host", whatever the libc (or, later, the CPU). - LoadProject derives the host triple from HostTarget() (clang's -print-target-triple), which is also more correct on glibc distros whose triple isn't x86_64-pc-linux-gnu. - project.cpp / the cmake libc++ flags treat any *-linux-gnu or *-linux-musl target as Linux. - build.sh targets the host toolchain's triple (CRAFTER_BUILD_TARGET overrides) and derives the per-target define from it. On Arch this is byte-for-byte the previous behaviour. - CI: a release-musl job bootstraps on alpine:edge, runs the tests, and attaches crafter-build-linux-x86_64-musl-v2.tar.gz to the rolling latest release (via the API, so the glibc assets survive). Dynamic against musl libc++: the launcher dlopens project.so, so it cannot be fully static. First consumer: imsd's package CI, which cross-compiles for aarch64 from an Alpine container. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
261 lines
13 KiB
YAML
261 lines
13 KiB
YAML
name: CI
|
||
|
||
on:
|
||
pull_request:
|
||
branches: [master]
|
||
push:
|
||
branches: [master]
|
||
workflow_dispatch:
|
||
|
||
jobs:
|
||
build-test-release:
|
||
runs-on: arch-latest
|
||
env:
|
||
# x86-64-v2 = SSE4.2 baseline; runs on the CI SBC (Intel N5105 / Tremont,
|
||
# no AVX) and on essentially every x86_64 CPU since ~2011. The bootstrap
|
||
# MUST stay v2 — anything higher would emit instructions the SBC can't
|
||
# execute and the bootstrap binary itself would crash before it could
|
||
# rebuild for higher levels. v3/v4 variants are produced from this v2
|
||
# bootstrap by re-running it with CRAFTER_BUILD_MARCH overridden.
|
||
CRAFTER_BUILD_MARCH: x86-64-v2
|
||
CRAFTER_BUILD_MTUNE: generic
|
||
steps:
|
||
- name: Install build dependencies
|
||
run: |
|
||
# The slim archlinux:latest image arrives without a populated
|
||
# pacman keyring AND without a local pacman master key, so:
|
||
# 1. --init generates the local signing key (needed when pacman
|
||
# itself rewrites the keyring during package upgrades)
|
||
# 2. --populate archlinux imports the upstream master keys so
|
||
# currently-shipping signatures verify
|
||
# After that we can refresh archlinux-keyring to pick up keys for
|
||
# packagers added after the image's snapshot, then -Syu the rest.
|
||
pacman-key --init
|
||
pacman-key --populate archlinux
|
||
pacman -Sy --noconfirm --needed archlinux-keyring
|
||
pacman -Syu --noconfirm --needed \
|
||
base-devel git zip tar jq \
|
||
clang llvm lld libc++ cmake \
|
||
mingw-w64-gcc \
|
||
wasi-libc wasi-libc++ wasi-libc++abi wasi-compiler-rt \
|
||
nodejs
|
||
# The container runs as root and the workspace may be owned by a
|
||
# different uid; tell git not to refuse operations on it.
|
||
git config --global --add safe.directory '*'
|
||
|
||
- name: Checkout
|
||
uses: actions/checkout@v4
|
||
with:
|
||
# Persist the auth token so the 'Update rolling latest tag' step
|
||
# below can push the tag back via the implicit GITHUB_TOKEN.
|
||
persist-credentials: true
|
||
|
||
- name: Cache glslang clone+build
|
||
# Cache covers crafter-build's per-(target, march) external dep cache.
|
||
# First run builds glslang up to 6 times (3 marches × 2 targets); the
|
||
# cache makes subsequent runs reuse those builds.
|
||
uses: actions/cache@v4
|
||
with:
|
||
path: |
|
||
~/.cache/crafter.build/external
|
||
key: glslang-${{ runner.os }}-v2
|
||
|
||
- name: Bootstrap (build.sh)
|
||
run: ./build.sh
|
||
|
||
# Full style gate: Report() findings (naming, enum-class, char*, …) AND
|
||
# would-reformat findings from the transform rules. A dirty tree means
|
||
# someone skipped `crafter-build format` — fail fast, before the
|
||
# (much slower) test suite.
|
||
# - name: Lint
|
||
# run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build lint
|
||
|
||
- name: Run tests
|
||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build test
|
||
|
||
# Everything below produces release artifacts and only runs on push to
|
||
# master. PRs stop after the test step above — the bootstrap binary is
|
||
# enough to validate the change; the per-march variant builds and the
|
||
# archive uploads are wasted work on a PR.
|
||
- name: Build Linux x86-64-v2 (matches bootstrap)
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build
|
||
|
||
- name: Build Linux x86-64-v3
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build CRAFTER_BUILD_MARCH=x86-64-v3 ./bin/crafter-build
|
||
|
||
- name: Build Linux x86-64-v4
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build CRAFTER_BUILD_MARCH=x86-64-v4 ./bin/crafter-build
|
||
|
||
# - name: Build Windows x86-64-v2 (mingw)
|
||
# if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
# run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build --target=x86_64-w64-mingw32
|
||
|
||
# - name: Build Windows x86-64-v3 (mingw)
|
||
# if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
# run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build CRAFTER_BUILD_MARCH=x86-64-v3 ./bin/crafter-build --target=x86_64-w64-mingw32
|
||
|
||
# - name: Build Windows x86-64-v4 (mingw)
|
||
# if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
# run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build CRAFTER_BUILD_MARCH=x86-64-v4 ./bin/crafter-build --target=x86_64-w64-mingw32
|
||
|
||
- name: Package artifacts
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
run: |
|
||
set -eux
|
||
mkdir -p dist
|
||
|
||
# 6 archives — 3 marches × 2 OSes. Each is self-contained: the
|
||
# Linux launcher is statically linked; the Windows launcher has
|
||
# crafter-build.dll + libcrafter-build.dll.a alongside the exe.
|
||
for march in x86-64-v2 x86-64-v3 x86-64-v4; do
|
||
# Short suffix (v2/v3/v4) for archive filenames; the upload-artifact
|
||
# steps below reference these short names.
|
||
short=${march##*-}
|
||
|
||
# VariantId is "<name>-<target>-<march>-<mtune>-<hash>"; CI pins
|
||
# mtune=generic so each (target, march) resolves to a single subdir
|
||
# but the trailing hash isn't known to the workflow, hence the glob.
|
||
stage_lin=$(mktemp -d)
|
||
mkdir -p "$stage_lin/bin" "$stage_lin/lib"
|
||
cp bin/crafter.build-exe-x86_64-pc-linux-gnu-$march-*/crafter-build "$stage_lin/bin/"
|
||
cp bin/crafter.build-lib-x86_64-pc-linux-gnu-$march-*/libcrafter-build.a "$stage_lin/lib/"
|
||
cp -r share "$stage_lin/"
|
||
tar czf "dist/crafter-build-linux-x86_64-$short.tar.gz" -C "$stage_lin" .
|
||
|
||
# stage_win=$(mktemp -d)
|
||
# mkdir -p "$stage_win/bin"
|
||
# cp bin/crafter.build-exe-x86_64-w64-mingw32-$march-*/* "$stage_win/bin/"
|
||
# cp -r share "$stage_win/"
|
||
# (cd "$stage_win" && zip -r "$GITHUB_WORKSPACE/dist/crafter-build-windows-x86_64-$short.zip" .)
|
||
done
|
||
|
||
ls -la dist/
|
||
|
||
# Upload each archive as its own workflow artifact (one upload per file)
|
||
# so users browsing the PR / run page get six small, individually-named
|
||
# downloads instead of one wrapper zip containing the lot.
|
||
- name: Upload Linux v2 artifact
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
uses: actions/upload-artifact@v3
|
||
with: { name: crafter-build-linux-x86_64-v2, path: dist/crafter-build-linux-x86_64-v2.tar.gz, if-no-files-found: error }
|
||
- name: Upload Linux v3 artifact
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
uses: actions/upload-artifact@v3
|
||
with: { name: crafter-build-linux-x86_64-v3, path: dist/crafter-build-linux-x86_64-v3.tar.gz, if-no-files-found: error }
|
||
- name: Upload Linux v4 artifact
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
uses: actions/upload-artifact@v3
|
||
with: { name: crafter-build-linux-x86_64-v4, path: dist/crafter-build-linux-x86_64-v4.tar.gz, if-no-files-found: error }
|
||
# - name: Upload Windows v2 artifact
|
||
# if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
# uses: actions/upload-artifact@v3
|
||
# with: { name: crafter-build-windows-x86_64-v2, path: dist/crafter-build-windows-x86_64-v2.zip, if-no-files-found: error }
|
||
# - name: Upload Windows v3 artifact
|
||
# if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
# uses: actions/upload-artifact@v3
|
||
# with: { name: crafter-build-windows-x86_64-v3, path: dist/crafter-build-windows-x86_64-v3.zip, if-no-files-found: error }
|
||
# - name: Upload Windows v4 artifact
|
||
# if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
# uses: actions/upload-artifact@v3
|
||
# with: { name: crafter-build-windows-x86_64-v4, path: dist/crafter-build-windows-x86_64-v4.zip, if-no-files-found: error }
|
||
|
||
- name: Update rolling 'latest' tag
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
run: |
|
||
git config user.email "ci@catcrafts.net"
|
||
git config user.name "Crafter Build CI"
|
||
git tag -f latest
|
||
git push origin latest --force
|
||
|
||
- name: Publish rolling 'latest' release
|
||
# forgejo-release uploads each file in release-dir as a separate
|
||
# release asset, so the Releases page shows six individually
|
||
# downloadable archives — pick the v-level matching your CPU.
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
uses: https://code.forgejo.org/actions/forgejo-release@v2
|
||
with:
|
||
direction: upload
|
||
url: ${{ github.server_url }}
|
||
repo: ${{ github.repository }}
|
||
tag: latest
|
||
title: Latest main build
|
||
prerelease: true
|
||
override: true
|
||
release-dir: dist
|
||
token: ${{ secrets.GITHUB_TOKEN }}
|
||
|
||
# musl build for Alpine / postmarketOS hosts: the glibc launcher above cannot
|
||
# run there (musl has no glibc loader and gcompat lacks libgcc_s + the
|
||
# __isoc23_* symbols). Bootstrap natively on alpine:edge and ship the
|
||
# bootstrap output — the same thing the Arch PKGBUILD installs. Dynamic
|
||
# against musl libc++ (crafter-build dlopens project.so, so it cannot be
|
||
# fully static); consumers need `apk add clang lld libc++-dev llvm-runtimes`.
|
||
release-musl:
|
||
needs: build-test-release
|
||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
||
runs-on: arch-latest
|
||
container:
|
||
image: alpine:edge
|
||
env:
|
||
CRAFTER_BUILD_MARCH: x86-64-v2
|
||
CRAFTER_BUILD_MTUNE: generic
|
||
steps:
|
||
- name: Install build dependencies
|
||
# nodejs: actions/checkout is a Node action. build-base: gcc's
|
||
# libgcc_s + crt objects, which clang's musl driver links against, plus
|
||
# make/binutils for the glslang cmake build (python3: its generators).
|
||
# clang-dev + llvm-dev: the libclang header Crafter.Build-Lint includes
|
||
# lives under llvm-config --includedir here, not /usr/include. llvm:
|
||
# llvm-ar for the LTO static archives the self-host/test builds write.
|
||
run: |
|
||
apk add -q nodejs git curl jq tar clang clang-dev lld libc++-dev \
|
||
llvm-libunwind-dev llvm-runtimes llvm llvm-dev cmake build-base python3
|
||
git config --global --add safe.directory '*'
|
||
|
||
- name: Checkout
|
||
uses: actions/checkout@v4
|
||
|
||
- name: Bootstrap (build.sh)
|
||
run: ./build.sh
|
||
|
||
- name: Run tests
|
||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build test
|
||
|
||
- name: Package artifact
|
||
run: |
|
||
set -eux
|
||
file bin/crafter-build || true
|
||
mkdir -p dist
|
||
stage=$(mktemp -d)
|
||
mkdir -p "$stage/bin" "$stage/lib"
|
||
cp bin/crafter-build "$stage/bin/"
|
||
cp lib/libcrafter-build.a "$stage/lib/"
|
||
cp -r share "$stage/"
|
||
tar czf dist/crafter-build-linux-x86_64-musl-v2.tar.gz -C "$stage" .
|
||
ls -la dist/
|
||
|
||
- name: Upload musl artifact
|
||
uses: actions/upload-artifact@v3
|
||
with: { name: crafter-build-linux-x86_64-musl-v2, path: dist/crafter-build-linux-x86_64-musl-v2.tar.gz, if-no-files-found: error }
|
||
|
||
# Add the asset to the rolling 'latest' release the first job just
|
||
# (re)published, via the API rather than forgejo-release: that action's
|
||
# override mode would drop the other job's assets.
|
||
- name: Attach to rolling 'latest' release
|
||
env:
|
||
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||
API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
|
||
run: |
|
||
set -eu
|
||
asset=crafter-build-linux-x86_64-musl-v2.tar.gz
|
||
rel=$(curl -fsS -H "Authorization: token $TOKEN" "$API/releases/tags/latest")
|
||
id=$(printf '%s' "$rel" | jq -r .id)
|
||
for old in $(printf '%s' "$rel" | jq -r --arg n "$asset" '.assets[] | select(.name==$n) | .id'); do
|
||
curl -fsS -X DELETE -H "Authorization: token $TOKEN" "$API/releases/$id/assets/$old"
|
||
done
|
||
curl -fsS -H "Authorization: token $TOKEN" -F "attachment=@dist/$asset" \
|
||
"$API/releases/$id/assets?name=$asset" > /dev/null
|
||
echo "attached $asset to release $id"
|