diff --git a/.forgejo/workflows/package.yml b/.forgejo/workflows/package.yml index f64b0a5..30c381d 100644 --- a/.forgejo/workflows/package.yml +++ b/.forgejo/workflows/package.yml @@ -8,16 +8,17 @@ name: package # Build: crafter-build cross-compiles against an Alpine aarch64 sysroot # (packaging/build-package.sh — AGENTS.md §4.3's cross-compiling flow), the # test suite runs natively, and packaging/APKBUILD.binary wraps the result. -# This needs no privileged runner: it runs in an alpine:edge container on -# the ordinary arch-latest runner. +# crafter-build is the Crafter.Build Flatpak, and bubblewrap inside the job +# container needs a privileged runner: the job runs on the `flatpak` label +# (a privileged runner registered to this repository), in an alpine:edge +# container. # # Release gating is the version: pkgver comes from implementations/main.cpp, # the registry answers 409 for an already-published version, and the publish # step treats that as "nothing to do" — so pushes only release when the # Version constant bumps. This is the only producer of the imsd apk: fp6-img -# images install the published package from the registry (pinned version, -# pinned sha256) instead of building their own, so image and 'apk upgrade' -# carry the same binary. +# images install the published package from the registry instead of +# building their own, so image and 'apk upgrade' carry the same binary. # # Publishing needs PACKAGE_TOKEN (catbot account, package:write scope) — an # org-level secret on Catcrafts since 2026-09-02, shared with fp6-img; @@ -30,7 +31,7 @@ on: jobs: package: - runs-on: arch-latest + runs-on: flatpak container: image: alpine:edge timeout-minutes: 90 diff --git a/AGENTS.md b/AGENTS.md index 5c9f446..e48a494 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -105,8 +105,24 @@ pdn/boot match, defaults for absent keys — proven on the phone only, captures ### 4.1 Native, with crafter-build -Prerequisites: crafter-build, clang with libc++, lld, and the gio-2.0 headers -(`glib2` on Arch, `glib-dev` on Alpine). +crafter-build is the Crafter.Build Flatpak. It carries its own clang, lld and +libc++, and its SDK (org.freedesktop.Sdk, from Flathub) has the gio-2.0 +headers, so the host needs Flatpak and nothing else: + +```sh +flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo +flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo +flatpak install catcrafts net.catcrafts.CrafterBuild//stable +mkdir -p ~/.local/bin +printf '#!/bin/sh\nexec flatpak run net.catcrafts.CrafterBuild//stable "$@"\n' > ~/.local/bin/crafter-build +chmod +x ~/.local/bin/crafter-build +``` + +`crafter-build` is a script and not an alias so that the name also resolves in +a shell that has not defined the alias (a fresh `bash -c`, a CI step); +`~/.local/bin` has to be on `PATH` (any directory on it will do). The +sandbox does not see the host's `/tmp`: keep the checkout under `/home`, or +grant its directory with `flatpak run --filesystem=`. ```sh crafter-build # bin/imsd----/imsd @@ -120,6 +136,14 @@ crafter-build lint # the house-style rules of lint-rules.h Output directories under `bin/` are named `----`; the `./bin/imsd-*/imsd` globs in §8.3 rely on that. +A host-native binary built this way links the Flatpak's libc++ through an +rpath into `/app/lib`: run it inside the sandbox (`flatpak run --command=sh +net.catcrafts.CrafterBuild//stable`, with `--socket=session-bus` for §8.3's +daemons — the sandbox's own session bus is a filter that refuses their names; +`crafter-build test` runs the suites there +anyway), or build with `make` (§4.2) for one that runs on the host. A sysroot +build (§4.3) gets no such rpath — it links the sysroot's libc++. + ### 4.2 With make (distro packaging) GNU make, `clang++` with libc++ and the libc++ std module sources (`std.cppm`: @@ -152,22 +176,30 @@ the `80-imsd-ofono.preset`. ### 4.3 Cross-compiling for a phone (aarch64 Alpine / postmarketOS) ```sh -packaging/make-sysroot.sh # one-time: Alpine aarch64 sysroot from the CDN +packaging/make-sysroot.sh # one-time: Alpine aarch64 sysroot from the CDN (edge) crafter-build -- --target=aarch64-alpine-linux-musl \ --sysroot=$HOME/.cache/imsd/sysroot-aarch64-alpine \ --march=armv8.6-a+fp16fml+aes+sha3+sm4 --mtune=cortex-a720 -# the same flags after `crafter-build test --target=aarch64-alpine-linux-musl` -# run the suite under qemu-aarch64 against the sysroot +# `crafter-build test` with the same flags fails: its runner is qemu-aarch64, +# which the Flatpak's sandbox does not have. Test host-native (§4.1). ``` `make-sysroot.sh` needs no root, qemu or device: it downloads `apk-tools-static` and the Alpine keys and populates a minimal rootfs (musl, -libc++, glib and their headers, compiler-rt, libunwind, gcc for its +libc++ with its std module source, glib and their headers, compiler-rt, +libunwind, gcc for its `crtbeginS.o`/libgcc, which clang's `*-alpine-linux-musl` driver links against). In a cross build `project.cpp` does not ask the host's pkg-config (it would answer for the wrong architecture); it adds `-I=/usr/include/glib-2.0` and `-I=/usr/lib/glib-2.0/include`, which clang resolves inside the sysroot. -This is the flow CI runs (§9.4). +crafter-build compiles the std module against the sysroot's libc++ headers +from the sysroot's own source — Alpine's `llvm-runtimes`, the same release +as its libc++, which `make-sysroot.sh` installs (`libc++-dev` ships none); +run over an existing sysroot the script upgrades its packages (`apk add +--upgrade`). +Without that source it falls back to the toolchain's source for the same libc++ +major release, then to the sources Crafter.Build carries for other +releases. This is the flow CI runs (§9.4). ## 5. Tests @@ -791,7 +823,9 @@ Known gaps, all inherited rather than introduced: (`imsd-ofono-restart-calls.sh`) restarts the Calls daemon of every user running one. The first start since boot leaves a running Calls alone. -Development without a phone or root: +Development without a phone or root (crafter-build's binaries inside the +sandbox, the `make` build's `build/make/imsd` and `build/make/imsd-ofonod` on +the host: §4.1): ```sh ./bin/imsd-*/imsd --session & # imsd's ABI on the session bus @@ -857,10 +891,15 @@ install finds no unit active and marks nothing. `imsd-systemd` marks ### 9.4 CI and what a push means `.forgejo/workflows/package.yml` runs on every push to `main` (and on -dispatch): in an `alpine:edge` container, `packaging/build-package.sh` -provisions clang/lld/llvm-runtimes/glib-dev/abuild, fetches the musl build of -crafter-build, builds the sysroot, cross-compiles the four products, runs the -suite natively, wraps the result with `APKBUILD.binary`, and the publish step +dispatch): on the `flatpak` runner label (a privileged runner registered to +this repository — bubblewrap inside the job container needs the privilege), in an +`alpine:edge` container, `packaging/build-package.sh` installs the +Crafter.Build Flatpak (`net.catcrafts.CrafterBuild//stable` from +flatpak.catcrafts.net — the branch Crafter.Build publishes from its release +tags, where `beta` follows every push to its master; the build log names its +commit) and llvm/abuild, builds +the sysroot, cross-compiles the four products, runs the suite natively inside +the Flatpak, wraps the result with `APKBUILD.binary`, and the publish step uploads the apks to the Forgejo Alpine registry (`…/api/packages/Catcrafts/alpine/edge/fp6`) with `PACKAGE_TOKEN`. The registry answers 409 for an already-published version and the step treats diff --git a/README.md b/README.md index ff99e53..81745cc 100644 --- a/README.md +++ b/README.md @@ -42,11 +42,11 @@ your carrier's P-CSCF address. On a Catcrafts Fairphone 6 image imsd is installed from the project's apk registry and upgrades with `apk upgrade` (an upgrade restarts the daemon; a -call in progress ends). Elsewhere, build it (below), `make install`, and -enable the units yourself — `systemctl enable imsd`, and `systemctl enable ---now imsd-ofonod` for GNOME Calls: imsd's preset, the -`multi-user.target.wants` link and the `/etc/imsd.env` condition below come -with the apk, not with `make install`. +call in progress ends). Elsewhere, build it with `make` (below), run +`make install`, and enable the units yourself — `systemctl enable imsd`, +and `systemctl enable --now imsd-ofonod` for GNOME Calls: imsd's preset, +the `multi-user.target.wants` link and the `/etc/imsd.env` condition +below come with the apk, not with `make install`. Configuration is environment variables in `/etc/imsd.env`; the one required setting is the P-CSCF: @@ -102,13 +102,23 @@ session and root may place, answer and end calls; an ssh login may not. ## Building -[Crafter Build](https://forgejo.catcrafts.net/Catcrafts/Crafter.Build) plus -clang/libc++/lld and the gio-2.0 headers: +With the [Crafter Build](https://forgejo.catcrafts.net/Catcrafts/Crafter.Build) +Flatpak, which carries the compiler: ```sh +flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo +flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo +flatpak install catcrafts net.catcrafts.CrafterBuild//stable +alias crafter-build='flatpak run net.catcrafts.CrafterBuild//stable' crafter-build # the daemon; -- --product=media|dialerd|ofonod for the others crafter-build test # the seven unit-test suites -make && make check # the same, without crafter-build (distro packaging) +``` + +Or without crafter-build (distro packaging), with clang/libc++/lld and the +gio-2.0 headers: + +```sh +make && make check ``` Cross-compiling for a phone, the Makefile's requirements, the package layout diff --git a/packaging/build-package.sh b/packaging/build-package.sh index e316a10..53c6153 100755 --- a/packaging/build-package.sh +++ b/packaging/build-package.sh @@ -5,44 +5,28 @@ # cross-compiling flow of AGENTS.md §4.3), run the test suite natively, and package # the result via packaging/APKBUILD.binary. Expects an x86_64 Alpine # environment with root — the workflow runs it in an alpine:edge job -# container on an ordinary runner. Root only installs packages and hands off -# to a scratch user: the sysroot is built with apk.static --usermode (which -# refuses root) and abuild wants a user too. +# container on the privileged `flatpak` runner: crafter-build is the +# Crafter.Build Flatpak, and bubblewrap inside the container needs the +# privilege. Root only installs packages and hands off to a scratch user: the +# sysroot is built with apk.static --usermode (which refuses root) and abuild +# wants a user too. # # Built packages land in /home/build/.local/share/abuild/*/aarch64/imsd*.apk; # the workflow's publish step uploads them to the Forgejo Alpine registry. set -eu -# The musl build of crafter-build (Crafter.Build CI's release-musl job): -# this container is Alpine, and the glibc launcher cannot run on musl. v2 = -# SSE4.2 baseline: the CI box is an Intel N5105 (no AVX). Overridable for -# local rehearsals (file:// works). -CRAFTER_URL=${CRAFTER_URL:-https://forgejo.catcrafts.net/Catcrafts/Crafter.Build/releases/download/latest/crafter-build-linux-x86_64-musl-v2.tar.gz} +# crafter-build is the Crafter.Build Flatpak: it carries its own clang, lld +# and libc++, and its SDK has glib, so nothing that compiles is installed +# here. `stable` is the branch Crafter.Build publishes from its release tags +# (`beta` follows every push to its master), so the toolchain of a release +# moves only with a release of the toolchain. The branch is overridable +# for a rehearsal against another one. +CRAFTER_REMOTE=${CRAFTER_REMOTE:-https://flatpak.catcrafts.net/catcrafts.flatpakrepo} +CRAFTER_REF=${CRAFTER_REF:-net.catcrafts.CrafterBuild//stable} SRC=$(CDPATH= cd -- "$(dirname "$0")/.." && pwd) -# clang cross-targets aarch64 natively and the target's libc++/glib come from -# the sysroot; llvm-runtimes/libc++-dev/glib-dev here serve the NATIVE -# test-suite run. build-base = Alpine's standard build environment (the one -# abuild implies): binutils' ld/ar for clang's default link driver, gcc's -# libgcc_s/crt objects the musl clang driver links against. -if [ "$(id -u)" = 0 ]; then - apk add -q git curl tar clang lld llvm llvm-runtimes libc++-dev llvm-libunwind-dev glib-dev \ - build-base abuild sudo - id build >/dev/null 2>&1 || adduser -D build - addgroup build abuild 2>/dev/null || true - echo 'build ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/build - # abuild in cross mode strips with $CHOST-strip; llvm-strip handles any - # ELF arch, so give it that name - ln -sf "$(command -v llvm-strip)" /usr/local/bin/aarch64-alpine-linux-musl-strip - # the CI checkout arrives root-owned; crafter-build writes bin/ into it - chown -R build "$SRC" - # -l: a login shell, so HOME really is /home/build (abuild keys + output); - # it scrubs the environment, so carry the one knob that matters across - exec su -l build -c "CRAFTER_URL='${CRAFTER_URL:-}' sh -eu '$SRC/packaging/build-package.sh'" -fi - retry() { # retry _desc=$1; shift for _i in 1 2 3; do @@ -54,26 +38,49 @@ retry() { # retry return 1 } +# llvm for llvm-strip (abuild in cross mode strips with $CHOST-strip); the +# compiler is the Flatpak's. +if [ "$(id -u)" = 0 ]; then + apk add -q git curl tar flatpak llvm abuild sudo + # the Flatpak as a system installation the build user runs; its SDK + # (org.freedesktop.Sdk) comes from Flathub + flatpak remote-add --if-not-exists catcrafts "$CRAFTER_REMOTE" + flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo + retry "install crafter-build" flatpak install -y --noninteractive catcrafts "$CRAFTER_REF" + id build >/dev/null 2>&1 || adduser -D build + addgroup build abuild 2>/dev/null || true + echo 'build ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/build + # abuild in cross mode strips with $CHOST-strip; llvm-strip handles any + # ELF arch, so give it that name + ln -sf "$(command -v llvm-strip)" /usr/local/bin/aarch64-alpine-linux-musl-strip + # the CI checkout arrives root-owned; crafter-build writes bin/ into it + chown -R build "$SRC" + # -l: a login shell, so HOME really is /home/build (abuild keys + output); + # it scrubs the environment, so carry the one knob that matters across + exec su -l build -c "CRAFTER_REF='$CRAFTER_REF' sh -eu '$SRC/packaging/build-package.sh'" +fi + # implementations/main.cpp is the version's single source of truth (same # derivation as make-bin-tarball.sh) VER=$(sed -n 's/.*char\* Version = "\(.*\)".*/\1/p' "$SRC/implementations/main.cpp") [ -n "$VER" ] || { echo "cannot read Version from implementations/main.cpp" >&2; exit 1; } echo ">> packaging imsd $VER" -# --- crafter-build: static launcher from the rolling release -mkdir -p "$HOME/crafter-build" -retry "fetch crafter-build" \ - sh -c "curl -fsSL '$CRAFTER_URL' | tar -xz -C '$HOME/crafter-build'" -PATH="$HOME/crafter-build/bin:$PATH" -export CRAFTER_BUILD_HOME="$HOME/crafter-build/share/crafter-build" +# --- crafter-build: the Flatpak, under the name the rest of this file uses. +# The build log names the build it ran with. +mkdir -p "$HOME/bin" +printf '#!/bin/sh\nexec flatpak run %s "$@"\n' "$CRAFTER_REF" > "$HOME/bin/crafter-build" +chmod +x "$HOME/bin/crafter-build" +PATH="$HOME/bin:$PATH" +flatpak info "$CRAFTER_REF" | sed -n 's/^ *\(Version\|Commit\|Date\): */crafter-build \1: /p' +crafter-build --version # --- aarch64 Alpine sysroot (unprivileged: apk.static --usermode) SYSROOT="$HOME/.cache/imsd/sysroot-aarch64-alpine" retry "make sysroot" "$SRC/packaging/make-sysroot.sh" "$SYSROOT" -# --- cross-compile the products; run the suites natively (same sources, -# host arch — the aarch64 binaries are additionally exercised by fp6-img's -# chroot `make check` whenever an image builds) +# --- cross-compile the products; run the suites natively, inside the Flatpak +# (same sources, host arch: no suite runs on the aarch64 binaries) cd "$SRC" # The phone is one SoC, 4x Cortex-A520 + 4x Cortex-A720, so build for it. # armv8.6-a is the highest level whose mandatory set it exposes (8.7 would diff --git a/packaging/make-sysroot.sh b/packaging/make-sysroot.sh index fdca8d8..112b7e2 100755 --- a/packaging/make-sysroot.sh +++ b/packaging/make-sysroot.sh @@ -8,7 +8,8 @@ # clang can use via --sysroot. No root, no qemu, no device needed: # # packaging/make-sysroot.sh [dir] # default: ~/.cache/imsd/sysroot-aarch64-alpine -# crafter-build --target=aarch64-alpine-linux-musl --sysroot= +# crafter-build -- --target=aarch64-alpine-linux-musl --sysroot= \ +# --march=… --mtune=… # the phone's values: AGENTS.md §4.3 set -eu SYSROOT="${1:-$HOME/.cache/imsd/sysroot-aarch64-alpine}" @@ -16,8 +17,13 @@ MIRROR="${MIRROR:-https://dl-cdn.alpinelinux.org/alpine/edge}" ARCH=aarch64 # gcc is in the list for its crtbeginS.o/libgcc: clang's driver for # *-alpine-linux-musl links against the GCC runtime it detects under -# /usr/lib/gcc// -PKGS="musl-dev libc++ libc++-dev glib-dev compiler-rt llvm-libunwind-dev gcc" +# /usr/lib/gcc//. llvm-runtimes is in it for the std module +# source of this very libc++ (usr/share/libc++/v1/std.cppm; libc++-dev ships +# none): crafter-build compiles the sysroot's own copy first. --upgrade below +# is for a run over an existing sysroot: added alone beside libc++ 22, +# llvm-runtimes 23's source did not compile against its headers; upgraded +# with it, it did. +PKGS="musl-dev libc++ libc++-dev llvm-runtimes glib-dev compiler-rt llvm-libunwind-dev gcc" work=$(mktemp -d) trap 'rm -rf "$work"' EXIT @@ -40,6 +46,6 @@ mkdir -p "$SYSROOT" --arch "$ARCH" --root "$SYSROOT" --initdb --no-scripts --no-cache \ --usermode --keys-dir "$work/usr/share/apk/keys/$ARCH" \ -X "$MIRROR/main" -X "$MIRROR/community" \ - add $PKGS + add --upgrade $PKGS echo ">> done: $SYSROOT"