Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
5244fa9120 |
|||
|
3334b0b254 |
4 changed files with 175 additions and 236 deletions
|
|
@ -1,229 +1,182 @@
|
||||||
name: CI
|
name: Flatpak
|
||||||
|
|
||||||
|
# One job, three outcomes:
|
||||||
|
# pull request -> build the Flatpak and run the suite inside it
|
||||||
|
# push master -> also publish the result as the `beta` branch of the
|
||||||
|
# Catcrafts repo and refresh the rolling `latest` release
|
||||||
|
# tag X.Y.Z -> publish as `stable` and create release X.Y.Z
|
||||||
|
#
|
||||||
|
# The `flatpak` runner label is the privileged runner on the Hetzner box
|
||||||
|
# (bubblewrap needs it) with three volumes allowed: the Sdk install, the
|
||||||
|
# flatpak-builder state (ccache and the LLVM module, so only crafter-build
|
||||||
|
# itself rebuilds on a normal run) and the published repo's web root.
|
||||||
on:
|
on:
|
||||||
pull_request:
|
pull_request:
|
||||||
branches: [master]
|
branches: [master]
|
||||||
push:
|
push:
|
||||||
branches: [master]
|
branches: [master]
|
||||||
|
tags: ['[0-9]+.[0-9]+.[0-9]+']
|
||||||
workflow_dispatch:
|
workflow_dispatch:
|
||||||
|
|
||||||
|
env:
|
||||||
|
APP_ID: net.catcrafts.CrafterBuild
|
||||||
|
MANIFEST: packaging/flatpak/net.catcrafts.CrafterBuild.yaml
|
||||||
|
CI_MANIFEST: packaging/flatpak/ci-manifest.yaml
|
||||||
|
RUNTIME: org.freedesktop.Sdk//26.08
|
||||||
|
REPO_DIR: /srv/flatpak/repo
|
||||||
|
WEB_DIR: /srv/flatpak
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
build-test-release:
|
flatpak:
|
||||||
runs-on: arch-latest
|
runs-on: flatpak
|
||||||
env:
|
container:
|
||||||
# x86-64-v2 = SSE4.2 baseline; runs on the CI SBC (Intel N5105 / Tremont,
|
image: archlinux:latest
|
||||||
# no AVX) and on essentially every x86_64 CPU since ~2011. The bootstrap
|
options: >-
|
||||||
# MUST stay v2 — anything higher would emit instructions the SBC can't
|
-v crafter-flatpak-system:/var/lib/flatpak
|
||||||
# execute and the bootstrap binary itself would crash before it could
|
-v crafter-flatpak-builder:/root/.cache/flatpak-builder
|
||||||
# rebuild for higher levels. v3/v4 variants are produced from this v2
|
-v /var/www/flatpak:/srv/flatpak
|
||||||
# bootstrap by re-running it with CRAFTER_BUILD_MARCH overridden.
|
|
||||||
CRAFTER_BUILD_MARCH: x86-64-v2
|
|
||||||
CRAFTER_BUILD_MTUNE: generic
|
|
||||||
steps:
|
steps:
|
||||||
- name: Install build dependencies
|
- name: Install tools
|
||||||
run: |
|
run: |
|
||||||
# The slim archlinux:latest image arrives without a populated
|
|
||||||
# pacman keyring AND without a local pacman master key, so:
|
|
||||||
# 1. --init generates the local signing key (needed when pacman
|
|
||||||
# itself rewrites the keyring during package upgrades)
|
|
||||||
# 2. --populate archlinux imports the upstream master keys so
|
|
||||||
# currently-shipping signatures verify
|
|
||||||
# After that we can refresh archlinux-keyring to pick up keys for
|
|
||||||
# packagers added after the image's snapshot, then -Syu the rest.
|
|
||||||
pacman-key --init
|
pacman-key --init
|
||||||
pacman-key --populate archlinux
|
pacman-key --populate archlinux
|
||||||
pacman -Sy --noconfirm --needed archlinux-keyring
|
pacman -Sy --noconfirm --needed archlinux-keyring
|
||||||
pacman -Syu --noconfirm --needed \
|
pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree
|
||||||
base-devel git zip tar jq \
|
|
||||||
clang llvm lld libc++ cmake \
|
|
||||||
mingw-w64-gcc \
|
|
||||||
wasi-libc wasi-libc++ wasi-libc++abi wasi-compiler-rt \
|
|
||||||
nodejs
|
|
||||||
# The container runs as root and the workspace may be owned by a
|
|
||||||
# different uid; tell git not to refuse operations on it.
|
|
||||||
git config --global --add safe.directory '*'
|
git config --global --add safe.directory '*'
|
||||||
|
|
||||||
|
- name: Sdk
|
||||||
|
# A no-op once the volume is warm; `update` follows point releases.
|
||||||
|
run: |
|
||||||
|
flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
|
||||||
|
flatpak install -y --noninteractive flathub "$RUNTIME"
|
||||||
|
flatpak update -y --noninteractive
|
||||||
|
|
||||||
- name: Checkout
|
- name: Checkout
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
with:
|
with:
|
||||||
# Persist the auth token so the 'Update rolling latest tag' step
|
fetch-depth: 0
|
||||||
# below can push the tag back via the implicit GITHUB_TOKEN.
|
|
||||||
persist-credentials: true
|
persist-credentials: true
|
||||||
|
|
||||||
- name: Cache glslang clone+build
|
- name: Version and channel
|
||||||
# Cache covers crafter-build's external dep cache. The first run builds
|
id: meta
|
||||||
# glslang; the cache makes subsequent runs reuse that build.
|
run: |
|
||||||
uses: actions/cache@v4
|
set -eu
|
||||||
with:
|
if [ "${GITHUB_REF_TYPE}" = tag ]; then
|
||||||
path: |
|
version="$GITHUB_REF_NAME"
|
||||||
~/.cache/crafter.build/external
|
branch=stable
|
||||||
key: glslang-${{ runner.os }}-v2
|
# The metainfo's newest release is the release notes and must be
|
||||||
|
# the version being tagged.
|
||||||
- name: Bootstrap (build.sh)
|
top=$(grep -m1 -oP '<release version="\K[^"]+' packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml)
|
||||||
run: ./build.sh
|
if [ "$top" != "$version" ]; then
|
||||||
|
echo "metainfo lists $top as the newest release, but the tag is $version" >&2
|
||||||
# Full style gate: Report() findings (naming, enum-class, char*, …) AND
|
exit 1
|
||||||
# would-reformat findings from the transform rules. A dirty tree means
|
fi
|
||||||
# someone skipped `crafter-build format` — fail fast, before the
|
else
|
||||||
# (much slower) test suite.
|
version="0.0.0-dev+$(git rev-parse --short HEAD)"
|
||||||
# - name: Lint
|
branch=beta
|
||||||
# run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build lint
|
fi
|
||||||
|
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||||
- name: Run tests
|
echo "branch=$branch" >> "$GITHUB_OUTPUT"
|
||||||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build test
|
echo "version $version, flatpak branch $branch"
|
||||||
|
|
||||||
# Everything below produces release artifacts and only runs on push to
|
- name: Build
|
||||||
# master. PRs stop after the test step above — the bootstrap binary is
|
# The manifest is rewritten beside the original so its relative
|
||||||
# enough to validate the change; the per-march variant builds and the
|
# source paths still resolve. The suite runs inside this build.
|
||||||
# archive uploads are wasted work on a PR.
|
run: |
|
||||||
- name: Build Linux x86-64-v2 (matches bootstrap)
|
set -eu
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
yq ".build-options.env.CRAFTER_BUILD_VERSION = \"${{ steps.meta.outputs.version }}\"" "$MANIFEST" > "$CI_MANIFEST"
|
||||||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build
|
flatpak-builder --force-clean --ccache --disable-rofiles-fuse \
|
||||||
|
--state-dir=/root/.cache/flatpak-builder \
|
||||||
- name: Build Linux x86-64-v3
|
--repo=repo --default-branch="${{ steps.meta.outputs.branch }}" \
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
build-dir "$CI_MANIFEST"
|
||||||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build CRAFTER_BUILD_MARCH=x86-64-v3 ./bin/crafter-build
|
|
||||||
|
- name: Smoke test
|
||||||
- name: Build Linux x86-64-v4
|
run: |
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
flatpak-builder --run build-dir "$CI_MANIFEST" crafter-build --version
|
||||||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build CRAFTER_BUILD_MARCH=x86-64-v4 ./bin/crafter-build
|
flatpak-builder --run build-dir "$CI_MANIFEST" sh -c 'cd examples/hello-world && crafter-build && ls bin/*/hello*'
|
||||||
|
|
||||||
- name: Package artifacts
|
# Everything below publishes and only runs on master and on tags.
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
- name: Bundle
|
||||||
|
if: github.event_name != 'pull_request'
|
||||||
run: |
|
run: |
|
||||||
set -eux
|
|
||||||
mkdir -p dist
|
mkdir -p dist
|
||||||
|
flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}"
|
||||||
|
cp packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref dist/
|
||||||
|
(cd dist && sha256sum * > SHA256SUMS)
|
||||||
|
ls -la dist
|
||||||
|
|
||||||
# 3 archives, one per march level. Each carries the executable, the
|
- name: Upload bundle
|
||||||
# static library and share/; libc++ comes from the host.
|
if: github.event_name != 'pull_request'
|
||||||
for march in x86-64-v2 x86-64-v3 x86-64-v4; do
|
|
||||||
# Short suffix (v2/v3/v4) for archive filenames; the upload-artifact
|
|
||||||
# steps below reference these short names.
|
|
||||||
short=${march##*-}
|
|
||||||
|
|
||||||
# VariantId is "<name>-<target>-<march>-<mtune>-<hash>"; CI pins
|
|
||||||
# mtune=generic so each (target, march) resolves to a single subdir
|
|
||||||
# but the trailing hash isn't known to the workflow, hence the glob.
|
|
||||||
stage_lin=$(mktemp -d)
|
|
||||||
mkdir -p "$stage_lin/bin" "$stage_lin/lib"
|
|
||||||
cp bin/crafter.build-exe-x86_64-pc-linux-gnu-$march-*/crafter-build "$stage_lin/bin/"
|
|
||||||
cp bin/crafter.build-lib-x86_64-pc-linux-gnu-$march-*/libcrafter-build.a "$stage_lin/lib/"
|
|
||||||
cp -r share "$stage_lin/"
|
|
||||||
tar czf "dist/crafter-build-linux-x86_64-$short.tar.gz" -C "$stage_lin" .
|
|
||||||
done
|
|
||||||
|
|
||||||
ls -la dist/
|
|
||||||
|
|
||||||
# Upload each archive as its own workflow artifact (one upload per file)
|
|
||||||
# so users browsing the PR / run page get small, individually-named
|
|
||||||
# downloads instead of one wrapper zip containing the lot.
|
|
||||||
- name: Upload Linux v2 artifact
|
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
|
||||||
uses: actions/upload-artifact@v3
|
uses: actions/upload-artifact@v3
|
||||||
with: { name: crafter-build-linux-x86_64-v2, path: dist/crafter-build-linux-x86_64-v2.tar.gz, if-no-files-found: error }
|
with:
|
||||||
- name: Upload Linux v3 artifact
|
name: crafter-build-${{ steps.meta.outputs.version }}
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
path: dist/*.flatpak
|
||||||
uses: actions/upload-artifact@v3
|
if-no-files-found: error
|
||||||
with: { name: crafter-build-linux-x86_64-v3, path: dist/crafter-build-linux-x86_64-v3.tar.gz, if-no-files-found: error }
|
|
||||||
- name: Upload Linux v4 artifact
|
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
|
||||||
uses: actions/upload-artifact@v3
|
|
||||||
with: { name: crafter-build-linux-x86_64-v4, path: dist/crafter-build-linux-x86_64-v4.tar.gz, if-no-files-found: error }
|
|
||||||
|
|
||||||
- name: Update rolling 'latest' tag
|
- name: Publish to the Catcrafts repo
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
if: github.event_name != 'pull_request'
|
||||||
|
env:
|
||||||
|
FLATPAK_GPG_PRIVATE_KEY: ${{ secrets.FLATPAK_GPG_PRIVATE_KEY }}
|
||||||
|
FLATPAK_GPG_KEY_ID: ${{ secrets.FLATPAK_GPG_KEY_ID }}
|
||||||
|
run: |
|
||||||
|
set -eu
|
||||||
|
export GNUPGHOME=$(mktemp -d)
|
||||||
|
printf '%s' "$FLATPAK_GPG_PRIVATE_KEY" | gpg --batch --quiet --import
|
||||||
|
# The live repo is the canonical copy; builds are promoted into it,
|
||||||
|
# never built into it.
|
||||||
|
[ -d "$REPO_DIR" ] || ostree init --repo="$REPO_DIR" --mode=archive
|
||||||
|
flatpak build-commit-from --src-repo=repo \
|
||||||
|
--gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
|
||||||
|
"$REPO_DIR" "app/$APP_ID/x86_64/${{ steps.meta.outputs.branch }}"
|
||||||
|
flatpak build-update-repo --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
|
||||||
|
--generate-static-deltas --prune "$REPO_DIR"
|
||||||
|
# The repo description and the app ref carry the public key.
|
||||||
|
key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0)
|
||||||
|
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo"
|
||||||
|
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref"
|
||||||
|
|
||||||
|
- name: Rolling 'latest' release
|
||||||
|
if: github.event_name != 'pull_request' && github.ref_type != 'tag'
|
||||||
run: |
|
run: |
|
||||||
git config user.email "ci@catcrafts.net"
|
git config user.email "ci@catcrafts.net"
|
||||||
git config user.name "Crafter Build CI"
|
git config user.name "Crafter Build CI"
|
||||||
git tag -f latest
|
git tag -f latest
|
||||||
git push origin latest --force
|
git push origin latest --force
|
||||||
|
|
||||||
- name: Publish rolling 'latest' release
|
- name: Publish rolling 'latest' release
|
||||||
# forgejo-release uploads each file in release-dir as a separate
|
if: github.event_name != 'pull_request' && github.ref_type != 'tag'
|
||||||
# release asset, so the Releases page shows individually
|
|
||||||
# downloadable archives — pick the v-level matching your CPU.
|
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
|
||||||
uses: https://code.forgejo.org/actions/forgejo-release@v2
|
uses: https://code.forgejo.org/actions/forgejo-release@v2
|
||||||
with:
|
with:
|
||||||
direction: upload
|
direction: upload
|
||||||
url: ${{ github.server_url }}
|
url: ${{ github.server_url }}
|
||||||
repo: ${{ github.repository }}
|
repo: ${{ github.repository }}
|
||||||
tag: latest
|
tag: latest
|
||||||
title: Latest main build
|
title: Latest master build
|
||||||
prerelease: true
|
prerelease: true
|
||||||
override: true
|
override: true
|
||||||
release-dir: dist
|
release-dir: dist
|
||||||
token: ${{ secrets.GITHUB_TOKEN }}
|
token: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
# musl build for Alpine / postmarketOS hosts: the glibc launcher above cannot
|
- name: Release notes
|
||||||
# run there (musl has no glibc loader and gcompat lacks libgcc_s + the
|
if: github.ref_type == 'tag'
|
||||||
# __isoc23_* symbols). Bootstrap natively on alpine:edge and ship the
|
|
||||||
# bootstrap output — the same thing the Arch PKGBUILD installs. Dynamic
|
|
||||||
# against musl libc++ (crafter-build dlopens project.so, so it cannot be
|
|
||||||
# fully static); consumers need `apk add clang lld libc++-dev llvm-runtimes`.
|
|
||||||
release-musl:
|
|
||||||
needs: build-test-release
|
|
||||||
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/master'
|
|
||||||
runs-on: arch-latest
|
|
||||||
container:
|
|
||||||
image: alpine:edge
|
|
||||||
env:
|
|
||||||
CRAFTER_BUILD_MARCH: x86-64-v2
|
|
||||||
CRAFTER_BUILD_MTUNE: generic
|
|
||||||
steps:
|
|
||||||
- name: Install build dependencies
|
|
||||||
# nodejs: actions/checkout is a Node action. build-base: gcc's
|
|
||||||
# libgcc_s + crt objects, which clang's musl driver links against, plus
|
|
||||||
# make/binutils for the glslang cmake build (python3: its generators).
|
|
||||||
# clang-dev + llvm-dev: the libclang header Crafter.Build-Lint includes
|
|
||||||
# lives under llvm-config --includedir here, not /usr/include. llvm:
|
|
||||||
# llvm-ar for the LTO static archives the self-host/test builds write.
|
|
||||||
run: |
|
run: |
|
||||||
apk add -q nodejs git curl jq tar clang clang-dev lld libc++-dev \
|
# The newest <release> block's description, as plain text.
|
||||||
llvm-libunwind-dev llvm-runtimes llvm llvm-dev cmake build-base python3
|
python3 - > release-notes.md <<'EOF'
|
||||||
git config --global --add safe.directory '*'
|
import re, html
|
||||||
|
text = open("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").read()
|
||||||
- name: Checkout
|
block = re.search(r"<release [^>]*>(.*?)</release>", text, re.S).group(1)
|
||||||
uses: actions/checkout@v4
|
paras = re.findall(r"<p>(.*?)</p>", block, re.S)
|
||||||
|
print("\n\n".join(html.unescape(" ".join(p.split())) for p in paras))
|
||||||
- name: Bootstrap (build.sh)
|
print("\nInstall:\n\n flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo\n flatpak install catcrafts net.catcrafts.CrafterBuild\n")
|
||||||
run: ./build.sh
|
EOF
|
||||||
|
- name: Publish release
|
||||||
- name: Run tests
|
if: github.ref_type == 'tag'
|
||||||
run: CRAFTER_BUILD_HOME=$PWD/share/crafter-build ./bin/crafter-build test
|
uses: https://code.forgejo.org/actions/forgejo-release@v2
|
||||||
|
with:
|
||||||
- name: Package artifact
|
direction: upload
|
||||||
run: |
|
url: ${{ github.server_url }}
|
||||||
set -eux
|
repo: ${{ github.repository }}
|
||||||
file bin/crafter-build || true
|
tag: ${{ github.ref_name }}
|
||||||
mkdir -p dist
|
title: crafter-build ${{ github.ref_name }}
|
||||||
stage=$(mktemp -d)
|
release-notes: release-notes.md
|
||||||
mkdir -p "$stage/bin" "$stage/lib"
|
release-dir: dist
|
||||||
cp bin/crafter-build "$stage/bin/"
|
token: ${{ secrets.GITHUB_TOKEN }}
|
||||||
cp lib/libcrafter-build.a "$stage/lib/"
|
|
||||||
cp -r share "$stage/"
|
|
||||||
tar czf dist/crafter-build-linux-x86_64-musl-v2.tar.gz -C "$stage" .
|
|
||||||
ls -la dist/
|
|
||||||
|
|
||||||
- name: Upload musl artifact
|
|
||||||
uses: actions/upload-artifact@v3
|
|
||||||
with: { name: crafter-build-linux-x86_64-musl-v2, path: dist/crafter-build-linux-x86_64-musl-v2.tar.gz, if-no-files-found: error }
|
|
||||||
|
|
||||||
# Add the asset to the rolling 'latest' release the first job just
|
|
||||||
# (re)published, via the API rather than forgejo-release: that action's
|
|
||||||
# override mode would drop the other job's assets.
|
|
||||||
- name: Attach to rolling 'latest' release
|
|
||||||
env:
|
|
||||||
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
||||||
API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
|
|
||||||
run: |
|
|
||||||
set -eu
|
|
||||||
asset=crafter-build-linux-x86_64-musl-v2.tar.gz
|
|
||||||
rel=$(curl -fsS -H "Authorization: token $TOKEN" "$API/releases/tags/latest")
|
|
||||||
id=$(printf '%s' "$rel" | jq -r .id)
|
|
||||||
for old in $(printf '%s' "$rel" | jq -r --arg n "$asset" '.assets[] | select(.name==$n) | .id'); do
|
|
||||||
curl -fsS -X DELETE -H "Authorization: token $TOKEN" "$API/releases/$id/assets/$old"
|
|
||||||
done
|
|
||||||
curl -fsS -H "Authorization: token $TOKEN" -F "attachment=@dist/$asset" \
|
|
||||||
"$API/releases/$id/assets?name=$asset" > /dev/null
|
|
||||||
echo "attached $asset to release $id"
|
|
||||||
|
|
|
||||||
18
AGENTS.md
18
AGENTS.md
|
|
@ -17,7 +17,7 @@ crafter-build is a C++26 modules build system. A project is described by a `proj
|
||||||
| `wasi-runtime/` | `runtime.js` and `index.html.in`, shipped for browser targets. |
|
| `wasi-runtime/` | `runtime.js` and `index.html.in`, shipped for browser targets. |
|
||||||
| `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. |
|
| `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. |
|
||||||
| `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. |
|
| `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. |
|
||||||
| `PKGBUILD`, `.forgejo/workflows/ci.yaml` | Arch package and the Forgejo Actions CI. |
|
| `packaging/flatpak/`, `.forgejo/workflows/ci.yaml` | The Flatpak manifest with its metainfo, repo and ref files, and the Forgejo Actions CI that builds and publishes it. |
|
||||||
|
|
||||||
Partitions and what they own:
|
Partitions and what they own:
|
||||||
|
|
||||||
|
|
@ -32,6 +32,8 @@ Partitions and what they own:
|
||||||
|
|
||||||
## Dev environment
|
## Dev environment
|
||||||
|
|
||||||
|
The shipped artifact is the Flatpak (see "CI and publishing"); the loop below is the native development loop.
|
||||||
|
|
||||||
Prerequisites on Linux: `clang` (with `libclang`, which the AST lint rules load at runtime), `lld`, `libc++` with its std module source (`share/libc++/v1/std.cppm`), `cmake`, `git`. `crafter-build --version` prints which compiler was resolved; `CRAFTER_BUILD_LLVM_DIR` points it at another LLVM prefix, and `build.sh` honours the same variable. Network access to `forgejo.catcrafts.net` and `github.com` is needed: loading `project.cpp` fetches Crafter.Math, Crafter.Asset and Crafter.LintRules into `~/.cache/crafter.build/external/`, and the bootstrap clones glslang. Those sources are unpinned, so every invocation runs `git pull` on them.
|
Prerequisites on Linux: `clang` (with `libclang`, which the AST lint rules load at runtime), `lld`, `libc++` with its std module source (`share/libc++/v1/std.cppm`), `cmake`, `git`. `crafter-build --version` prints which compiler was resolved; `CRAFTER_BUILD_LLVM_DIR` points it at another LLVM prefix, and `build.sh` honours the same variable. Network access to `forgejo.catcrafts.net` and `github.com` is needed: loading `project.cpp` fetches Crafter.Math, Crafter.Asset and Crafter.LintRules into `~/.cache/crafter.build/external/`, and the bootstrap clones glslang. Those sources are unpinned, so every invocation runs `git pull` on them.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -116,7 +118,7 @@ These were decided deliberately. Do not undo them without the maintainer's say-s
|
||||||
|
|
||||||
1. **Configuration is C++.** No JSON, YAML, TOML or DSL for project, runner or lint configuration. Lint rules are C++ callbacks; there is no clang-format or clang-tidy integration and no separate "format rule" type. A rule that calls `ctx.SetContent` is a transform, and that is the whole mechanism.
|
1. **Configuration is C++.** No JSON, YAML, TOML or DSL for project, runner or lint configuration. Lint rules are C++ callbacks; there is no clang-format or clang-tidy integration and no separate "format rule" type. A rule that calls `ctx.SetContent` is a transform, and that is the whole mechanism.
|
||||||
2. **Per-project artifacts go in `<project>/build` and `<project>/bin`. Only cross-project things go in `~/.cache/crafter.build/`:** the std PCM and Crafter.Build PCMs per target and march, and external clones keyed by their spec hash.
|
2. **Per-project artifacts go in `<project>/build` and `<project>/bin`. Only cross-project things go in `~/.cache/crafter.build/`:** the std PCM and Crafter.Build PCMs per target and march, and external clones keyed by their spec hash.
|
||||||
3. **Sources ship, PCMs build locally.** Packages install `*.cppm` under `share/crafter-build/`. Nothing target-baked ships except the executable itself, and that defaults to a portable `-march` in `PKGBUILD` and CI.
|
3. **Sources ship, PCMs build locally.** The Flatpak installs `*.cppm` under `/app/share/crafter-build/`. Nothing target-baked ships except the executable and the toolchain beside it, and the executable is built for a portable `-march`.
|
||||||
4. **crafter-build is a monolithic executable with `--export-dynamic`.** `project.so` resolves `Crafter::` symbols from the running binary at dlopen time. There is no Windows host; `x86_64-w64-mingw32` is a cross-compile target for user projects only.
|
4. **crafter-build is a monolithic executable with `--export-dynamic`.** `project.so` resolves `Crafter::` symbols from the running binary at dlopen time. There is no Windows host; `x86_64-w64-mingw32` is a cross-compile target for user projects only.
|
||||||
5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point.
|
5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point.
|
||||||
6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch.
|
6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch.
|
||||||
|
|
@ -131,9 +133,9 @@ These were decided deliberately. Do not undo them without the maintainer's say-s
|
||||||
- Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice.
|
- Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice.
|
||||||
- Separate commits for the fix, its test and the README change when each stands on its own.
|
- Separate commits for the fix, its test and the README change when each stands on its own.
|
||||||
- Update the README in the same PR when user-visible behaviour changes.
|
- Update the README in the same PR when user-visible behaviour changes.
|
||||||
- `format`, `lint` and `test` green before pushing. CI runs the bootstrap and the test suite on PRs and publishes release archives on `master`.
|
- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on PRs, publishes the `beta` branch of the Catcrafts repo on every push to `master`, and the `stable` branch plus a release on an `X.Y.Z` tag.
|
||||||
- Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`.
|
- Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`.
|
||||||
- Keep CI's `CRAFTER_BUILD_MARCH=x86-64-v2` for the bootstrap: the runner's CPU cannot execute v3 code, and the v3 and v4 artifacts are produced from the v2 binary.
|
- Keep `CRAFTER_BUILD_MARCH=x86-64-v2` in the manifest for the shipped binary: it has to run on any x86_64 machine. User projects still default to `native`.
|
||||||
|
|
||||||
## Security considerations
|
## Security considerations
|
||||||
|
|
||||||
|
|
@ -141,9 +143,15 @@ These were decided deliberately. Do not undo them without the maintainer's say-s
|
||||||
- Unpinned dependencies `git pull` on every invocation, so builds and tests need network. A missing or private repo surfaces as a clone failure before anything is built.
|
- Unpinned dependencies `git pull` on every invocation, so builds and tests need network. A missing or private repo surfaces as a clone failure before anything is built.
|
||||||
- `--runner=cmd:<command>` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests.
|
- `--runner=cmd:<command>` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests.
|
||||||
- The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library.
|
- The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library.
|
||||||
- CI force-pushes the rolling `latest` tag and publishes a prerelease on every push to `master` with the workflow's own token. Do not add secrets to the workflow or echo the environment in it.
|
- CI force-pushes the rolling `latest` tag and publishes a prerelease on every push to `master` with the workflow's own token, and signs the Flatpak repo with the key in the `FLATPAK_GPG_PRIVATE_KEY` secret. Do not add other secrets to the workflow or echo the environment in it.
|
||||||
- Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way.
|
- Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way.
|
||||||
|
|
||||||
|
## CI and publishing
|
||||||
|
|
||||||
|
`.forgejo/workflows/ci.yaml` runs on the `flatpak` runner label: the privileged forgejo-runner on the Hetzner box (bubblewrap inside the job container needs it). Its `config.yml` allows three volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, so the LLVM and WASI modules are rebuilt only when their sources change) and `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`. Secrets: `FLATPAK_GPG_PRIVATE_KEY` (armored, no passphrase) and `FLATPAK_GPG_KEY_ID`. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it.
|
||||||
|
|
||||||
|
To release: add a `<release>` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), merge, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry. Local builds: `flatpak-builder --user --force-clean --ccache --repo=repo build-dir packaging/flatpak/net.catcrafts.CrafterBuild.yaml`, then `flatpak-builder --run build-dir <manifest> crafter-build --version`.
|
||||||
|
|
||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
- A `#pragma crafter dependency` target is rebuilt on every load on purpose: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the depfile check is what catches a regenerated host cache. Do not optimise the rebuild away.
|
- A `#pragma crafter dependency` target is rebuilt on every load on purpose: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the depfile check is what catches a regenerated host cache. Do not optimise the rebuild away.
|
||||||
|
|
|
||||||
35
PKGBUILD
35
PKGBUILD
|
|
@ -1,35 +0,0 @@
|
||||||
#SPDX-License-Identifier: LGPL-3.0-only
|
|
||||||
#SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
|
|
||||||
|
|
||||||
pkgname=crafter-build
|
|
||||||
pkgver=0.1.0
|
|
||||||
pkgrel=1
|
|
||||||
pkgdesc='C++26 modules build system'
|
|
||||||
arch=('x86_64')
|
|
||||||
url='https://forgejo.catcrafts.net/Catcrafts/Crafter.Build'
|
|
||||||
license=('LGPL-3.0-only')
|
|
||||||
depends=('clang' 'libc++' 'lld')
|
|
||||||
optdepends=('wasi-libc: WASI/WebAssembly target support'
|
|
||||||
'wasi-libc++: C++ standard library for WASI'
|
|
||||||
'wasi-libc++abi: C++ ABI support for WASI')
|
|
||||||
makedepends=('cmake' 'git')
|
|
||||||
source=()
|
|
||||||
sha256sums=()
|
|
||||||
options=('!strip' '!debug' '!lto')
|
|
||||||
|
|
||||||
build() {
|
|
||||||
cd "$startdir"
|
|
||||||
rm -rf bin build share/crafter-build
|
|
||||||
CRAFTER_BUILD_MARCH=x86-64-v3 CRAFTER_BUILD_MTUNE=generic ./build.sh
|
|
||||||
}
|
|
||||||
|
|
||||||
package() {
|
|
||||||
cd "$startdir"
|
|
||||||
install -Dm755 bin/crafter-build "$pkgdir/usr/bin/crafter-build"
|
|
||||||
install -Dm644 lib/libcrafter-build.a "$pkgdir/usr/lib/libcrafter-build.a"
|
|
||||||
install -dm755 "$pkgdir/usr/share/crafter-build"
|
|
||||||
install -m644 share/crafter-build/*.cppm "$pkgdir/usr/share/crafter-build/"
|
|
||||||
install -dm755 "$pkgdir/usr/share/crafter-build/wasi-runtime"
|
|
||||||
install -m644 share/crafter-build/wasi-runtime/runtime.js "$pkgdir/usr/share/crafter-build/wasi-runtime/"
|
|
||||||
install -m644 share/crafter-build/wasi-runtime/index.html.in "$pkgdir/usr/share/crafter-build/wasi-runtime/"
|
|
||||||
}
|
|
||||||
39
README.md
39
README.md
|
|
@ -2,9 +2,21 @@
|
||||||
|
|
||||||
A C++26 modules build system. Project descriptions are written in C++ with no JSON, no Lua, no embedded DSL. You write a `project.cpp` that constructs a `Configuration` and returns it; Crafter Build compiles, loads, and executes it.
|
A C++26 modules build system. Project descriptions are written in C++ with no JSON, no Lua, no embedded DSL. You write a `project.cpp` that constructs a `Configuration` and returns it; Crafter Build compiles, loads, and executes it.
|
||||||
|
|
||||||
## Quick start (Linux)
|
## Install
|
||||||
|
|
||||||
Bootstrap requires `clang`, `cmake`, `git`, `lld`, and `libc++`.
|
crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo
|
||||||
|
flatpak install catcrafts net.catcrafts.CrafterBuild
|
||||||
|
alias crafter-build='flatpak run net.catcrafts.CrafterBuild'
|
||||||
|
```
|
||||||
|
|
||||||
|
Pre-release builds of `master` are the `beta` branch (`flatpak install catcrafts net.catcrafts.CrafterBuild//beta`), and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-<version>.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host.
|
||||||
|
|
||||||
|
## Building from source (Linux)
|
||||||
|
|
||||||
|
The development loop is native. It needs `clang`, `cmake`, `git`, `lld` and `libc++` with its std module source:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build.sh # produces bin/crafter-build
|
./build.sh # produces bin/crafter-build
|
||||||
|
|
@ -13,13 +25,7 @@ Bootstrap requires `clang`, `cmake`, `git`, `lld`, and `libc++`.
|
||||||
|
|
||||||
The compiler is resolved once per run: `CRAFTER_BUILD_LLVM_DIR` if set, else a `clang++` installed beside crafter-build (`<prefix>/bin`), else PATH. `crafter-build --version` prints the version and the compiler in use; `crafter-build --help` lists the other overrides (`CRAFTER_BUILD_LIBCXX_DIR`, `CRAFTER_BUILD_LIBCLANG`, `CRAFTER_BUILD_WASI_SYSROOT`, `CRAFTER_MINGW_DIR`). A `Configuration` targets the host triple unless `cfg.target` says otherwise.
|
The compiler is resolved once per run: `CRAFTER_BUILD_LLVM_DIR` if set, else a `clang++` installed beside crafter-build (`<prefix>/bin`), else PATH. `crafter-build --version` prints the version and the compiler in use; `crafter-build --help` lists the other overrides (`CRAFTER_BUILD_LIBCXX_DIR`, `CRAFTER_BUILD_LIBCLANG`, `CRAFTER_BUILD_WASI_SYSROOT`, `CRAFTER_MINGW_DIR`). A `Configuration` targets the host triple unless `cfg.target` says otherwise.
|
||||||
|
|
||||||
The bootstrap targets the host toolchain's own triple (`clang++ -print-target-triple`): `x86_64-pc-linux-gnu` on glibc distros, `x86_64-alpine-linux-musl` on Alpine/postmarketOS (`apk add clang clang-dev lld libc++-dev llvm-libunwind-dev llvm-runtimes llvm llvm-dev cmake build-base python3 git`). Override with `CRAFTER_BUILD_TARGET=`. CI ships both: `crafter-build-linux-x86_64-v{2,3,4}.tar.gz` (glibc) and `crafter-build-linux-x86_64-musl-v2.tar.gz` (musl, needs `libc++` at runtime — the launcher dlopens `project.so`, so it cannot be static).
|
The bootstrap targets the toolchain's own triple (`clang++ -print-target-triple`); override with `CRAFTER_BUILD_TARGET=`. The Flatpak is built from `packaging/flatpak/net.catcrafts.CrafterBuild.yaml`, which runs this same bootstrap, the self-host build and the test suite inside the sandbox against the bundled toolchain (`CRAFTER_BUILD_LLVM_DIR=/app`).
|
||||||
|
|
||||||
To build the system as a distro package on Arch:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
makepkg -si # uses CRAFTER_BUILD_MARCH=x86-64-v3 by default for portability
|
|
||||||
```
|
|
||||||
|
|
||||||
## Writing a project.cpp
|
## Writing a project.cpp
|
||||||
|
|
||||||
|
|
@ -103,13 +109,20 @@ import Crafter.LintRules;
|
||||||
|
|
||||||
## Install layout
|
## Install layout
|
||||||
|
|
||||||
|
Inside the Flatpak:
|
||||||
|
|
||||||
```
|
```
|
||||||
<prefix>/bin/crafter-build # the executable
|
/app/bin/crafter-build # the executable
|
||||||
<prefix>/lib/libcrafter-build.a # static archive for downstream consumers
|
/app/bin/clang++ lld llvm-ar wasmtime # the bundled toolchain the executable resolves first
|
||||||
<prefix>/share/crafter-build/*.cppm # module sources (rebuilt on user machine on first run)
|
/app/lib/libcrafter-build.a # static archive for downstream consumers
|
||||||
~/.cache/crafter.build/<target>-<march>-<clang>-<hash>/ # std.pcm + Crafter.Build*.pcm, built locally per toolchain
|
/app/share/crafter-build/*.cppm # module sources (rebuilt on the user's machine on first run)
|
||||||
|
/app/share/wasi-sysroot/ # WASI sysroot for wasm32 targets
|
||||||
|
~/.var/app/net.catcrafts.CrafterBuild/cache/crafter.build/<target>-<march>-<clang>-<hash>/
|
||||||
|
# std.pcm + Crafter.Build*.pcm, built locally per toolchain
|
||||||
```
|
```
|
||||||
|
|
||||||
|
A source build uses the same shape under the checkout (`bin/`, `lib/`, `share/crafter-build/`) with the system compiler and `~/.cache/crafter.build/`.
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
Tests live under `tests/<Name>/`. The simplest case is a single C++ file:
|
Tests live under `tests/<Name>/`. The simplest case is a single C++ file:
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue