From a1bc96a323818d2e4eb451dbdf9eb377f5cfbc78 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 14:10:27 +0100 Subject: [PATCH 1/6] fix(review): name each lens section, as crafter-review 1.0 requires crafter-review 1.0.1, the released build installed on this machine, and the newer build installed beside it as //local refuse the config key lensSections: a lens's repository section is named per lens, as lenses..section, instead of being found by listing a directory. With the key in place `crafter-review config` reports it as a problem, and freeze and run stop at "preflight: config: unknown key lensSections". The three files in .review-gates/lenses/ are named under lenses, one entry each, the migration the Crafter.Review session gave; with them `crafter-review config` reports no problem. --- .review-gates/config.json | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/.review-gates/config.json b/.review-gates/config.json index f452056..115ee6c 100644 --- a/.review-gates/config.json +++ b/.review-gates/config.json @@ -6,7 +6,17 @@ ], "aiTrailers": "forbid", "requireSigned": true, - "lensSections": ".review-gates/lenses", + "lenses": { + "repo-better-solution": { + "section": ".review-gates/lenses/repo-better-solution.md" + }, + "repo-bug-prover": { + "section": ".review-gates/lenses/repo-bug-prover.md" + }, + "repo-facts": { + "section": ".review-gates/lenses/repo-facts.md" + } + }, "modes": { "commits": { "kind": "commits", From 2447d6ec4f6f8111fe1310967dff50031c9d0d62 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:39:01 +0100 Subject: [PATCH 2/6] fix(ci): attach the flatpakref with the repo's key to releases The Bundle step copied packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref into dist/ as the checkout has it, with GPGKey=GPG_KEY_BASE64_PLACEHOLDER, so all six releases, 2.0.0 to 5.0.0, attach a .flatpakref that does not install: `flatpak install --from` it stops with "GPG: Unable to export keys: GPGME: No data" (Flatpak 1.18.4) before it adds the remote. The copy the publish step writes to the web root has the key filled in, and installing from it goes ahead. The key is known only from the publish step on, so the release's copy, and SHA256SUMS over it, are made there from the web root's file instead of in the Bundle step. --- .forgejo/workflows/ci.yaml | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/.forgejo/workflows/ci.yaml b/.forgejo/workflows/ci.yaml index 73f1449..086d3dc 100644 --- a/.forgejo/workflows/ci.yaml +++ b/.forgejo/workflows/ci.yaml @@ -120,8 +120,6 @@ jobs: run: | mkdir -p dist flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}" - cp packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref dist/ - (cd dist && sha256sum * > SHA256SUMS) ls -la dist - name: Upload bundle @@ -160,6 +158,12 @@ jobs: key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0) sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" + # The release attaches this .flatpakref, not the one in the + # checkout: that one has a placeholder where the key goes, and + # flatpak cannot install from it. + cp "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" dist/ + (cd dist && sha256sum * > SHA256SUMS) + ls -la dist - name: Rolling 'latest' release if: github.event_name != 'pull_request' && github.ref_type != 'tag' From 2babe94a6aa004c7b29092a4db2a16ab019cb991 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 12:04:06 +0100 Subject: [PATCH 3/6] fix(ci): publish under the lock Crafter.Review's workflow takes Crafter.Review's workflow publishes its Flatpak into the same repo, the web root's repo/, and holds an flock on .publish.lock in the web root while it commits and updates the summary. Its comment says that this protects both only once Crafter.Build's workflow takes the lock too, and ours never did, so a publish of each could interleave: a build-commit-from and a build-update-repo --generate-static-deltas --prune of each workflow on one repo at the same time. The Publish step now holds the same lock from its commit to the ref files it writes into the web root, and the job installs util-linux for flock, as Crafter.Review's jobs do. AGENTS.template.md says that the repo is shared and the lock taken. Neither it nor the workflow's header comment says any longer that the web root and the signing key are this runner's alone, or that the organisation-wide `flatpak` runner mounts none of them. That runner, which serves every Catcrafts repository and refused all of this job's volumes when it last took it, in October, now mounts the web root and the key for Crafter.Review's publish job, and the volumes a runner allows are open to every job it runs. The owner has ruled the sharing intended. The header also stops saying that a job of ours that lands there fails before building: that held while the runner refused every volume, and AGENTS.template.md keeps it as the October history. --- .forgejo/workflows/ci.yaml | 16 ++++++++++++---- AGENTS.template.md | 4 ++-- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/.forgejo/workflows/ci.yaml b/.forgejo/workflows/ci.yaml index 086d3dc..644009f 100644 --- a/.forgejo/workflows/ci.yaml +++ b/.forgejo/workflows/ci.yaml @@ -11,9 +11,10 @@ name: Flatpak # flatpak-builder state (ccache and the LLVM module, so only crafter-build # itself rebuilds on a normal run), the published repo's web root, and the # repo signing key, which lives on the runner host and never in the forge. -# No other runner carries that label. Plain `flatpak` also matches the -# organisation's runner, which mounts none of those volumes; a job it picks -# up fails before building anything. +# No other runner carries that label. Plain `flatpak` is the label of the +# organisation's runner, which serves every Catcrafts repository and also +# mounts the web root and the key, for Crafter.Review's publish job, so any +# job it runs can mount them (AGENTS.template.md, CI and publishing). on: pull_request: branches: [master] @@ -45,11 +46,12 @@ jobs: -v /etc/crafter-flatpak:/run/crafter-flatpak:ro steps: - name: Install tools + # util-linux: flock, for the Publish step. run: | pacman-key --init pacman-key --populate archlinux pacman -Sy --noconfirm --needed archlinux-keyring - pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python + pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python util-linux git config --global --add safe.directory '*' - name: Sdk @@ -144,6 +146,11 @@ jobs: export GNUPGHOME=$(mktemp -d) gpg --batch --quiet --import /run/crafter-flatpak/signing-key.asc FLATPAK_GPG_KEY_ID=$(cat /run/crafter-flatpak/key-id) + # Crafter.Review's workflow publishes into the same repo and holds + # this lock while it does, so a commit and the summary update of + # one workflow never interleave with the other's. + exec 9> "$WEB_DIR/.publish.lock" + flock 9 # The live repo is the canonical copy; builds are promoted into it, # never built into it. # Keyed on the repo's config file, not the directory: the web root @@ -158,6 +165,7 @@ jobs: key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0) sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" + flock -u 9 # The release attaches this .flatpakref, not the one in the # checkout: that one has a placeholder where the key goes, and # flatpak cannot install from it. diff --git a/AGENTS.template.md b/AGENTS.template.md index 68b35bd..1db87b0 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -160,9 +160,9 @@ These were decided deliberately. Do not undo them without the maintainer's say-s ## CI and publishing -`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. +`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and no other repository's job runs on it. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The web root and the key are not this runner's alone: the organisation-wide `flatpak` runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository, mounts both for Crafter.Review's publish job, and the volumes a runner allows (`valid_volumes` in its config) are open to every job it runs, so a job of any Catcrafts repository that runs there can mount them as well. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. Crafter.Review's workflow publishes into the same repo, and each workflow holds an flock on `/var/www/flatpak/.publish.lock` while it commits and updates the summary. -Keep the label unique to that runner. Plain `flatpak` also belongs to the organisation-wide runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository and on purpose allows no volumes. Whenever this job lands there, flatpak-builder stops with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of the job log shows which runner took it and lists every volume it ignored. +Keep the label unique to that runner. Plain `flatpak` belongs to the organisation-wide runner. When this job still asked for `flatpak` and that runner took it (runs #368 and #370, in October 2026), the runner allowed none of the job's volumes, and flatpak-builder stopped with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of such a job's log shows which runner took it and lists every volume it ignored. The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing: From be81fc098e6d02eb528ef35654318284029bdec1 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:42:58 +0100 Subject: [PATCH 4/6] feat(ci): drop beta and latest, release signed tags on master A push to master published the `beta` branch and moved the rolling `latest` tag and its "Latest master build" prerelease, and a pushed X.Y.Z tag made the release and published `stable`. The owner wants a release for every push to master, made by the signed tag that goes with it, and nothing for a push without one. So `beta`, `latest` and the branch trigger are gone: the workflow runs for pushed tags, pull requests and manual runs only, and the `latest` tag and its prerelease have been deleted from the forge. `beta` gets no more builds, and its installations would not hear of it: the release notes ship in stable builds only. The Publish step marks beta's last build end-of-life, once, with a reason that names stable, and the next `flatpak update` of a beta installation prints it. The reason also names the branch, because a non-interactive update prints only the app's name with it. Before building, a tag run now checks the tag: it has to be annotated and signed by the owner as Forgejo verifies it, its commit has to be on master, its version has to be X.Y.Z and newer than every release the forge has published, its own included (a later tag whose run still queues does not count), and the metainfo's newest entry has to be that version. The tag run checked only the last of these. It reads the tag from the forge's API, because the checkout fetches a pushed tag again as a plain ref to its commit, so the job's clone never has the annotated tag, and Forgejo verifies a signed tag's own signature but falls back to the commit's for an unsigned one, so the step also requires that the verdict covers the tag's payload. The metainfo's entry is read once, as the first element: the Version step takes its version and renders its notes into release-notes.md with the renderer the Release notes step had, and stops the run when they come out empty, so a comment or another attribute order cannot give a release one entry's version and another's notes. It does so on every run, pull requests and manual runs included, so a broken entry shows before a release. Release runs share a concurrency group, so they queue one behind another; a pull request still cancels its own outdated run. A failed release run can be re-run while no newer release exists, since its tag is still on its commit; a release that an attempt left half made has to be deleted first. The release step runs without `override`, which would delete a tag that has moved off the run's commit since the Version step and tag the old commit instead. A manual run no longer publishes anything; it used to publish `beta` and move `latest` from whichever branch it was started on. No step pushes to git, so the checkout keeps no credentials. git describe in build.sh, project.cpp and the workflow has no `latest` tag left to exclude. --- .forgejo/workflows/ci.yaml | 259 ++++++++++++++++++++++++------------- build.sh | 7 +- project.cpp | 8 +- 3 files changed, 173 insertions(+), 101 deletions(-) diff --git a/.forgejo/workflows/ci.yaml b/.forgejo/workflows/ci.yaml index 644009f..dcdff57 100644 --- a/.forgejo/workflows/ci.yaml +++ b/.forgejo/workflows/ci.yaml @@ -1,10 +1,15 @@ name: Flatpak -# One job, three outcomes: -# pull request -> build the Flatpak and run the suite inside it -# push master -> also publish the result as the `beta` branch of the -# Catcrafts repo and refresh the rolling `latest` release -# tag X.Y.Z -> publish as `stable` and create release X.Y.Z +# One job, two outcomes: +# pull request, manual run -> build the Flatpak and run the suite inside it +# pushed tag X.Y.Z -> also release the build: publish it as the +# `stable` branch of the Catcrafts repo and +# create release X.Y.Z +# +# A release is a tag that the owner makes and signs locally and pushes with +# the master it releases; a push to master without one runs nothing. The tag +# names the version, and the metainfo's newest entry has to be that +# version: its description is the release notes. # # The `crafter-flatpak` runner label is the privileged runner on the Hetzner # box (bubblewrap needs it) with four volumes allowed: the Sdk install, the @@ -19,20 +24,29 @@ on: pull_request: branches: [master] push: - branches: [master] # Bare semver tags. Spelled with globs so it means the same whether the - # forge reads it as a workflow pattern or as a plain glob; `latest`, the - # rolling tag this workflow moves itself, does not start with a digit. + # forge reads it as a workflow pattern or as a plain glob. tags: ['[0-9]*.[0-9]*.[0-9]*'] workflow_dispatch: +# Pushes are tags only, so every release run is in one group, and a release +# starts only once the one before it has finished. A pull request's group +# also holds its number, which only a pull request's event carries, so a +# new push to it cancels its own outdated run and no other. +concurrency: + group: flatpak-${{ github.event_name }}-${{ github.event.pull_request.number }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + env: APP_ID: net.catcrafts.CrafterBuild + BRANCH: stable MANIFEST: packaging/flatpak/net.catcrafts.CrafterBuild.yaml CI_MANIFEST: packaging/flatpak/ci-manifest.yaml RUNTIME: org.freedesktop.Sdk//26.08 REPO_DIR: /srv/flatpak/repo WEB_DIR: /srv/flatpak + # A release tag carries a signature Forgejo verifies for this address. + RELEASE_SIGNER: jorijnvdgraaf@catcrafts.net jobs: flatpak: @@ -62,40 +76,137 @@ jobs: flatpak update -y --noninteractive - name: Checkout + # Every branch and tag, for git describe and the release checks; no + # step pushes, so the token is not kept in the checkout. uses: actions/checkout@v4 with: fetch-depth: 0 - persist-credentials: true + persist-credentials: false - - name: Version and channel + - name: Version id: meta run: | set -eu - if [ "${GITHUB_REF_TYPE}" = tag ]; then - version="$GITHUB_REF_NAME" - branch=stable - # The metainfo's newest release is the release notes and must be - # the version being tagged. - top=$(grep -m1 -oP '&2 + # The metainfo's newest entry is its first element, the + # one parse that gives both its version and the release notes, + # before anything is built: a comment or another attribute order + # would mislead a text search, and an entry whose notes render + # empty stops the run here. The notes go to release-notes.md, for + # the release page, as Markdown. AppStream has no headings, so a + # paragraph that introduces a list is one:

New

    ...
+ # comes out as "## New" over its items, the shape the notes take + # since 3.0.0. A paragraph on its own stays a paragraph, as the + # older entries are written. and get their Markdown + # spelling; a code span also keeps text such as lib//eh + # from being read as an HTML tag by the forge. + newest=$(python3 - <<'EOF' + import sys + import xml.etree.ElementTree as ET + + release = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release") + desc = None if release is None else release.find("description") + + def text(el): + parts = [el.text or ""] + for child in el: + inner = text(child) + parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner)) + parts.append(child.tail or "") + return " ".join("".join(parts).split()) + + blocks = [] if desc is None else list(desc) + out = [] + for i, el in enumerate(blocks): + heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol") + if el.tag == "p": + out.append(f"## {text(el)}" if heads_list else text(el)) + elif el.tag == "ul": + out.append("\n".join(f"- {text(li)}" for li in el)) + elif el.tag == "ol": + out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1))) + notes = "\n\n".join(out) + if not notes.strip(): + sys.exit("the metainfo needs a at the top of whose renders release notes") + with open("release-notes.md", "w") as f: + print(notes, file=f) + print(release.get("version", "")) + EOF + ) + if [ "$GITHUB_EVENT_NAME" = push ]; then + # A release: the only pushes this workflow runs for are tags. + # The tag is the version; it has to be annotated, signed by the + # owner as Forgejo verifies it against the keys its users added, + # on master, newer than every published release, and the version + # of the metainfo's newest entry. The tag itself is read from the + # forge: the checkout fetches a pushed tag again as a plain ref + # to its commit, so the local one is never the annotated tag. + version=$GITHUB_REF_NAME + if ! printf '%s\n' "$version" | grep -qxE '[0-9]+\.[0-9]+\.[0-9]+'; then + echo "the tag $version is not a version of the form X.Y.Z" >&2 + exit 1 + fi + api="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY" + tag=$(curl -sSf "$api/tags/$version") + object=$(printf '%s' "$tag" | jq -r .id) + commit=$(printf '%s' "$tag" | jq -r .commit.sha) + if [ "$commit" != "$GITHUB_SHA" ]; then + echo "the forge has the tag $version on $commit, not on this run's commit $GITHUB_SHA" >&2 + exit 1 + fi + if [ "$object" = "$commit" ]; then + echo "$version is a lightweight tag; a release tag is annotated and signed: git tag -s $version -m 'crafter-build $version'" >&2 + exit 1 + fi + # Forgejo verifies a signed tag's own signature, and for an + # unsigned one falls back to its commit's, so the verdict also + # has to cover the tag's payload, not the commit's. + signed=$(curl -sSf "$api/git/tags/$object" | + jq -r --arg c "$commit" --arg v "$version" '"\(.verification.verified) \(.verification.signer.email // "") \(.verification.payload // "" | startswith("object \($c)\ntype commit\ntag \($v)\n"))"') + if [ "$signed" != "true $RELEASE_SIGNER true" ]; then + echo "Forgejo does not verify the tag $version itself as signed by $RELEASE_SIGNER (verified, signer, over the tag's payload: $signed)" >&2 + exit 1 + fi + if ! git merge-base --is-ancestor "$GITHUB_SHA" origin/master; then + echo "the tag $version is on $(git rev-parse --short "$GITHUB_SHA"), which is not on master: push the tag with the master it releases" >&2 + exit 1 + fi + # Newer than every release the forge has published, its own + # included, and not every tag: a later tag whose run still + # queues behind this one has no release yet, and a tag that is + # no X.Y.Z version never gets one. + for t in $(git tag --list | awk '/^[0-9]+\.[0-9]+\.[0-9]+$/' | sort -V); do + [ "$(printf '%s\n%s\n' "$t" "$version" | sort -V | tail -n 1)" = "$t" ] || continue + code=$(curl -s -o /dev/null -w '%{http_code}' "$api/releases/tags/$t") + case "$code" in + 404) ;; + 200) + if [ "$t" = "$version" ]; then + echo "$version is released already; to re-run after a release left half made, delete that release first" >&2 + else + echo "$version is not newer than the release $t" >&2 + fi + exit 1 ;; + *) echo "the forge answered $code when asked whether $t is released" >&2; exit 1 ;; + esac + done + if [ "$newest" != "$version" ]; then + echo "the metainfo's newest release is \"$newest\", but the tag is $version: the tagged commit needs the entry for $version at the top of packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml" >&2 exit 1 fi else - # A master build names itself after the release it follows, the - # way a local checkout does: 2.0.0-3-g21746b1 is three commits - # past 2.0.0. Before the first tag there is nothing to describe - # from and the bare hash gets a placeholder in front. - described=$(git describe --tags --always --exclude=latest) + # Nothing is published. The build names itself after the + # release it follows, the way a local checkout does: + # 5.0.0-3-g21746b1 is three commits past 5.0.0. Before the first + # tag there is nothing to describe from and the bare hash gets a + # placeholder in front. + described=$(git describe --tags --always) case "$described" in [0-9]*.[0-9]*.[0-9]*) version="$described" ;; *) version="0.0.0-dev+$described" ;; esac - branch=beta fi echo "version=$version" >> "$GITHUB_OUTPUT" - echo "branch=$branch" >> "$GITHUB_OUTPUT" - echo "version $version, flatpak branch $branch" + echo "version $version" - name: Build # The manifest is rewritten beside the original so its relative @@ -108,7 +219,7 @@ jobs: grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST" flatpak-builder --force-clean --ccache --disable-rofiles-fuse \ --state-dir=/root/.cache/flatpak-builder \ - --repo=repo --default-branch="${{ steps.meta.outputs.branch }}" \ + --repo=repo --default-branch="$BRANCH" \ build-dir "$CI_MANIFEST" - name: Smoke test @@ -116,16 +227,16 @@ jobs: flatpak-builder --run build-dir "$CI_MANIFEST" crafter-build --version flatpak-builder --run build-dir "$CI_MANIFEST" sh -c 'cd examples/hello-world && crafter-build && ls bin/*/hello*' - # Everything below publishes and only runs on master and on tags. + # Everything below releases the build and runs only for a pushed tag. - name: Bundle - if: github.event_name != 'pull_request' + if: github.event_name == 'push' run: | mkdir -p dist - flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}" + flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "$BRANCH" ls -la dist - name: Upload bundle - if: github.event_name != 'pull_request' + if: github.event_name == 'push' uses: actions/upload-artifact@v3 with: name: crafter-build-${{ steps.meta.outputs.version }} @@ -133,7 +244,7 @@ jobs: if-no-files-found: error - name: Publish to the Catcrafts repo - if: github.event_name != 'pull_request' + if: github.event_name == 'push' run: | set -eu # The signing key is generated once on the runner host @@ -158,7 +269,20 @@ jobs: [ -f "$REPO_DIR/config" ] || ostree init --repo="$REPO_DIR" --mode=archive flatpak build-commit-from --src-repo=repo \ --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ - "$REPO_DIR" "app/$APP_ID/x86_64/${{ steps.meta.outputs.branch }}" + "$REPO_DIR" "app/$APP_ID/x86_64/$BRANCH" + # Every push to master used to be published as `beta`. Its last + # build is marked end-of-life, once, so that the next + # `flatpak update` of a beta installation says why nothing follows. + # The reason names the branch: with --noninteractive, flatpak's + # message names only the app. + beta="app/$APP_ID/x86_64/beta" + if ostree rev-parse --repo="$REPO_DIR" "$beta" >/dev/null 2>&1 && + ! ostree show --repo="$REPO_DIR" --print-metadata-key=ostree.endoflife "$beta" >/dev/null 2>&1; then + flatpak build-commit-from --src-ref="$beta" \ + --end-of-life="The beta branch gets no more updates: every change now comes as a release on the stable branch. Install stable with flatpak install catcrafts net.catcrafts.CrafterBuild//stable, then remove beta with flatpak uninstall net.catcrafts.CrafterBuild//beta." \ + --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ + "$REPO_DIR" "$beta" + fi flatpak build-update-repo --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ --generate-static-deltas --prune "$REPO_DIR" # The repo description and the app ref carry the public key. @@ -173,72 +297,21 @@ jobs: (cd dist && sha256sum * > SHA256SUMS) ls -la dist - - name: Rolling 'latest' release - if: github.event_name != 'pull_request' && github.ref_type != 'tag' - run: | - git config user.email "ci@catcrafts.net" - git config user.name "Crafter Build CI" - git tag -f latest - git push origin latest --force - - name: Publish rolling 'latest' release - if: github.event_name != 'pull_request' && github.ref_type != 'tag' - uses: https://code.forgejo.org/actions/forgejo-release@v2 - with: - direction: upload - url: ${{ github.server_url }} - repo: ${{ github.repository }} - tag: latest - title: Latest master build - prerelease: true - override: true - release-dir: dist - token: ${{ secrets.GITHUB_TOKEN }} - - - name: Release notes - if: github.ref_type == 'tag' - run: | - # The newest block's description as Markdown, for the - # release page. AppStream has no headings, so a paragraph that - # introduces a list is one:

New

    ...
comes out as - # "## New" over its items, the shape the notes take since 3.0.0. - # A paragraph on its own stays a paragraph, as the older entries - # are written. and get their Markdown spelling; a code - # span also keeps text such as lib//eh from being read as - # an HTML tag by the forge. - python3 - > release-notes.md <<'EOF' - import xml.etree.ElementTree as ET - - desc = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release/description") - - def text(el): - parts = [el.text or ""] - for child in el: - inner = text(child) - parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner)) - parts.append(child.tail or "") - return " ".join("".join(parts).split()) - - blocks = list(desc) - out = [] - for i, el in enumerate(blocks): - heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol") - if el.tag == "p": - out.append(f"## {text(el)}" if heads_list else text(el)) - elif el.tag == "ul": - out.append("\n".join(f"- {text(li)}" for li in el)) - elif el.tag == "ol": - out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1))) - print("\n\n".join(out)) - EOF - name: Publish release - if: github.ref_type == 'tag' + if: github.event_name == 'push' + # The notes are release-notes.md, which the Version step wrote. No + # `override`: it would delete a tag that has moved off this commit + # since the Version step and tag the old commit instead. Without it + # the action refuses such a tag, and a release that an earlier + # attempt left half made, such as a draft whose upload failed, has + # to be deleted before a re-run. uses: https://code.forgejo.org/actions/forgejo-release@v2 with: direction: upload url: ${{ github.server_url }} repo: ${{ github.repository }} - tag: ${{ github.ref_name }} - title: crafter-build ${{ github.ref_name }} + tag: ${{ steps.meta.outputs.version }} + title: crafter-build ${{ steps.meta.outputs.version }} release-notes-file: release-notes.md release-dir: dist token: ${{ secrets.GITHUB_TOKEN }} diff --git a/build.sh b/build.sh index 02b089b..dcb8696 100755 --- a/build.sh +++ b/build.sh @@ -36,10 +36,9 @@ else STD_CPPM=/usr/share/libc++/v1/std.cppm fi -# The version baked into the binary: the environment (CI sets it from the -# tag), else the nearest tag, else a development marker. `latest` is the -# rolling CI tag and never a version. -VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty --exclude=latest 2>/dev/null || echo 0.0.0-dev)}" +# The version baked into the binary: the environment (CI sets it), else the +# nearest tag, else a development marker. +VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo 0.0.0-dev)}" mkdir -p build bin lib share/crafter-build share/crafter-build/wasi-runtime diff --git a/project.cpp b/project.cpp index 9fc8e43..0f10375 100644 --- a/project.cpp +++ b/project.cpp @@ -157,14 +157,14 @@ extern "C" Configuration CrafterBuildProject(std::span a if (const fs::path& llvmInclude = GetToolchain().llvmIncludeDir; !llvmInclude.empty()) { CrafterBuildLib->compileFlags.push_back(std::format("-I{}", llvmInclude.string())); } - // The version the binary reports: CI sets it from the tag, a checkout - // describes itself, anything else is a development build. A define on - // the library, so each version is its own library variant. + // The version the binary reports: CI sets it, a checkout describes + // itself, anything else is a development build. A define on the + // library, so each version is its own library variant. std::string version; if (const char* env = std::getenv("CRAFTER_BUILD_VERSION"); env && *env) { version = env; } else { - auto described = RunCommandWithTimeout("git describe --tags --always --dirty --exclude=latest 2>/dev/null", std::chrono::seconds(10)); + auto described = RunCommandWithTimeout("git describe --tags --always --dirty 2>/dev/null", std::chrono::seconds(10)); version = described.output; while (!version.empty() && (version.back() == '\n' || version.back() == '\r')) version.pop_back(); if (described.exitCode != 0 || version.empty()) version = "0.0.0-dev"; From 54aaf078b20f5c899a7ea63f88dae0d2ac8fe3b1 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:44:35 +0100 Subject: [PATCH 5/6] docs: a release is a signed tag pushed with master The README offered master builds as the `beta` branch, AGENTS.template.md described the `latest` tag CI force-pushed, a release as a tag pushed after master, and CI as running on every push to master, and the repo-better-solution lens told reviewers that users get master through `beta`. The README now says that releases go to `stable`, the branch its install commands pick even while the repo still holds the old `beta` branch. AGENTS.template.md says that every pushed range adds a release entry and goes out in one push with a signed tag on the commit master receives, its last commit or, on the pull-request path, the merge, how to write the entry, what every run checks before building and what a tag run also checks, publishes and creates, what follows when one fails, what CI runs for, and what becomes of `beta`; the lens says users get master through releases on `stable`. The entry's rules are the owner's: at most three sections, New, Fixes and Breaking, the most important item first and nothing after them, and a version that is a major for a Breaking section, a minor for New items and a patch for Fixes alone. The release paragraph also gets the blank line that keeps it apart from the one on local builds. --- .review-gates/lenses/repo-better-solution.md | 4 ++-- AGENTS.template.md | 10 ++++++---- README.md | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/.review-gates/lenses/repo-better-solution.md b/.review-gates/lenses/repo-better-solution.md index 65bd330..278b12f 100644 --- a/.review-gates/lenses/repo-better-solution.md +++ b/.review-gates/lenses/repo-better-solution.md @@ -37,6 +37,6 @@ checkout (`../Crafter.*/project.cpp`) and of every user compiles against what `interfaces/` exports and is loaded by the binary; the CLI, the `CRAFTER_BUILD_*` variables, the cache an older version left under `~/.cache/crafter.build/` and the `bin/` layout CI globs are contracts too. -Users get a push to master through the Flatpak's `beta` branch and a tag -through `stable`. A change an existing `project.cpp` does not survive is a +Users get master through releases on the Flatpak's `stable` branch. A +change an existing `project.cpp` does not survive is a finding unless the range documents, in the README, what users must change. diff --git a/AGENTS.template.md b/AGENTS.template.md index 1db87b0..f275a93 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -69,7 +69,7 @@ Three things to know about this loop: ./bin/crafter-build clean ``` -Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` on every push to `master` and on pull requests, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/.log`. +Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` for every release tag, every pull request into `master` and every manual run, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/.log`. ## Testing instructions @@ -142,7 +142,8 @@ These were decided deliberately. Do not undo them without the maintainer's say-s - Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice. - Separate commits for the fix, its test and the README change when each stands on its own. - Update the README in the same push when user-visible behaviour changes. -- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on every push to `master` and on pull requests; a push to `master` then publishes the `beta` branch of the Catcrafts repo, and an `X.Y.Z` tag the `stable` branch plus a release. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push. +- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it for every release tag, every pull request into `master` and every manual run. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push. +- Every push to `master` is a release, made by the signed tag pushed with it (see "CI and publishing"). Every pushed range adds a `` entry for its version at the top of `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, in a commit of its own (`docs(packaging): add the X.Y.Z release entry`), dated the day of the push in UTC. Its notes have up to three sections, `

New

`, `

Fixes

` and `

Breaking

` in that order, each followed by a `
    ` of its items, the most important first, and nothing after the last section. The version follows from them: a Breaking section makes it a major, New items a minor, and Fixes alone a patch. Once the owner has approved the frozen hash, the commit `master` is to receive gets the signed tag of that version, made locally, and both go out in one push: `git tag -s X.Y.Z -m 'crafter-build X.Y.Z'`, then `git push --atomic origin HEAD:master X.Y.Z`. That commit is the range's last one, or, when the owner asks for a pull request instead, the merge of the reviewed branch into `master`. - Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits: a range reaches `master` as a fast-forward, and when the owner asks for a pull request instead, the merge of the reviewed branch into `master` is pushed without a freeze of its own. - Every commit of a pushed range is GPG-signed, has `jorijnvdgraaf@catcrafts.net` as its author and committer, and carries no AI attribution trailer in its message, neither `Co-Authored-By:` nor `Assisted-by:`. `freeze` refuses a range with a commit that breaks any of these (`requireSigned`, `authors` and `aiTrailers` in `.review-gates/config.json`). - Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a`, `AGENTS.md` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`. @@ -155,7 +156,7 @@ These were decided deliberately. Do not undo them without the maintainer's say-s - `--runner=cmd:` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests. - `crafter-build run ` executes whatever the script does, with your permissions. Only the scripts of the project crafter-build was pointed at can be run, never a dependency's; keep it that way. - The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library. -- CI force-pushes the rolling `latest` tag and publishes a prerelease on every push to `master` with the workflow's own token, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it. +- CI creates a release for each release tag with the workflow's own token, pushes nothing to git, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it. - Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way. ## CI and publishing @@ -174,7 +175,8 @@ gpg --batch --pinentry-mode loopback --passphrase '' --export-secret-keys --armo printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id ``` -To release: add a `` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), push it to `master`, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry. +A release is a tag `X.Y.Z`, annotated and signed, pushed with the `master` it releases; a push to `master` without one runs nothing. Before building, every run renders the release notes from the newest `` entry in `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, and stops when the metainfo does not parse or the notes come out empty. A tag run also checks that Forgejo verifies the tag as signed by the address the workflow's `RELEASE_SIGNER` names, that the tag's commit is on `master`, that its version has the form `X.Y.Z` and is newer than every release the forge has published, its own included, and that the newest entry is that version. Once the build and the suite inside it pass, the tag run publishes `stable` and creates the release, with the notes, the bundle, the `.flatpakref` and `SHA256SUMS` attached. Release runs queue one behind another, and a failed one can be re-run while no newer release exists, once a release it left half made is deleted; when the fix needs a commit of its own, that push is the next version's release. Pull requests and manual runs publish nothing. The `beta` branch, which every push to `master` used to go to, gets no more builds: the publish step marks its last build end-of-life, once, and the next `flatpak update` of a beta installation prints the reason. + Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive. ```bash diff --git a/README.md b/README.md index f9238e0..cfd2bb1 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ alias crafter-build='flatpak run --die-with-parent net.catcrafts.CrafterBuild' `--die-with-parent` ends the sandbox with the `flatpak run` that started it. Without the flag, killing only the `flatpak run` process leaves the sandbox running: a build carries on writing into `bin/` and `build/`, and a program started with `-r` keeps running. With it, bubblewrap kills everything in the sandbox with SIGKILL when the `flatpak run` process dies or when that process's parent does. For `crafter-build …` or `crafter-build … &` typed at a bash or zsh prompt, that parent is the shell, so the sandbox ends with the shell, even after a `disown`. Ctrl-C reaches the program either way. -Pre-release builds of `master` are the `beta` branch (`flatpak install catcrafts net.catcrafts.CrafterBuild//beta`), and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host. +Releases are published to the `stable` branch that the commands above install, and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host. ## Writing a project.cpp From 0cc699139cb5d2802a62e90f6033193101bf4219 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:46:11 +0100 Subject: [PATCH 6/6] docs(packaging): add the 6.0.0 release entry The release of this push and of everything on master since 5.0.0: lldb, the Khronos validation layer and the SPIRV-Tools programs in the Flatpak, --die-with-parent in the README's alias, and the release .flatpakref with the repository's key in it. Its Breaking item, the end of beta and of the rolling release, makes it a major. --- .../net.catcrafts.CrafterBuild.metainfo.xml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml index d962c8f..38d6b97 100644 --- a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml +++ b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml @@ -25,6 +25,25 @@ + + +

    New

    +
      +
    • The Flatpak carries lldb, with lldb-server, lldb-dap and lldb-mcp, from the same LLVM build as the compiler, so programs that -r and the test runner start can be debugged inside the sandbox they run in. The README's new Debugging section says how, including flatpak enter to reach a program that is already running.
      • +
    • The Flatpak carries the Khronos Vulkan validation layer, the Vulkan SDK release of the runtime's own loader. VK_INSTANCE_LAYERS or VK_LOADER_LAYERS_ENABLE turns it on from the environment, and VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1 adds GPU-assisted validation. A layer installed on the host cannot stand in: flatpak run clears VK_LAYER_PATH and VK_ADD_LAYER_PATH.
      • +
    • The Flatpak carries the SPIRV-Tools programs (spirv-val, spirv-opt, spirv-dis and the rest), so a test that requires tool:spirv-val runs inside it instead of being skipped.
      • +
    • The alias in the README's install commands passes --die-with-parent, which ends the sandbox when its flatpak run process dies and, for a command typed at a bash or zsh prompt, when that shell does. Without it, killing flatpak run leaves a build writing into bin/ and build/ and a program started with -r running. An alias set up from an earlier README does not have it.
      • +
    +

    Fixes

    +
      +
    • The .flatpakref attached to a release carries the repository's signing key. Releases 2.0.0 to 5.0.0 attach one with a placeholder in its place, and installing from it fails with GPG: Unable to export keys; the one on flatpak.catcrafts.net has the key.
      • +
    +

    Breaking

    +
      +
    • Master has no builds of its own any more: the beta branch gets no new builds and the rolling "Latest master build" release is gone, and every change reaches users as a release on the stable branch. Beta's last build is marked end-of-life, and the next flatpak update of a beta installation says so: install stable with flatpak install catcrafts net.catcrafts.CrafterBuild//stable and remove beta with flatpak uninstall net.catcrafts.CrafterBuild//beta.
      • +
    +     +

    Fixes