Catcrafts/Crafter.Build
Watch
2
0
Fork
You've already forked Crafter.Build
 0
Code Issues 5 Pull requests Projects Releases 7 Packages Wiki Activity Actions

fix(ci): publish under the lock Crafter.Review's workflow takes

Browse source 
Crafter.Review's workflow publishes its Flatpak into the same repo, the
web root's repo/, and holds an flock on .publish.lock in the web root
while it commits and updates the summary. Its comment says that this
protects both only once Crafter.Build's workflow takes the lock too, and
ours never did, so a publish of each could interleave: a
build-commit-from and a build-update-repo --generate-static-deltas
--prune of each workflow on one repo at the same time.

The Publish step now holds the same lock from its commit to the ref
files it writes into the web root, and the job installs util-linux for
flock, as Crafter.Review's jobs do.

AGENTS.template.md says that the repo is shared and the lock taken.
Neither it nor the workflow's header comment says any longer that the
web root and the signing key are this runner's alone, or that the
organisation-wide `flatpak` runner mounts none of them. That runner,
which serves every Catcrafts repository and refused all of this job's
volumes when it last took it, in October, now mounts the web root and
the key for Crafter.Review's publish job, and the volumes a runner
allows are open to every job it runs. The owner has ruled the sharing
intended. The header also stops saying that a job of ours that lands
there fails before building: that held while the runner refused every
volume, and AGENTS.template.md keeps it as the October history.
This commit is contained in:
Jorijn van der Graaf 2026-11-02 12:04:06 +01:00
commit 2babe94a6a
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
2 changed files with 14 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

16
.forgejo/workflows/ci.yaml
View file

@ -11,9 +11,10 @@ name: Flatpak
# flatpak-builder state (ccache and the LLVM module, so only crafter-build # flatpak-builder state (ccache and the LLVM module, so only crafter-build
# itself rebuilds on a normal run), the published repo's web root, and the # itself rebuilds on a normal run), the published repo's web root, and the
# repo signing key, which lives on the runner host and never in the forge. # repo signing key, which lives on the runner host and never in the forge.
# No other runner carries that label. Plain `flatpak` also matches the # No other runner carries that label. Plain `flatpak` is the label of the
# organisation's runner, which mounts none of those volumes; a job it picks # organisation's runner, which serves every Catcrafts repository and also
# up fails before building anything. # mounts the web root and the key, for Crafter.Review's publish job, so any
# job it runs can mount them (AGENTS.template.md, CI and publishing).
on: on:
pull_request: pull_request:
branches: [master] branches: [master]
@ -45,11 +46,12 @@ jobs:
-v /etc/crafter-flatpak:/run/crafter-flatpak:ro -v /etc/crafter-flatpak:/run/crafter-flatpak:ro
steps: steps:
- name: Install tools - name: Install tools
# util-linux: flock, for the Publish step.
run: | run: |
pacman-key --init pacman-key --init
pacman-key --populate archlinux pacman-key --populate archlinux
pacman -Sy --noconfirm --needed archlinux-keyring pacman -Sy --noconfirm --needed archlinux-keyring
pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python util-linux
git config --global --add safe.directory '*' git config --global --add safe.directory '*'
- name: Sdk - name: Sdk
@ -144,6 +146,11 @@ jobs:
export GNUPGHOME=$(mktemp -d) export GNUPGHOME=$(mktemp -d)
gpg --batch --quiet --import /run/crafter-flatpak/signing-key.asc gpg --batch --quiet --import /run/crafter-flatpak/signing-key.asc
FLATPAK_GPG_KEY_ID=$(cat /run/crafter-flatpak/key-id) FLATPAK_GPG_KEY_ID=$(cat /run/crafter-flatpak/key-id)
# Crafter.Review's workflow publishes into the same repo and holds
# this lock while it does, so a commit and the summary update of
# one workflow never interleave with the other's.
exec 9> "$WEB_DIR/.publish.lock"
flock 9
# The live repo is the canonical copy; builds are promoted into it, # The live repo is the canonical copy; builds are promoted into it,
# never built into it. # never built into it.
# Keyed on the repo's config file, not the directory: the web root # Keyed on the repo's config file, not the directory: the web root
@ -158,6 +165,7 @@ jobs:
key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0) key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0)
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo"
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref"
flock -u 9
# The release attaches this .flatpakref, not the one in the # The release attaches this .flatpakref, not the one in the
# checkout: that one has a placeholder where the key goes, and # checkout: that one has a placeholder where the key goes, and
# flatpak cannot install from it. # flatpak cannot install from it.

4
AGENTS.template.md
View file

@ -160,9 +160,9 @@ These were decided deliberately. Do not undo them without the maintainer's say-s
## CI and publishing ## CI and publishing
`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. `.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and no other repository's job runs on it. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The web root and the key are not this runner's alone: the organisation-wide `flatpak` runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository, mounts both for Crafter.Review's publish job, and the volumes a runner allows (`valid_volumes` in its config) are open to every job it runs, so a job of any Catcrafts repository that runs there can mount them as well. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. Crafter.Review's workflow publishes into the same repo, and each workflow holds an flock on `/var/www/flatpak/.publish.lock` while it commits and updates the summary.
Keep the label unique to that runner. Plain `flatpak` also belongs to the organisation-wide runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository and on purpose allows no volumes. Whenever this job lands there, flatpak-builder stops with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of the job log shows which runner took it and lists every volume it ignored. Keep the label unique to that runner. Plain `flatpak` belongs to the organisation-wide runner. When this job still asked for `flatpak` and that runner took it (runs #368 and #370, in October 2026), the runner allowed none of the job's volumes, and flatpak-builder stopped with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of such a job's log shows which runner took it and lists every volume it ignored.
The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing: The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing: