Jorijn van der Graaf
2babe94a6a
Crafter.Review's workflow publishes its Flatpak into the same repo, the web root's repo/, and holds an flock on .publish.lock in the web root while it commits and updates the summary. Its comment says that this protects both only once Crafter.Build's workflow takes the lock too, and ours never did, so a publish of each could interleave: a build-commit-from and a build-update-repo --generate-static-deltas --prune of each workflow on one repo at the same time. The Publish step now holds the same lock from its commit to the ref files it writes into the web root, and the job installs util-linux for flock, as Crafter.Review's jobs do. AGENTS.template.md says that the repo is shared and the lock taken. Neither it nor the workflow's header comment says any longer that the web root and the signing key are this runner's alone, or that the organisation-wide `flatpak` runner mounts none of them. That runner, which serves every Catcrafts repository and refused all of this job's volumes when it last took it, in October, now mounts the web root and the key for Crafter.Review's publish job, and the volumes a runner allows are open to every job it runs. The owner has ruled the sharing intended. The header also stops saying that a job of ours that lands there fails before building: that held while the runner refused every volume, and AGENTS.template.md keeps it as the October history.
244 lines
11 KiB
YAML
244 lines
11 KiB
YAML
name: Flatpak
|
|
|
|
# One job, three outcomes:
|
|
# pull request -> build the Flatpak and run the suite inside it
|
|
# push master -> also publish the result as the `beta` branch of the
|
|
# Catcrafts repo and refresh the rolling `latest` release
|
|
# tag X.Y.Z -> publish as `stable` and create release X.Y.Z
|
|
#
|
|
# The `crafter-flatpak` runner label is the privileged runner on the Hetzner
|
|
# box (bubblewrap needs it) with four volumes allowed: the Sdk install, the
|
|
# flatpak-builder state (ccache and the LLVM module, so only crafter-build
|
|
# itself rebuilds on a normal run), the published repo's web root, and the
|
|
# repo signing key, which lives on the runner host and never in the forge.
|
|
# No other runner carries that label. Plain `flatpak` is the label of the
|
|
# organisation's runner, which serves every Catcrafts repository and also
|
|
# mounts the web root and the key, for Crafter.Review's publish job, so any
|
|
# job it runs can mount them (AGENTS.template.md, CI and publishing).
|
|
on:
|
|
pull_request:
|
|
branches: [master]
|
|
push:
|
|
branches: [master]
|
|
# Bare semver tags. Spelled with globs so it means the same whether the
|
|
# forge reads it as a workflow pattern or as a plain glob; `latest`, the
|
|
# rolling tag this workflow moves itself, does not start with a digit.
|
|
tags: ['[0-9]*.[0-9]*.[0-9]*']
|
|
workflow_dispatch:
|
|
|
|
env:
|
|
APP_ID: net.catcrafts.CrafterBuild
|
|
MANIFEST: packaging/flatpak/net.catcrafts.CrafterBuild.yaml
|
|
CI_MANIFEST: packaging/flatpak/ci-manifest.yaml
|
|
RUNTIME: org.freedesktop.Sdk//26.08
|
|
REPO_DIR: /srv/flatpak/repo
|
|
WEB_DIR: /srv/flatpak
|
|
|
|
jobs:
|
|
flatpak:
|
|
runs-on: crafter-flatpak
|
|
container:
|
|
image: archlinux:latest
|
|
options: >-
|
|
-v crafter-flatpak-system:/var/lib/flatpak
|
|
-v crafter-flatpak-builder:/root/.cache/flatpak-builder
|
|
-v /var/www/flatpak:/srv/flatpak
|
|
-v /etc/crafter-flatpak:/run/crafter-flatpak:ro
|
|
steps:
|
|
- name: Install tools
|
|
# util-linux: flock, for the Publish step.
|
|
run: |
|
|
pacman-key --init
|
|
pacman-key --populate archlinux
|
|
pacman -Sy --noconfirm --needed archlinux-keyring
|
|
pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python util-linux
|
|
git config --global --add safe.directory '*'
|
|
|
|
- name: Sdk
|
|
# A no-op once the volume is warm; `update` follows point releases.
|
|
run: |
|
|
flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
|
|
flatpak install -y --noninteractive flathub "$RUNTIME"
|
|
flatpak update -y --noninteractive
|
|
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: true
|
|
|
|
- name: Version and channel
|
|
id: meta
|
|
run: |
|
|
set -eu
|
|
if [ "${GITHUB_REF_TYPE}" = tag ]; then
|
|
version="$GITHUB_REF_NAME"
|
|
branch=stable
|
|
# The metainfo's newest release is the release notes and must be
|
|
# the version being tagged.
|
|
top=$(grep -m1 -oP '<release version="\K[^"]+' packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml)
|
|
if [ "$top" != "$version" ]; then
|
|
echo "metainfo lists $top as the newest release, but the tag is $version" >&2
|
|
exit 1
|
|
fi
|
|
else
|
|
# A master build names itself after the release it follows, the
|
|
# way a local checkout does: 2.0.0-3-g21746b1 is three commits
|
|
# past 2.0.0. Before the first tag there is nothing to describe
|
|
# from and the bare hash gets a placeholder in front.
|
|
described=$(git describe --tags --always --exclude=latest)
|
|
case "$described" in
|
|
[0-9]*.[0-9]*.[0-9]*) version="$described" ;;
|
|
*) version="0.0.0-dev+$described" ;;
|
|
esac
|
|
branch=beta
|
|
fi
|
|
echo "version=$version" >> "$GITHUB_OUTPUT"
|
|
echo "branch=$branch" >> "$GITHUB_OUTPUT"
|
|
echo "version $version, flatpak branch $branch"
|
|
|
|
- name: Build
|
|
# The manifest is rewritten beside the original so its relative
|
|
# source paths still resolve. The suite runs inside this build.
|
|
env:
|
|
VERSION: ${{ steps.meta.outputs.version }}
|
|
run: |
|
|
set -eu
|
|
yq '(.modules[] | select(.name == "crafter-build") | ."build-options".env.CRAFTER_BUILD_VERSION) = strenv(VERSION)' "$MANIFEST" > "$CI_MANIFEST"
|
|
grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST"
|
|
flatpak-builder --force-clean --ccache --disable-rofiles-fuse \
|
|
--state-dir=/root/.cache/flatpak-builder \
|
|
--repo=repo --default-branch="${{ steps.meta.outputs.branch }}" \
|
|
build-dir "$CI_MANIFEST"
|
|
|
|
- name: Smoke test
|
|
run: |
|
|
flatpak-builder --run build-dir "$CI_MANIFEST" crafter-build --version
|
|
flatpak-builder --run build-dir "$CI_MANIFEST" sh -c 'cd examples/hello-world && crafter-build && ls bin/*/hello*'
|
|
|
|
# Everything below publishes and only runs on master and on tags.
|
|
- name: Bundle
|
|
if: github.event_name != 'pull_request'
|
|
run: |
|
|
mkdir -p dist
|
|
flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}"
|
|
ls -la dist
|
|
|
|
- name: Upload bundle
|
|
if: github.event_name != 'pull_request'
|
|
uses: actions/upload-artifact@v3
|
|
with:
|
|
name: crafter-build-${{ steps.meta.outputs.version }}
|
|
path: dist/*.flatpak
|
|
if-no-files-found: error
|
|
|
|
- name: Publish to the Catcrafts repo
|
|
if: github.event_name != 'pull_request'
|
|
run: |
|
|
set -eu
|
|
# The signing key is generated once on the runner host
|
|
# (/etc/crafter-flatpak: signing-key.asc and key-id) and mounted
|
|
# read-only; it is imported into a throwaway keyring per run.
|
|
if [ ! -s /run/crafter-flatpak/signing-key.asc ] || [ ! -s /run/crafter-flatpak/key-id ]; then
|
|
echo "no signing key under /etc/crafter-flatpak on the runner host; generate it as described in AGENTS.template.md (CI and publishing)" >&2
|
|
exit 1
|
|
fi
|
|
export GNUPGHOME=$(mktemp -d)
|
|
gpg --batch --quiet --import /run/crafter-flatpak/signing-key.asc
|
|
FLATPAK_GPG_KEY_ID=$(cat /run/crafter-flatpak/key-id)
|
|
# Crafter.Review's workflow publishes into the same repo and holds
|
|
# this lock while it does, so a commit and the summary update of
|
|
# one workflow never interleave with the other's.
|
|
exec 9> "$WEB_DIR/.publish.lock"
|
|
flock 9
|
|
# The live repo is the canonical copy; builds are promoted into it,
|
|
# never built into it.
|
|
# Keyed on the repo's config file, not the directory: the web root
|
|
# may exist as an empty directory before the first publish.
|
|
[ -f "$REPO_DIR/config" ] || ostree init --repo="$REPO_DIR" --mode=archive
|
|
flatpak build-commit-from --src-repo=repo \
|
|
--gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
|
|
"$REPO_DIR" "app/$APP_ID/x86_64/${{ steps.meta.outputs.branch }}"
|
|
flatpak build-update-repo --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
|
|
--generate-static-deltas --prune "$REPO_DIR"
|
|
# The repo description and the app ref carry the public key.
|
|
key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0)
|
|
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo"
|
|
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref"
|
|
flock -u 9
|
|
# The release attaches this .flatpakref, not the one in the
|
|
# checkout: that one has a placeholder where the key goes, and
|
|
# flatpak cannot install from it.
|
|
cp "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" dist/
|
|
(cd dist && sha256sum * > SHA256SUMS)
|
|
ls -la dist
|
|
|
|
- name: Rolling 'latest' release
|
|
if: github.event_name != 'pull_request' && github.ref_type != 'tag'
|
|
run: |
|
|
git config user.email "ci@catcrafts.net"
|
|
git config user.name "Crafter Build CI"
|
|
git tag -f latest
|
|
git push origin latest --force
|
|
- name: Publish rolling 'latest' release
|
|
if: github.event_name != 'pull_request' && github.ref_type != 'tag'
|
|
uses: https://code.forgejo.org/actions/forgejo-release@v2
|
|
with:
|
|
direction: upload
|
|
url: ${{ github.server_url }}
|
|
repo: ${{ github.repository }}
|
|
tag: latest
|
|
title: Latest master build
|
|
prerelease: true
|
|
override: true
|
|
release-dir: dist
|
|
token: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- name: Release notes
|
|
if: github.ref_type == 'tag'
|
|
run: |
|
|
# The newest <release> block's description as Markdown, for the
|
|
# release page. AppStream has no headings, so a paragraph that
|
|
# introduces a list is one: <p>New</p><ul>...</ul> comes out as
|
|
# "## New" over its items, the shape the notes take since 3.0.0.
|
|
# A paragraph on its own stays a paragraph, as the older entries
|
|
# are written. <em> and <code> get their Markdown spelling; a code
|
|
# span also keeps text such as lib/<triple>/eh from being read as
|
|
# an HTML tag by the forge.
|
|
python3 - > release-notes.md <<'EOF'
|
|
import xml.etree.ElementTree as ET
|
|
|
|
desc = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release/description")
|
|
|
|
def text(el):
|
|
parts = [el.text or ""]
|
|
for child in el:
|
|
inner = text(child)
|
|
parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner))
|
|
parts.append(child.tail or "")
|
|
return " ".join("".join(parts).split())
|
|
|
|
blocks = list(desc)
|
|
out = []
|
|
for i, el in enumerate(blocks):
|
|
heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol")
|
|
if el.tag == "p":
|
|
out.append(f"## {text(el)}" if heads_list else text(el))
|
|
elif el.tag == "ul":
|
|
out.append("\n".join(f"- {text(li)}" for li in el))
|
|
elif el.tag == "ol":
|
|
out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1)))
|
|
print("\n\n".join(out))
|
|
EOF
|
|
- name: Publish release
|
|
if: github.ref_type == 'tag'
|
|
uses: https://code.forgejo.org/actions/forgejo-release@v2
|
|
with:
|
|
direction: upload
|
|
url: ${{ github.server_url }}
|
|
repo: ${{ github.repository }}
|
|
tag: ${{ github.ref_name }}
|
|
title: crafter-build ${{ github.ref_name }}
|
|
release-notes-file: release-notes.md
|
|
release-dir: dist
|
|
token: ${{ secrets.GITHUB_TOKEN }}
|