feat(ci): drop beta and latest, release signed tags on master

A push to master published the `beta` branch and moved the rolling
`latest` tag and its "Latest master build" prerelease, and a pushed
X.Y.Z tag made the release and published `stable`. The owner wants a
release for every push to master, made by the signed tag that goes with
it, and nothing for a push without one. So `beta`, `latest` and the
branch trigger are gone: the workflow runs for pushed tags, pull
requests and manual runs only, and the `latest` tag and its prerelease
have been deleted from the forge.

`beta` gets no more builds, and its installations would not hear of it:
the release notes ship in stable builds only. The Publish step marks
beta's last build end-of-life, once, with a reason that names stable,
and the next `flatpak update` of a beta installation prints it. The
reason also names the branch, because a non-interactive update prints
only the app's name with it.

Before building, a tag run now checks the tag: it has to be annotated
and signed by the owner as Forgejo verifies it, its commit has to be on
master, its version has to be X.Y.Z and newer than every release the
forge has published, its own included (a later tag whose run still
queues does not count), and the metainfo's newest <release> entry has to
be that version. The tag run checked only the last of these. It reads
the tag from the forge's API, because the checkout fetches a pushed tag
again as a plain ref to its commit, so the job's clone never has the
annotated tag, and Forgejo verifies a signed tag's own signature but
falls back to the commit's for an unsigned one, so the step also
requires that the verdict covers the tag's payload. The metainfo's entry
is read once, as the first <release> element: the Version step takes its
version and renders its notes into release-notes.md with the renderer
the Release notes step had, and stops the run when they come out empty,
so a comment or another attribute order cannot give a release one
entry's version and another's notes. It does so on every run, pull
requests and manual runs included, so a broken entry shows before a
release.

Release runs share a concurrency group, so they queue one behind
another; a pull request still cancels its own outdated run. A failed
release run can be re-run while no newer release exists, since its tag
is still on its commit; a release that an attempt left half made has to
be deleted first. The release step runs without `override`, which would
delete a tag that has moved off the run's commit since the Version step
and tag the old commit instead.

A manual run no longer publishes anything; it used to publish `beta`
and move `latest` from whichever branch it was started on. No step
pushes to git, so the checkout keeps no credentials. git describe in
build.sh, project.cpp and the workflow has no `latest` tag left to
exclude.

.forgejo/workflows/ci.yaml

@@ -1,10 +1,15 @@
 name: Flatpak
 
-# One job, three outcomes:
+# One job, two outcomes:
-#   pull request  -> build the Flatpak and run the suite inside it
+#   pull request, manual run -> build the Flatpak and run the suite inside it
-#   push master   -> also publish the result as the `beta` branch of the
+#   pushed tag X.Y.Z         -> also release the build: publish it as the
-#                    Catcrafts repo and refresh the rolling `latest` release
+#                               `stable` branch of the Catcrafts repo and
-#   tag X.Y.Z     -> publish as `stable` and create release X.Y.Z
+#                               create release X.Y.Z
+#
+# A release is a tag that the owner makes and signs locally and pushes with
+# the master it releases; a push to master without one runs nothing. The tag
+# names the version, and the metainfo's newest <release> entry has to be that
+# version: its description is the release notes.
 #
 # The `crafter-flatpak` runner label is the privileged runner on the Hetzner
 # box (bubblewrap needs it) with four volumes allowed: the Sdk install, the
@@ -19,20 +24,29 @@ on:
   pull_request:
     branches: [master]
   push:
-    branches: [master]
     # Bare semver tags. Spelled with globs so it means the same whether the
-    # forge reads it as a workflow pattern or as a plain glob; `latest`, the
+    # forge reads it as a workflow pattern or as a plain glob.
-    # rolling tag this workflow moves itself, does not start with a digit.
     tags: ['[0-9]*.[0-9]*.[0-9]*']
   workflow_dispatch:
 
+# Pushes are tags only, so every release run is in one group, and a release
+# starts only once the one before it has finished. A pull request's group
+# also holds its number, which only a pull request's event carries, so a
+# new push to it cancels its own outdated run and no other.
+concurrency:
+  group: flatpak-${{ github.event_name }}-${{ github.event.pull_request.number }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
 env:
   APP_ID: net.catcrafts.CrafterBuild
+  BRANCH: stable
   MANIFEST: packaging/flatpak/net.catcrafts.CrafterBuild.yaml
   CI_MANIFEST: packaging/flatpak/ci-manifest.yaml
   RUNTIME: org.freedesktop.Sdk//26.08
   REPO_DIR: /srv/flatpak/repo
   WEB_DIR: /srv/flatpak
+  # A release tag carries a signature Forgejo verifies for this address.
+  RELEASE_SIGNER: jorijnvdgraaf@catcrafts.net
 
 jobs:
   flatpak:
@@ -62,40 +76,137 @@ jobs:
           flatpak update -y --noninteractive
 
       - name: Checkout
+        # Every branch and tag, for git describe and the release checks; no
+        # step pushes, so the token is not kept in the checkout.
         uses: actions/checkout@v4
         with:
           fetch-depth: 0
-          persist-credentials: true
+          persist-credentials: false
 
-      - name: Version and channel
+      - name: Version
         id: meta
         run: |
           set -eu
-          if [ "${GITHUB_REF_TYPE}" = tag ]; then
+          # The metainfo's newest entry is its first <release> element, the
-            version="$GITHUB_REF_NAME"
+          # one parse that gives both its version and the release notes,
-            branch=stable
+          # before anything is built: a comment or another attribute order
-            # The metainfo's newest release is the release notes and must be
+          # would mislead a text search, and an entry whose notes render
-            # the version being tagged.
+          # empty stops the run here. The notes go to release-notes.md, for
-            top=$(grep -m1 -oP '<release version="\K[^"]+' packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml)
+          # the release page, as Markdown. AppStream has no headings, so a
-            if [ "$top" != "$version" ]; then
+          # paragraph that introduces a list is one: <p>New</p><ul>...</ul>
-              echo "metainfo lists $top as the newest release, but the tag is $version" >&2
+          # comes out as "## New" over its items, the shape the notes take
+          # since 3.0.0. A paragraph on its own stays a paragraph, as the
+          # older entries are written. <em> and <code> get their Markdown
+          # spelling; a code span also keeps text such as lib/<triple>/eh
+          # from being read as an HTML tag by the forge.
+          newest=$(python3 - <<'EOF'
+          import sys
+          import xml.etree.ElementTree as ET
+
+          release = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release")
+          desc = None if release is None else release.find("description")
+
+          def text(el):
+              parts = [el.text or ""]
+              for child in el:
+                  inner = text(child)
+                  parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner))
+                  parts.append(child.tail or "")
+              return " ".join("".join(parts).split())
+
+          blocks = [] if desc is None else list(desc)
+          out = []
+          for i, el in enumerate(blocks):
+              heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol")
+              if el.tag == "p":
+                  out.append(f"## {text(el)}" if heads_list else text(el))
+              elif el.tag == "ul":
+                  out.append("\n".join(f"- {text(li)}" for li in el))
+              elif el.tag == "ol":
+                  out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1)))
+          notes = "\n\n".join(out)
+          if not notes.strip():
+              sys.exit("the metainfo needs a <release> at the top of <releases> whose <description> renders release notes")
+          with open("release-notes.md", "w") as f:
+              print(notes, file=f)
+          print(release.get("version", ""))
+          EOF
+          )
+          if [ "$GITHUB_EVENT_NAME" = push ]; then
+            # A release: the only pushes this workflow runs for are tags.
+            # The tag is the version; it has to be annotated, signed by the
+            # owner as Forgejo verifies it against the keys its users added,
+            # on master, newer than every published release, and the version
+            # of the metainfo's newest entry. The tag itself is read from the
+            # forge: the checkout fetches a pushed tag again as a plain ref
+            # to its commit, so the local one is never the annotated tag.
+            version=$GITHUB_REF_NAME
+            if ! printf '%s\n' "$version" | grep -qxE '[0-9]+\.[0-9]+\.[0-9]+'; then
+              echo "the tag $version is not a version of the form X.Y.Z" >&2
+              exit 1
+            fi
+            api="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY"
+            tag=$(curl -sSf "$api/tags/$version")
+            object=$(printf '%s' "$tag" | jq -r .id)
+            commit=$(printf '%s' "$tag" | jq -r .commit.sha)
+            if [ "$commit" != "$GITHUB_SHA" ]; then
+              echo "the forge has the tag $version on $commit, not on this run's commit $GITHUB_SHA" >&2
+              exit 1
+            fi
+            if [ "$object" = "$commit" ]; then
+              echo "$version is a lightweight tag; a release tag is annotated and signed: git tag -s $version -m 'crafter-build $version'" >&2
+              exit 1
+            fi
+            # Forgejo verifies a signed tag's own signature, and for an
+            # unsigned one falls back to its commit's, so the verdict also
+            # has to cover the tag's payload, not the commit's.
+            signed=$(curl -sSf "$api/git/tags/$object" |
+              jq -r --arg c "$commit" --arg v "$version" '"\(.verification.verified) \(.verification.signer.email // "") \(.verification.payload // "" | startswith("object \($c)\ntype commit\ntag \($v)\n"))"')
+            if [ "$signed" != "true $RELEASE_SIGNER true" ]; then
+              echo "Forgejo does not verify the tag $version itself as signed by $RELEASE_SIGNER (verified, signer, over the tag's payload: $signed)" >&2
+              exit 1
+            fi
+            if ! git merge-base --is-ancestor "$GITHUB_SHA" origin/master; then
+              echo "the tag $version is on $(git rev-parse --short "$GITHUB_SHA"), which is not on master: push the tag with the master it releases" >&2
+              exit 1
+            fi
+            # Newer than every release the forge has published, its own
+            # included, and not every tag: a later tag whose run still
+            # queues behind this one has no release yet, and a tag that is
+            # no X.Y.Z version never gets one.
+            for t in $(git tag --list | awk '/^[0-9]+\.[0-9]+\.[0-9]+$/' | sort -V); do
+              [ "$(printf '%s\n%s\n' "$t" "$version" | sort -V | tail -n 1)" = "$t" ] || continue
+              code=$(curl -s -o /dev/null -w '%{http_code}' "$api/releases/tags/$t")
+              case "$code" in
+                404) ;;
+                200)
+                  if [ "$t" = "$version" ]; then
+                    echo "$version is released already; to re-run after a release left half made, delete that release first" >&2
+                  else
+                    echo "$version is not newer than the release $t" >&2
+                  fi
+                  exit 1 ;;
+                *) echo "the forge answered $code when asked whether $t is released" >&2; exit 1 ;;
+              esac
+            done
+            if [ "$newest" != "$version" ]; then
+              echo "the metainfo's newest release is \"$newest\", but the tag is $version: the tagged commit needs the <release> entry for $version at the top of packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml" >&2
               exit 1
             fi
           else
-            # A master build names itself after the release it follows, the
+            # Nothing is published. The build names itself after the
-            # way a local checkout does: 2.0.0-3-g21746b1 is three commits
+            # release it follows, the way a local checkout does:
-            # past 2.0.0. Before the first tag there is nothing to describe
+            # 5.0.0-3-g21746b1 is three commits past 5.0.0. Before the first
-            # from and the bare hash gets a placeholder in front.
+            # tag there is nothing to describe from and the bare hash gets a
-            described=$(git describe --tags --always --exclude=latest)
+            # placeholder in front.
+            described=$(git describe --tags --always)
             case "$described" in
               [0-9]*.[0-9]*.[0-9]*) version="$described" ;;
               *) version="0.0.0-dev+$described" ;;
             esac
-            branch=beta
           fi
           echo "version=$version" >> "$GITHUB_OUTPUT"
-          echo "branch=$branch" >> "$GITHUB_OUTPUT"
+          echo "version $version"
-          echo "version $version, flatpak branch $branch"
 
       - name: Build
         # The manifest is rewritten beside the original so its relative
@@ -108,7 +219,7 @@ jobs:
           grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST"
           flatpak-builder --force-clean --ccache --disable-rofiles-fuse \
             --state-dir=/root/.cache/flatpak-builder \
-            --repo=repo --default-branch="${{ steps.meta.outputs.branch }}" \
+            --repo=repo --default-branch="$BRANCH" \
             build-dir "$CI_MANIFEST"
 
       - name: Smoke test
@@ -116,16 +227,16 @@ jobs:
           flatpak-builder --run build-dir "$CI_MANIFEST" crafter-build --version
           flatpak-builder --run build-dir "$CI_MANIFEST" sh -c 'cd examples/hello-world && crafter-build && ls bin/*/hello*'
 
-      # Everything below publishes and only runs on master and on tags.
+      # Everything below releases the build and runs only for a pushed tag.
       - name: Bundle
-        if: github.event_name != 'pull_request'
+        if: github.event_name == 'push'
         run: |
           mkdir -p dist
-          flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}"
+          flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "$BRANCH"
           ls -la dist
 
       - name: Upload bundle
-        if: github.event_name != 'pull_request'
+        if: github.event_name == 'push'
         uses: actions/upload-artifact@v3
         with:
           name: crafter-build-${{ steps.meta.outputs.version }}
@@ -133,7 +244,7 @@ jobs:
           if-no-files-found: error
 
       - name: Publish to the Catcrafts repo
-        if: github.event_name != 'pull_request'
+        if: github.event_name == 'push'
         run: |
           set -eu
           # The signing key is generated once on the runner host
@@ -158,7 +269,20 @@ jobs:
           [ -f "$REPO_DIR/config" ] || ostree init --repo="$REPO_DIR" --mode=archive
           flatpak build-commit-from --src-repo=repo \
             --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
-            "$REPO_DIR" "app/$APP_ID/x86_64/${{ steps.meta.outputs.branch }}"
+            "$REPO_DIR" "app/$APP_ID/x86_64/$BRANCH"
+          # Every push to master used to be published as `beta`. Its last
+          # build is marked end-of-life, once, so that the next
+          # `flatpak update` of a beta installation says why nothing follows.
+          # The reason names the branch: with --noninteractive, flatpak's
+          # message names only the app.
+          beta="app/$APP_ID/x86_64/beta"
+          if ostree rev-parse --repo="$REPO_DIR" "$beta" >/dev/null 2>&1 &&
+             ! ostree show --repo="$REPO_DIR" --print-metadata-key=ostree.endoflife "$beta" >/dev/null 2>&1; then
+            flatpak build-commit-from --src-ref="$beta" \
+              --end-of-life="The beta branch gets no more updates: every change now comes as a release on the stable branch. Install stable with flatpak install catcrafts net.catcrafts.CrafterBuild//stable, then remove beta with flatpak uninstall net.catcrafts.CrafterBuild//beta." \
+              --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
+              "$REPO_DIR" "$beta"
+          fi
           flatpak build-update-repo --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
             --generate-static-deltas --prune "$REPO_DIR"
           # The repo description and the app ref carry the public key.
@@ -173,72 +297,21 @@ jobs:
           (cd dist && sha256sum * > SHA256SUMS)
           ls -la dist
 
-      - name: Rolling 'latest' release
-        if: github.event_name != 'pull_request' && github.ref_type != 'tag'
-        run: |
-          git config user.email "ci@catcrafts.net"
-          git config user.name "Crafter Build CI"
-          git tag -f latest
-          git push origin latest --force
-      - name: Publish rolling 'latest' release
-        if: github.event_name != 'pull_request' && github.ref_type != 'tag'
-        uses: https://code.forgejo.org/actions/forgejo-release@v2
-        with:
-          direction: upload
-          url: ${{ github.server_url }}
-          repo: ${{ github.repository }}
-          tag: latest
-          title: Latest master build
-          prerelease: true
-          override: true
-          release-dir: dist
-          token: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Release notes
-        if: github.ref_type == 'tag'
-        run: |
-          # The newest <release> block's description as Markdown, for the
-          # release page. AppStream has no headings, so a paragraph that
-          # introduces a list is one: <p>New</p><ul>...</ul> comes out as
-          # "## New" over its items, the shape the notes take since 3.0.0.
-          # A paragraph on its own stays a paragraph, as the older entries
-          # are written. <em> and <code> get their Markdown spelling; a code
-          # span also keeps text such as lib/<triple>/eh from being read as
-          # an HTML tag by the forge.
-          python3 - > release-notes.md <<'EOF'
-          import xml.etree.ElementTree as ET
-
-          desc = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release/description")
-
-          def text(el):
-              parts = [el.text or ""]
-              for child in el:
-                  inner = text(child)
-                  parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner))
-                  parts.append(child.tail or "")
-              return " ".join("".join(parts).split())
-
-          blocks = list(desc)
-          out = []
-          for i, el in enumerate(blocks):
-              heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol")
-              if el.tag == "p":
-                  out.append(f"## {text(el)}" if heads_list else text(el))
-              elif el.tag == "ul":
-                  out.append("\n".join(f"- {text(li)}" for li in el))
-              elif el.tag == "ol":
-                  out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1)))
-          print("\n\n".join(out))
-          EOF
       - name: Publish release
-        if: github.ref_type == 'tag'
+        if: github.event_name == 'push'
+        # The notes are release-notes.md, which the Version step wrote. No
+        # `override`: it would delete a tag that has moved off this commit
+        # since the Version step and tag the old commit instead. Without it
+        # the action refuses such a tag, and a release that an earlier
+        # attempt left half made, such as a draft whose upload failed, has
+        # to be deleted before a re-run.
         uses: https://code.forgejo.org/actions/forgejo-release@v2
         with:
           direction: upload
           url: ${{ github.server_url }}
           repo: ${{ github.repository }}
-          tag: ${{ github.ref_name }}
+          tag: ${{ steps.meta.outputs.version }}
-          title: crafter-build ${{ github.ref_name }}
+          title: crafter-build ${{ steps.meta.outputs.version }}
           release-notes-file: release-notes.md
           release-dir: dist
           token: ${{ secrets.GITHUB_TOKEN }}

build.sh

@@ -36,10 +36,9 @@ else
     STD_CPPM=/usr/share/libc++/v1/std.cppm
 fi
 
-# The version baked into the binary: the environment (CI sets it from the
+# The version baked into the binary: the environment (CI sets it), else the
-# tag), else the nearest tag, else a development marker. `latest` is the
+# nearest tag, else a development marker.
-# rolling CI tag and never a version.
+VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo 0.0.0-dev)}"
-VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty --exclude=latest 2>/dev/null || echo 0.0.0-dev)}"
 
 mkdir -p build bin lib share/crafter-build share/crafter-build/wasi-runtime
 

project.cpp

@@ -157,14 +157,14 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
     if (const fs::path& llvmInclude = GetToolchain().llvmIncludeDir; !llvmInclude.empty()) {
         CrafterBuildLib->compileFlags.push_back(std::format("-I{}", llvmInclude.string()));
     }
-    // The version the binary reports: CI sets it from the tag, a checkout
+    // The version the binary reports: CI sets it, a checkout describes
-    // describes itself, anything else is a development build. A define on
+    // itself, anything else is a development build. A define on the
-    // the library, so each version is its own library variant.
+    // library, so each version is its own library variant.
     std::string version;
     if (const char* env = std::getenv("CRAFTER_BUILD_VERSION"); env && *env) {
         version = env;
     } else {
-        auto described = RunCommandWithTimeout("git describe --tags --always --dirty --exclude=latest 2>/dev/null", std::chrono::seconds(10));
+        auto described = RunCommandWithTimeout("git describe --tags --always --dirty 2>/dev/null", std::chrono::seconds(10));
         version = described.output;
         while (!version.empty() && (version.back() == '\n' || version.back() == '\r')) version.pop_back();
         if (described.exitCode != 0 || version.empty()) version = "0.0.0-dev";