From a3fe1bdda79b50078d51cb41d2a41e0e4c67bc75 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sat, 31 Oct 2026 23:55:28 +0100 Subject: [PATCH 01/14] docs(packaging): add the 5.0.0 release entry The fix for #43, a mingw build that saw the host's glibc headers and Linux libraries through a prefix like /usr as clang's sysroot, and what the review of it brought along: units that recompile after a std module rebuilt in place, CMake externals built again when their configure inputs change, std modules keyed on every flag they are compiled with, and lint that fails rather than passing files it could not parse. A major, because Toolchain::mingwSysroot is now mingwPrefix and every build directory moves once. Co-Authored-By: Claude Opus 5.5 --- .../net.catcrafts.CrafterBuild.metainfo.xml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml index 95a0a53..d962c8f 100644 --- a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml +++ b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml @@ -25,6 +25,23 @@ + + +

Fixes

+
    +
  • A mingw build no longer sees the Linux system's headers and libraries. Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot, so clang searched /usr/include and /usr/lib for the Windows build as well: libstdc++ found glibc's sys/single_threaded.h and had shared_ptr read __libc_single_threaded, and every program that used one failed to link. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak. A prefix that also holds a Linux system (the host's /usr, or any prefix whose include/ has a C library's features.h) now reaches clang through a view of its mingw-w64 directories in the cache, and compiler messages still name its headers where they are installed. A prefix that holds mingw-w64 alone is the sysroot as before, so Windows libraries installed into its include/ and lib/ are found (#43).
    • +
  • Units that import a std module rebuilt in place, after a libc++ update or a mingw-w64 update under the same compiler, are compiled again. They used to be relinked unchanged.
    • +
  • A CMake external whose configure inputs changed, such as another compiler or --sysroot, is built again from an empty build directory. It used to be reconfigured in place, keeping the objects and check results of the old configure, so a mingw external built by 4.0.0 would have kept failing to link after the fix above.
    • +
  • Configurations that compile the std module differently no longer share one. Two that differed only in -mtune (a project built with CRAFTER_BUILD_MTUNE set, next to its own project.cpp) or only in sysroot took turns rebuilding it, and every module built against it, on every run.
    • +
  • crafter-build lint fails, and says why, when a configuration's compile command cannot be assembled or its std module cannot be built. Its files used to pass every rule that needs an AST as if they were clean.
    • +
+

Breaking

+
    +
  • Toolchain::mingwSysroot is now mingwPrefix, beside mingwPrefixIsSystem, because clang no longer always gets the prefix itself. A project.cpp that passes the mingw sysroot to a command of its own calls MingwSysroot(); passing mingwPrefix brings the bug above back for a prefix like /usr.
    • +
  • The std module's cache directory is now keyed on every flag it is compiled with, and every build directory on the std module it imports, so the first build of each project after updating starts in fresh directories and compiles everything, as after a compiler update. crafter-build clean removes the old build directories; the old std module directories stay in the cache until it is cleared by hand.
    • +
+ +

New

From e2bd0b14ee74c86633647657c8b78f772301b5c3 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sun, 1 Nov 2026 02:55:03 +0100 Subject: [PATCH 02/14] feat(review): set up Crafter.Review's pre-push review Crafter.Review reviews a frozen commit range before it is pushed: the approach, a static proof of the code's bugs, and every claim in the messages and in the documentation the range changes, one claude process per lens and unit, or per whole-range pass, and the owner approves the exact frozen hash before the push. This sets it up for this repository. .review-gates/config.json turns on three lenses in the commits mode, at xhigh: repo-better-solution, repo-bug-prover and repo-facts. The functional lens is off: it exercises a web application through a browser, and this repository's product is a command-line tool. The security lens is off by the owner's ruling: crafter-build stores no secret of its own, and while its Flatpak runs with the user's SSH agent (--socket=ssh-auth) and the whole host filesystem (--filesystem=host), everything it runs is code its user chose to run. The rules CI keeps around its token and signing key go to the approach lens's section instead. gate0 lists the commands run by hand before a freeze. build.sh comes first, because bin/crafter-build compiles project.cpp against the interfaces it was bootstrapped with, and lint comes after the self-host build, because lint's AST rules parse with the module PCMs a build leaves and lint builds only the std PCM itself. .review-gates/lenses/ holds this repository's section of each lens's charter, appended to the text Crafter.Review ships: that AGENTS.md is absent from the frozen tree, the layers and the consumers of interfaces/, the execution contexts a bug can hide in, where libc++ and clang are readable from the review's sandbox, and what a "passes the suite" claim traces to. The /prepush skill that crafter-review init writes stays untracked. It is generated from the installed Crafter.Review and refreshed by every init, so a committed copy would lag the binary it describes. One init per checkout restores it, which AGENTS.template.md now says, together with the rules freeze holds every commit of a range to: signed (requireSigned), with the owner's address as its author and committer (authors), and no AI attribution trailer (aiTrailers forbid). --- .gitignore | 2 + .review-gates/config.json | 28 ++++++++++++ .review-gates/lenses/repo-better-solution.md | 42 ++++++++++++++++++ .review-gates/lenses/repo-bug-prover.md | 46 ++++++++++++++++++++ .review-gates/lenses/repo-facts.md | 18 ++++++++ AGENTS.template.md | 3 ++ 6 files changed, 139 insertions(+) create mode 100644 .review-gates/config.json create mode 100644 .review-gates/lenses/repo-better-solution.md create mode 100644 .review-gates/lenses/repo-bug-prover.md create mode 100644 .review-gates/lenses/repo-facts.md diff --git a/.gitignore b/.gitignore index bb506eb..f92f1f1 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,5 @@ share/crafter-build/ packaging/flatpak/ci-manifest.yaml # written by every build and test run from AGENTS.template.md /AGENTS.md +.review-gates/push/ +.review-gates/trees/ diff --git a/.review-gates/config.json b/.review-gates/config.json new file mode 100644 index 0000000..f452056 --- /dev/null +++ b/.review-gates/config.json @@ -0,0 +1,28 @@ +{ + "base": "origin/master", + "rangeFrom": "upstream", + "authors": [ + "jorijnvdgraaf@catcrafts.net" + ], + "aiTrailers": "forbid", + "requireSigned": true, + "lensSections": ".review-gates/lenses", + "modes": { + "commits": { + "kind": "commits", + "lenses": [ + "repo-better-solution", + "repo-bug-prover", + "repo-facts" + ], + "effort": "xhigh" + } + }, + "gate0": [ + "./build.sh", + "./bin/crafter-build", + "./bin/crafter-build test", + "./bin/crafter-build format --check", + "./bin/crafter-build lint" + ] +} diff --git a/.review-gates/lenses/repo-better-solution.md b/.review-gates/lenses/repo-better-solution.md new file mode 100644 index 0000000..65bd330 --- /dev/null +++ b/.review-gates/lenses/repo-better-solution.md @@ -0,0 +1,42 @@ +## This repository: Crafter.Build + +A C++26 modules build system that builds itself. The stated design is +`README.md` (the manual) and `AGENTS.template.md` (the contributor guide; +`AGENTS.md` is written from it by a crafter-build build or test run, not +by `build.sh`, and is not in the tree). +Its "Design rules" and "Security considerations" are the owner's +decisions: a range that departs from one without saying so is a finding; +whether one should change is the owner's call, not a defect of the range. +Among them, for CI, which holds the workflow's token and the signing key +its runner mounts: no secret is added to the workflow, and the +environment is never echoed in it. + +The layers: the library (`interfaces/`, `implementations/`), which the +executable wraps and every `project.cpp` is compiled against; this +repository's own `project.cpp`; `build.sh`, the bootstrap that builds the +tool without the tool; the Flatpak (`packaging/flatpak/`), which bundles +the LLVM users build with; CI (`.forgejo/workflows/ci.yaml`). A target +difference branches on `cfg.target` at run time, never on the host; an +output path goes through `BinDir()`, `BuildDir()` or `PcmDir()`; whatever +changes what gets compiled or bundled reaches `VariantId()`. + +Search first: the partitions' own helpers (`RunBuildCommand`, +`RunCommandChecked`, `Progress::Diagnostic`, `ApplyStandardArgs`, +`HostTarget`, `GitProject`, `LocalProject`); the Crafter.* libraries the +tree already uses, cloned under `~/.cache/crafter.build/external/`; libc++ +and clang as the host installs them (`/run/host/usr/include/c++/v1`, +`/run/host/usr/lib/clang/`); git, cmake and flatpak-builder. + +Commit shape here: a fix, its test and its README change are separate +commits when each stands on its own, in that order, and a bug fix ships +with a test that fails before it. That split is not a later commit +repairing an earlier one. + +Compatibility: the `project.cpp` of every sibling repository beside this +checkout (`../Crafter.*/project.cpp`) and of every user compiles against +what `interfaces/` exports and is loaded by the binary; the CLI, the +`CRAFTER_BUILD_*` variables, the cache an older version left under +`~/.cache/crafter.build/` and the `bin/` layout CI globs are contracts too. +Users get a push to master through the Flatpak's `beta` branch and a tag +through `stable`. A change an existing `project.cpp` does not survive is a +finding unless the range documents, in the README, what users must change. diff --git a/.review-gates/lenses/repo-bug-prover.md b/.review-gates/lenses/repo-bug-prover.md new file mode 100644 index 0000000..13c9531 --- /dev/null +++ b/.review-gates/lenses/repo-bug-prover.md @@ -0,0 +1,46 @@ +## This repository: Crafter.Build + +The execution contexts to enumerate, beside the charter's: +- `project.cpp` is compiled to a shared object and loaded into the running + crafter-build, resolving `Crafter::` symbols from the executable + (`--export-dynamic`); its static initialisers run at load. Its sibling + projects (`GitProject`, `LocalProject`) load the same way into the same + process, and its `#pragma crafter dependency` targets are built for the + host first, on every load. +- The clang++, clang and llvm-ar the tool spawns, and the git and cmake it + runs, are shell commands (`popen`, `std::system` in `:Platform`), many in + parallel; such a step fails on its exit code, never because it printed. + Running a test or a built artifact is a shell command too (`:Test`, + `:Clang`). A path, file name or flag that reaches one with a space or a + shell metacharacter in it is an input to attack. Other steps run inside + the process, among them glslang's shader compiles, on a thread of their + own per stale shader (`:Shader`), and lint's parse through the libclang + the linter dlopens (`:Lint`). +- Several crafter-build processes share `~/.cache/crafter.build/` at once: + the std and Crafter.Build PCMs per target and march (the + `ConcurrentCacheRace` test), and the external clones with their `.lock` + files. +- Three builds of the tool: the bootstrap from `build.sh` (without + `CRAFTER_BUILD_HAS_ASSET`), the self-hosted binary on the host's LLVM, and + the Flatpak on its own LLVM under `/app`, in a sandbox whose `/usr` is the + Freedesktop SDK's. Their toolchains can be a release apart. +- Targets: the host triple, a sysroot (aarch64, musl), `x86_64-w64-mingw32` + under wine, `wasm32-wasi*` under wasmtime (or wasmer for `-r`), and + browser builds through the runtime in `wasi-runtime/`. +- Incremental builds: a first build, a rebuild after an edit, a rebuild + after a toolchain or flag change and a build after `clean` start from + different states; whatever changes what gets compiled has to reach + `VariantId()`, or a stale artifact is reused. +- Tests: each its own subprocess, in parallel, with a 60 s timeout. A test + that edits or touches files stages a copy under + `fs::temp_directory_path()`; one that only builds its fixture builds it in + place, into the fixture's own `bin/` and `build/` in the checkout. + +Semantics to read on disk rather than recall: libc++ and clang as the host +installs them (`/run/host/usr/include/c++/v1`, +`/run/host/usr/share/libc++/v1/std.cppm`, +`/run/host/usr/lib/clang//include`), the dependencies as cloned +(`~/.cache/crafter.build/external/-/`), the vendored +`lib/DirStackFileIncluder.h`. The Flatpak's own LLVM is not readable from +here: a semantic that may differ between LLVM releases is unverified for +the Flatpak build. diff --git a/.review-gates/lenses/repo-facts.md b/.review-gates/lenses/repo-facts.md new file mode 100644 index 0000000..7129581 --- /dev/null +++ b/.review-gates/lenses/repo-facts.md @@ -0,0 +1,18 @@ +## This repository: Crafter.Build + +- A claim that a tree passes `build.sh`, the self-host build, the test + suite, `format --check` or `lint` traces to the Gate 0 log among the + evidence paths INPUTS names: each command with its exit code and the sha + it ran at. A claim of a run anywhere else (the Flatpak through + `run flatpak`, CI, wine, qemu, node, a browser) traces to a capture of + that run that names the commit, or is unverifiable. +- A test a message names exists as a `cfg.AddTest("")` line in + `project.cpp` and a `tests//main.cpp` whose checks pin what the + message says. +- `AGENTS.md` is generated from `AGENTS.template.md` and the dependencies' + `AGENTS.consumer.md` and is not in the tree: a claim about it is checked + against those sources. +- A `` entry in + `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` is the + release notes users read: every sentence in it is a claim about the code + since the previous tag. diff --git a/AGENTS.template.md b/AGENTS.template.md index 882b551..d375b03 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -19,6 +19,7 @@ crafter-build is a C++26 modules build system. A project is described by a `proj | `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. | | `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. | | `packaging/flatpak/`, `.forgejo/workflows/ci.yaml` | The Flatpak manifest with its metainfo, repo and ref files, and the Forgejo Actions CI that builds and publishes it. | +| `.review-gates/` | The pre-push review's `config.json` and this repository's lens sections (`lenses/`), read by Crafter.Review. Its `push/` and `trees/` are generated and gitignored. | Partitions and what they own: @@ -142,6 +143,8 @@ These were decided deliberately. Do not undo them without the maintainer's say-s - Separate commits for the fix, its test and the README change when each stands on its own. - Update the README in the same PR when user-visible behaviour changes. - `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on PRs, publishes the `beta` branch of the Catcrafts repo on every push to `master`, and the `stable` branch plus a release on an `X.Y.Z` tag. +- Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits, so the merge of a reviewed branch into `master` is pushed without a freeze of its own. +- Every commit of a pushed range is GPG-signed, has `jorijnvdgraaf@catcrafts.net` as its author and committer, and carries no AI attribution trailer in its message, neither `Co-Authored-By:` nor `Assisted-by:`. `freeze` refuses a range with a commit that breaks any of these (`requireSigned`, `authors` and `aiTrailers` in `.review-gates/config.json`). - Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a`, `AGENTS.md` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`. - Keep `CRAFTER_BUILD_MARCH=x86-64-v2` in the manifest for the shipped binary: it has to run on any x86_64 machine. User projects still default to `native`. From 16db37bbe6f02b1121635ccc0dd5f333da46fa04 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sun, 1 Nov 2026 19:20:22 +0100 Subject: [PATCH 03/14] feat(flatpak): ship lldb and the Khronos validation layers Programs that -r and the test runner start run inside the Flatpak's sandbox, and the Sdk it runs on has gdb but neither lldb nor the Khronos validation layer: in the sandbox of beta 5.0.0, a vkCreateInstance that asks for VK_LAYER_KHRONOS_validation returns VK_ERROR_LAYER_NOT_PRESENT. flatpak run clears VK_LAYER_PATH and VK_ADD_LAYER_PATH, so the layer the host has cannot stand in. lldb becomes a project of the llvm module and brings lldb-server, lldb-dap and lldb-mcp. Its Python, libedit, curses, libxml2 and lzma support are set ON instead of being left to detection, so a dependency that goes missing fails the configure step rather than shipping an lldb without it. The Sdk has neither SWIG nor libedit, so both become modules before llvm, SWIG as build-only. The layer is Vulkan-ValidationLayers vulkan-sdk-1.4.357.0, the Vulkan SDK release of the Sdk's loader (libvulkan.so.1.4.357), built from Vulkan-Headers, Vulkan-Utility-Libraries, SPIRV-Headers and SPIRV-Tools at the revisions that release's scripts/known_good.json names. Those four are built into /app at those revisions and removed afterwards. All five come after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. UPDATE_DEPS is turned off: it defaults to ON, which has update_deps.py clone the dependencies, and of all the modules only crafter-build's build-args share the network. --- .../flatpak/net.catcrafts.CrafterBuild.yaml | 146 +++++++++++++++++- 1 file changed, 141 insertions(+), 5 deletions(-) diff --git a/packaging/flatpak/net.catcrafts.CrafterBuild.yaml b/packaging/flatpak/net.catcrafts.CrafterBuild.yaml index 562a3fb..905485f 100644 --- a/packaging/flatpak/net.catcrafts.CrafterBuild.yaml +++ b/packaging/flatpak/net.catcrafts.CrafterBuild.yaml @@ -5,9 +5,14 @@ # Platform: a build tool needs glibc's headers and crt objects, gcc's # libgcc_s, cmake, ninja, make, git and python at run time. Everything # compiler-shaped is ours and lives under /app: clang, lld, libclang, libc++, -# compiler-rt (native and wasm32) and the WASI sysroot, all built from one -# LLVM source tree so the modules crafter-build builds and the compiler that -# consumes them can never drift apart. +# compiler-rt (native and wasm32), lldb and the WASI sysroot, all built from +# one LLVM source tree so the modules crafter-build builds and the compiler +# that consumes them can never drift apart. +# +# Programs that `-r` and the test runner start run in this sandbox, so the +# tools to debug them live here too: lldb (above), the Sdk's gdb, valgrind +# and strace, and the Khronos Vulkan validation layers, GPU-assisted +# validation included, which neither the Sdk nor its GL extensions carry. app-id: net.catcrafts.CrafterBuild runtime: org.freedesktop.Sdk runtime-version: '26.08' @@ -25,13 +30,38 @@ finish-args: - --socket=fallback-x11 - --socket=pulseaudio - --device=all - # gdb and friends from the Sdk + # lldb, and gdb and friends from the Sdk - --allow=devel - --env=PATH=/app/bin:/usr/bin cleanup: - /share/man - /share/doc modules: + - name: swig + # Generates lldb's Python bindings while the llvm module builds; nothing + # of it ships. + cleanup: + - '*' + sources: + - type: archive + url: https://downloads.sourceforge.net/project/swig/swig/swig-4.4.1/swig-4.4.1.tar.gz + sha256: 40162a706c56f7592d08fd52ef5511cb7ac191f3593cf07306a0a554c6281fcf + + - name: libedit + # Line editing, history and completion at lldb's prompt. The Sdk has the + # ncurses it needs but not libedit itself; liblldb links the shared + # library, so that ships and the rest goes. + config-opts: + - --disable-static + cleanup: + - /include + - /lib/pkgconfig + - /lib/*.la + sources: + - type: archive + url: https://thrysoee.dk/editline/libedit-20260512-3.1.tar.gz + sha256: 432d5e7ea8b0116dd39f2eca7bc11d0eed77faa6b77ea526ace89907c23ea4a0 + - name: llvm buildsystem: cmake-ninja builddir: true @@ -50,8 +80,20 @@ modules: config-opts: - -DCMAKE_BUILD_TYPE=Release - -DLLVM_ENABLE_ASSERTIONS=OFF - - -DLLVM_ENABLE_PROJECTS=clang;lld + - -DLLVM_ENABLE_PROJECTS=clang;lld;lldb - -DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind + # lldb with Python scripting (its bindings generated by the swig module + # above, against the Sdk's Python), libedit (the module above), curses, + # libxml2 and lzma, each set ON rather than left to the default of "if + # found", so a dependency gone missing fails the configure step instead + # of shipping an lldb without it. lldb-server, lldb-dap and lldb-mcp + # come with it. + - -DLLDB_ENABLE_PYTHON=ON + - -DLLDB_ENABLE_LIBEDIT=ON + - -DLLDB_ENABLE_CURSES=ON + - -DLLDB_ENABLE_LIBXML2=ON + - -DLLDB_ENABLE_LZMA=ON + - -DLLDB_ENABLE_LUA=OFF - -DLLVM_TARGETS_TO_BUILD=X86;AArch64;WebAssembly # Classic layout: libc++ in lib/ and include/c++/v1, not under a # per-target subdirectory, the same shape as a distro install. @@ -82,6 +124,7 @@ modules: - /include/clang - /include/clang-tidy - /include/lld + - /include/lldb - /lib/cmake - /lib/*.a - /share/opt-viewer @@ -219,6 +262,99 @@ modules: url: https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.8/llvm-project-22.1.8.src.tar.xz sha256: 922f1817a0df7b1489272d18134ee0087a8b068828f87ac63b9861b1a9965888 + # The Khronos validation layers and the four modules they are built from, + # after llvm, so that bumping them reuses the cached LLVM build. All five + # are pinned to the Vulkan SDK release of the Sdk's own Vulkan loader and + # headers (1.4.357), at the dependency revisions that release's + # scripts/known_good.json names. When the runtime moves to a newer loader, + # move the layers to the SDK release of that loader and take the other + # four revisions from its known_good.json again. The four are built into + # /app at those revisions rather than taken from the Sdk, whose versions + # follow the runtime, and none of them ships. + - name: vulkan-headers + buildsystem: cmake-ninja + builddir: true + cleanup: + - '*' + sources: + - type: archive + url: https://github.com/KhronosGroup/Vulkan-Headers/archive/refs/tags/v1.4.357.tar.gz + sha256: 7dc0dbcf1d49dd3d7da3761c251c6097dfbaac475321a4a8a99269d3d5abecdc + + - name: vulkan-utility-libraries + buildsystem: cmake-ninja + builddir: true + config-opts: + - -DCMAKE_BUILD_TYPE=Release + # CMake searches its install prefix after /usr, where the Sdk has + # Vulkan headers of its own. + - -DCMAKE_PREFIX_PATH=/app + cleanup: + - '*' + sources: + - type: archive + url: https://github.com/KhronosGroup/Vulkan-Utility-Libraries/archive/refs/tags/v1.4.357.tar.gz + sha256: e8688013cdc67dad26f15dcece91fa6b80ad595d551fceb56397fc4795efd00a + + - name: spirv-headers + buildsystem: cmake-ninja + builddir: true + cleanup: + - '*' + sources: + - type: archive + url: https://github.com/KhronosGroup/SPIRV-Headers/archive/29981f65241605e08b0ede4cfeb999fe3b723c6a.tar.gz + sha256: 232899f1ad4104fb5bc377b94596c7621575eee62ad9a9e8f929b63a7dd8a7ad + + - name: spirv-tools + # The optimizer GPU-assisted validation instruments shaders with, linked + # into the layer statically. + buildsystem: cmake-ninja + builddir: true + build-options: + no-debuginfo: true + config-opts: + - -DCMAKE_BUILD_TYPE=Release + # The installed SPIRV-Headers: its include/ holds the grammar files + # SPIRV-Tools generates its tables from, as the source tree does. + - -DSPIRV-Headers_SOURCE_DIR=/app + - -DSPIRV_SKIP_EXECUTABLES=ON + - -DSPIRV_WERROR=OFF + cleanup: + - '*' + sources: + - type: archive + url: https://github.com/KhronosGroup/SPIRV-Tools/archive/b707790a898e44038547df54580022fc1cf89c3d.tar.gz + sha256: 05d8af89737bde57571c48dbd36714c9f520a69623e14de72c3be6b600e277d6 + + - name: vulkan-validation-layers + # VK_LAYER_KHRONOS_validation. The loader finds its manifest under + # /app/share/vulkan/explicit_layer.d through XDG_DATA_DIRS, and the + # manifest names the library without a path, which the sandbox's loader + # finds in /app/lib. GPU-AV is a setting of this layer, not a layer of + # its own (VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1, or the layer settings). + buildsystem: cmake-ninja + builddir: true + build-options: + no-debuginfo: true + strip: true + config-opts: + - -DCMAKE_BUILD_TYPE=Release + - -DCMAKE_PREFIX_PATH=/app + # ON by default: it clones every dependency with update_deps.py, which + # the build sandbox has no network for, instead of using the four + # modules above. + - -DUPDATE_DEPS=OFF + - -DBUILD_TESTS=OFF + - -DBUILD_WERROR=OFF + post-install: + - test -f /app/share/vulkan/explicit_layer.d/VkLayer_khronos_validation.json + - test -f /app/lib/libVkLayer_khronos_validation.so + sources: + - type: archive + url: https://github.com/KhronosGroup/Vulkan-ValidationLayers/archive/refs/tags/vulkan-sdk-1.4.357.0.tar.gz + sha256: 73180b11992a3554e97ebc18e6bf2b45ff9790ded9d87fc526c7bab865d1303f + - name: crafter-build buildsystem: simple build-options: From a32b8c898a7da7430d098defd450e32ad36d21d4 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sun, 1 Nov 2026 22:28:08 +0100 Subject: [PATCH 04/14] docs: debugging inside the Flatpak The README gets a Debugging section: the debugging tools the sandbox carries, that a program to debug needs cfg.debug (which --debug sets where the project calls ApplyStandardArgs), how to start the tools with --command, why a debugger in one flatpak run cannot see a program that another one started (each has process IDs of its own), and flatpak enter as the way into a running program's sandbox, with the ptrace opt-in that takes where ptrace_scope is 1. It warns off lldb's gdb-remote from a second sandbox: when a process in lldb's own sandbox has the remote program's process ID, lldb takes that process's executable for the program's. And it shows which variables turn the layer and GPU-AV on from the environment, and that VK_LAYER_PATH and VK_ADD_LAYER_PATH do not reach the sandbox. AGENTS.template.md gets the module layout and the rule that the layer follows the Sdk's Vulkan loader. Its CI section says that a change to a module's sources or build-commands builds it again and every module after it, so that a new version of swig or libedit, which come before LLVM, now rebuilds it too, and loses the cold-run time it gave, written on 2026-10-03, before any of these modules were in the manifest. --- AGENTS.template.md | 3 ++- README.md | 28 +++++++++++++++++++++++++++- 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/AGENTS.template.md b/AGENTS.template.md index d375b03..e19f846 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -160,7 +160,7 @@ These were decided deliberately. Do not undo them without the maintainer's say-s ## CI and publishing -`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, so the LLVM and WASI modules are rebuilt only when their sources change; a cold run takes about 80 minutes on 8 cores, a warm one a fraction of that), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. +`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. Keep the label unique to that runner. Plain `flatpak` also belongs to the organisation-wide runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository and on purpose allows no volumes. Whenever this job lands there, flatpak-builder stops with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of the job log shows which runner took it and lists every volume it ignored. @@ -197,6 +197,7 @@ Local builds are two scripts in `project.cpp`. Run them with the native binary: - The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop. - A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone. - A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules//`, one `libcxx--modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot. +- The Flatpak carries the debugging tools for what it builds, because those programs run in its sandbox: lldb is a project of the `llvm` module, with `LLDB_ENABLE_PYTHON`, `LIBEDIT`, `CURSES`, `LIBXML2` and `LZMA` set ON so that a missing dependency fails the configure step (SWIG through Python, which needs it), and the `swig` and `libedit` modules come before it. The Khronos validation layer is the `vulkan-validation-layers` module plus four build-only modules before it, all after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. The layer follows the Sdk's Vulkan loader: when `runtime-version` moves to a runtime with a newer loader, move the layer to the Vulkan SDK release of that loader and take the other four revisions from that release's `scripts/known_good.json`. - The wasm compile flags live in one place, `WasmBaselineFlags` in `Crafter.Build-Clang.cppm`: the std module and every unit have to agree on them, and `VariantId()` and the std PCM directory hash them, so a change rebuilds instead of mixing old and new BMIs. They enable C++ exceptions, so the link takes the exception-enabled libc++ and libunwind from the sysroot's `lib//eh/` (wasi-sdk 33 and later). The Flatpak's sysroot has them and a distro's may not; where they are missing the wasm tests skip, so run those through the Flatpak. A browser page checks the engine with the `WasmBaselineProbes` named in `variants.json`. - `WasiBrowserRuntimeExit` runs `wasi-runtime/runtime.js` under node and skips where there is none. The Flatpak has no node, so CI never runs it: after touching `runtime.js`, run the suite in the native loop with node on `PATH`. - `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe---*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both. diff --git a/README.md b/README.md index 95d936f..11d4269 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ A C++26 modules build system. Project descriptions are written in C++ with no JS ## Install -crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler: +crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler, and lldb and the Vulkan validation layers to debug the result (see [Debugging](#debugging)): ```bash flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo @@ -364,6 +364,32 @@ Things to know: - Inside the Flatpak, commands run in its sandbox: the host's files are visible, but the tools on `PATH` are the ones the Flatpak ships (`git`, `cmake`, `python3`, `tar`, ...), not everything installed on the host. - A return value outside 0 to 255 is not an exit status and becomes 1. So does a script that throws, with the exception's message printed. +## Debugging + +Programs that `-r` and the test runner start run inside the Flatpak's sandbox, and so do the tools to debug them: lldb, with `lldb-server`, `lldb-dap` and `lldb-mcp`, from the same LLVM build as the compiler; `gdb`, `gdbserver`, `valgrind` and `strace` from the Freedesktop Sdk; and the Khronos Vulkan validation layer. A default build is optimised (`-O3`, and ThinLTO except for wasm) and carries no debug information: set `cfg.debug = true` for a program you want to debug, which compiles its C++ units with `-g` and links it without LTO; its C and CUDA sources (`cfg.cFiles`, `cfg.cuda`) stay at `-O3` without debug information. A `project.cpp` that calls `ApplyStandardArgs` sets it from the `--debug` argument, as in `crafter-build --debug -r`. Start the tools with `--command`: + +```bash +flatpak run --die-with-parent --command=lldb net.catcrafts.CrafterBuild ./bin/myapp-/myapp +flatpak run --die-with-parent --command=valgrind net.catcrafts.CrafterBuild ./bin/myapp-/myapp +``` + +Every `flatpak run` is a sandbox of its own, with process IDs of its own, so a debugger in one cannot see a program that another one, such as `crafter-build -r`, started. Either start the program under the debugger, or run the debugger inside that program's sandbox with `flatpak enter`, giving the instance `flatpak ps` lists for net.catcrafts.CrafterBuild and the debugger's full path: + +```bash +flatpak ps --columns=instance,application +flatpak enter /app/bin/lldb -n myapp +``` + +Where `/proc/sys/kernel/yama/ptrace_scope` is 1, a debugger in the sandbox, where processes hold no capabilities, can attach to a program that is not its descendant only when the program allowed it with `prctl(PR_SET_PTRACER, …)`: with the debugger's process ID, or with `PR_SET_PTRACER_ANY` for any process. Otherwise the attach fails. A program started under the debugger is its descendant. Prefer `flatpak enter` to `lldb-server gdbserver` in one sandbox and lldb's `gdb-remote` from another: when a process in lldb's own sandbox has the remote program's process ID, lldb takes that process's executable for the program's: symbols come out wrong, and a breakpoint in the program's source finds no location. + +The validation layer is enabled the usual ways, by the program itself or from the environment: `flatpak run` passes `VK_INSTANCE_LAYERS`, `VK_LOADER_LAYERS_ENABLE` and the layer's `VK_KHRONOS_VALIDATION_*` settings into the sandbox. `VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1` adds GPU-assisted validation (GPU-AV): + +```bash +VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1 crafter-build -r +``` + +The layer is the Vulkan SDK release of the runtime's own Vulkan loader. `VK_LAYER_PATH` and `VK_ADD_LAYER_PATH` are cleared on the way into the sandbox, so a layer installed on the host is out of reach. + ## Examples [examples/](examples/) contains progressively larger self-contained projects: From c8517a41cf1cb97065b1c3d770de9cccc8e3c7ab Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sun, 1 Nov 2026 22:28:08 +0100 Subject: [PATCH 05/14] docs(readme): --die-with-parent in the install alias flatpak run becomes bubblewrap, and without --die-with-parent the sandbox outlives it. With crafter-build started through the alias in an interactive bash or zsh on a terminal, SIGKILL to its flatpak run process alone left a -r program running and a build writing into bin/ and build/. With the flag, both ended with it. bubblewrap kills the sandbox when flatpak run or its parent dies, and for crafter-build typed at the prompt, in the foreground or with &, that parent is the shell: with the flag, the sandbox also ended when the shell was SIGKILLed, or exited after a disown. In a (crafter-build -r &) subshell, flatpak run outlives the subshell and is adopted by another process, so its sandbox outlived the shell with the flag as well. Ctrl-C ended the program either way. The README's alias gets the flag, and the paragraph after it says what that changes: a program started through the alias, as crafter-build ... or crafter-build ... & at a bash or zsh prompt, ends with its shell, even after a disown. --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 11d4269..2f71543 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,11 @@ crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WAS ```bash flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo flatpak install catcrafts net.catcrafts.CrafterBuild -alias crafter-build='flatpak run net.catcrafts.CrafterBuild' +alias crafter-build='flatpak run --die-with-parent net.catcrafts.CrafterBuild' ``` +`--die-with-parent` ends the sandbox with the `flatpak run` that started it. Without the flag, killing only the `flatpak run` process leaves the sandbox running: a build carries on writing into `bin/` and `build/`, and a program started with `-r` keeps running. With it, bubblewrap kills everything in the sandbox with SIGKILL when the `flatpak run` process dies or when that process's parent does. For `crafter-build …` or `crafter-build … &` typed at a bash or zsh prompt, that parent is the shell, so the sandbox ends with the shell, even after a `disown`. Ctrl-C reaches the program either way. + Pre-release builds of `master` are the `beta` branch (`flatpak install catcrafts net.catcrafts.CrafterBuild//beta`), and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host. ## Writing a project.cpp From 40dcb18a1ead083ec629ce04b28d9d55db50a5e1 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sun, 1 Nov 2026 22:28:08 +0100 Subject: [PATCH 06/14] feat(flatpak): ship the SPIRV-Tools programs Crafter.Graphics' PushConstantRewrite and UIFusedShader tests require tool:spirv-val, and the Sdk has no spirv-val, so inside the Flatpak they are skipped. The spirv-tools module the validation layer needs built only the libraries, and removed them afterwards. It now builds the programs as well, spirv-val, spirv-opt, spirv-dis and the rest, which link SPIRV-Tools statically, and ships them. The headers, the static libraries, the CMake and pkg-config files and libSPIRV-Tools-shared.so, which nothing here links, are still removed, and a post-install step runs spirv-val --version. --- AGENTS.template.md | 2 +- README.md | 2 +- .../flatpak/net.catcrafts.CrafterBuild.yaml | 25 ++++++++++++++----- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/AGENTS.template.md b/AGENTS.template.md index e19f846..af78d5b 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -197,7 +197,7 @@ Local builds are two scripts in `project.cpp`. Run them with the native binary: - The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop. - A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone. - A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules//`, one `libcxx--modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot. -- The Flatpak carries the debugging tools for what it builds, because those programs run in its sandbox: lldb is a project of the `llvm` module, with `LLDB_ENABLE_PYTHON`, `LIBEDIT`, `CURSES`, `LIBXML2` and `LZMA` set ON so that a missing dependency fails the configure step (SWIG through Python, which needs it), and the `swig` and `libedit` modules come before it. The Khronos validation layer is the `vulkan-validation-layers` module plus four build-only modules before it, all after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. The layer follows the Sdk's Vulkan loader: when `runtime-version` moves to a runtime with a newer loader, move the layer to the Vulkan SDK release of that loader and take the other four revisions from that release's `scripts/known_good.json`. +- The Flatpak carries the debugging tools for what it builds, because those programs run in its sandbox: lldb is a project of the `llvm` module, with `LLDB_ENABLE_PYTHON`, `LIBEDIT`, `CURSES`, `LIBXML2` and `LZMA` set ON so that a missing dependency fails the configure step (SWIG through Python, which needs it), and the `swig` and `libedit` modules come before it. The Khronos validation layer is the `vulkan-validation-layers` module plus four modules before it, build-only except for SPIRV-Tools' programs (`spirv-val` and the rest), all after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. The layer follows the Sdk's Vulkan loader: when `runtime-version` moves to a runtime with a newer loader, move the layer to the Vulkan SDK release of that loader and take the other four revisions from that release's `scripts/known_good.json`. - The wasm compile flags live in one place, `WasmBaselineFlags` in `Crafter.Build-Clang.cppm`: the std module and every unit have to agree on them, and `VariantId()` and the std PCM directory hash them, so a change rebuilds instead of mixing old and new BMIs. They enable C++ exceptions, so the link takes the exception-enabled libc++ and libunwind from the sysroot's `lib//eh/` (wasi-sdk 33 and later). The Flatpak's sysroot has them and a distro's may not; where they are missing the wasm tests skip, so run those through the Flatpak. A browser page checks the engine with the `WasmBaselineProbes` named in `variants.json`. - `WasiBrowserRuntimeExit` runs `wasi-runtime/runtime.js` under node and skips where there is none. The Flatpak has no node, so CI never runs it: after touching `runtime.js`, run the suite in the native loop with node on `PATH`. - `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe---*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both. diff --git a/README.md b/README.md index 2f71543..f9238e0 100644 --- a/README.md +++ b/README.md @@ -368,7 +368,7 @@ Things to know: ## Debugging -Programs that `-r` and the test runner start run inside the Flatpak's sandbox, and so do the tools to debug them: lldb, with `lldb-server`, `lldb-dap` and `lldb-mcp`, from the same LLVM build as the compiler; `gdb`, `gdbserver`, `valgrind` and `strace` from the Freedesktop Sdk; and the Khronos Vulkan validation layer. A default build is optimised (`-O3`, and ThinLTO except for wasm) and carries no debug information: set `cfg.debug = true` for a program you want to debug, which compiles its C++ units with `-g` and links it without LTO; its C and CUDA sources (`cfg.cFiles`, `cfg.cuda`) stay at `-O3` without debug information. A `project.cpp` that calls `ApplyStandardArgs` sets it from the `--debug` argument, as in `crafter-build --debug -r`. Start the tools with `--command`: +Programs that `-r` and the test runner start run inside the Flatpak's sandbox, and so do the tools to debug them: lldb, with `lldb-server`, `lldb-dap` and `lldb-mcp`, from the same LLVM build as the compiler; `gdb`, `gdbserver`, `valgrind` and `strace` from the Freedesktop Sdk; the Khronos Vulkan validation layer; and the SPIRV-Tools programs (`spirv-val`, `spirv-opt`, `spirv-dis`, ...). A default build is optimised (`-O3`, and ThinLTO except for wasm) and carries no debug information: set `cfg.debug = true` for a program you want to debug, which compiles its C++ units with `-g` and links it without LTO; its C and CUDA sources (`cfg.cFiles`, `cfg.cuda`) stay at `-O3` without debug information. A `project.cpp` that calls `ApplyStandardArgs` sets it from the `--debug` argument, as in `crafter-build --debug -r`. Start the tools with `--command`: ```bash flatpak run --die-with-parent --command=lldb net.catcrafts.CrafterBuild ./bin/myapp-/myapp diff --git a/packaging/flatpak/net.catcrafts.CrafterBuild.yaml b/packaging/flatpak/net.catcrafts.CrafterBuild.yaml index 905485f..fa926fb 100644 --- a/packaging/flatpak/net.catcrafts.CrafterBuild.yaml +++ b/packaging/flatpak/net.catcrafts.CrafterBuild.yaml @@ -11,8 +11,9 @@ # # Programs that `-r` and the test runner start run in this sandbox, so the # tools to debug them live here too: lldb (above), the Sdk's gdb, valgrind -# and strace, and the Khronos Vulkan validation layers, GPU-assisted -# validation included, which neither the Sdk nor its GL extensions carry. +# and strace, the Khronos Vulkan validation layers, GPU-assisted validation +# included, which neither the Sdk nor its GL extensions carry, and the +# SPIRV-Tools programs such as spirv-val. app-id: net.catcrafts.CrafterBuild runtime: org.freedesktop.Sdk runtime-version: '26.08' @@ -270,7 +271,7 @@ modules: # move the layers to the SDK release of that loader and take the other # four revisions from its known_good.json again. The four are built into # /app at those revisions rather than taken from the Sdk, whose versions - # follow the runtime, and none of them ships. + # follow the runtime, and of them only SPIRV-Tools' programs ship. - name: vulkan-headers buildsystem: cmake-ninja builddir: true @@ -308,20 +309,32 @@ modules: - name: spirv-tools # The optimizer GPU-assisted validation instruments shaders with, linked - # into the layer statically. + # into the layer statically, and SPIRV-Tools' programs (spirv-val, + # spirv-opt, spirv-dis and the rest), which ship: a program or test that + # checks its SPIR-V with spirv-val, as Crafter.Graphics' do, finds it in + # the sandbox. The libraries and headers stay build-only. buildsystem: cmake-ninja builddir: true build-options: no-debuginfo: true + strip: true config-opts: - -DCMAKE_BUILD_TYPE=Release # The installed SPIRV-Headers: its include/ holds the grammar files # SPIRV-Tools generates its tables from, as the source tree does. - -DSPIRV-Headers_SOURCE_DIR=/app - - -DSPIRV_SKIP_EXECUTABLES=ON + - -DSPIRV_SKIP_TESTS=ON - -DSPIRV_WERROR=OFF cleanup: - - '*' + - /include/spirv-tools + - /lib/libSPIRV-Tools*.a + # Always built besides the static libraries, and linked by nothing + # here: the programs and the layer take SPIRV-Tools statically. + - /lib/libSPIRV-Tools-shared.so + - /lib/cmake/SPIRV-Tools* + - /lib/pkgconfig/SPIRV-Tools* + post-install: + - spirv-val --version sources: - type: archive url: https://github.com/KhronosGroup/SPIRV-Tools/archive/b707790a898e44038547df54580022fc1cf89c3d.tar.gz From 03951ff45833e84d84b7e48b783f4dcf4bccba76 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sun, 1 Nov 2026 22:28:08 +0100 Subject: [PATCH 07/14] docs: update the first local Flatpak build's time and size AGENTS.template.md and project.cpp gave the first run flatpak as about 25 minutes on a 56-core machine and its state as about 2 GB, figures written on 2026-10-06, before lldb and the Vulkan and SPIR-V modules were in the manifest. A first run of this commit's manifest, in a fresh worktree, took 30 min 36 s with 56 CPU threads, 14 min of it for LLVM with lldb, and left build/flatpak at 4.1 GB, 3.3 GB of it state/. The figures become those, and the line on later runs says that they rebuild only the crafter-build module while the manifest is unchanged. --- AGENTS.template.md | 4 ++-- project.cpp | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/AGENTS.template.md b/AGENTS.template.md index af78d5b..7dc020f 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -51,7 +51,7 @@ Three things to know about this loop: - Changes under `implementations/` are picked up by the self-host build and by the tests, which link the local library. Changes under `interfaces/` are not: the harness compiles against the copies in `share/crafter-build/` and the PCM cache built from them, so a new or changed declaration is invisible until you run `./build.sh` again. The symptom is `use of undeclared identifier` for the symbol you just added. A changed default member initialiser has no symptom at all: `project.cpp` compiles against the old one and quietly gets the old default, as `project.cpp`'s sibling loads do with a harness that predates `GitProjectSpec::args = ProjectArgs()`. - A new exported symbol that a `project.cpp` reaches on its default path also needs `./build.sh`: the old binary does not export it, and `LoadProject` fails with an undefined symbol. -`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. That takes `bin/crafter-build` itself with it, and `build/flatpak`, the state of the local Flatpak build (see "CI and publishing"): about 2 GB, and the next `run flatpak` rebuilds LLVM. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`. +`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. That takes `bin/crafter-build` itself with it, and `build/flatpak`, the state of the local Flatpak build (see "CI and publishing"): about 4 GB after a first run, and the next `run flatpak` rebuilds LLVM. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`. ## Build, test and style commands @@ -182,7 +182,7 @@ Local builds are two scripts in `project.cpp`. Run them with the native binary: ./bin/crafter-build run install # install that build as branch `local` and make it current ``` -`run flatpak` needs `flatpak-builder` or the `org.flatpak.Builder` app, and the Sdk (`flatpak install flathub org.freedesktop.Sdk//26.08`). It builds the working tree, uncommitted changes included, runs the whole suite inside the sandbox and reports version `0.0.0-dev`. Its state is `build/flatpak/` (`state/`, `repo/`, `build-dir/`). The first run builds LLVM and the WASI sysroot, about 25 minutes on a 56-core machine; later runs rebuild only the crafter-build module, a few minutes, because flatpak-builder never caches a `type: dir` source. The checkout must not be under `/tmp`, which `org.flatpak.Builder` cannot see. Use it for anything toolchain-sensitive: the native loop compiles with the distro's clang and the Flatpak with its bundled LLVM, and the two can be a major version apart. +`run flatpak` needs `flatpak-builder` or the `org.flatpak.Builder` app, and the Sdk (`flatpak install flathub org.freedesktop.Sdk//26.08`). It builds the working tree, uncommitted changes included, runs the whole suite inside the sandbox and reports version `0.0.0-dev`. Its state is `build/flatpak/` (`state/`, `repo/`, `build-dir/`). The first run builds every module, LLVM with lldb, the WASI sysroot and the Vulkan validation layer among them, in about 30 minutes with 56 CPU threads; later runs rebuild only the crafter-build module while the manifest is unchanged, a few minutes, because flatpak-builder never caches a `type: dir` source. The checkout must not be under `/tmp`, which `org.flatpak.Builder` cannot see. Use it for anything toolchain-sensitive: the native loop compiles with the distro's clang and the Flatpak with its bundled LLVM, and the two can be a major version apart. `run install` does not build; it installs what `run flatpak` last exported. It points the user remote `crafter-build-local` at `build/flatpak/repo`, installs `net.catcrafts.CrafterBuild//local` from it next to whichever published branch is installed, and makes it the branch `flatpak run net.catcrafts.CrafterBuild` starts. `flatpak make-current --user net.catcrafts.CrafterBuild ` switches back. Before `crafter-build clean`, or before deleting the checkout, run `flatpak uninstall --user net.catcrafts.CrafterBuild//local` and `flatpak remote-delete --user crafter-build-local`; otherwise the remote is left pointing at a repo that is gone. diff --git a/project.cpp b/project.cpp index 38db3a4..9fc8e43 100644 --- a/project.cpp +++ b/project.cpp @@ -16,7 +16,8 @@ namespace { struct FlatpakPaths { fs::path manifest; // flatpak-builder's downloads, ccache and per-module cache. The - // LLVM build lives here: about 2 GB, and 25 minutes to rebuild. + // LLVM build lives here: about 3 GB in all after a first run, + // which takes about 30 minutes with 56 CPU threads. fs::path state; // The OSTree repo the build is exported to; `install` reads it. fs::path repo; From 00f07dae88e393a7d2fcfa9d0b17ad6c9c39022e Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sun, 1 Nov 2026 23:20:10 +0100 Subject: [PATCH 08/14] docs: push reviewed ranges straight to master AGENTS.template.md had every change go through a pull request against master. The owner has made pushing straight to master the default. The commit and push guidelines say so: an approved range is pushed to master as a fast-forward, with no merge commit, and a pull request is opened only when the owner asks for one; when master has moved since the freeze, the range is rebased, frozen and reviewed again. What follows from that: the README changes in the same push; without a pull request CI does not run on a range before it reaches master unless someone starts it by hand, so the review's gate0 is the check before the push; and the freeze exception for a merge commit applies only to a pull request. Design rule 8, the testing note and the release steps lose their pull-request wording, and the testing note no longer speaks of a commented-out lint step: ci.yaml has no lint step at all. --- AGENTS.template.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/AGENTS.template.md b/AGENTS.template.md index 7dc020f..68b35bd 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -69,7 +69,7 @@ Three things to know about this loop: ./bin/crafter-build clean ``` -Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` on every PR; its lint step is currently commented out, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/.log`. +Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` on every push to `master` and on pull requests, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/.log`. ## Testing instructions @@ -132,18 +132,18 @@ These were decided deliberately. Do not undo them without the maintainer's say-s 5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point. 6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch. 7. **The house-style rules stay in their own repo.** They are not library API of crafter-build and are not forced on downstream projects. -8. **Fix warts inline; do not stack workarounds.** The previous version of this tool died of accumulated hacks. If a change needs a shortcut, say so in the PR rather than hiding it. +8. **Fix warts inline; do not stack workarounds.** The previous version of this tool died of accumulated hacks. If a change needs a shortcut, say so in its commit message rather than hiding it. 9. **`project.cpp` keeps the library/executable split.** `crafter.build-lib` owns the sources, dependencies and glslang; `crafter.build-exe` is `main.cpp` plus link flags. 10. **Host-tooling builds** (the target of a `#pragma crafter dependency`) get `-fPIC` and the host triple scoped to that build. Keep it scoped. -## Commit and PR guidelines +## Commit and push guidelines -- Branch from `master` and open the PR against `master` on the project's Forgejo at `forgejo.catcrafts.net/Catcrafts/Crafter.Build`. Issue work uses `claude/issue-` branch names. Pushes go over SSH; HTTPS push is disabled on the server. +- Branch from `master`; issue work uses `claude/issue-` branch names. Once the owner approves its frozen hash, a range goes straight to `master` on the project's Forgejo at `forgejo.catcrafts.net/Catcrafts/Crafter.Build`, as a fast-forward push with no merge commit and no pull request; open a pull request against `master` only when the owner asks for one. If `master` has moved since the freeze, rebase onto it, then freeze and review the rebased range again. Pushes go over SSH; HTTPS push is disabled on the server. - Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice. - Separate commits for the fix, its test and the README change when each stands on its own. -- Update the README in the same PR when user-visible behaviour changes. -- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on PRs, publishes the `beta` branch of the Catcrafts repo on every push to `master`, and the `stable` branch plus a release on an `X.Y.Z` tag. -- Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits, so the merge of a reviewed branch into `master` is pushed without a freeze of its own. +- Update the README in the same push when user-visible behaviour changes. +- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on every push to `master` and on pull requests; a push to `master` then publishes the `beta` branch of the Catcrafts repo, and an `X.Y.Z` tag the `stable` branch plus a release. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push. +- Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits: a range reaches `master` as a fast-forward, and when the owner asks for a pull request instead, the merge of the reviewed branch into `master` is pushed without a freeze of its own. - Every commit of a pushed range is GPG-signed, has `jorijnvdgraaf@catcrafts.net` as its author and committer, and carries no AI attribution trailer in its message, neither `Co-Authored-By:` nor `Assisted-by:`. `freeze` refuses a range with a commit that breaks any of these (`requireSigned`, `authors` and `aiTrailers` in `.review-gates/config.json`). - Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a`, `AGENTS.md` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`. - Keep `CRAFTER_BUILD_MARCH=x86-64-v2` in the manifest for the shipped binary: it has to run on any x86_64 machine. User projects still default to `native`. @@ -174,7 +174,7 @@ gpg --batch --pinentry-mode loopback --passphrase '' --export-secret-keys --armo printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id ``` -To release: add a `` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), merge, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry. +To release: add a `` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), push it to `master`, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry. Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive. ```bash From a1bc96a323818d2e4eb451dbdf9eb377f5cfbc78 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 14:10:27 +0100 Subject: [PATCH 09/14] fix(review): name each lens section, as crafter-review 1.0 requires crafter-review 1.0.1, the released build installed on this machine, and the newer build installed beside it as //local refuse the config key lensSections: a lens's repository section is named per lens, as lenses..section, instead of being found by listing a directory. With the key in place `crafter-review config` reports it as a problem, and freeze and run stop at "preflight: config: unknown key lensSections". The three files in .review-gates/lenses/ are named under lenses, one entry each, the migration the Crafter.Review session gave; with them `crafter-review config` reports no problem. --- .review-gates/config.json | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/.review-gates/config.json b/.review-gates/config.json index f452056..115ee6c 100644 --- a/.review-gates/config.json +++ b/.review-gates/config.json @@ -6,7 +6,17 @@ ], "aiTrailers": "forbid", "requireSigned": true, - "lensSections": ".review-gates/lenses", + "lenses": { + "repo-better-solution": { + "section": ".review-gates/lenses/repo-better-solution.md" + }, + "repo-bug-prover": { + "section": ".review-gates/lenses/repo-bug-prover.md" + }, + "repo-facts": { + "section": ".review-gates/lenses/repo-facts.md" + } + }, "modes": { "commits": { "kind": "commits", From 2447d6ec4f6f8111fe1310967dff50031c9d0d62 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:39:01 +0100 Subject: [PATCH 10/14] fix(ci): attach the flatpakref with the repo's key to releases The Bundle step copied packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref into dist/ as the checkout has it, with GPGKey=GPG_KEY_BASE64_PLACEHOLDER, so all six releases, 2.0.0 to 5.0.0, attach a .flatpakref that does not install: `flatpak install --from` it stops with "GPG: Unable to export keys: GPGME: No data" (Flatpak 1.18.4) before it adds the remote. The copy the publish step writes to the web root has the key filled in, and installing from it goes ahead. The key is known only from the publish step on, so the release's copy, and SHA256SUMS over it, are made there from the web root's file instead of in the Bundle step. --- .forgejo/workflows/ci.yaml | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/.forgejo/workflows/ci.yaml b/.forgejo/workflows/ci.yaml index 73f1449..086d3dc 100644 --- a/.forgejo/workflows/ci.yaml +++ b/.forgejo/workflows/ci.yaml @@ -120,8 +120,6 @@ jobs: run: | mkdir -p dist flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}" - cp packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref dist/ - (cd dist && sha256sum * > SHA256SUMS) ls -la dist - name: Upload bundle @@ -160,6 +158,12 @@ jobs: key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0) sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" + # The release attaches this .flatpakref, not the one in the + # checkout: that one has a placeholder where the key goes, and + # flatpak cannot install from it. + cp "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" dist/ + (cd dist && sha256sum * > SHA256SUMS) + ls -la dist - name: Rolling 'latest' release if: github.event_name != 'pull_request' && github.ref_type != 'tag' From 2babe94a6aa004c7b29092a4db2a16ab019cb991 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 12:04:06 +0100 Subject: [PATCH 11/14] fix(ci): publish under the lock Crafter.Review's workflow takes Crafter.Review's workflow publishes its Flatpak into the same repo, the web root's repo/, and holds an flock on .publish.lock in the web root while it commits and updates the summary. Its comment says that this protects both only once Crafter.Build's workflow takes the lock too, and ours never did, so a publish of each could interleave: a build-commit-from and a build-update-repo --generate-static-deltas --prune of each workflow on one repo at the same time. The Publish step now holds the same lock from its commit to the ref files it writes into the web root, and the job installs util-linux for flock, as Crafter.Review's jobs do. AGENTS.template.md says that the repo is shared and the lock taken. Neither it nor the workflow's header comment says any longer that the web root and the signing key are this runner's alone, or that the organisation-wide `flatpak` runner mounts none of them. That runner, which serves every Catcrafts repository and refused all of this job's volumes when it last took it, in October, now mounts the web root and the key for Crafter.Review's publish job, and the volumes a runner allows are open to every job it runs. The owner has ruled the sharing intended. The header also stops saying that a job of ours that lands there fails before building: that held while the runner refused every volume, and AGENTS.template.md keeps it as the October history. --- .forgejo/workflows/ci.yaml | 16 ++++++++++++---- AGENTS.template.md | 4 ++-- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/.forgejo/workflows/ci.yaml b/.forgejo/workflows/ci.yaml index 086d3dc..644009f 100644 --- a/.forgejo/workflows/ci.yaml +++ b/.forgejo/workflows/ci.yaml @@ -11,9 +11,10 @@ name: Flatpak # flatpak-builder state (ccache and the LLVM module, so only crafter-build # itself rebuilds on a normal run), the published repo's web root, and the # repo signing key, which lives on the runner host and never in the forge. -# No other runner carries that label. Plain `flatpak` also matches the -# organisation's runner, which mounts none of those volumes; a job it picks -# up fails before building anything. +# No other runner carries that label. Plain `flatpak` is the label of the +# organisation's runner, which serves every Catcrafts repository and also +# mounts the web root and the key, for Crafter.Review's publish job, so any +# job it runs can mount them (AGENTS.template.md, CI and publishing). on: pull_request: branches: [master] @@ -45,11 +46,12 @@ jobs: -v /etc/crafter-flatpak:/run/crafter-flatpak:ro steps: - name: Install tools + # util-linux: flock, for the Publish step. run: | pacman-key --init pacman-key --populate archlinux pacman -Sy --noconfirm --needed archlinux-keyring - pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python + pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python util-linux git config --global --add safe.directory '*' - name: Sdk @@ -144,6 +146,11 @@ jobs: export GNUPGHOME=$(mktemp -d) gpg --batch --quiet --import /run/crafter-flatpak/signing-key.asc FLATPAK_GPG_KEY_ID=$(cat /run/crafter-flatpak/key-id) + # Crafter.Review's workflow publishes into the same repo and holds + # this lock while it does, so a commit and the summary update of + # one workflow never interleave with the other's. + exec 9> "$WEB_DIR/.publish.lock" + flock 9 # The live repo is the canonical copy; builds are promoted into it, # never built into it. # Keyed on the repo's config file, not the directory: the web root @@ -158,6 +165,7 @@ jobs: key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0) sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" + flock -u 9 # The release attaches this .flatpakref, not the one in the # checkout: that one has a placeholder where the key goes, and # flatpak cannot install from it. diff --git a/AGENTS.template.md b/AGENTS.template.md index 68b35bd..1db87b0 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -160,9 +160,9 @@ These were decided deliberately. Do not undo them without the maintainer's say-s ## CI and publishing -`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. +`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and no other repository's job runs on it. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The web root and the key are not this runner's alone: the organisation-wide `flatpak` runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository, mounts both for Crafter.Review's publish job, and the volumes a runner allows (`valid_volumes` in its config) are open to every job it runs, so a job of any Catcrafts repository that runs there can mount them as well. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. Crafter.Review's workflow publishes into the same repo, and each workflow holds an flock on `/var/www/flatpak/.publish.lock` while it commits and updates the summary. -Keep the label unique to that runner. Plain `flatpak` also belongs to the organisation-wide runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository and on purpose allows no volumes. Whenever this job lands there, flatpak-builder stops with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of the job log shows which runner took it and lists every volume it ignored. +Keep the label unique to that runner. Plain `flatpak` belongs to the organisation-wide runner. When this job still asked for `flatpak` and that runner took it (runs #368 and #370, in October 2026), the runner allowed none of the job's volumes, and flatpak-builder stopped with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of such a job's log shows which runner took it and lists every volume it ignored. The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing: From be81fc098e6d02eb528ef35654318284029bdec1 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:42:58 +0100 Subject: [PATCH 12/14] feat(ci): drop beta and latest, release signed tags on master A push to master published the `beta` branch and moved the rolling `latest` tag and its "Latest master build" prerelease, and a pushed X.Y.Z tag made the release and published `stable`. The owner wants a release for every push to master, made by the signed tag that goes with it, and nothing for a push without one. So `beta`, `latest` and the branch trigger are gone: the workflow runs for pushed tags, pull requests and manual runs only, and the `latest` tag and its prerelease have been deleted from the forge. `beta` gets no more builds, and its installations would not hear of it: the release notes ship in stable builds only. The Publish step marks beta's last build end-of-life, once, with a reason that names stable, and the next `flatpak update` of a beta installation prints it. The reason also names the branch, because a non-interactive update prints only the app's name with it. Before building, a tag run now checks the tag: it has to be annotated and signed by the owner as Forgejo verifies it, its commit has to be on master, its version has to be X.Y.Z and newer than every release the forge has published, its own included (a later tag whose run still queues does not count), and the metainfo's newest entry has to be that version. The tag run checked only the last of these. It reads the tag from the forge's API, because the checkout fetches a pushed tag again as a plain ref to its commit, so the job's clone never has the annotated tag, and Forgejo verifies a signed tag's own signature but falls back to the commit's for an unsigned one, so the step also requires that the verdict covers the tag's payload. The metainfo's entry is read once, as the first element: the Version step takes its version and renders its notes into release-notes.md with the renderer the Release notes step had, and stops the run when they come out empty, so a comment or another attribute order cannot give a release one entry's version and another's notes. It does so on every run, pull requests and manual runs included, so a broken entry shows before a release. Release runs share a concurrency group, so they queue one behind another; a pull request still cancels its own outdated run. A failed release run can be re-run while no newer release exists, since its tag is still on its commit; a release that an attempt left half made has to be deleted first. The release step runs without `override`, which would delete a tag that has moved off the run's commit since the Version step and tag the old commit instead. A manual run no longer publishes anything; it used to publish `beta` and move `latest` from whichever branch it was started on. No step pushes to git, so the checkout keeps no credentials. git describe in build.sh, project.cpp and the workflow has no `latest` tag left to exclude. --- .forgejo/workflows/ci.yaml | 259 ++++++++++++++++++++++++------------- build.sh | 7 +- project.cpp | 8 +- 3 files changed, 173 insertions(+), 101 deletions(-) diff --git a/.forgejo/workflows/ci.yaml b/.forgejo/workflows/ci.yaml index 644009f..dcdff57 100644 --- a/.forgejo/workflows/ci.yaml +++ b/.forgejo/workflows/ci.yaml @@ -1,10 +1,15 @@ name: Flatpak -# One job, three outcomes: -# pull request -> build the Flatpak and run the suite inside it -# push master -> also publish the result as the `beta` branch of the -# Catcrafts repo and refresh the rolling `latest` release -# tag X.Y.Z -> publish as `stable` and create release X.Y.Z +# One job, two outcomes: +# pull request, manual run -> build the Flatpak and run the suite inside it +# pushed tag X.Y.Z -> also release the build: publish it as the +# `stable` branch of the Catcrafts repo and +# create release X.Y.Z +# +# A release is a tag that the owner makes and signs locally and pushes with +# the master it releases; a push to master without one runs nothing. The tag +# names the version, and the metainfo's newest entry has to be that +# version: its description is the release notes. # # The `crafter-flatpak` runner label is the privileged runner on the Hetzner # box (bubblewrap needs it) with four volumes allowed: the Sdk install, the @@ -19,20 +24,29 @@ on: pull_request: branches: [master] push: - branches: [master] # Bare semver tags. Spelled with globs so it means the same whether the - # forge reads it as a workflow pattern or as a plain glob; `latest`, the - # rolling tag this workflow moves itself, does not start with a digit. + # forge reads it as a workflow pattern or as a plain glob. tags: ['[0-9]*.[0-9]*.[0-9]*'] workflow_dispatch: +# Pushes are tags only, so every release run is in one group, and a release +# starts only once the one before it has finished. A pull request's group +# also holds its number, which only a pull request's event carries, so a +# new push to it cancels its own outdated run and no other. +concurrency: + group: flatpak-${{ github.event_name }}-${{ github.event.pull_request.number }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + env: APP_ID: net.catcrafts.CrafterBuild + BRANCH: stable MANIFEST: packaging/flatpak/net.catcrafts.CrafterBuild.yaml CI_MANIFEST: packaging/flatpak/ci-manifest.yaml RUNTIME: org.freedesktop.Sdk//26.08 REPO_DIR: /srv/flatpak/repo WEB_DIR: /srv/flatpak + # A release tag carries a signature Forgejo verifies for this address. + RELEASE_SIGNER: jorijnvdgraaf@catcrafts.net jobs: flatpak: @@ -62,40 +76,137 @@ jobs: flatpak update -y --noninteractive - name: Checkout + # Every branch and tag, for git describe and the release checks; no + # step pushes, so the token is not kept in the checkout. uses: actions/checkout@v4 with: fetch-depth: 0 - persist-credentials: true + persist-credentials: false - - name: Version and channel + - name: Version id: meta run: | set -eu - if [ "${GITHUB_REF_TYPE}" = tag ]; then - version="$GITHUB_REF_NAME" - branch=stable - # The metainfo's newest release is the release notes and must be - # the version being tagged. - top=$(grep -m1 -oP '&2 + # The metainfo's newest entry is its first element, the + # one parse that gives both its version and the release notes, + # before anything is built: a comment or another attribute order + # would mislead a text search, and an entry whose notes render + # empty stops the run here. The notes go to release-notes.md, for + # the release page, as Markdown. AppStream has no headings, so a + # paragraph that introduces a list is one:

New

    ...
+ # comes out as "## New" over its items, the shape the notes take + # since 3.0.0. A paragraph on its own stays a paragraph, as the + # older entries are written. and get their Markdown + # spelling; a code span also keeps text such as lib//eh + # from being read as an HTML tag by the forge. + newest=$(python3 - <<'EOF' + import sys + import xml.etree.ElementTree as ET + + release = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release") + desc = None if release is None else release.find("description") + + def text(el): + parts = [el.text or ""] + for child in el: + inner = text(child) + parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner)) + parts.append(child.tail or "") + return " ".join("".join(parts).split()) + + blocks = [] if desc is None else list(desc) + out = [] + for i, el in enumerate(blocks): + heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol") + if el.tag == "p": + out.append(f"## {text(el)}" if heads_list else text(el)) + elif el.tag == "ul": + out.append("\n".join(f"- {text(li)}" for li in el)) + elif el.tag == "ol": + out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1))) + notes = "\n\n".join(out) + if not notes.strip(): + sys.exit("the metainfo needs a at the top of whose renders release notes") + with open("release-notes.md", "w") as f: + print(notes, file=f) + print(release.get("version", "")) + EOF + ) + if [ "$GITHUB_EVENT_NAME" = push ]; then + # A release: the only pushes this workflow runs for are tags. + # The tag is the version; it has to be annotated, signed by the + # owner as Forgejo verifies it against the keys its users added, + # on master, newer than every published release, and the version + # of the metainfo's newest entry. The tag itself is read from the + # forge: the checkout fetches a pushed tag again as a plain ref + # to its commit, so the local one is never the annotated tag. + version=$GITHUB_REF_NAME + if ! printf '%s\n' "$version" | grep -qxE '[0-9]+\.[0-9]+\.[0-9]+'; then + echo "the tag $version is not a version of the form X.Y.Z" >&2 + exit 1 + fi + api="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY" + tag=$(curl -sSf "$api/tags/$version") + object=$(printf '%s' "$tag" | jq -r .id) + commit=$(printf '%s' "$tag" | jq -r .commit.sha) + if [ "$commit" != "$GITHUB_SHA" ]; then + echo "the forge has the tag $version on $commit, not on this run's commit $GITHUB_SHA" >&2 + exit 1 + fi + if [ "$object" = "$commit" ]; then + echo "$version is a lightweight tag; a release tag is annotated and signed: git tag -s $version -m 'crafter-build $version'" >&2 + exit 1 + fi + # Forgejo verifies a signed tag's own signature, and for an + # unsigned one falls back to its commit's, so the verdict also + # has to cover the tag's payload, not the commit's. + signed=$(curl -sSf "$api/git/tags/$object" | + jq -r --arg c "$commit" --arg v "$version" '"\(.verification.verified) \(.verification.signer.email // "") \(.verification.payload // "" | startswith("object \($c)\ntype commit\ntag \($v)\n"))"') + if [ "$signed" != "true $RELEASE_SIGNER true" ]; then + echo "Forgejo does not verify the tag $version itself as signed by $RELEASE_SIGNER (verified, signer, over the tag's payload: $signed)" >&2 + exit 1 + fi + if ! git merge-base --is-ancestor "$GITHUB_SHA" origin/master; then + echo "the tag $version is on $(git rev-parse --short "$GITHUB_SHA"), which is not on master: push the tag with the master it releases" >&2 + exit 1 + fi + # Newer than every release the forge has published, its own + # included, and not every tag: a later tag whose run still + # queues behind this one has no release yet, and a tag that is + # no X.Y.Z version never gets one. + for t in $(git tag --list | awk '/^[0-9]+\.[0-9]+\.[0-9]+$/' | sort -V); do + [ "$(printf '%s\n%s\n' "$t" "$version" | sort -V | tail -n 1)" = "$t" ] || continue + code=$(curl -s -o /dev/null -w '%{http_code}' "$api/releases/tags/$t") + case "$code" in + 404) ;; + 200) + if [ "$t" = "$version" ]; then + echo "$version is released already; to re-run after a release left half made, delete that release first" >&2 + else + echo "$version is not newer than the release $t" >&2 + fi + exit 1 ;; + *) echo "the forge answered $code when asked whether $t is released" >&2; exit 1 ;; + esac + done + if [ "$newest" != "$version" ]; then + echo "the metainfo's newest release is \"$newest\", but the tag is $version: the tagged commit needs the entry for $version at the top of packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml" >&2 exit 1 fi else - # A master build names itself after the release it follows, the - # way a local checkout does: 2.0.0-3-g21746b1 is three commits - # past 2.0.0. Before the first tag there is nothing to describe - # from and the bare hash gets a placeholder in front. - described=$(git describe --tags --always --exclude=latest) + # Nothing is published. The build names itself after the + # release it follows, the way a local checkout does: + # 5.0.0-3-g21746b1 is three commits past 5.0.0. Before the first + # tag there is nothing to describe from and the bare hash gets a + # placeholder in front. + described=$(git describe --tags --always) case "$described" in [0-9]*.[0-9]*.[0-9]*) version="$described" ;; *) version="0.0.0-dev+$described" ;; esac - branch=beta fi echo "version=$version" >> "$GITHUB_OUTPUT" - echo "branch=$branch" >> "$GITHUB_OUTPUT" - echo "version $version, flatpak branch $branch" + echo "version $version" - name: Build # The manifest is rewritten beside the original so its relative @@ -108,7 +219,7 @@ jobs: grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST" flatpak-builder --force-clean --ccache --disable-rofiles-fuse \ --state-dir=/root/.cache/flatpak-builder \ - --repo=repo --default-branch="${{ steps.meta.outputs.branch }}" \ + --repo=repo --default-branch="$BRANCH" \ build-dir "$CI_MANIFEST" - name: Smoke test @@ -116,16 +227,16 @@ jobs: flatpak-builder --run build-dir "$CI_MANIFEST" crafter-build --version flatpak-builder --run build-dir "$CI_MANIFEST" sh -c 'cd examples/hello-world && crafter-build && ls bin/*/hello*' - # Everything below publishes and only runs on master and on tags. + # Everything below releases the build and runs only for a pushed tag. - name: Bundle - if: github.event_name != 'pull_request' + if: github.event_name == 'push' run: | mkdir -p dist - flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}" + flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "$BRANCH" ls -la dist - name: Upload bundle - if: github.event_name != 'pull_request' + if: github.event_name == 'push' uses: actions/upload-artifact@v3 with: name: crafter-build-${{ steps.meta.outputs.version }} @@ -133,7 +244,7 @@ jobs: if-no-files-found: error - name: Publish to the Catcrafts repo - if: github.event_name != 'pull_request' + if: github.event_name == 'push' run: | set -eu # The signing key is generated once on the runner host @@ -158,7 +269,20 @@ jobs: [ -f "$REPO_DIR/config" ] || ostree init --repo="$REPO_DIR" --mode=archive flatpak build-commit-from --src-repo=repo \ --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ - "$REPO_DIR" "app/$APP_ID/x86_64/${{ steps.meta.outputs.branch }}" + "$REPO_DIR" "app/$APP_ID/x86_64/$BRANCH" + # Every push to master used to be published as `beta`. Its last + # build is marked end-of-life, once, so that the next + # `flatpak update` of a beta installation says why nothing follows. + # The reason names the branch: with --noninteractive, flatpak's + # message names only the app. + beta="app/$APP_ID/x86_64/beta" + if ostree rev-parse --repo="$REPO_DIR" "$beta" >/dev/null 2>&1 && + ! ostree show --repo="$REPO_DIR" --print-metadata-key=ostree.endoflife "$beta" >/dev/null 2>&1; then + flatpak build-commit-from --src-ref="$beta" \ + --end-of-life="The beta branch gets no more updates: every change now comes as a release on the stable branch. Install stable with flatpak install catcrafts net.catcrafts.CrafterBuild//stable, then remove beta with flatpak uninstall net.catcrafts.CrafterBuild//beta." \ + --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ + "$REPO_DIR" "$beta" + fi flatpak build-update-repo --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ --generate-static-deltas --prune "$REPO_DIR" # The repo description and the app ref carry the public key. @@ -173,72 +297,21 @@ jobs: (cd dist && sha256sum * > SHA256SUMS) ls -la dist - - name: Rolling 'latest' release - if: github.event_name != 'pull_request' && github.ref_type != 'tag' - run: | - git config user.email "ci@catcrafts.net" - git config user.name "Crafter Build CI" - git tag -f latest - git push origin latest --force - - name: Publish rolling 'latest' release - if: github.event_name != 'pull_request' && github.ref_type != 'tag' - uses: https://code.forgejo.org/actions/forgejo-release@v2 - with: - direction: upload - url: ${{ github.server_url }} - repo: ${{ github.repository }} - tag: latest - title: Latest master build - prerelease: true - override: true - release-dir: dist - token: ${{ secrets.GITHUB_TOKEN }} - - - name: Release notes - if: github.ref_type == 'tag' - run: | - # The newest block's description as Markdown, for the - # release page. AppStream has no headings, so a paragraph that - # introduces a list is one:

New

    ...
comes out as - # "## New" over its items, the shape the notes take since 3.0.0. - # A paragraph on its own stays a paragraph, as the older entries - # are written. and get their Markdown spelling; a code - # span also keeps text such as lib//eh from being read as - # an HTML tag by the forge. - python3 - > release-notes.md <<'EOF' - import xml.etree.ElementTree as ET - - desc = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release/description") - - def text(el): - parts = [el.text or ""] - for child in el: - inner = text(child) - parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner)) - parts.append(child.tail or "") - return " ".join("".join(parts).split()) - - blocks = list(desc) - out = [] - for i, el in enumerate(blocks): - heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol") - if el.tag == "p": - out.append(f"## {text(el)}" if heads_list else text(el)) - elif el.tag == "ul": - out.append("\n".join(f"- {text(li)}" for li in el)) - elif el.tag == "ol": - out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1))) - print("\n\n".join(out)) - EOF - name: Publish release - if: github.ref_type == 'tag' + if: github.event_name == 'push' + # The notes are release-notes.md, which the Version step wrote. No + # `override`: it would delete a tag that has moved off this commit + # since the Version step and tag the old commit instead. Without it + # the action refuses such a tag, and a release that an earlier + # attempt left half made, such as a draft whose upload failed, has + # to be deleted before a re-run. uses: https://code.forgejo.org/actions/forgejo-release@v2 with: direction: upload url: ${{ github.server_url }} repo: ${{ github.repository }} - tag: ${{ github.ref_name }} - title: crafter-build ${{ github.ref_name }} + tag: ${{ steps.meta.outputs.version }} + title: crafter-build ${{ steps.meta.outputs.version }} release-notes-file: release-notes.md release-dir: dist token: ${{ secrets.GITHUB_TOKEN }} diff --git a/build.sh b/build.sh index 02b089b..dcb8696 100755 --- a/build.sh +++ b/build.sh @@ -36,10 +36,9 @@ else STD_CPPM=/usr/share/libc++/v1/std.cppm fi -# The version baked into the binary: the environment (CI sets it from the -# tag), else the nearest tag, else a development marker. `latest` is the -# rolling CI tag and never a version. -VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty --exclude=latest 2>/dev/null || echo 0.0.0-dev)}" +# The version baked into the binary: the environment (CI sets it), else the +# nearest tag, else a development marker. +VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo 0.0.0-dev)}" mkdir -p build bin lib share/crafter-build share/crafter-build/wasi-runtime diff --git a/project.cpp b/project.cpp index 9fc8e43..0f10375 100644 --- a/project.cpp +++ b/project.cpp @@ -157,14 +157,14 @@ extern "C" Configuration CrafterBuildProject(std::span a if (const fs::path& llvmInclude = GetToolchain().llvmIncludeDir; !llvmInclude.empty()) { CrafterBuildLib->compileFlags.push_back(std::format("-I{}", llvmInclude.string())); } - // The version the binary reports: CI sets it from the tag, a checkout - // describes itself, anything else is a development build. A define on - // the library, so each version is its own library variant. + // The version the binary reports: CI sets it, a checkout describes + // itself, anything else is a development build. A define on the + // library, so each version is its own library variant. std::string version; if (const char* env = std::getenv("CRAFTER_BUILD_VERSION"); env && *env) { version = env; } else { - auto described = RunCommandWithTimeout("git describe --tags --always --dirty --exclude=latest 2>/dev/null", std::chrono::seconds(10)); + auto described = RunCommandWithTimeout("git describe --tags --always --dirty 2>/dev/null", std::chrono::seconds(10)); version = described.output; while (!version.empty() && (version.back() == '\n' || version.back() == '\r')) version.pop_back(); if (described.exitCode != 0 || version.empty()) version = "0.0.0-dev"; From 54aaf078b20f5c899a7ea63f88dae0d2ac8fe3b1 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:44:35 +0100 Subject: [PATCH 13/14] docs: a release is a signed tag pushed with master The README offered master builds as the `beta` branch, AGENTS.template.md described the `latest` tag CI force-pushed, a release as a tag pushed after master, and CI as running on every push to master, and the repo-better-solution lens told reviewers that users get master through `beta`. The README now says that releases go to `stable`, the branch its install commands pick even while the repo still holds the old `beta` branch. AGENTS.template.md says that every pushed range adds a release entry and goes out in one push with a signed tag on the commit master receives, its last commit or, on the pull-request path, the merge, how to write the entry, what every run checks before building and what a tag run also checks, publishes and creates, what follows when one fails, what CI runs for, and what becomes of `beta`; the lens says users get master through releases on `stable`. The entry's rules are the owner's: at most three sections, New, Fixes and Breaking, the most important item first and nothing after them, and a version that is a major for a Breaking section, a minor for New items and a patch for Fixes alone. The release paragraph also gets the blank line that keeps it apart from the one on local builds. --- .review-gates/lenses/repo-better-solution.md | 4 ++-- AGENTS.template.md | 10 ++++++---- README.md | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/.review-gates/lenses/repo-better-solution.md b/.review-gates/lenses/repo-better-solution.md index 65bd330..278b12f 100644 --- a/.review-gates/lenses/repo-better-solution.md +++ b/.review-gates/lenses/repo-better-solution.md @@ -37,6 +37,6 @@ checkout (`../Crafter.*/project.cpp`) and of every user compiles against what `interfaces/` exports and is loaded by the binary; the CLI, the `CRAFTER_BUILD_*` variables, the cache an older version left under `~/.cache/crafter.build/` and the `bin/` layout CI globs are contracts too. -Users get a push to master through the Flatpak's `beta` branch and a tag -through `stable`. A change an existing `project.cpp` does not survive is a +Users get master through releases on the Flatpak's `stable` branch. A +change an existing `project.cpp` does not survive is a finding unless the range documents, in the README, what users must change. diff --git a/AGENTS.template.md b/AGENTS.template.md index 1db87b0..f275a93 100644 --- a/AGENTS.template.md +++ b/AGENTS.template.md @@ -69,7 +69,7 @@ Three things to know about this loop: ./bin/crafter-build clean ``` -Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` on every push to `master` and on pull requests, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/.log`. +Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` for every release tag, every pull request into `master` and every manual run, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/.log`. ## Testing instructions @@ -142,7 +142,8 @@ These were decided deliberately. Do not undo them without the maintainer's say-s - Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice. - Separate commits for the fix, its test and the README change when each stands on its own. - Update the README in the same push when user-visible behaviour changes. -- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on every push to `master` and on pull requests; a push to `master` then publishes the `beta` branch of the Catcrafts repo, and an `X.Y.Z` tag the `stable` branch plus a release. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push. +- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it for every release tag, every pull request into `master` and every manual run. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push. +- Every push to `master` is a release, made by the signed tag pushed with it (see "CI and publishing"). Every pushed range adds a `` entry for its version at the top of `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, in a commit of its own (`docs(packaging): add the X.Y.Z release entry`), dated the day of the push in UTC. Its notes have up to three sections, `

New

`, `

Fixes

` and `

Breaking

` in that order, each followed by a `
    ` of its items, the most important first, and nothing after the last section. The version follows from them: a Breaking section makes it a major, New items a minor, and Fixes alone a patch. Once the owner has approved the frozen hash, the commit `master` is to receive gets the signed tag of that version, made locally, and both go out in one push: `git tag -s X.Y.Z -m 'crafter-build X.Y.Z'`, then `git push --atomic origin HEAD:master X.Y.Z`. That commit is the range's last one, or, when the owner asks for a pull request instead, the merge of the reviewed branch into `master`. - Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits: a range reaches `master` as a fast-forward, and when the owner asks for a pull request instead, the merge of the reviewed branch into `master` is pushed without a freeze of its own. - Every commit of a pushed range is GPG-signed, has `jorijnvdgraaf@catcrafts.net` as its author and committer, and carries no AI attribution trailer in its message, neither `Co-Authored-By:` nor `Assisted-by:`. `freeze` refuses a range with a commit that breaks any of these (`requireSigned`, `authors` and `aiTrailers` in `.review-gates/config.json`). - Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a`, `AGENTS.md` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`. @@ -155,7 +156,7 @@ These were decided deliberately. Do not undo them without the maintainer's say-s - `--runner=cmd:` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests. - `crafter-build run ` executes whatever the script does, with your permissions. Only the scripts of the project crafter-build was pointed at can be run, never a dependency's; keep it that way. - The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library. -- CI force-pushes the rolling `latest` tag and publishes a prerelease on every push to `master` with the workflow's own token, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it. +- CI creates a release for each release tag with the workflow's own token, pushes nothing to git, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it. - Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way. ## CI and publishing @@ -174,7 +175,8 @@ gpg --batch --pinentry-mode loopback --passphrase '' --export-secret-keys --armo printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id ``` -To release: add a `` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), push it to `master`, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry. +A release is a tag `X.Y.Z`, annotated and signed, pushed with the `master` it releases; a push to `master` without one runs nothing. Before building, every run renders the release notes from the newest `` entry in `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, and stops when the metainfo does not parse or the notes come out empty. A tag run also checks that Forgejo verifies the tag as signed by the address the workflow's `RELEASE_SIGNER` names, that the tag's commit is on `master`, that its version has the form `X.Y.Z` and is newer than every release the forge has published, its own included, and that the newest entry is that version. Once the build and the suite inside it pass, the tag run publishes `stable` and creates the release, with the notes, the bundle, the `.flatpakref` and `SHA256SUMS` attached. Release runs queue one behind another, and a failed one can be re-run while no newer release exists, once a release it left half made is deleted; when the fix needs a commit of its own, that push is the next version's release. Pull requests and manual runs publish nothing. The `beta` branch, which every push to `master` used to go to, gets no more builds: the publish step marks its last build end-of-life, once, and the next `flatpak update` of a beta installation prints the reason. + Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive. ```bash diff --git a/README.md b/README.md index f9238e0..cfd2bb1 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ alias crafter-build='flatpak run --die-with-parent net.catcrafts.CrafterBuild' `--die-with-parent` ends the sandbox with the `flatpak run` that started it. Without the flag, killing only the `flatpak run` process leaves the sandbox running: a build carries on writing into `bin/` and `build/`, and a program started with `-r` keeps running. With it, bubblewrap kills everything in the sandbox with SIGKILL when the `flatpak run` process dies or when that process's parent does. For `crafter-build …` or `crafter-build … &` typed at a bash or zsh prompt, that parent is the shell, so the sandbox ends with the shell, even after a `disown`. Ctrl-C reaches the program either way. -Pre-release builds of `master` are the `beta` branch (`flatpak install catcrafts net.catcrafts.CrafterBuild//beta`), and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host. +Releases are published to the `stable` branch that the commands above install, and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host. ## Writing a project.cpp From 0cc699139cb5d2802a62e90f6033193101bf4219 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 2 Nov 2026 11:46:11 +0100 Subject: [PATCH 14/14] docs(packaging): add the 6.0.0 release entry The release of this push and of everything on master since 5.0.0: lldb, the Khronos validation layer and the SPIRV-Tools programs in the Flatpak, --die-with-parent in the README's alias, and the release .flatpakref with the repository's key in it. Its Breaking item, the end of beta and of the rolling release, makes it a major. --- .../net.catcrafts.CrafterBuild.metainfo.xml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml index d962c8f..38d6b97 100644 --- a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml +++ b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml @@ -25,6 +25,25 @@ + + +

    New

    +
      +
    • The Flatpak carries lldb, with lldb-server, lldb-dap and lldb-mcp, from the same LLVM build as the compiler, so programs that -r and the test runner start can be debugged inside the sandbox they run in. The README's new Debugging section says how, including flatpak enter to reach a program that is already running.
      • +
    • The Flatpak carries the Khronos Vulkan validation layer, the Vulkan SDK release of the runtime's own loader. VK_INSTANCE_LAYERS or VK_LOADER_LAYERS_ENABLE turns it on from the environment, and VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1 adds GPU-assisted validation. A layer installed on the host cannot stand in: flatpak run clears VK_LAYER_PATH and VK_ADD_LAYER_PATH.
      • +
    • The Flatpak carries the SPIRV-Tools programs (spirv-val, spirv-opt, spirv-dis and the rest), so a test that requires tool:spirv-val runs inside it instead of being skipped.
      • +
    • The alias in the README's install commands passes --die-with-parent, which ends the sandbox when its flatpak run process dies and, for a command typed at a bash or zsh prompt, when that shell does. Without it, killing flatpak run leaves a build writing into bin/ and build/ and a program started with -r running. An alias set up from an earlier README does not have it.
      • +
    +

    Fixes

    +
      +
    • The .flatpakref attached to a release carries the repository's signing key. Releases 2.0.0 to 5.0.0 attach one with a placeholder in its place, and installing from it fails with GPG: Unable to export keys; the one on flatpak.catcrafts.net has the key.
      • +
    +

    Breaking

    +
      +
    • Master has no builds of its own any more: the beta branch gets no new builds and the rolling "Latest master build" release is gone, and every change reaches users as a release on the stable branch. Beta's last build is marked end-of-life, and the next flatpak update of a beta installation says so: install stable with flatpak install catcrafts net.catcrafts.CrafterBuild//stable and remove beta with flatpak uninstall net.catcrafts.CrafterBuild//beta.
      • +
    +     +

    Fixes