feat(packaging): run flatpak and run install for a local Flatpak

The local Flatpak build was one long flatpak-builder line in AGENTS.md,
with its state wherever the caller happened to stand, and installing
the result was not written down anywhere. It is not a rare chore: the
development box compiles with the distro's clang and the Flatpak with
its bundled LLVM, a major version apart at the moment, so anything
toolchain-sensitive has to be tried in the Flatpak too.

Both are now scripts in project.cpp, the first users of cfg.scripts.

`run flatpak` builds the manifest as it is, from the working tree, and
ends with `crafter-build --version` from inside the result. Its state
is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and
not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in
the sandbox, so it does not depend on the host variant, and a
variant-keyed directory would strand the LLVM cache on every host
compiler update. The manifest already skips build/ when it copies the
sources. flatpak-builder keys its cache on the manifest's path, so the
manifest is never rewritten into a copy the way CI stamps the version
in, and a local build reports 0.0.0-dev. The only argument accepted is
--jobs=<N>: anything that ends the run without a build would leave the
smoke test checking the previous one.

`run install` installs the last export as branch `local` from a user
remote of its own and makes it current, leaving the published branch
installed. It does not build first: flatpak-builder never caches a
`type: dir` module, so every build reruns build.sh and the suite.
remote-add --if-not-exists is followed by remote-modify --url, because
the remote may exist and point at another checkout.

Both refuse to run from the Flatpak build of crafter-build, which has
no flatpak to drive.

The smoke test goes through `flatpak build` and not `<builder> --run`,
which is what CI uses. flatpak-builder 1.4.9 run as the
org.flatpak.Builder app exits 1 when the command it ran succeeded and 0
when it failed (`true` gives 1, `exit 3` gives 0), so the script
reported a good build as a failure.

Measured on the 56-core box with LLVM and the WASI sysroot cached: 260
seconds, the crafter-build module only, 29 tests passing in the sandbox.
A second `run install` after a rebuild moved the installed commit.

`clean` now also drops that state, about 2 GB and a cold LLVM build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

AGENTS.md

@@ -10,7 +10,7 @@ crafter-build is a C++26 modules build system. A project is described by a `proj
 |---|---|
 | `interfaces/Crafter.Build-<Partition>.cppm` | Module interface partitions. `Crafter.Build.cppm` is the umbrella that re-exports them. |
 | `implementations/Crafter.Build-<Partition>.cpp` | Implementation units, one per partition, named `Crafter.Build:<Partition>_impl`. `main.cpp` is a thin wrapper around `Crafter::Run`. |
-| `project.cpp` | The self-host build description: a `crafter.build-lib` library, a `crafter.build-exe` executable, the test declarations and the lint rules. |
+| `project.cpp` | The self-host build description: a `crafter.build-lib` library, a `crafter.build-exe` executable, the test declarations, the lint rules and the `flatpak` and `install` scripts. |
 | `tests/<Name>/main.cpp` | One self-test per directory; `tests/<Name>/fixture/` holds the sample project it builds. |
 | `examples/` | Standalone user-facing projects, MIT licensed. |
 | `build.sh` | Bootstrap script. It compiles everything by hand, without crafter-build. |
@@ -48,7 +48,7 @@ Three things to know about this loop:
 - Changes under `implementations/` are picked up by the self-host build and by the tests, which link the local library. Changes under `interfaces/` are not: the harness compiles against the copies in `share/crafter-build/` and the PCM cache built from them, so a new or changed declaration is invisible until you run `./build.sh` again. The symptom is `use of undeclared identifier` for the symbol you just added.
 - A new exported symbol that a `project.cpp` reaches on its default path also needs `./build.sh`: the old binary does not export it, and `LoadProject` fails with an undefined symbol.
 
-`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`.
+`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. That takes `bin/crafter-build` itself with it, and `build/flatpak`, the state of the local Flatpak build (see "CI and publishing"): about 2 GB, and the next `run flatpak` rebuilds LLVM. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`.
 
 ## Build, test and style commands
 
@@ -164,7 +164,17 @@ gpg --batch --pinentry-mode loopback --passphrase '' --export-secret-keys --armo
 printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id
 ```
 
-To release: add a `<release>` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), merge, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry. Local builds: `flatpak-builder --user --force-clean --ccache --repo=repo build-dir packaging/flatpak/net.catcrafts.CrafterBuild.yaml`, then `flatpak-builder --run build-dir <manifest> crafter-build --version`.
+To release: add a `<release>` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), merge, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry.
+Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive.
+
+```bash
+./bin/crafter-build run flatpak [--jobs=<N>]   # build the Flatpak, then print its version from inside the sandbox
+./bin/crafter-build run install                # install that build as branch `local` and make it current
+```
+
+`run flatpak` needs `flatpak-builder` or the `org.flatpak.Builder` app, and the Sdk (`flatpak install flathub org.freedesktop.Sdk//26.08`). It builds the working tree, uncommitted changes included, runs the whole suite inside the sandbox and reports version `0.0.0-dev`. Its state is `build/flatpak/` (`state/`, `repo/`, `build-dir/`). The first run builds LLVM and the WASI sysroot, about 25 minutes on a 56-core machine; later runs rebuild only the crafter-build module, a few minutes, because flatpak-builder never caches a `type: dir` source. The checkout must not be under `/tmp`, which `org.flatpak.Builder` cannot see. Use it for anything toolchain-sensitive: the native loop compiles with the distro's clang and the Flatpak with its bundled LLVM, and the two can be a major version apart.
+
+`run install` does not build; it installs what `run flatpak` last exported. It points the user remote `crafter-build-local` at `build/flatpak/repo`, installs `net.catcrafts.CrafterBuild//local` from it next to whichever published branch is installed, and makes it the branch `flatpak run net.catcrafts.CrafterBuild` starts. `flatpak make-current --user net.catcrafts.CrafterBuild <branch>` switches back. Before `crafter-build clean`, or before deleting the checkout, run `flatpak uninstall --user net.catcrafts.CrafterBuild//local` and `flatpak remote-delete --user crafter-build-local`; otherwise the remote is left pointing at a repo that is gone.
 
 ## Gotchas
 

project.cpp

@@ -8,6 +8,134 @@ import Crafter.LintRules;
 namespace fs = std::filesystem;
 using namespace Crafter;
 
+// The local Flatpak: `crafter-build run flatpak` builds it from this checkout
+// and `crafter-build run install` installs that build. The development box
+// compiles with its own clang, the shipped artifact with the LLVM bundled in
+// the Flatpak, so this is how a change is tried against the real thing.
+namespace {
+    struct FlatpakPaths {
+        fs::path manifest;
+        // flatpak-builder's downloads, ccache and per-module cache. The
+        // LLVM build lives here: about 2 GB, and 25 minutes to rebuild.
+        fs::path state;
+        // The OSTree repo the build is exported to; `install` reads it.
+        fs::path repo;
+        fs::path buildDir;
+    };
+
+    // Everything under <project>/build/flatpak, and a fixed name on purpose
+    // instead of a BuildDir(): the Flatpak is compiled by the toolchain
+    // inside the sandbox, so it does not depend on the host variant, and a
+    // variant-keyed directory would strand the LLVM cache on every host
+    // compiler update. The manifest's `skip: build` keeps it out of the
+    // sources the sandbox copies. flatpak-builder keys its cache on the
+    // manifest's absolute path, so the manifest is used where it is, never
+    // through a rewritten copy the way CI stamps the version in; a local
+    // build therefore reports version 0.0.0-dev.
+    FlatpakPaths FlatpakLayout(const fs::path& projectDir) {
+        const fs::path root = projectDir / "build" / "flatpak";
+        return {
+            .manifest = projectDir / "packaging" / "flatpak" / "net.catcrafts.CrafterBuild.yaml",
+            .state = root / "state",
+            .repo = root / "repo",
+            .buildDir = root / "build-dir",
+        };
+    }
+
+    // Both scripts drive the host's flatpak, and the sandbox crafter-build
+    // itself runs in as a Flatpak has none.
+    bool RefusedInSandbox(std::string_view script) {
+        if (!fs::exists("/.flatpak-info")) return false;
+        std::println(std::cerr, "run {0}: needs the host's flatpak, which the Flatpak build of crafter-build cannot reach. Use a native build: ./build.sh, then ./bin/crafter-build run {0}", script);
+        return true;
+    }
+
+    // flatpak-builder as a host package, else the org.flatpak.Builder app.
+    // Empty when neither is installed.
+    std::string FlatpakBuilder() {
+        if (RunCommandWithTimeout("flatpak-builder --version", std::chrono::seconds(10)).exitCode == 0) return "flatpak-builder";
+        if (RunCommandWithTimeout("flatpak info org.flatpak.Builder", std::chrono::seconds(10)).exitCode == 0) return "flatpak run org.flatpak.Builder";
+        return "";
+    }
+
+    // A repo path as the file:// URL of a remote. Percent-encoded, because
+    // `remote-modify --url=` stores what it is given as it is.
+    std::string FileUrl(const fs::path& path) {
+        std::string url = "file://";
+        for (const char c : path.string()) {
+            const bool plain = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || std::string_view("/-._~").contains(c);
+            if (plain) url.push_back(c);
+            else url += std::format("%{:02X}", static_cast<std::uint8_t>(c));
+        }
+        return url;
+    }
+
+    std::int32_t BuildFlatpak(ScriptContext& ctx) {
+        if (RefusedInSandbox("flatpak")) return 1;
+        // Nothing else is passed through: a flag such as --stop-at or
+        // --download-only would end the run without a build and leave the
+        // smoke test below running against the previous one.
+        std::string jobs;
+        for (const std::string& arg : ctx.args) {
+            const std::string_view count = arg.starts_with("--jobs=") ? std::string_view(arg).substr(7) : std::string_view();
+            if (count.empty() || !std::ranges::all_of(count, [](char c) { return c >= '0' && c <= '9'; })) {
+                std::println(std::cerr, "run flatpak: unknown argument '{}' (only --jobs=<N> is accepted)", arg);
+                return 1;
+            }
+            jobs = std::format(" --jobs={}", count);
+        }
+        const std::string builder = FlatpakBuilder();
+        if (builder.empty()) {
+            std::println(std::cerr, "run flatpak: neither flatpak-builder nor the org.flatpak.Builder app is installed (flatpak install flathub org.flatpak.Builder)");
+            return 1;
+        }
+        const FlatpakPaths paths = FlatpakLayout(ctx.projectDir);
+        const std::string buildDir = ShellQuote(paths.buildDir.string());
+        if (const std::int32_t code = ctx.Run(std::format("{}{} --user --force-clean --ccache --state-dir={} --repo={} --default-branch=local {} {}", builder, jobs, ShellQuote(paths.state.string()), ShellQuote(paths.repo.string()), buildDir, ShellQuote(paths.manifest.string()))); code != 0) return code;
+        // The smoke test CI runs on its build, but through `flatpak build`
+        // and not `<builder> --run`: the org.flatpak.Builder app (1.4.9)
+        // exits 1 when the command it ran succeeded and 0 when it failed,
+        // and --run also wants the state dir again or it drops a
+        // .flatpak-builder/ into the working directory.
+        return ctx.Run(std::format("flatpak build {} crafter-build --version", buildDir));
+    }
+
+    std::int32_t InstallFlatpak(ScriptContext& ctx) {
+        if (RefusedInSandbox("install")) return 1;
+        if (!ctx.args.empty()) {
+            std::println(std::cerr, "run install: takes no arguments (got '{}')", ctx.args.front());
+            return 1;
+        }
+        const FlatpakPaths paths = FlatpakLayout(ctx.projectDir);
+        // Deliberately not built from here: flatpak-builder never caches a
+        // `type: dir` module, so every build reruns build.sh and the whole
+        // suite inside the sandbox. Installing is the cheap half.
+        if (!fs::exists(paths.repo / "config")) {
+            std::println(std::cerr, "run install: nothing built yet, run: crafter-build run flatpak");
+            return 1;
+        }
+        const std::string url = ShellQuote(FileUrl(paths.repo));
+        // Its own branch from its own remote, next to whichever published
+        // branch is installed. The remote may already exist and point at
+        // another checkout's repo; --if-not-exists would keep that silently.
+        if (const std::int32_t code = ctx.Run(std::format("flatpak remote-add --user --if-not-exists --no-gpg-verify crafter-build-local {}", url)); code != 0) return code;
+        if (const std::int32_t code = ctx.Run(std::format("flatpak remote-modify --user --url={} crafter-build-local", url)); code != 0) return code;
+        if (const std::int32_t code = ctx.Run("flatpak install --user --noninteractive --or-update crafter-build-local net.catcrafts.CrafterBuild//local"); code != 0) return code;
+        // Only a first install becomes the branch `flatpak run` starts; an
+        // update leaves that choice where it was.
+        if (const std::int32_t code = ctx.Run("flatpak make-current --user net.catcrafts.CrafterBuild local"); code != 0) return code;
+        std::println(
+R"(
+`flatpak run net.catcrafts.CrafterBuild` now starts this build.
+Back to a published branch, keeping this one installed:
+    flatpak make-current --user net.catcrafts.CrafterBuild <branch>
+Remove it (do this before `crafter-build clean`, which deletes the repo the remote points at):
+    flatpak uninstall --user net.catcrafts.CrafterBuild//local
+    flatpak remote-delete --user crafter-build-local)");
+        return 0;
+    }
+}
+
 extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
     std::vector<std::string> depArgs(args.begin(), args.end());
     Configuration* math = GitProject({
@@ -163,5 +291,9 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
     // rules also auto-fix under `crafter-build format`.
     LintRules::AddRules(cfg);
 
+    // See the top of this file.
+    cfg.scripts.push_back({"flatpak", BuildFlatpak});
+    cfg.scripts.push_back({"install", InstallFlatpak});
+
     return cfg;
 }