From b5a366d060e26d2f88aad4c976f39f598718f0e8 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sat, 3 Oct 2026 20:09:22 +0200 Subject: [PATCH 1/4] fix(incremental): key build outputs on the compiler A project built with one clang kept its outputs when crafter-build started using another. The output directory is named by VariantId(), which hashed every input that changes what is compiled except the compiler itself, and staleness is decided by mtimes, which a compiler update leaves alone. A PCM can only be read by the clang release that wrote it, so after the bundled LLVM moved from 22 to 23 the first build found nothing stale, and the first edit failed with "module file Greeter.pcm uses an older format that is no longer supported" until the project was cleaned by hand. A project with a `#pragma crafter dependency` did not get that far: its project file imports the dependency's PCM, so it failed to load, and for a git dependency that PCM lives in the cache, where `crafter-build clean` does not reach. The same happens natively when the distro updates clang. The toolchain's key now goes into the hash, as the host PCM cache was already keyed on it. An update starts an empty variant directory and rebuilds into it; the previous one stays until `clean`, as after any other change of variant, and two crafter-build versions can work in one checkout without disturbing each other. VariantId() gains an overload taking the toolchain so the dependence can be tested in-process, the way ResolveToolchain and SysrootStdModule take theirs. External cmake builds are not covered: their directory is fixed per clone and keyed on the compiler's path, not its release. Objects of two clang releases link together on the host, so nothing fails there, but they are not rebuilt either. Verified by replaying the update: three projects built with the 2.0.0 Flatpak, then built, edited and run with this one without cleaning, then built with 2.0.0 again. Co-Authored-By: Claude Fable 5.1 --- interfaces/Crafter.Build-Clang.cppm | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/interfaces/Crafter.Build-Clang.cppm b/interfaces/Crafter.Build-Clang.cppm index 1149661..8a306be 100644 --- a/interfaces/Crafter.Build-Clang.cppm +++ b/interfaces/Crafter.Build-Clang.cppm @@ -7,6 +7,7 @@ import :Shader; import :Interface; import :Implementation; import :External; +import :Toolchain; namespace fs = std::filesystem; export namespace Crafter { @@ -488,10 +489,21 @@ export namespace Crafter { void AddAstLintRule(std::string name, std::function check); // Suffix that uniquely identifies this Configuration's compile state. // target+march+mtune are spelled out for readability; the rest - // (type, debug, sysroot, defines, compileFlags) collapse into a short - // hash so two Configurations sharing a path but with different - // compile state can't clobber each other's outputs. - std::string VariantId() const { + // (type, debug, sysroot, defines, compileFlags, the toolchain) + // collapse into a short hash so two Configurations sharing a path but + // with different compile state can't clobber each other's outputs. + std::string VariantId() const { return VariantId(GetToolchain()); } + // The variant under an explicit toolchain; VariantId() is this for + // the running one. The compiler is compile state like any flag: a + // PCM can only be read by the clang release that wrote it, and + // staleness is decided by mtimes, which a compiler update does not + // touch. Before the toolchain was hashed, a project built with clang + // 22 kept its PCMs when crafter-build moved to clang 23, and the + // first edit afterwards failed with "module file ... uses an older + // format that is no longer supported" until the project was cleaned. + // Keyed here, an update starts an empty directory and rebuilds; the + // old one stays until `crafter-build clean`. + std::string VariantId(const Toolchain& toolchain) const { std::string compileKey; compileKey += std::to_string(static_cast(type)); compileKey += '|'; @@ -521,6 +533,8 @@ export namespace Crafter { compileKey += "|A:"; compileKey += a; } + compileKey += "|T:"; + compileKey += toolchain.Key(); std::size_t configHash = std::hash{}(compileKey); return std::format("{}-{}-{}-{}-{:08x}", name, target, march, mtune, configHash); } From 995a40e7dd627b9920c1b00214844d0f039bc0a1 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sat, 3 Oct 2026 20:09:22 +0200 Subject: [PATCH 2/4] test(incremental): a compiler update perturbs VariantId Two toolchains that differ only in the clang release, as the Flatpak's do across an update, must give one Configuration two variants; so must the same release installed under another prefix. The no-argument form is checked to be the variant under the running toolchain, and BuildDir to be named for it. Co-Authored-By: Claude Fable 5.1 --- tests/VariantId/main.cpp | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/tests/VariantId/main.cpp b/tests/VariantId/main.cpp index 022b0d6..e4d2a36 100644 --- a/tests/VariantId/main.cpp +++ b/tests/VariantId/main.cpp @@ -133,6 +133,33 @@ int main() { Check(a.BuildDir() != b.BuildDir(), "different VariantId yields different BuildDir"); Check(a.BinDir() != b.BinDir(), "different VariantId yields different BinDir"); } + { + // The compiler is compile state as much as any flag: a PCM can only + // be read by the clang release that wrote it, and nothing in a + // project's sources changes when the compiler does. With the + // toolchain left out of the id, a project built with clang 22 kept + // its PCMs after crafter-build moved to clang 23; the build after + // the update found nothing stale, and the first edit failed with + // "module file ... uses an older format that is no longer supported" + // until the project was cleaned by hand. + Configuration cfg = MakeBase(); + Toolchain older; + older.clangxx = "/app/bin/clang++"; + older.clangVersion = "clang version 22.1.8"; + Toolchain newer = older; + newer.clangVersion = "clang version 23.1.2"; + Check(cfg.VariantId(older) != cfg.VariantId(newer), "a compiler update at the same path perturbs VariantId"); + Check(cfg.VariantId(older) == cfg.VariantId(older), "VariantId under one toolchain is deterministic"); + + // Two installs of one release are still two toolchains: their + // libc++ need not be the same build. + Toolchain elsewhere = older; + elsewhere.clangxx = "/usr/bin/clang++"; + Check(cfg.VariantId(older) != cfg.VariantId(elsewhere), "the same release installed elsewhere perturbs VariantId"); + + Check(cfg.VariantId() == cfg.VariantId(GetToolchain()), "VariantId() is the variant under the running toolchain"); + Check(cfg.BuildDir().filename() == cfg.VariantId(GetToolchain()), "BuildDir is named for the running toolchain's variant"); + } if (Failures > 0) { std::println(std::cerr, "{} assertions failed", Failures); From 03728fd4a7b1372628a829f12e7a16922bd95b15 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sat, 3 Oct 2026 20:09:22 +0200 Subject: [PATCH 3/4] docs: describe the output directory name and what a compiler update does The README still gave the output path as bin/--/, which has not been its shape since the variant hash was introduced. It now shows the real one and says what the hash stands for, including that a compiler update starts a new directory and `clean` removes the old. AGENTS.md lists the toolchain among VariantId()'s inputs and notes that a glob over variant directories matches two after an update. Co-Authored-By: Claude Fable 5.1 --- AGENTS.md | 4 ++-- README.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 091807a..574abeb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -110,7 +110,7 @@ Beyond the rules: - There is no host-platform preprocessor split: the host is Linux. Target differences (mingw, wasm) branch on `cfg.target` strings at runtime, in the library as well as in `project.cpp` and tests. - A build step fails when its command exits nonzero, never because it printed. Use `RunBuildCommand` or `RunCommandChecked`; warnings go to `Progress::Diagnostic`. - Every output path goes through `Configuration::BinDir()`, `BuildDir()` and `PcmDir()`. Hand-rolled `path / "bin" / ...` clobbered variants before; do not add new ones. -- Anything that changes what gets compiled or bundled belongs in `VariantId()`. It already hashes type, debug, sysroot, defines, compile flags, wasm variant flags and uninterpreted project args. +- Anything that changes what gets compiled or bundled belongs in `VariantId()`. It already hashes type, debug, sysroot, defines, compile flags, wasm variant flags, uninterpreted project args and the toolchain (`Toolchain::Key()`). ## Design rules @@ -170,4 +170,4 @@ To release: add a `` to `packaging/flatpak/net.catcrafts.CrafterBuild.m - The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop. - A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone. - A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules//`, one `libcxx--modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot. -- `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe---*/` rather than predicting the hash. +- `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe---*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both. diff --git a/README.md b/README.md index c195fb2..07ae901 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ extern "C" Configuration CrafterBuildProject(std::span a } ``` -Run `crafter-build` in that directory; outputs land at `bin/myapp--/myapp` and intermediates at `build/myapp--/`. Add `-r` to run the produced executable straight after a successful build. +Run `crafter-build` in that directory; outputs land at `bin/myapp----/myapp` and intermediates at `build/myapp----/`. The hash covers everything that changes what is compiled (build type, defines, flags, sysroot, the compiler itself), so two such builds never share a directory. After a compiler update, a new crafter-build release included, the first build starts a fresh directory and recompiles; `crafter-build clean` removes the ones left behind. Add `-r` to run the produced executable straight after a successful build. ## Dependencies From 486935bce862623279737db5b14da70d0d07104e Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Sat, 3 Oct 2026 20:09:22 +0200 Subject: [PATCH 4/4] docs(packaging): add the 2.1.0 release entry CI takes the newest as the notes of a tag and refuses a tag that does not match it. 2.1.0 covers what changed since 2.0.0: the bundled LLVM 23.1.2, build outputs kept per compiler, and the std module source chosen by the sysroot's libc++ release. Co-Authored-By: Claude Fable 5.1 --- packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml index afe8428..e63ee54 100644 --- a/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml +++ b/packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml @@ -25,6 +25,12 @@ + + +

The bundled toolchain is now LLVM 23.1.2. Build outputs are kept per compiler, so the first build after updating recompiles each project into a fresh directory and nothing has to be cleaned by hand.

+

Cross builds pick the std module source that matches the sysroot's libc++ release. The source for libc++ 22 ships beside the bundled 23 for sysroots that carry libc++ without one, such as Alpine, and a sysroot nothing matches fails with one line naming both releases. crafter-build --version prints the libc++ release in use.

+ +

First release distributed as a Flatpak, with a bundled LLVM 22 toolchain and WASI sysroot. Windows host support is removed; crafter-build gains --version and resolves its compiler from one place.