fix: key the std module's directory on everything it is compiled with

StdPcmDir named the directory by target, march and toolchain, then
grew a case per target for the rest: the wasm flags, and with the first
fix for #43 the mingw prefix. Every other flag BuildStdPcm passes went
unkeyed, so configurations that differed only there took turns
rebuilding one std PCM in place, under the PCMs built against it: a
project with -mtune=generic against project.cpp's -mtune=native on every
run, or two sysroots for one target. And the mingw prefix was keyed only
when there was one, so a build without a prefix landed in the directory
an older release had filled with a CRAFTER_MINGW_DIR=/usr build, and
relinked its objects.

StdPcmFlags is now the one source of the flags BuildStdPcm compiles
with, and StdPcmKey names the directory by target, march and toolchain
followed by a hash of those flags and, for the mingw target, of the
prefix behind its --sysroot. VariantId hashes StdPcmKey in place of the
mingw prefix: units do not notice a std module that moved directories,
so a build has to move with it. Every std PCM directory and build
directory moves once, as after a compiler update; the old ones stay
until `clean` and the cache is cleared by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

implementations/Crafter.Build-Clang.cpp

@@ -247,36 +247,51 @@ void Configuration::GetInterfacesAndImplementations(std::span<fs::path> interfac
     }
 }
 
+std::string Crafter::StdPcmFlags(const Configuration& config) {
+    if (config.target == "x86_64-w64-mingw32") {
+        // -femulated-tls keeps PCM TLS access matching libstdc++'s emutls
+        // definitions; a mismatch surfaces as undefined std::__once_callable.
+        return std::format(" --target={} -march={} -mtune={} -femulated-tls -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier", config.target, config.march, config.mtune);
+    }
+    const bool isWasm = config.target.starts_with("wasm32");
+    const std::string sysrootFlag = config.sysroot.empty() ? std::string() : std::format(" --sysroot={}", config.sysroot);
+    // wasm32 rejects -march, and the std module has to be compiled the way
+    // the units importing it are, a variant pass's flags (e.g.
+    // -mrelaxed-simd) included, or the BMI is not feature-compatible.
+    std::string archFlags = isWasm ? WasmCompileFlags(config) : std::format(" -march={} -mtune={}", config.march, config.mtune);
+    // non-wasm cross builds: the driver ranks the host toolchain's own
+    // libc++ headers (<install>/../include/c++/v1) above --sysroot, so the
+    // PCM would mix host libc++ with the foreign libc. Force the sysroot's
+    // copy.
+    if (!isWasm && !config.sysroot.empty()) {
+        archFlags += std::format(" -nostdinc++ -isystem {}/usr/include/c++/v1", config.sysroot);
+    }
+    return std::format(" --target={} -std=c++26{}{} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier", config.target, sysrootFlag, archFlags);
+}
+
+std::string Crafter::StdPcmKey(const Configuration& config, const Toolchain& toolchain) {
+    // A PCM is only valid for the clang that wrote it and the flags it was
+    // written with, so a compiler bump starts a cold directory instead of
+    // handing the new clang a PCM it will crash on, and so does any flag.
+    // With only some of them in the name, configurations that differed in
+    // the rest took turns rebuilding one std PCM in place: -mtune=generic
+    // against project.cpp's -mtune=native on every run, two sysroots, or two
+    // releases sharing a cache, as the Flatpak's branches do, each time under
+    // the PCMs built against the previous one, which clang crashes on rather
+    // than rejecting.
+    std::string compiled = StdPcmFlags(config);
+    // The base command's part: the toolchain's key covers the compiler and
+    // libc++, and for the mingw target what clang finds mingw-w64 through is
+    // the prefix, given whole or through a view (MingwSysroot), named here by
+    // the prefix rather than by the view's own path.
+    if (config.target == "x86_64-w64-mingw32" && !toolchain.mingwPrefix.empty()) {
+        compiled += std::format("|mingw:{}|{}", toolchain.mingwPrefix.string(), toolchain.mingwPrefixIsSystem ? "view" : "whole");
+    }
+    return std::format("{}-{}-{}-{:08x}", config.target, config.march, toolchain.Key(), static_cast<std::uint32_t>(std::hash<std::string>{}(compiled)));
+}
+
 fs::path Crafter::StdPcmDir(const Configuration& config) {
-    // The std PCM is cached per target+march+toolchain: a PCM is only valid
-    // for the exact clang that wrote it, so a compiler bump starts a cold
-    // directory instead of handing the new clang a PCM it will crash on. A
-    // wasm variant pass compiles it with extra codegen flags (e.g.
-    // -mrelaxed-simd) — its BMI must not be shared with the baseline's, or
-    // the consuming TUs see a target-feature mismatch. Suffix the cache dir
-    // with the variant flags when present.
-    std::string stdPcmKey = std::format("{}-{}-{}", config.target, config.march, GetToolchain().Key());
-    // The flags every wasm unit gets are compiled into the std PCM as well,
-    // and the importing units have to match them. Two releases that share a
-    // cache and a compiler, as the Flatpak's branches do, would otherwise
-    // take turns rebuilding this one std PCM, each time underneath the PCMs
-    // built against the previous one, and clang crashes on those rather than
-    // rejecting them.
-    if (config.target.starts_with("wasm32")) {
-        stdPcmKey += std::format("-{:08x}", static_cast<std::uint32_t>(std::hash<std::string_view>{}(WasmBaselineFlags)));
-    }
-    // The same goes for the mingw-w64 prefix the mingw std module is
-    // compiled against: releases up to the fix for issue #43 gave clang the
-    // prefix itself as the sysroot and the ones after give it a view of it,
-    // and two prefixes are two sets of headers.
-    if (config.target == "x86_64-w64-mingw32" && !GetToolchain().mingwPrefix.empty()) {
-        stdPcmKey += std::format("-{:08x}", static_cast<std::uint32_t>(std::hash<std::string>{}(GetToolchain().mingwPrefix.string())));
-    }
-    for (const std::string& f : config.wasmVariantFlags) {
-        stdPcmKey += "+";
-        for (char c : f) stdPcmKey += (c == '/' || c == '\\') ? '_' : c;
-    }
-    return GetCacheDir()/stdPcmKey;
+    return GetCacheDir() / StdPcmKey(config, GetToolchain());
 }
 
 std::string Crafter::WasmCompileFlags(const Configuration& config) {

implementations/Crafter.Build-Platform.cpp

@@ -790,12 +790,10 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
         const fs::path stdCc = (manifestPath.parent_path() / *source).lexically_normal();
         if (!fs::exists(stdCc)) return std::format("{}, which {} names as the std module, is missing", stdCc.string(), manifestPath.string());
 
-        // -femulated-tls keeps PCM TLS access matching libstdc++'s emutls
-        // definitions; mismatch surfaces as undefined std::__once_callable.
         // The std.cc → std.cppm copy uses a plain filename: held under the
         // cache lock, no other builder writes this path concurrently.
         fs::path stdCppm = stdPcm.parent_path() / "std.cppm";
-        std::string command = std::format("{} --target={} -march={} -mtune={} -femulated-tls -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, config.march, config.mtune, stdCppm.string(), stdPcm.string());
+        std::string command = std::format("{}{} --precompile {} -o {}", GetBaseCommand(config), StdPcmFlags(config), stdCppm.string(), stdPcm.string());
         std::string stamp = HashStamp(command + '\0' + FileBytes(stdCc));
         if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return "";
         std::error_code copyEc;
@@ -825,27 +823,11 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
             if (!source.error.empty()) return source.error;
             stdCppm = source.stdCppm.string();
         }
-        std::string sysrootFlag = config.sysroot.empty()
-            ? std::string()
-            : std::format(" --sysroot={}", config.sysroot);
-        // wasm32 rejects -march, and the std module has to be compiled the way
-        // the units importing it are, a variant pass's flags (e.g.
-        // -mrelaxed-simd) included, or the BMI is not feature-compatible.
-        std::string archFlags = isWasm
-            ? WasmCompileFlags(config)
-            : std::format(" -march={} -mtune={}", config.march, config.mtune);
-        // non-wasm cross builds: the driver ranks the host toolchain's own
-        // libc++ headers (<install>/../include/c++/v1) above --sysroot, so
-        // the PCM would mix host libc++ with the foreign libc — force the
-        // sysroot's copy.
-        if (!isWasm && !config.sysroot.empty()) {
-            archFlags += std::format(" -nostdinc++ -isystem {}/usr/include/c++/v1", config.sysroot);
-        }
         // Keyed on the command and the module source's bytes, never on
         // mtimes: an installed std.cppm can be dated at the epoch (OSTree),
         // and a libc++ update under the same clang changes the bytes but
         // not the path.
-        std::string command = std::format("{} --target={} -std=c++26{}{} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, sysrootFlag, archFlags, stdCppm, stdPcm.string());
+        std::string command = std::format("{}{} --precompile {} -o {}", GetBaseCommand(config), StdPcmFlags(config), stdCppm, stdPcm.string());
         std::string stamp = HashStamp(command + '\0' + FileBytes(stdCppm));
         if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return "";
         std::string result = RunBuildCommand(command);

interfaces/Crafter.Build-Clang.cppm

@@ -392,6 +392,15 @@ export namespace Crafter {
     };
 
     struct Configuration;
+    // The name StdPcmDir gives the directory of this configuration's std
+    // module under `toolchain`: target, march and the toolchain's key, which
+    // say what it is, then a hash of everything else it is compiled with,
+    // each flag BuildStdPcm passes (StdPcmFlags) and for the mingw target the
+    // prefix behind its --sysroot. Two configurations that would compile the
+    // std module differently never share a directory, so neither rebuilds it
+    // under the PCMs the other compiled against it. Declared ahead of
+    // Configuration, whose VariantId keys on it.
+    std::string StdPcmKey(const Configuration& config, const Toolchain& toolchain);
 
     // What a Script's callback is handed when `crafter-build run <name>`
     // starts it.
@@ -580,7 +589,8 @@ export namespace Crafter {
         void AddAstLintRule(std::string name, std::function<void(LintContext&)> check);
         // Suffix that uniquely identifies this Configuration's compile state.
         // target+march+mtune are spelled out for readability; the rest
-        // (type, debug, sysroot, defines, compileFlags, the toolchain)
+        // (type, debug, sysroot, defines, compileFlags, the std module, the
+        // toolchain)
         // collapse into a short hash so two Configurations sharing a path but
         // with different compile state can't clobber each other's outputs.
         std::string VariantId() const { return VariantId(GetToolchain()); }
@@ -628,17 +638,15 @@ export namespace Crafter {
                 compileKey += "|B:";
                 compileKey += WasmBaselineFlags;
             }
-            // The mingw-w64 prefix decides what the mingw target compiles
-            // against, and until issue #43 clang got it as the sysroot
-            // itself, which put the host's own headers on the include path.
-            // Unkeyed, the release that fixed it kept the objects compiled
-            // against glibc's headers: the build after the update only
-            // relinked them, and failed to link as before. No prefix keeps
-            // the id as it was, since clang's own discovery has not changed.
-            if (target == "x86_64-w64-mingw32" && !toolchain.mingwPrefix.empty()) {
-                compileKey += "|M:";
-                compileKey += toolchain.mingwPrefix.string();
-            }
+            // The std module the units import. Its PCMs and objects record
+            // it by path, and units do not notice it moving: when the mingw
+            // prefix stopped being clang's sysroot as is (issue #43), the
+            // std module was rebuilt and the objects compiled against the
+            // host's glibc headers were only relinked, failing as before. A
+            // std module that moves directories, because anything it is
+            // compiled with changed, moves the build with it.
+            compileKey += "|S:";
+            compileKey += StdPcmKey(*this, toolchain);
             // Sorted by ApplyStandardArgs, so flag order on the command line
             // doesn't split the cache.
             for (const std::string& a : projectArgs) {
@@ -673,10 +681,9 @@ export namespace Crafter {
         std::vector<std::string> requires_;
     };
 
-    // Directory holding the std module PCM for this Configuration. Keyed on
-    // target+march because a BMI is only loadable by a TU with matching target
-    // features, and suffixed with the wasm flags (WasmBaselineFlags, then the
-    // variant's) for the same reason.
+    // Directory holding the std module PCM for this Configuration,
+    // GetCacheDir() / StdPcmKey(config, GetToolchain()): a BMI is only
+    // loadable by a unit compiled the way it was.
     fs::path StdPcmDir(const Configuration& config);
 
     // Where Build leaves this Configuration's linked artifact: the executable
@@ -805,4 +812,9 @@ namespace Crafter {
     // compile command and of the std module's, which have to agree flag for
     // flag.
     std::string WasmCompileFlags(const Configuration& config);
+    // The flags BuildStdPcm compiles the std module with, after the base
+    // command and before the source, each with a leading space. StdPcmKey
+    // hashes the same string, so the command and the directory it writes to
+    // cannot disagree about what decides the BMI.
+    std::string StdPcmFlags(const Configuration& config);
 }

tests/VariantId/main.cpp

@@ -178,6 +178,46 @@ int main() {
         Check(mingw.VariantId(hostUsr) != mingw.VariantId(optMingw), "two mingw prefixes are two variants");
         Configuration host = MakeBase();
         Check(host.VariantId(discovered) == host.VariantId(hostUsr), "the mingw prefix leaves other targets' variants alone");
+
+        // The std module's directory is keyed the same way, and so is a
+        // prefix given whole against one reached through a view.
+        Check(StdPcmKey(mingw, discovered) != StdPcmKey(mingw, hostUsr), "a mingw prefix moves the mingw std module");
+        Check(StdPcmKey(mingw, hostUsr) != StdPcmKey(mingw, optMingw), "two mingw prefixes are two std modules");
+        Toolchain hostUsrWhole = hostUsr;
+        hostUsrWhole.mingwPrefixIsSystem = !hostUsr.mingwPrefixIsSystem;
+        Check(StdPcmKey(mingw, hostUsr) != StdPcmKey(mingw, hostUsrWhole), "a prefix given whole and one shown through a view are two std modules");
+        Check(mingw.VariantId(hostUsr) != mingw.VariantId(hostUsrWhole), "and two variants, through the std module");
+        Check(StdPcmKey(host, discovered) == StdPcmKey(host, hostUsr), "the mingw prefix leaves other targets' std modules alone");
+    }
+    {
+        // Everything the std module is compiled with keys its directory, not
+        // only target and march. -mtune was left out, so a project built
+        // with -mtune=generic shared one std PCM with project.cpp's
+        // -mtune=native, and the two rebuilt it in turn on every run, under
+        // every module built against it; two sysroots did the same.
+        Toolchain tc;
+        tc.clangxx = "/app/bin/clang++";
+        tc.clangVersion = "clang version 23.1.2";
+        Configuration native = MakeBase();
+        Configuration generic = MakeBase();
+        generic.mtune = "generic";
+        Check(StdPcmKey(native, tc) != StdPcmKey(generic, tc), "-mtune keys the std module");
+        Configuration sysrootA = MakeBase();
+        sysrootA.sysroot = "/opt/sysroot-a";
+        Configuration sysrootB = MakeBase();
+        sysrootB.sysroot = "/opt/sysroot-b";
+        Check(StdPcmKey(sysrootA, tc) != StdPcmKey(sysrootB, tc) && StdPcmKey(sysrootA, tc) != StdPcmKey(native, tc), "the sysroot keys the std module");
+        Configuration wasm = MakeBase();
+        wasm.target = "wasm32-wasip1";
+        wasm.sysroot = "/opt/wasi-sysroot";
+        Configuration relaxed = MakeBase();
+        relaxed.target = wasm.target;
+        relaxed.sysroot = wasm.sysroot;
+        relaxed.wasmVariantFlags = {"-mrelaxed-simd"};
+        Check(StdPcmKey(wasm, tc) != StdPcmKey(relaxed, tc), "a wasm variant pass's flags key the std module");
+        Check(StdPcmKey(native, tc) == StdPcmKey(MakeBase(), tc), "one configuration, one std module");
+        Check(StdPcmKey(native, tc).starts_with(std::format("{}-{}-{}-", native.target, native.march, tc.Key())), std::format("the directory still says target, march and toolchain (got '{}')", StdPcmKey(native, tc)));
+        Check(native.VariantId(tc) != generic.VariantId(tc) && sysrootA.VariantId(tc) != sysrootB.VariantId(tc), "a build moves with its std module");
     }
 
     if (Failures > 0) {