Compare commits

...
Author SHA1 Message Date
c3485b7ab1
docs: name the toolchain in the cache directory layout
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 23:18:09 +02:00
59fc9501f1
fix: key PCM and project.so freshness on content, not installed mtimes
std.pcm was rebuilt when it was older than std.cppm, and project.so when
it was older than the crafter-build executable. Both inputs are installed
files, and an OSTree deployment (the Flatpak) gives every installed file
mtime 0: std.cppm and the executable always looked older than the cache,
so a clang bump would never rebuild std.pcm and a crafter-build update
would never recompile project.so. clang crashes on a PCM from another
compiler instead of reporting it, so the failure would have been a
backtrace on the user's first build after an update.

Each artifact now has a one-line stamp beside it. std.pcm's covers the
precompile command and the module source's bytes, which also catches a
libc++ update under the same clang. project.so's covers the crafter-build
version, the toolchain key, the module source stamp and the compile
command; project.cpp's and the pre-dependency artifacts' mtimes remain
inputs, since those live next to it with real timestamps. The module
source stamp stands in for the executable: project.so resolves Crafter::
symbols from the running binary, so only declarations matter, and an
implementation-only rebuild of crafter-build no longer recompiles it.
The std.pcm stamp folds into the module cache stamp, so a rebuilt
std.pcm rebuilds the PCMs that import it.

The cache directory carries the toolchain key
(<target>-<march>-<clang version>-<hash>), so a compiler bump starts
cold instead of handing the new clang PCMs it will crash on. LoadProject
and Build() now derive the directory from the same StdPcmDir. Old
<target>-<march> directories are left behind; `clean` never touched the
cache and still does not.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 23:18:09 +02:00
d22a8603df
fix(toolchain): pass -L as well as an rpath for a toolchain outside /usr
Inside the Flatpak the bundled toolchain compiled every bootstrap object,
then glslang's configure failed its test link with `unable to find
library -lc++`. clang searches its per-target runtime directory
(lib/<triple>) and the system library paths for libc++, not
<prefix>/lib, so with the classic layout a toolchain outside /usr needs
the search path spelled out, just as its loader needs the rpath.
LibcxxLinkFlags and build.sh now emit both, the same pair an explicit
CRAFTER_BUILD_LIBCXX_DIR already got.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 23:18:07 +02:00
9 changed files with 282 additions and 76 deletions

View file

@ -107,7 +107,7 @@ import Crafter.LintRules;
<prefix>/bin/crafter-build # the executable <prefix>/bin/crafter-build # the executable
<prefix>/lib/libcrafter-build.a # static archive for downstream consumers <prefix>/lib/libcrafter-build.a # static archive for downstream consumers
<prefix>/share/crafter-build/*.cppm # module sources (rebuilt on user machine on first run) <prefix>/share/crafter-build/*.cppm # module sources (rebuilt on user machine on first run)
~/.cache/crafter.build/<target>-<march>/ # std.pcm + Crafter.Build*.pcm, built locally ~/.cache/crafter.build/<target>-<march>-<clang>-<hash>/ # std.pcm + Crafter.Build*.pcm, built locally per toolchain
``` ```
## Tests ## Tests

View file

@ -14,12 +14,13 @@ if [ -n "${CRAFTER_BUILD_LLVM_DIR:-}" ]; then
CC="$CRAFTER_BUILD_LLVM_DIR/bin/clang" CC="$CRAFTER_BUILD_LLVM_DIR/bin/clang"
AR="$CRAFTER_BUILD_LLVM_DIR/bin/llvm-ar" AR="$CRAFTER_BUILD_LLVM_DIR/bin/llvm-ar"
LLVM_CONFIG="$CRAFTER_BUILD_LLVM_DIR/bin/llvm-config" LLVM_CONFIG="$CRAFTER_BUILD_LLVM_DIR/bin/llvm-config"
# A prefix other than the system one keeps its libc++ off the loader's # A prefix other than the system one keeps its libc++ off both the
# default path; the rpath is what lets the binary start. # linker's search path (clang looks in lib/<triple> and the system dirs,
# not <prefix>/lib) and the loader's; -L and an rpath cover the two.
if [ "$CRAFTER_BUILD_LLVM_DIR" != /usr ]; then if [ "$CRAFTER_BUILD_LLVM_DIR" != /usr ]; then
RPATH_FLAG="-Wl,-rpath,$CRAFTER_BUILD_LLVM_DIR/lib" LIBCXX_LINK_FLAGS="-L$CRAFTER_BUILD_LLVM_DIR/lib -Wl,-rpath,$CRAFTER_BUILD_LLVM_DIR/lib"
else else
RPATH_FLAG="" LIBCXX_LINK_FLAGS=""
fi fi
if [ -f "$CRAFTER_BUILD_LLVM_DIR/share/libc++/v1/std.cppm" ]; then if [ -f "$CRAFTER_BUILD_LLVM_DIR/share/libc++/v1/std.cppm" ]; then
STD_CPPM="$CRAFTER_BUILD_LLVM_DIR/share/libc++/v1/std.cppm" STD_CPPM="$CRAFTER_BUILD_LLVM_DIR/share/libc++/v1/std.cppm"
@ -31,7 +32,7 @@ else
CC=clang CC=clang
AR=llvm-ar AR=llvm-ar
LLVM_CONFIG=llvm-config LLVM_CONFIG=llvm-config
RPATH_FLAG="" LIBCXX_LINK_FLAGS=""
STD_CPPM=/usr/share/libc++/v1/std.cppm STD_CPPM=/usr/share/libc++/v1/std.cppm
fi fi
@ -65,8 +66,8 @@ cmake -B build \
-DCMAKE_C_COMPILER="$CC" \ -DCMAKE_C_COMPILER="$CC" \
-DCMAKE_CXX_COMPILER="$CXX" \ -DCMAKE_CXX_COMPILER="$CXX" \
-DCMAKE_CXX_FLAGS="-stdlib=libc++" \ -DCMAKE_CXX_FLAGS="-stdlib=libc++" \
-DCMAKE_EXE_LINKER_FLAGS="-stdlib=libc++ $RPATH_FLAG" \ -DCMAKE_EXE_LINKER_FLAGS="-stdlib=libc++ $LIBCXX_LINK_FLAGS" \
-DCMAKE_SHARED_LINKER_FLAGS="-stdlib=libc++ $RPATH_FLAG" \ -DCMAKE_SHARED_LINKER_FLAGS="-stdlib=libc++ $LIBCXX_LINK_FLAGS" \
-DCMAKE_BUILD_TYPE=Release \ -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY="$current_dir/build" \ -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY="$current_dir/build" \
-DCMAKE_LIBRARY_OUTPUT_DIRECTORY="$current_dir/build" \ -DCMAKE_LIBRARY_OUTPUT_DIRECTORY="$current_dir/build" \
@ -117,7 +118,7 @@ rm -f ./lib/libcrafter-build.a
"$AR" rcs ./lib/libcrafter-build.a $OBJECTS "$AR" rcs ./lib/libcrafter-build.a $OBJECTS
"$CXX" --target="$TARGET" -std=c++26 -stdlib=libc++ -O3 -march="$MARCH" -mtune="$MTUNE" -fuse-ld=lld \ "$CXX" --target="$TARGET" -std=c++26 -stdlib=libc++ -O3 -march="$MARCH" -mtune="$MTUNE" -fuse-ld=lld \
-Wl,--export-dynamic $RPATH_FLAG \ -Wl,--export-dynamic $LIBCXX_LINK_FLAGS \
-L./build \ -L./build \
$OBJECTS \ $OBJECTS \
./build/main.o \ ./build/main.o \

View file

@ -246,11 +246,14 @@ void Configuration::GetInterfacesAndImplementations(std::span<fs::path> interfac
} }
fs::path Crafter::StdPcmDir(const Configuration& config) { fs::path Crafter::StdPcmDir(const Configuration& config) {
// The std PCM is cached per target+march, but a wasm variant pass compiles // The std PCM is cached per target+march+toolchain: a PCM is only valid
// it with extra codegen flags (e.g. -mrelaxed-simd) — its BMI must not be // for the exact clang that wrote it, so a compiler bump starts a cold
// shared with the baseline's, or the consuming TUs see a target-feature // directory instead of handing the new clang a PCM it will crash on. A
// mismatch. Suffix the cache dir with the variant flags when present. // wasm variant pass compiles it with extra codegen flags (e.g.
std::string stdPcmKey = std::format("{}-{}", config.target, config.march); // -mrelaxed-simd) — its BMI must not be shared with the baseline's, or
// the consuming TUs see a target-feature mismatch. Suffix the cache dir
// with the variant flags when present.
std::string stdPcmKey = std::format("{}-{}-{}", config.target, config.march, GetToolchain().Key());
for (const std::string& f : config.wasmVariantFlags) { for (const std::string& f : config.wasmVariantFlags) {
stdPcmKey += "+"; stdPcmKey += "+";
for (char c : f) stdPcmKey += (c == '/' || c == '\\') ? '_' : c; for (char c : f) stdPcmKey += (c == '/' || c == '\\') ? '_' : c;

View file

@ -90,15 +90,35 @@ namespace {
return cacheDir / "crafter-build-sources.stamp"; return cacheDir / "crafter-build-sources.stamp";
} }
std::string ReadCacheStamp(const fs::path& cacheDir) { // A stamp is a one-line content hash kept next to the artifact it
std::ifstream in(CacheStampPath(cacheDir), std::ios::binary); // describes. Written last, after the artifact, so an interrupted build
// leaves a stamp that disagrees and the next run starts over.
std::string ReadStamp(const fs::path& file) {
std::ifstream in(file, std::ios::binary);
std::string stamp; std::string stamp;
std::getline(in, stamp); std::getline(in, stamp);
return stamp; return stamp;
} }
void WriteCacheStamp(const fs::path& cacheDir, const std::string& stamp) { void WriteStamp(const fs::path& file, std::string_view stamp) {
std::ofstream(CacheStampPath(cacheDir), std::ios::binary | std::ios::trunc) << stamp << '\n'; std::ofstream(file, std::ios::binary | std::ios::trunc) << stamp << '\n';
}
fs::path StampPath(const fs::path& artifact) {
fs::path stamp = artifact;
stamp += ".stamp";
return stamp;
}
std::string HashStamp(std::string_view content) {
return std::format("{:016x}", std::hash<std::string_view>{}(content));
}
std::string FileBytes(const fs::path& file) {
std::ifstream in(file, std::ios::binary);
std::ostringstream buffer;
buffer << in.rdbuf();
return buffer.str();
} }
} }
@ -495,12 +515,17 @@ namespace {
return out; return out;
} }
// project.so predates its source, the crafter-build that will load it, or // project.so is stale when its stamp (crafter-build version, toolchain,
// a pre-dependency artifact — any of them means recompiling. // module sources and compile command) disagrees, or when it predates its
bool ProjectFileStale(const fs::path& artifact, const fs::path& project, const fs::path& hostExe, const ProjectFileDependencies& deps) { // source or a pre-dependency artifact. The executable's mtime is not an
// input any more: an OSTree deployment dates every installed file at the
// epoch, so a comparison against it never fired after an update and the
// old project.so was loaded into a binary it did not match.
bool ProjectFileStale(const fs::path& artifact, const fs::path& project, std::string_view stamp, const ProjectFileDependencies& deps) {
if (!fs::exists(artifact)) return true; if (!fs::exists(artifact)) return true;
if (ReadStamp(StampPath(artifact)) != stamp) return true;
const fs::file_time_type built = fs::last_write_time(artifact); const fs::file_time_type built = fs::last_write_time(artifact);
return built < fs::last_write_time(project) || built < fs::last_write_time(hostExe) || built < deps.newestArtifact; return built < fs::last_write_time(project) || built < deps.newestArtifact;
} }
} }
@ -611,21 +636,22 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
fs::path stdCc = GetToolchain().mingwSysroot / "include" / "c++" / mingWversion / "bits" / "std.cc"; fs::path stdCc = GetToolchain().mingwSysroot / "include" / "c++" / mingWversion / "bits" / "std.cc";
if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time(stdCc)) {
// -femulated-tls keeps PCM TLS access matching libstdc++'s emutls // -femulated-tls keeps PCM TLS access matching libstdc++'s emutls
// definitions; mismatch surfaces as undefined std::__once_callable. // definitions; mismatch surfaces as undefined std::__once_callable.
// The std.cc → std.cppm copy uses a plain filename: held under the // The std.cc → std.cppm copy uses a plain filename: held under the
// cache lock, no other builder writes this path concurrently. // cache lock, no other builder writes this path concurrently.
fs::path stdCppm = stdPcm.parent_path() / "std.cppm"; fs::path stdCppm = stdPcm.parent_path() / "std.cppm";
std::string command = std::format("{} --target={} -march={} -mtune={} -femulated-tls -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, config.march, config.mtune, stdCppm.string(), stdPcm.string());
std::string stamp = HashStamp(command + '\0' + FileBytes(stdCc));
if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return "";
std::error_code copyEc; std::error_code copyEc;
fs::copy_file(stdCc, stdCppm, fs::copy_options::overwrite_existing, copyEc); fs::copy_file(stdCc, stdCppm, fs::copy_options::overwrite_existing, copyEc);
if (copyEc) { if (copyEc) {
return std::format("copy {} -> {}: {}", stdCc.string(), stdCppm.string(), copyEc.message()); return std::format("copy {} -> {}: {}", stdCc.string(), stdCppm.string(), copyEc.message());
} }
return RunBuildCommand(std::format("{} --target={} -march={} -mtune={} -femulated-tls -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, config.march, config.mtune, stdCppm.string(), stdPcm.string())); std::string result = RunBuildCommand(command);
} else { if (result.empty()) WriteStamp(StampPath(stdPcm), stamp);
return ""; return result;
}
} else { } else {
const bool isWasm = config.target.starts_with("wasm32"); const bool isWasm = config.target.starts_with("wasm32");
// wasi-sdk drops std.cppm at <sysroot>/share/libc++/v1/, the rest of // wasi-sdk drops std.cppm at <sysroot>/share/libc++/v1/, the rest of
@ -669,11 +695,16 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
if (!isWasm && !config.sysroot.empty()) { if (!isWasm && !config.sysroot.empty()) {
archFlags += std::format(" -nostdinc++ -isystem {}/usr/include/c++/v1", config.sysroot); archFlags += std::format(" -nostdinc++ -isystem {}/usr/include/c++/v1", config.sysroot);
} }
if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time(stdCppm)) { // Keyed on the command and the module source's bytes, never on
return RunBuildCommand(std::format("{} --target={} -std=c++26{}{} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, sysrootFlag, archFlags, stdCppm, stdPcm.string())); // mtimes: an installed std.cppm can be dated at the epoch (OSTree),
} else { // and a libc++ update under the same clang changes the bytes but
return ""; // not the path.
} std::string command = std::format("{} --target={} -std=c++26{}{} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, sysrootFlag, archFlags, stdCppm, stdPcm.string());
std::string stamp = HashStamp(command + '\0' + FileBytes(stdCppm));
if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return "";
std::string result = RunBuildCommand(command);
if (result.empty()) WriteStamp(StampPath(stdPcm), stamp);
return result;
} }
} }
@ -721,12 +752,17 @@ namespace {
"Crafter.Build", "Crafter.Build",
}; };
void EnsureCrafterBuildPcms(const fs::path& sourceDir, const fs::path& cacheDir, const Configuration& host) { // Returns the stamp of the module sources the cache was built from; it is
// one of project.so's inputs.
std::string EnsureCrafterBuildPcms(const fs::path& sourceDir, const fs::path& cacheDir, const Configuration& host) {
CacheLock lock(cacheDir); CacheLock lock(cacheDir);
// Re-derived under the lock: another builder may have refreshed the // Re-derived under the lock: another builder may have refreshed the
// cache from its own sources while we waited. // cache from its own sources while we waited.
std::string stamp = CrafterBuildSourceStamp(sourceDir, CrafterBuildModules); // The std PCM is an input to every module PCM: its stamp is folded in
const bool upToDate = ReadCacheStamp(cacheDir) == stamp; // so a rebuilt std.pcm (a libc++ update under the same clang) rebuilds
// them too, instead of leaving PCMs that import a std.pcm that is gone.
std::string stamp = HashStamp(CrafterBuildSourceStamp(sourceDir, CrafterBuildModules) + '\0' + ReadStamp(StampPath(cacheDir / "std.pcm")));
const bool upToDate = ReadStamp(CacheStampPath(cacheDir)) == stamp;
for (std::string_view name : CrafterBuildModules) { for (std::string_view name : CrafterBuildModules) {
fs::path cppmPath = sourceDir / std::format("{}.cppm", name); fs::path cppmPath = sourceDir / std::format("{}.cppm", name);
fs::path pcmPath = cacheDir / std::format("{}.pcm", name); fs::path pcmPath = cacheDir / std::format("{}.pcm", name);
@ -751,7 +787,8 @@ namespace {
// Last, so an interrupted rebuild leaves the stamp disagreeing with the // Last, so an interrupted rebuild leaves the stamp disagreeing with the
// sources and the next run starts over rather than trusting a half-built // sources and the next run starts over rather than trusting a half-built
// set of PCMs. // set of PCMs.
if (!upToDate) WriteCacheStamp(cacheDir, stamp); if (!upToDate) WriteStamp(CacheStampPath(cacheDir), stamp);
return stamp;
} }
} }
@ -765,8 +802,6 @@ Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const
} }
fs::path soPath = buildDir / std::format("{}.so", absProject.stem().string()); fs::path soPath = buildDir / std::format("{}.so", absProject.stem().string());
fs::path hostExe = fs::read_symlink("/proc/self/exe");
fs::path sourceDir = Crafter::GetCrafterBuildHome(); fs::path sourceDir = Crafter::GetCrafterBuildHome();
// The host toolchain's own triple (x86_64-pc-linux-gnu on glibc distros, // The host toolchain's own triple (x86_64-pc-linux-gnu on glibc distros,
@ -776,7 +811,9 @@ Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const
hostConfig.target = HostTarget(); hostConfig.target = HostTarget();
hostConfig.march = "native"; hostConfig.march = "native";
hostConfig.mtune = "native"; hostConfig.mtune = "native";
fs::path cacheDir = GetCacheDir() / std::format("{}-{}", hostConfig.target, hostConfig.march); // The same directory Build() resolves the std PCM from, toolchain key
// included (see StdPcmDir).
fs::path cacheDir = StdPcmDir(hostConfig);
if (!fs::exists(cacheDir)) fs::create_directories(cacheDir); if (!fs::exists(cacheDir)) fs::create_directories(cacheDir);
std::string stdResult = BuildStdPcm(hostConfig, cacheDir / "std.pcm"); std::string stdResult = BuildStdPcm(hostConfig, cacheDir / "std.pcm");
@ -784,7 +821,7 @@ Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const
throw std::runtime_error(std::format("Failed to build std.pcm: {}", stdResult)); throw std::runtime_error(std::format("Failed to build std.pcm: {}", stdResult));
} }
EnsureCrafterBuildPcms(sourceDir, cacheDir, hostConfig); const std::string sourcesStamp = EnsureCrafterBuildPcms(sourceDir, cacheDir, hostConfig);
// After the host PCMs exist (a dependency's build imports Crafter.Build // After the host PCMs exist (a dependency's build imports Crafter.Build
// from that cache) and before the staleness check (its artifacts are // from that cache) and before the staleness check (its artifacts are
@ -793,9 +830,6 @@ Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const
// clang crashes on it. // clang crashes on it.
ProjectFileDependencies preDeps = ResolveProjectFileDependencies(absProject, hostConfig); ProjectFileDependencies preDeps = ResolveProjectFileDependencies(absProject, hostConfig);
const bool stale = ProjectFileStale(soPath, absProject, hostExe, preDeps);
if (stale) {
// -fuse-ld=lld like every other link the tool performs: lld is a hard // -fuse-ld=lld like every other link the tool performs: lld is a hard
// dependency, binutils' ld is not (Alpine's clang would default to it), // dependency, binutils' ld is not (Alpine's clang would default to it),
// and a dependency archive built in release holds ThinLTO bitcode, // and a dependency archive built in release holds ThinLTO bitcode,
@ -808,11 +842,18 @@ Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const
"{} -o {}{}{}", "{} -o {}{}{}",
GetBaseCommand(hostConfig), hostConfig.target, cacheDir.string(), preDeps.compileFlags, GetBaseCommand(hostConfig), hostConfig.target, cacheDir.string(), preDeps.compileFlags,
absProject.string(), soPath.string(), preDeps.linkFlags, GetToolchain().LibcxxLinkFlags()); absProject.string(), soPath.string(), preDeps.linkFlags, GetToolchain().LibcxxLinkFlags());
// What project.so was compiled against. The module source stamp stands
// in for the executable: project.so resolves Crafter:: symbols from the
// running binary, so only the declarations (the .cppm files) matter,
// and an implementation-only rebuild of crafter-build leaves it valid.
const std::string projectStamp = HashStamp(std::format("{}\0{}\0{}\0{}", CrafterBuildVersion(), GetToolchain().Key(), sourcesStamp, compileCmd));
if (ProjectFileStale(soPath, absProject, projectStamp, preDeps)) {
std::string result = RunBuildCommand(compileCmd); std::string result = RunBuildCommand(compileCmd);
if (!result.empty()) { if (!result.empty()) {
throw std::runtime_error(std::format("Failed to compile project {}: {}", absProject.string(), result)); throw std::runtime_error(std::format("Failed to compile project {}: {}", absProject.string(), result));
} }
WriteStamp(StampPath(soPath), projectStamp);
} }
void* handle = dlopen(soPath.c_str(), RTLD_NOW | RTLD_GLOBAL); void* handle = dlopen(soPath.c_str(), RTLD_NOW | RTLD_GLOBAL);

View file

@ -51,14 +51,17 @@ std::string Toolchain::LibcxxCompileFlags() const {
} }
std::string Toolchain::LibcxxLinkFlags() const { std::string Toolchain::LibcxxLinkFlags() const {
std::string flags; // clang searches its per-target runtime directory and the system library
// paths for -lc++, not <prefix>/lib, so a toolchain outside /usr with the
// classic layout needs the search path as well as the run-time path.
fs::path lib;
if (!libcxxDir.empty()) { if (!libcxxDir.empty()) {
fs::path lib = libcxxDir / "lib"; lib = libcxxDir / "lib";
flags += std::format(" -L{} -Wl,-rpath,{}", lib.string(), lib.string());
} else if (!prefix.empty() && prefix != "/usr") { } else if (!prefix.empty() && prefix != "/usr") {
flags += std::format(" -Wl,-rpath,{}", (prefix / "lib").string()); lib = prefix / "lib";
} }
return flags; if (lib.empty()) return {};
return std::format(" -L{} -Wl,-rpath,{}", lib.string(), lib.string());
} }
std::string Toolchain::Key() const { std::string Toolchain::Key() const {

View file

@ -44,10 +44,12 @@ export namespace Crafter {
// Flags for compiling against libcxxDir; empty without one. // Flags for compiling against libcxxDir; empty without one.
std::string LibcxxCompileFlags() const; std::string LibcxxCompileFlags() const;
// Flags for linking against the libc++ the compiler will use: an // Flags for linking against the libc++ the compiler will use: a -L
// rpath into the prefix when that is not the system one (a bundled // and an rpath for <prefix>/lib when the prefix is not the system one
// toolchain's libc++.so.1 is not on the loader's default path), plus // (clang searches only its per-target runtime directory and the
// -L and an rpath for an explicit libcxxDir. // system paths, and a bundled toolchain's libc++.so.1 is on neither
// the linker's nor the loader's default path), or for an explicit
// libcxxDir.
std::string LibcxxLinkFlags() const; std::string LibcxxLinkFlags() const;
// Names this toolchain for cache directory names: the clang version // Names this toolchain for cache directory names: the clang version
// followed by a hash of what decides which headers and libraries a // followed by a hash of what decides which headers and libraries a

View file

@ -142,6 +142,8 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
// Same LoadProject wiring as ConcurrentCacheRace above. // Same LoadProject wiring as ConcurrentCacheRace above.
cfg.AddTest("HostCacheSourceStamp").Dependencies({ CrafterBuildLib.get() }) cfg.AddTest("HostCacheSourceStamp").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl"); .LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
cfg.AddTest("HostCacheToolchainKey").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
cfg.AddTest("ProjectDependency").Dependencies({ CrafterBuildLib.get() }) cfg.AddTest("ProjectDependency").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl"); .LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
cfg.AddTest("ConcurrentDependencyReset").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ConcurrentDependencyReset").Dependencies({ CrafterBuildLib.get() });

View file

@ -0,0 +1,154 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
#include <stdlib.h>
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// Freshness of the std PCM and of project.so used to be an mtime comparison
// against files that come with the install: std.cppm and the crafter-build
// executable. A Flatpak (any OSTree deployment) gives every installed file
// mtime 0, so those inputs always looked older than the cache: a clang bump
// would never rebuild std.pcm, a crafter-build update would never recompile
// project.so, and clang crashes on the stale PCM instead of saying so.
//
// Both are now keyed on content stamps (the compile command, the std module
// source, the crafter-build version, the toolchain key and the module source
// stamp) and the cache directory carries the toolchain key in its name. This
// test drives the shapes an mtime rule gets wrong: a std.pcm dated before its
// source must still be reused, a warm load must leave project.so alone, and a
// stamp that disagrees must rebuild even when the artifact is dated after
// every input.
namespace {
std::int32_t Failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++Failures;
}
}
fs::path FindModuleCacheDir(const fs::path& cacheRoot) {
for (const fs::directory_entry& entry : fs::directory_iterator(cacheRoot / "crafter.build")) {
if (entry.is_directory()) return entry.path();
}
return {};
}
bool Load(const fs::path& projectFile, std::string_view what) {
std::array<std::string_view, 0> noArgs = {};
try {
(void)LoadProject(projectFile, noArgs);
return true;
} catch (const std::exception& e) {
std::println(std::cerr, "FAIL: LoadProject ({}) threw: {}", what, e.what());
++Failures;
return false;
}
}
fs::path StampOf(const fs::path& artifact) {
fs::path stamp = artifact;
stamp += ".stamp";
return stamp;
}
}
int main() {
fs::path installedHome = GetCrafterBuildHome();
if (!fs::exists(installedHome / "Crafter.Build.cppm")) {
std::println(std::cerr, "SKIP: no module sources at {}", installedHome.string());
return 77;
}
fs::path scratch = fs::temp_directory_path() / "crafter-build-host-cache-toolchain-key";
std::error_code ec;
fs::remove_all(scratch, ec);
fs::create_directories(scratch);
// A private copy of the module sources and a cold private cache, as in
// HostCacheSourceStamp.
fs::path home = scratch / "share" / "crafter-build";
fs::create_directories(home.parent_path());
fs::copy(installedHome, home, fs::copy_options::recursive);
setenv("CRAFTER_BUILD_HOME", home.string().c_str(), 1);
fs::path cacheRoot = scratch / "cache";
fs::create_directories(cacheRoot);
setenv("XDG_CACHE_HOME", cacheRoot.string().c_str(), 1);
fs::path projectDir = scratch / "project";
fs::create_directories(projectDir);
fs::path projectFile = projectDir / "project.cpp";
std::ofstream(projectFile) <<
"import std;\n"
"import Crafter.Build;\n"
"using namespace Crafter;\n"
"extern \"C\" Configuration CrafterBuildProject(std::span<const std::string_view>) {\n"
" Configuration cfg;\n"
" cfg.path = \"./\";\n"
" cfg.name = \"keyed\";\n"
" cfg.outputName = \"keyed\";\n"
" cfg.type = ConfigurationType::Executable;\n"
" return cfg;\n"
"}\n";
if (!Load(projectFile, "cold")) return 1;
const fs::path cacheDir = FindModuleCacheDir(cacheRoot);
Check(!cacheDir.empty(), "a module cache directory was created");
if (cacheDir.empty()) return 1;
Check(cacheDir.filename().string().contains(GetToolchain().Key()), std::format("the cache directory is named for the toolchain (got '{}', key '{}')", cacheDir.filename().string(), GetToolchain().Key()));
const fs::path stdPcm = cacheDir / "std.pcm";
const fs::path projectSo = projectDir / "build" / "project.so";
Check(fs::exists(stdPcm) && fs::exists(StampOf(stdPcm)), "std.pcm and its stamp exist after a cold load");
Check(fs::exists(projectSo) && fs::exists(StampOf(projectSo)), "project.so and its stamp exist after a cold load");
if (!fs::exists(StampOf(stdPcm)) || !fs::exists(StampOf(projectSo))) return 1;
// The OSTree shape, inverted: std.pcm dated before its source. A stamp
// rule reuses it; an mtime rule would rebuild it on every run. project.so
// keeps two mtime inputs that live next to it (project.cpp and dependency
// artifacts) but the executable is no longer one of them, so a warm load
// with nothing changed leaves it alone.
const fs::file_time_type epoch{};
fs::last_write_time(stdPcm, epoch);
const fs::file_time_type afterCold = fs::last_write_time(projectSo);
if (!Load(projectFile, "std.pcm dated at the epoch")) return 1;
Check(fs::last_write_time(stdPcm) == epoch, "std.pcm dated before its source is still reused");
Check(fs::last_write_time(projectSo) == afterCold, "project.so is left alone when nothing changed");
// The other half: artifacts dated after every input, with a stamp that
// disagrees. An mtime rule keeps them; the stamp rule rebuilds.
const fs::file_time_type future = fs::file_time_type::clock::now() + std::chrono::hours(24);
std::ofstream(StampOf(stdPcm)) << "stale";
fs::last_write_time(stdPcm, future);
if (!Load(projectFile, "stale std.pcm stamp")) return 1;
Check(fs::last_write_time(stdPcm) != future, "a disagreeing stamp rebuilds std.pcm even when it is dated after its source");
{
std::ifstream in(StampOf(stdPcm));
std::string stamp;
std::getline(in, stamp);
Check(stamp != "stale" && !stamp.empty(), "the rebuilt std.pcm gets its stamp back");
}
std::ofstream(StampOf(projectSo)) << "stale";
fs::last_write_time(projectSo, future);
if (!Load(projectFile, "stale project.so stamp")) return 1;
Check(fs::last_write_time(projectSo) != future, "a disagreeing stamp recompiles project.so even when it is dated after its inputs");
// The one mtime input that legitimately remains: the project file itself.
const fs::file_time_type beforeEdit = fs::last_write_time(projectSo);
{
std::ofstream append(projectFile, std::ios::app);
append << "// edited\n";
}
fs::last_write_time(projectFile, fs::file_time_type::clock::now() + std::chrono::seconds(2));
if (!Load(projectFile, "edited project.cpp")) return 1;
Check(fs::last_write_time(projectSo) != beforeEdit, "an edited project.cpp recompiles project.so");
fs::remove_all(scratch, ec);
return Failures == 0 ? 0 : 1;
}

View file

@ -73,7 +73,7 @@ int main() {
Check(tc.stdCppm == full / "share/libc++/v1/std.cppm", "the prefix's std module source is used"); Check(tc.stdCppm == full / "share/libc++/v1/std.cppm", "the prefix's std module source is used");
Check(tc.wasiSysroot == full / "share/wasi-sysroot", "the prefix's wasi sysroot is used"); Check(tc.wasiSysroot == full / "share/wasi-sysroot", "the prefix's wasi sysroot is used");
Check(tc.LibcxxCompileFlags().empty(), "the compiler finds its own libc++ headers without flags"); Check(tc.LibcxxCompileFlags().empty(), "the compiler finds its own libc++ headers without flags");
Check(Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (full / "lib").string())), "binaries get an rpath into the prefix's lib dir"); Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (full / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (full / "lib").string())), "binaries get -L and an rpath for the prefix's lib dir");
} }
{ {
ToolchainInputs in; ToolchainInputs in;
@ -95,7 +95,7 @@ int main() {
Check(tc.llvmIncludeDir.empty(), "no clang-c headers in the prefix: nothing added"); Check(tc.llvmIncludeDir.empty(), "no clang-c headers in the prefix: nothing added");
Check(tc.stdCppm == "/usr/share/libc++/v1/std.cppm", "no std module in the prefix: the system copy"); Check(tc.stdCppm == "/usr/share/libc++/v1/std.cppm", "no std module in the prefix: the system copy");
Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "no wasi sysroot in the prefix: the system one"); Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "no wasi sysroot in the prefix: the system one");
Check(Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (bare / "lib").string())), "the rpath follows the explicit prefix"); Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (bare / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (bare / "lib").string())), "the link flags follow the explicit prefix");
} }
{ {
ToolchainInputs in; ToolchainInputs in;