Catcrafts/Crafter.Build
Watch
2
0
Fork
You've already forked Crafter.Build
 0
Code Issues 3 Pull requests Projects Releases 4 Packages Wiki Activity Actions

fix: key PCM and project.so freshness on content, not installed mtimes

Browse source 
std.pcm was rebuilt when it was older than std.cppm, and project.so when
it was older than the crafter-build executable. Both inputs are installed
files, and an OSTree deployment (the Flatpak) gives every installed file
mtime 0: std.cppm and the executable always looked older than the cache,
so a clang bump would never rebuild std.pcm and a crafter-build update
would never recompile project.so. clang crashes on a PCM from another
compiler instead of reporting it, so the failure would have been a
backtrace on the user's first build after an update.

Each artifact now has a one-line stamp beside it. std.pcm's covers the
precompile command and the module source's bytes, which also catches a
libc++ update under the same clang. project.so's covers the crafter-build
version, the toolchain key, the module source stamp and the compile
command; project.cpp's and the pre-dependency artifacts' mtimes remain
inputs, since those live next to it with real timestamps. The module
source stamp stands in for the executable: project.so resolves Crafter::
symbols from the running binary, so only declarations matter, and an
implementation-only rebuild of crafter-build no longer recompiles it.
The std.pcm stamp folds into the module cache stamp, so a rebuilt
std.pcm rebuilds the PCMs that import it.

The cache directory carries the toolchain key
(<target>-<march>-<clang version>-<hash>), so a compiler bump starts
cold instead of handing the new clang PCMs it will crash on. LoadProject
and Build() now derive the directory from the same StdPcmDir. Old
<target>-<march> directories are left behind; `clean` never touched the
cache and still does not.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jorijn van der Graaf 2026-10-02 22:32:35 +02:00
commit 59fc9501f1
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
4 changed files with 256 additions and 56 deletions
Download patch file Download diff file Expand all files Collapse all files

154
tests/HostCacheToolchainKey/main.cpp Normal file
View file

@ -0,0 +1,154 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
#include <stdlib.h>
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// Freshness of the std PCM and of project.so used to be an mtime comparison
// against files that come with the install: std.cppm and the crafter-build
// executable. A Flatpak (any OSTree deployment) gives every installed file
// mtime 0, so those inputs always looked older than the cache: a clang bump
// would never rebuild std.pcm, a crafter-build update would never recompile
// project.so, and clang crashes on the stale PCM instead of saying so.
//
// Both are now keyed on content stamps (the compile command, the std module
// source, the crafter-build version, the toolchain key and the module source
// stamp) and the cache directory carries the toolchain key in its name. This
// test drives the shapes an mtime rule gets wrong: a std.pcm dated before its
// source must still be reused, a warm load must leave project.so alone, and a
// stamp that disagrees must rebuild even when the artifact is dated after
// every input.
namespace {
std::int32_t Failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++Failures;
}
}
fs::path FindModuleCacheDir(const fs::path& cacheRoot) {
for (const fs::directory_entry& entry : fs::directory_iterator(cacheRoot / "crafter.build")) {
if (entry.is_directory()) return entry.path();
}
return {};
}
bool Load(const fs::path& projectFile, std::string_view what) {
std::array<std::string_view, 0> noArgs = {};
try {
(void)LoadProject(projectFile, noArgs);
return true;
} catch (const std::exception& e) {
std::println(std::cerr, "FAIL: LoadProject ({}) threw: {}", what, e.what());
++Failures;
return false;
}
}
fs::path StampOf(const fs::path& artifact) {
fs::path stamp = artifact;
stamp += ".stamp";
return stamp;
}
}
int main() {
fs::path installedHome = GetCrafterBuildHome();
if (!fs::exists(installedHome / "Crafter.Build.cppm")) {
std::println(std::cerr, "SKIP: no module sources at {}", installedHome.string());
return 77;
}
fs::path scratch = fs::temp_directory_path() / "crafter-build-host-cache-toolchain-key";
std::error_code ec;
fs::remove_all(scratch, ec);
fs::create_directories(scratch);
// A private copy of the module sources and a cold private cache, as in
// HostCacheSourceStamp.
fs::path home = scratch / "share" / "crafter-build";
fs::create_directories(home.parent_path());
fs::copy(installedHome, home, fs::copy_options::recursive);
setenv("CRAFTER_BUILD_HOME", home.string().c_str(), 1);
fs::path cacheRoot = scratch / "cache";
fs::create_directories(cacheRoot);
setenv("XDG_CACHE_HOME", cacheRoot.string().c_str(), 1);
fs::path projectDir = scratch / "project";
fs::create_directories(projectDir);
fs::path projectFile = projectDir / "project.cpp";
std::ofstream(projectFile) <<
"import std;\n"
"import Crafter.Build;\n"
"using namespace Crafter;\n"
"extern \"C\" Configuration CrafterBuildProject(std::span<const std::string_view>) {\n"
" Configuration cfg;\n"
" cfg.path = \"./\";\n"
" cfg.name = \"keyed\";\n"
" cfg.outputName = \"keyed\";\n"
" cfg.type = ConfigurationType::Executable;\n"
" return cfg;\n"
"}\n";
if (!Load(projectFile, "cold")) return 1;
const fs::path cacheDir = FindModuleCacheDir(cacheRoot);
Check(!cacheDir.empty(), "a module cache directory was created");
if (cacheDir.empty()) return 1;
Check(cacheDir.filename().string().contains(GetToolchain().Key()), std::format("the cache directory is named for the toolchain (got '{}', key '{}')", cacheDir.filename().string(), GetToolchain().Key()));
const fs::path stdPcm = cacheDir / "std.pcm";
const fs::path projectSo = projectDir / "build" / "project.so";
Check(fs::exists(stdPcm) && fs::exists(StampOf(stdPcm)), "std.pcm and its stamp exist after a cold load");
Check(fs::exists(projectSo) && fs::exists(StampOf(projectSo)), "project.so and its stamp exist after a cold load");
if (!fs::exists(StampOf(stdPcm)) || !fs::exists(StampOf(projectSo))) return 1;
// The OSTree shape, inverted: std.pcm dated before its source. A stamp
// rule reuses it; an mtime rule would rebuild it on every run. project.so
// keeps two mtime inputs that live next to it (project.cpp and dependency
// artifacts) but the executable is no longer one of them, so a warm load
// with nothing changed leaves it alone.
const fs::file_time_type epoch{};
fs::last_write_time(stdPcm, epoch);
const fs::file_time_type afterCold = fs::last_write_time(projectSo);
if (!Load(projectFile, "std.pcm dated at the epoch")) return 1;
Check(fs::last_write_time(stdPcm) == epoch, "std.pcm dated before its source is still reused");
Check(fs::last_write_time(projectSo) == afterCold, "project.so is left alone when nothing changed");
// The other half: artifacts dated after every input, with a stamp that
// disagrees. An mtime rule keeps them; the stamp rule rebuilds.
const fs::file_time_type future = fs::file_time_type::clock::now() + std::chrono::hours(24);
std::ofstream(StampOf(stdPcm)) << "stale";
fs::last_write_time(stdPcm, future);
if (!Load(projectFile, "stale std.pcm stamp")) return 1;
Check(fs::last_write_time(stdPcm) != future, "a disagreeing stamp rebuilds std.pcm even when it is dated after its source");
{
std::ifstream in(StampOf(stdPcm));
std::string stamp;
std::getline(in, stamp);
Check(stamp != "stale" && !stamp.empty(), "the rebuilt std.pcm gets its stamp back");
}
std::ofstream(StampOf(projectSo)) << "stale";
fs::last_write_time(projectSo, future);
if (!Load(projectFile, "stale project.so stamp")) return 1;
Check(fs::last_write_time(projectSo) != future, "a disagreeing stamp recompiles project.so even when it is dated after its inputs");
// The one mtime input that legitimately remains: the project file itself.
const fs::file_time_type beforeEdit = fs::last_write_time(projectSo);
{
std::ofstream append(projectFile, std::ios::app);
append << "// edited\n";
}
fs::last_write_time(projectFile, fs::file_time_type::clock::now() + std::chrono::seconds(2));
if (!Load(projectFile, "edited project.cpp")) return 1;
Check(fs::last_write_time(projectSo) != beforeEdit, "an edited project.cpp recompiles project.so");
fs::remove_all(scratch, ec);
return Failures == 0 ? 0 : 1;
}