Compare commits

...
Author SHA1 Message Date
4e5d17b9e9
docs: describe how a sysroot build finds its std module source
All checks were successful
Flatpak / flatpak (push) Successful in 1h30m57s
The README said a cross build precompiles the sysroot's own std.cppm,
which was never the whole story for sysroots that do not ship one. It
now states the order (the sysroot's, the toolchain's for the same major
release, the sources crafter-build carries for other releases), what
the failure looks like when nothing matches and how to resolve it, and
lists the new directory in the install layout.

AGENTS.md records the rule for the next LLVM bump: the release the
bundled toolchain leaves behind stays available as a module-source
module, because stable distros remain on it long after.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 17:01:21 +02:00
4159ee4117
feat(packaging): ship libc++ 22's std module source for sysroots
With LLVM 23 bundled, a sysroot still on libc++ 22 without its own
module source (Alpine v3.24) is the mirror image of the case that
started this: the toolchain's source is a release ahead of the
sysroot's headers. The new libcxx-22-modules module installs libc++
22.1.8's module source under share/crafter-build/libc++-modules/22,
where SysrootStdModule looks for the release a sysroot is on.

It only configures libc++'s runtimes build and installs the cxx-modules
component: std.cppm as configure generates it and the .inc files from
the source tree, nothing compiled, about 600 KB installed. The
component is staged and copied because it also writes
lib/libc++.modules.json, which belongs to the bundled libc++.

Verified in the built Flatpak: `crafter-build --version` lists release
22, and a cross build against an Alpine v3.24 aarch64 sysroot (libc++
22.1.3) precompiles that source with clang 23, links and runs under
qemu.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 17:01:21 +02:00
8dccb88b78
feat(packaging): bundle LLVM 23.1.2
LLVM 23 has been the stable series since September, and the distros
crafter-build cross-builds for have moved to it: Alpine edge ships
libc++ 23.1.2, which is what surfaced the std module mismatch fixed in
the previous commits. wasi-sdk 34 already builds against a newer LLVM
than 22 upstream, so the WASI sysroot loses nothing by following.

The bump exposed a patch step that never ran. wasi-sdk patches its LLVM
tree with `git apply` in src/llvm-project, which upstream is a
submodule. Here that directory is a tarball unpacked inside the
wasi-sdk checkout, and git, finding the outer repository, treats every
patched path as outside the current directory, prints "Skipped patch"
and exits 0. LLVM 22 built without the three patches; libc++ 23 does
not compile for WASI without the one that keeps the z/OS locale shim
out of other platforms' builds (strtod_l redeclared with an abi_tag).
`git init` on the unpacked tree makes it a repository of its own, so
wasi-sdk's step applies whatever patch list the pinned tag carries, as
it does upstream, and fails loudly when a patch no longer fits.

Built locally: the whole manifest, the suite inside the sandbox (26
passed, WasmHelloRuns under the bundled wasmtime included), and aarch64
musl cross builds against Alpine edge and v3.24 sysroots that run under
qemu. The native development loop is unaffected and stays on whatever
clang the machine has.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 17:01:21 +02:00
cbf2b94f06
test(sysroot): cover the std module source choice for a sysroot build
SysrootStdModule runs over staged layouts that need no compiler: a
sysroot with its own module source, one on a point release of the
toolchain's libc++, one on the next major release with a shipped
source, one nothing was shipped for, one without libc++ at all, and the
two cases where a release cannot be read. The last block drives Build()
against a sysroot claiming libc++ 99 and expects the one-line
explanation back instead of compiler output, which is the failure the
fix replaces.

ToolchainResolve additionally checks that the release is read from the
libc++ the module source comes from (the prefix's, an explicit libc++
install's, the system's) and where the shipped sources are looked up.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
e15cd3c14d
fix(sysroot): take the std module source from the sysroot's libc++ release
A cross build against a sysroot whose libc++ differs from the
toolchain's failed while precompiling the std module, with a page of
"no member named 'runtime_format' in namespace 'std'". The module source
is a list of using-declarations for what one libc++ release declares, so
it only compiles against that release's headers. BuildStdPcm wants the
sysroot's own <sysroot>/usr/share/libc++/v1/std.cppm, but Alpine
packages the headers without it, and the fallback was the toolchain's
copy whatever its release. That held while the compiler came from the
same distro as the sysroot; with the Flatpak's libc++ 22.1.8 and Alpine
edge on 23.1.2 it no longer does.

The release is now read from _LIBCPP_VERSION in the sysroot's and the
toolchain's <__config>, and SysrootStdModule picks in this order: the
sysroot's own source, the toolchain's when both are the same major
release (a release branch only takes fixes), the copy crafter-build
ships for that release under share/crafter-build/libc++-modules/<major>.
When none fits the build stops with one line naming both releases and
the directory the matching source belongs in, before clang is run. If
either release cannot be read the toolchain's source is tried as
before, since there is nothing to compare.

Matching sources live in crafter-build's own share directory rather
than the LLVM prefix: they are data this tool ships for sysroots, not
part of any one compiler install, and "sources ship" already describes
that directory. `--version` prints the toolchain's libc++ release and
the releases sources are shipped for, the first thing to look at when
such a build fails.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
adc91ed9e2
fix(packaging): keep the per-commit version out of every module's cache key
flatpak-builder printed "Building module llvm" on every CI run instead
of "Cache hit". The build environment, CRAFTER_BUILD_VERSION included,
sat in the manifest's top-level build options, and those are part of
every module's cache key; the version changes with each commit, so no
module ever hit. Runs still took twelve minutes rather than eighty only
because ccache made the LLVM rebuild cheap.

The variables only matter to crafter-build's own module, so they move
there, and CI rewrites the version on that module. LLVM, the WASI
sysroot and wasmtime are now keyed on nothing that changes between
commits.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:04:43 +02:00
11 changed files with 340 additions and 31 deletions

View file

@ -99,7 +99,7 @@ jobs:
VERSION: ${{ steps.meta.outputs.version }} VERSION: ${{ steps.meta.outputs.version }}
run: | run: |
set -eu set -eu
yq '."build-options".env.CRAFTER_BUILD_VERSION = strenv(VERSION)' "$MANIFEST" > "$CI_MANIFEST" yq '(.modules[] | select(.name == "crafter-build") | ."build-options".env.CRAFTER_BUILD_VERSION) = strenv(VERSION)' "$MANIFEST" > "$CI_MANIFEST"
grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST" grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST"
flatpak-builder --force-clean --ccache --disable-rofiles-fuse \ flatpak-builder --force-clean --ccache --disable-rofiles-fuse \
--state-dir=/root/.cache/flatpak-builder \ --state-dir=/root/.cache/flatpak-builder \

View file

@ -21,7 +21,7 @@ crafter-build is a C++26 modules build system. A project is described by a `proj
Partitions and what they own: Partitions and what they own:
- `:Toolchain` resolves the compiler once per process (`GetToolchain`): clang++, clang, llvm-ar, libclang, libc++'s std module source and the default sysroots, from `CRAFTER_BUILD_LLVM_DIR`, else a toolchain installed beside crafter-build, else PATH. It also holds the version baked in at build time. - `:Toolchain` resolves the compiler once per process (`GetToolchain`): clang++, clang, llvm-ar, libclang, libc++'s std module source and the default sysroots, from `CRAFTER_BUILD_LLVM_DIR`, else a toolchain installed beside crafter-build, else PATH. It also holds the version baked in at build time. `SysrootStdModule` picks the std module source for a sysroot build by the sysroot's libc++ release.
- `:Clang` holds `Configuration`, `Build()`, `VariantId()`, `ApplyStandardArgs`, `HostTarget`, the CLI entry point `Crafter::Run`, and the lint and test data types. - `:Clang` holds `Configuration`, `Build()`, `VariantId()`, `ApplyStandardArgs`, `HostTarget`, the CLI entry point `Crafter::Run`, and the lint and test data types.
- `:Platform` runs commands, implements `LoadProject` (compile and load `project.cpp`), the host PCM cache, `#pragma crafter dependency`, and all OS-specific code. - `:Platform` runs commands, implements `LoadProject` (compile and load `project.cpp`), the host PCM cache, `#pragma crafter dependency`, and all OS-specific code.
- `:Interface` and `:Implementation` model module units, their staleness checks and their compiles. - `:Interface` and `:Implementation` model module units, their staleness checks and their compiles.
@ -169,4 +169,5 @@ To release: add a `<release>` to `packaging/flatpak/net.catcrafts.CrafterBuild.m
- `crafter-build clean` leaves a pragma dependency's build alone; it lives with the clone in the cache. - `crafter-build clean` leaves a pragma dependency's build alone; it lives with the clone in the cache.
- The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop. - The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop.
- A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone. - A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone.
- A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules/<major>/`, one `libcxx-<major>-modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot.
- `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash. - `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash.

View file

@ -116,6 +116,8 @@ Inside the Flatpak:
/app/bin/clang++ lld llvm-ar wasmtime # the bundled toolchain the executable resolves first /app/bin/clang++ lld llvm-ar wasmtime # the bundled toolchain the executable resolves first
/app/lib/libcrafter-build.a # static archive for downstream consumers /app/lib/libcrafter-build.a # static archive for downstream consumers
/app/share/crafter-build/*.cppm # module sources (rebuilt on the user's machine on first run) /app/share/crafter-build/*.cppm # module sources (rebuilt on the user's machine on first run)
/app/share/crafter-build/libc++-modules/<major>/
# std module sources of other libc++ releases, for sysroots without their own
/app/share/wasi-sysroot/ # WASI sysroot for wasm32 targets /app/share/wasi-sysroot/ # WASI sysroot for wasm32 targets
~/.var/app/net.catcrafts.CrafterBuild/cache/crafter.build/<target>-<march>-<clang>-<hash>/ ~/.var/app/net.catcrafts.CrafterBuild/cache/crafter.build/<target>-<march>-<clang>-<hash>/
# std.pcm + Crafter.Build*.pcm, built locally per toolchain # std.pcm + Crafter.Build*.pcm, built locally per toolchain
@ -174,7 +176,7 @@ TestRunner::Cmd("wine") // local exec via prefix co
For elaborate setups (rsync, ssh ControlMaster, env vars on the spawn, `QEMU_LD_PREFIX` for sysroots), construct `TestRunner` directly — `copy`/`exec`/`cleanup`/`probe` are public string fields with `{bundle}`, `{remote_bundle}`, `{bin}`, `{bin_name}`, `{args}` placeholders. For elaborate setups (rsync, ssh ControlMaster, env vars on the spawn, `QEMU_LD_PREFIX` for sysroots), construct `TestRunner` directly — `copy`/`exec`/`cleanup`/`probe` are public string fields with `{bundle}`, `{remote_bundle}`, `{bin}`, `{bin_name}`, `{args}` placeholders.
For cross-arch builds, set `cfg.sysroot` on the `Configuration` to point at a sysroot for the target. `BuildStdPcm` and the compile/link commands both consume it: `--sysroot=<path>` is passed to clang, and `<sysroot>/usr/share/libc++/v1/std.cppm` is precompiled instead of the host's. The cheapest aarch64 sysroot is an extracted [Arch Linux ARM rootfs](http://os.archlinuxarm.org/) plus `libc++` + `libc++abi` + `gcc` installed into it via `pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy …` (point pacman at ALARM's mirrors via explicit config to avoid pulling host-arch packages). For cross-arch builds, set `cfg.sysroot` on the `Configuration` to point at a sysroot for the target. `BuildStdPcm` and the compile/link commands both consume it: `--sysroot=<path>` is passed to clang, and the std module is precompiled against the sysroot's libc++ headers. The module source has to come from the same libc++ release as those headers, so it is taken from `<sysroot>/usr/share/libc++/v1/std.cppm` when the sysroot ships it, else from the toolchain when both are the same major release, else from the sources crafter-build carries for other releases (`share/crafter-build/libc++-modules/<major>/`; `crafter-build --version` lists them). Alpine is the common case for the last two: its `libc++-dev` has the headers and no module source. When nothing matches, the build stops with one line naming both releases; installing that release's `share/libc++/v1` directory into the sysroot resolves it. The cheapest aarch64 sysroot is an extracted [Arch Linux ARM rootfs](http://os.archlinuxarm.org/) plus `libc++` + `libc++abi` + `gcc` installed into it via `pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy …` (point pacman at ALARM's mirrors via explicit config to avoid pulling host-arch packages).
For `--target=x86_64-w64-mingw32` (Windows from Linux), the pipeline cross-compiles as long as `mingw-w64` is installed. Pair it with `--runner=cmd:wine` to run the result. Only the MinGW ABI is supported; there is no MSVC target and no Windows host. `-fuse-ld=lld` is auto-injected on every link, and `-lstdc++exp` is auto-linked for the mingw target so user projects don't need to declare those flags. For `--target=x86_64-w64-mingw32` (Windows from Linux), the pipeline cross-compiles as long as `mingw-w64` is installed. Pair it with `--runner=cmd:wine` to run the result. Only the MinGW ABI is supported; there is no MSVC target and no Windows host. `-fuse-ld=lld` is auto-injected on every link, and `-lstdc++exp` is auto-linked for the mingw target so user projects don't need to declare those flags.

View file

@ -1537,6 +1537,20 @@ static void PrintVersion() {
return; return;
} }
std::println("compiler {} ({})", tc.clangVersion, tc.clangxx.string()); std::println("compiler {} ({})", tc.clangVersion, tc.clangxx.string());
if (tc.libcxxVersion != 0) {
// A sysroot on another libc++ release builds only when its std
// module source is in the sysroot or among these; printed here
// because that is the first question when such a build fails.
std::vector<std::string> shipped;
std::error_code ec;
for (const fs::directory_entry& entry : fs::directory_iterator(tc.stdModulesDir, ec)) {
if (fs::exists(entry.path() / "std.cppm")) shipped.push_back(entry.path().filename().string());
}
std::ranges::sort(shipped);
std::string also;
for (const std::string& release : shipped) also += std::format(", {}", release);
std::println("libc++ {}{}", LibcxxVersionString(tc.libcxxVersion), also.empty() ? std::string() : std::format(" (std module sources for sysroots also for{})", also.substr(1)));
}
std::println("host {}", tc.hostTriple); std::println("host {}", tc.hostTriple);
} }

View file

@ -663,14 +663,13 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
} else if (config.sysroot.empty()) { } else if (config.sysroot.empty()) {
stdCppm = GetToolchain().stdCppm.string(); stdCppm = GetToolchain().stdCppm.string();
} else { } else {
stdCppm = std::format("{}/usr/share/libc++/v1/std.cppm", config.sysroot); // The source has to belong to the libc++ release whose headers
// not every distro ships the std module sources (Alpine's // the sysroot carries. Handing clang the toolchain's own copy
// libc++-dev doesn't); the host copy is interchangeable as long // regardless used to end in a page of "no member named ... in
// as the libc++ versions line up, which the compile of the PCM // namespace 'std'" once the two releases differed.
// against the sysroot's headers verifies anyway StdModuleSource source = SysrootStdModule(GetToolchain(), config.sysroot);
if (!fs::exists(stdCppm) && fs::exists(GetToolchain().stdCppm)) { if (!source.error.empty()) return source.error;
stdCppm = GetToolchain().stdCppm.string(); stdCppm = source.stdCppm.string();
}
} }
std::string sysrootFlag = config.sysroot.empty() std::string sysrootFlag = config.sysroot.empty()
? std::string() ? std::string()

View file

@ -112,15 +112,19 @@ Toolchain Crafter::ResolveToolchain(const ToolchainInputs& in) {
// 3. libc++ and its std module source. The compiler's own copy lives at // 3. libc++ and its std module source. The compiler's own copy lives at
// <prefix>/share/libc++/v1 when the prefix carries libc++; the system // <prefix>/share/libc++/v1 when the prefix carries libc++; the system
// copy is the last resort either way. // copy is the last resort either way. The headers of the same install
const fs::path systemStdCppm = "/usr/share/libc++/v1/std.cppm"; // say which release the source belongs to.
fs::path libcxxRoot = "/usr";
if (std::string_view dir = Env(in, "CRAFTER_BUILD_LIBCXX_DIR"); !dir.empty()) { if (std::string_view dir = Env(in, "CRAFTER_BUILD_LIBCXX_DIR"); !dir.empty()) {
tc.libcxxDir = fs::path(dir); tc.libcxxDir = fs::path(dir);
tc.stdCppm = tc.libcxxDir / "share" / "libc++" / "v1" / "std.cppm"; libcxxRoot = tc.libcxxDir;
} else if (!tc.prefix.empty() && fs::exists(tc.prefix / "share" / "libc++" / "v1" / "std.cppm")) { } else if (!tc.prefix.empty() && fs::exists(tc.prefix / "share" / "libc++" / "v1" / "std.cppm")) {
tc.stdCppm = tc.prefix / "share" / "libc++" / "v1" / "std.cppm"; libcxxRoot = tc.prefix;
} else { }
tc.stdCppm = systemStdCppm; tc.stdCppm = libcxxRoot / "share" / "libc++" / "v1" / "std.cppm";
tc.libcxxVersion = LibcxxVersion(libcxxRoot / "include" / "c++" / "v1");
if (!in.crafterPrefix.empty()) {
tc.stdModulesDir = in.crafterPrefix / "share" / "crafter-build" / "libc++-modules";
} }
// 4. Sysroots for the cross targets that have a conventional location. // 4. Sysroots for the cross targets that have a conventional location.
@ -180,6 +184,56 @@ const Toolchain& Crafter::RequireToolchain() {
return tc; return tc;
} }
std::int32_t Crafter::LibcxxVersion(const fs::path& includeDir) {
// "# define _LIBCPP_VERSION 230102", the one place libc++ states its
// release in a form the preprocessor does not have to be run for.
constexpr std::string_view Marker = "define _LIBCPP_VERSION ";
std::ifstream config(includeDir / "__config");
std::string line;
while (std::getline(config, line)) {
const std::size_t at = line.find(Marker);
if (at == std::string::npos) continue;
const std::string_view digits = std::string_view(line).substr(at + Marker.size());
std::int32_t version = 0;
if (std::from_chars(digits.data(), digits.data() + digits.size(), version).ec == std::errc{}) return version;
}
return 0;
}
std::string Crafter::LibcxxVersionString(std::int32_t version) {
return std::format("{}.{}.{}", version / 10000, version / 100 % 100, version % 100);
}
StdModuleSource Crafter::SysrootStdModule(const Toolchain& tc, const fs::path& sysroot) {
const fs::path own = sysroot / "usr" / "share" / "libc++" / "v1" / "std.cppm";
if (fs::exists(own)) return {own, {}};
const fs::path includeDir = sysroot / "usr" / "include" / "c++" / "v1";
if (!fs::exists(includeDir / "__config")) {
return {{}, std::format("sysroot {} has no libc++ headers at {}; install its libc++ development package", sysroot.string(), includeDir.string())};
}
const std::int32_t wanted = LibcxxVersion(includeDir);
const bool comparable = wanted != 0 && tc.libcxxVersion != 0;
// Releases are matched on the major version: a release branch only takes
// fixes, so 22.1.3 headers and the 22.1.8 module source name the same
// declarations.
if (comparable && wanted / 10000 == tc.libcxxVersion / 10000 && fs::exists(tc.stdCppm)) return {tc.stdCppm, {}};
if (wanted != 0 && !tc.stdModulesDir.empty()) {
const fs::path shipped = tc.stdModulesDir / std::to_string(wanted / 10000) / "std.cppm";
if (fs::exists(shipped)) return {shipped, {}};
}
// One of the two releases is unknown, so there is nothing to compare:
// hand over the toolchain's source and let the compile decide, which is
// what happened for every sysroot before releases were compared.
if (!comparable && fs::exists(tc.stdCppm)) return {tc.stdCppm, {}};
const std::string sysrootRelease = wanted != 0 ? LibcxxVersionString(wanted) : std::string("of an unknown release");
const std::string toolchainSource = !fs::exists(tc.stdCppm)
? std::string("the toolchain has no std module source at all")
: std::format("the toolchain's is for libc++ {}", LibcxxVersionString(tc.libcxxVersion));
return {{}, std::format("sysroot {} carries libc++ {} without its std module source, and crafter-build has none for that release ({}); install the share/libc++/v1 directory of that libc++ release (std.cppm and std/) as {}", sysroot.string(), sysrootRelease, toolchainSource, own.parent_path().string())};
}
std::string_view Crafter::CrafterBuildVersion() { std::string_view Crafter::CrafterBuildVersion() {
return CRAFTER_BUILD_VERSION; return CRAFTER_BUILD_VERSION;
} }

View file

@ -28,6 +28,14 @@ export namespace Crafter {
fs::path libcxxDir; fs::path libcxxDir;
// libc++'s std module source for host builds. // libc++'s std module source for host builds.
fs::path stdCppm; fs::path stdCppm;
// The _LIBCPP_VERSION of the libc++ stdCppm belongs to (220108 for
// 22.1.8); 0 when its <__config> could not be read.
std::int32_t libcxxVersion = 0;
// Std module sources for libc++ releases other than the toolchain's
// own, one <major>/std.cppm each, for sysroots that carry libc++
// without them: <crafter prefix>/share/crafter-build/libc++-modules.
// Empty when the crafter prefix is unknown; need not exist.
fs::path stdModulesDir;
fs::path wasiSysroot; fs::path wasiSysroot;
// Holds include/c++/<ver>/bits/std.cc for the mingw target. Passed to // Holds include/c++/<ver>/bits/std.cc for the mingw target. Passed to
// clang as --sysroot only when it was set explicitly: clang's own // clang as --sysroot only when it was set explicitly: clang's own
@ -74,6 +82,29 @@ export namespace Crafter {
const Toolchain& GetToolchain(); const Toolchain& GetToolchain();
// GetToolchain(), or a std::runtime_error carrying its error. // GetToolchain(), or a std::runtime_error carrying its error.
const Toolchain& RequireToolchain(); const Toolchain& RequireToolchain();
// _LIBCPP_VERSION as the <__config> in `includeDir` (an include/c++/v1)
// defines it, XXYYZZ; 0 when the file is missing or has no such line.
std::int32_t LibcxxVersion(const fs::path& includeDir);
// "23.1.2" for 230102.
std::string LibcxxVersionString(std::int32_t version);
struct StdModuleSource {
fs::path stdCppm;
// One line saying why no source fits; empty when stdCppm is set.
std::string error;
};
// The std module source to precompile for a native (non-wasm) build
// against `sysroot`. The module source is a list of using-declarations
// for names the headers of the same libc++ release declare, so one from
// another release fails on every name that release added or dropped. In
// order: the sysroot's own <sysroot>/usr/share/libc++/v1/std.cppm, the
// toolchain's when both are the same major release, the copy for the
// sysroot's release under toolchain.stdModulesDir. Not every distro
// packages the module sources with its libc++ (Alpine does not), which
// is why the sysroot's own copy cannot be the only answer. When the
// sysroot's release cannot be read, the toolchain's source is returned
// and the compile is the check.
StdModuleSource SysrootStdModule(const Toolchain& toolchain, const fs::path& sysroot);
// The version baked in when crafter-build was built; "0.0.0-dev" for a // The version baked in when crafter-build was built; "0.0.0-dev" for a
// build that was given none. // build that was given none.
std::string_view CrafterBuildVersion(); std::string_view CrafterBuildVersion();

View file

@ -28,15 +28,6 @@ finish-args:
# gdb and friends from the Sdk # gdb and friends from the Sdk
- --allow=devel - --allow=devel
- --env=PATH=/app/bin:/usr/bin - --env=PATH=/app/bin:/usr/bin
build-options:
env:
CRAFTER_BUILD_LLVM_DIR: /app
# The shipped binary runs on any x86_64 from the last decade; user
# projects still build with -march=native by default.
CRAFTER_BUILD_MARCH: x86-64-v2
CRAFTER_BUILD_MTUNE: generic
# CI replaces this with the tag; a local build is a development build.
CRAFTER_BUILD_VERSION: 0.0.0-dev
cleanup: cleanup:
- /share/man - /share/man
- /share/doc - /share/doc
@ -115,8 +106,8 @@ modules:
- /bin/scan-view - /bin/scan-view
sources: sources:
- type: archive - type: archive
url: https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.8/llvm-project-22.1.8.src.tar.xz url: https://github.com/llvm/llvm-project/releases/download/llvmorg-23.1.2/llvm-project-23.1.2.src.tar.xz
sha256: 922f1817a0df7b1489272d18134ee0087a8b068828f87ac63b9861b1a9965888 sha256: c98bbef08a2b4c2613cd50e9aa9ae7b69b1fe6c16b2c40373bc0ab6116fdf78a
- name: wasi-sysroot - name: wasi-sysroot
# wasi-sdk's sysroot-only mode, driven by the clang built above: # wasi-sdk's sysroot-only mode, driven by the clang built above:
@ -164,9 +155,21 @@ modules:
commit: 2e6fb9d8ee0cdf9e431fbcabe8af3115de000a13 commit: 2e6fb9d8ee0cdf9e431fbcabe8af3115de000a13
dest: src/wasi-libc dest: src/wasi-libc
- type: archive - type: archive
url: https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.8/llvm-project-22.1.8.src.tar.xz url: https://github.com/llvm/llvm-project/releases/download/llvmorg-23.1.2/llvm-project-23.1.2.src.tar.xz
sha256: 922f1817a0df7b1489272d18134ee0087a8b068828f87ac63b9861b1a9965888 sha256: c98bbef08a2b4c2613cd50e9aa9ae7b69b1fe6c16b2c40373bc0ab6116fdf78a
dest: src/llvm-project dest: src/llvm-project
# wasi-sdk patches its LLVM tree before building libc++ (`git apply` in
# src/llvm-project), and upstream that directory is a submodule, a
# repository of its own. Unpacked from a tarball it is a plain
# subdirectory of the wasi-sdk checkout, where git treats the patched
# paths as outside the current directory, skips every one of them and
# exits 0. LLVM 22 happened to build without the patches; LLVM 23's
# libc++ does not compile for WASI without one of them. Making the tree
# a repository lets wasi-sdk's own patch step do what it does upstream,
# with whatever patch list the pinned tag carries.
- type: shell
commands:
- git init -q src/llvm-project
- type: file - type: file
path: wasi-host-toolchain.cmake path: wasi-host-toolchain.cmake
@ -180,6 +183,33 @@ modules:
url: https://github.com/bytecodealliance/wasmtime/releases/download/v49.0.2/wasmtime-v49.0.2-x86_64-linux.tar.xz url: https://github.com/bytecodealliance/wasmtime/releases/download/v49.0.2/wasmtime-v49.0.2-x86_64-linux.tar.xz
sha256: a4d6e9e3a5a60f527cf7793d674c48930c80c2e8977995b8a275cad3254b9322 sha256: a4d6e9e3a5a60f527cf7793d674c48930c80c2e8977995b8a275cad3254b9322
- name: libcxx-22-modules
# The std module source of the libc++ release before the bundled one, for
# cross sysroots that carry that libc++ without it: Alpine packages the
# headers and the library but not share/libc++/v1, and a module source
# only compiles against the headers of its own release. crafter-build
# picks <major>/std.cppm here by the sysroot's _LIBCPP_VERSION
# (SysrootStdModule in Crafter.Build:Toolchain). Add a module like this
# one per release worth supporting; the bundled clang has to be able to
# compile that release's headers.
buildsystem: simple
build-commands:
# Configure only. The cxx-modules component is std.cppm as configure
# generates it plus the .inc files straight from the source tree, so
# nothing is compiled, and libc++'s own install rule stays the one
# place that knows what a module source install consists of.
- cmake -S runtimes -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=/app/bin/clang -DCMAKE_CXX_COMPILER=/app/bin/clang++ "-DLLVM_ENABLE_RUNTIMES=libcxx;libcxxabi" -DLIBCXXABI_USE_LLVM_UNWINDER=OFF -DLIBCXX_INSTALL_MODULES=ON
# Staged, not installed to /app: the component also writes
# lib/libc++.modules.json, which would replace the bundled libc++'s.
- cmake --install build --component cxx-modules --prefix "$PWD/stage"
- grep -q "_LIBCPP_VERSION 22" libcxx/include/__config
- mkdir -p /app/share/crafter-build/libc++-modules
- cp -r stage/share/libc++/v1 /app/share/crafter-build/libc++-modules/22
sources:
- type: archive
url: https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.8/llvm-project-22.1.8.src.tar.xz
sha256: 922f1817a0df7b1489272d18134ee0087a8b068828f87ac63b9861b1a9965888
- name: crafter-build - name: crafter-build
buildsystem: simple buildsystem: simple
build-options: build-options:
@ -188,6 +218,18 @@ modules:
# (sibling repos tracking their default branch, see AGENTS.md). # (sibling repos tracking their default branch, see AGENTS.md).
build-args: build-args:
- --share=network - --share=network
# On this module, not at the top of the manifest: top-level build
# options are part of every module's cache key, and the version changes
# with every commit, which made flatpak-builder rebuild LLVM each run.
env:
CRAFTER_BUILD_LLVM_DIR: /app
# The shipped binary runs on any x86_64 from the last decade; user
# projects still build with -march=native by default.
CRAFTER_BUILD_MARCH: x86-64-v2
CRAFTER_BUILD_MTUNE: generic
# CI replaces this with the version; a local build is a development
# build.
CRAFTER_BUILD_VERSION: 0.0.0-dev
build-commands: build-commands:
# Bootstrap by hand, then let the result build itself: only the # Bootstrap by hand, then let the result build itself: only the
# self-hosted binary has asset support and ThinLTO. # self-hosted binary has asset support and ThinLTO.

View file

@ -129,6 +129,7 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
cfg.AddTest("TestRunnerSpec").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("TestRunnerSpec").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("VariantId").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("VariantId").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ToolchainResolve").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ToolchainResolve").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("SysrootStdModule").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("VersionFlag").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("VersionFlag").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasiBrowserRuntime").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("WasiBrowserRuntime").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasmVariants").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("WasmVariants").Dependencies({ CrafterBuildLib.get() });

View file

@ -0,0 +1,156 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// A sysroot build precompiles the std module from a module source that has to
// belong to the libc++ release whose headers the sysroot carries. Not every
// distro packages that source with its libc++ (Alpine does not), and the
// fallback used to be the toolchain's own copy whatever its release: with a
// libc++ 22 toolchain and a libc++ 23 sysroot the build died on a page of "no
// member named 'runtime_format' in namespace 'std'" that named neither
// release nor what to do about it.
//
// Staged, compiler-free layouts cover the order SysrootStdModule picks in: the
// sysroot's own source, the toolchain's for the same major release, the copy
// crafter-build ships for another release, and otherwise one line naming both
// releases and where the matching source belongs. The last block drives
// Build() against a sysroot nothing matches and expects that line back in
// place of compiler output.
namespace {
std::int32_t Failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++Failures;
}
}
void Write(const fs::path& file, std::string_view content) {
fs::create_directories(file.parent_path());
std::ofstream(file) << content;
}
// What libc++ installs as include/c++/v1/__config, reduced to the line
// that states the release.
void StageHeaders(const fs::path& includeDir, std::int32_t version) {
Write(includeDir / "__config", std::format("#ifndef _LIBCPP___CONFIG\n# define _LIBCPP___CONFIG\n# define _LIBCPP_VERSION {}\n#endif\n", version));
}
fs::path StageSysroot(const fs::path& sysroot, std::int32_t version) {
StageHeaders(sysroot / "usr" / "include" / "c++" / "v1", version);
return sysroot;
}
}
int main() {
const fs::path scratch = fs::temp_directory_path() / "crafter-build-sysroot-std-module";
std::error_code ec;
fs::remove_all(scratch, ec);
fs::create_directories(scratch);
{
StageHeaders(scratch / "headers", 230102);
Check(LibcxxVersion(scratch / "headers") == 230102, "the release is read from __config");
Check(LibcxxVersion(scratch / "no-such-dir") == 0, "missing headers read as an unknown release");
Write(scratch / "other" / "__config", "#define _LIBCPP_ABI_VERSION 1\n");
Check(LibcxxVersion(scratch / "other") == 0, "a __config without the version line reads as an unknown release");
Check(LibcxxVersionString(230102) == "23.1.2" && LibcxxVersionString(220108) == "22.1.8", "XXYYZZ is spelled as a dotted release");
}
// A toolchain on libc++ 22.1.8 that also ships the sources for 23.
Toolchain tc;
tc.stdCppm = scratch / "toolchain" / "share" / "libc++" / "v1" / "std.cppm";
tc.libcxxVersion = 220108;
tc.stdModulesDir = scratch / "shipped";
Write(tc.stdCppm, "");
Write(tc.stdModulesDir / "23" / "std.cppm", "");
{
// The sysroot's own source matches its headers by construction, so
// it wins whatever the releases are.
const fs::path sysroot = StageSysroot(scratch / "sysroot-own", 240100);
const fs::path own = sysroot / "usr" / "share" / "libc++" / "v1" / "std.cppm";
Write(own, "");
StdModuleSource source = SysrootStdModule(tc, sysroot);
Check(source.error.empty() && source.stdCppm == own, "a sysroot that ships the module source uses its own");
}
{
const fs::path sysroot = StageSysroot(scratch / "sysroot-same-major", 220103);
StdModuleSource source = SysrootStdModule(tc, sysroot);
Check(source.error.empty() && source.stdCppm == tc.stdCppm, "a point release of the toolchain's libc++ uses the toolchain's source");
}
{
// The reported case: libc++ 23 headers, a libc++ 22 toolchain.
const fs::path sysroot = StageSysroot(scratch / "sysroot-next-major", 230102);
StdModuleSource source = SysrootStdModule(tc, sysroot);
Check(source.error.empty() && source.stdCppm == tc.stdModulesDir / "23" / "std.cppm", "another release uses the source shipped for it, not the toolchain's");
}
{
const fs::path sysroot = StageSysroot(scratch / "sysroot-unshipped", 210108);
StdModuleSource source = SysrootStdModule(tc, sysroot);
Check(source.stdCppm.empty() && !source.error.empty(), "a release nothing was shipped for is an error, not a compile against the wrong source");
Check(source.error.contains("21.1.8") && source.error.contains("22.1.8"), std::format("the error names both releases, got '{}'", source.error));
Check(source.error.contains((sysroot / "usr" / "share" / "libc++" / "v1").string()), std::format("the error says where the matching source belongs, got '{}'", source.error));
Check(!source.error.contains('\n'), "the error is one line");
}
{
const fs::path sysroot = scratch / "sysroot-no-libcxx";
fs::create_directories(sysroot / "usr" / "include");
StdModuleSource source = SysrootStdModule(tc, sysroot);
Check(source.stdCppm.empty() && source.error.contains("no libc++ headers"), std::format("a sysroot without libc++ says so, got '{}'", source.error));
}
{
// Nothing to compare when either release is unknown: the toolchain's
// source goes to the compiler, as it did before releases were read.
const fs::path sysroot = scratch / "sysroot-unreadable";
Write(sysroot / "usr" / "include" / "c++" / "v1" / "__config", "#define _LIBCPP_ABI_VERSION 1\n");
Check(SysrootStdModule(tc, sysroot).stdCppm == tc.stdCppm, "an unreadable sysroot release falls back to the toolchain's source");
Toolchain unknown = tc;
unknown.libcxxVersion = 0;
unknown.stdModulesDir.clear();
Check(SysrootStdModule(unknown, StageSysroot(scratch / "sysroot-vs-unknown", 230102)).stdCppm == tc.stdCppm, "an unreadable toolchain release falls back to the toolchain's source");
}
{
Toolchain bare;
bare.stdCppm = scratch / "nowhere" / "std.cppm";
bare.libcxxVersion = 220108;
StdModuleSource source = SysrootStdModule(bare, StageSysroot(scratch / "sysroot-vs-bare", 220108));
Check(source.stdCppm.empty() && source.error.contains("no std module source"), std::format("a toolchain without a module source is an error too, got '{}'", source.error));
}
{
// End to end with the real toolchain: a sysroot on a libc++ release
// that does not exist fails the build with the one-line explanation
// before clang is run. The target is the host's so the std PCM cache
// directory is the one every other test already uses.
const Toolchain& real = GetToolchain();
Check(real.libcxxVersion != 0, "the running toolchain's libc++ release is read from its headers");
const fs::path project = scratch / "project";
Write(project / "main.cpp", "int main() { return 0; }\n");
Configuration app;
app.path = project;
app.name = "sysroot-std-module";
app.outputName = "sysroot-std-module";
app.target = HostTarget();
app.type = ConfigurationType::Executable;
app.sysroot = StageSysroot(scratch / "sysroot-future", 990000).string();
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
app.GetInterfacesAndImplementations(ifaces, impls);
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
BuildResult r = Build(app, depResults, depMutex);
Check(r.result.contains("libc++ 99.0.0"), std::format("the build fails naming the sysroot's release, got '{}'", r.result));
Check(r.result.contains(LibcxxVersionString(real.libcxxVersion)), std::format("the build fails naming the toolchain's release, got '{}'", r.result));
Check(!r.result.contains('\n') && !r.result.contains("error:"), std::format("the failure is one line, not compiler output, got '{}'", r.result));
}
return Failures == 0 ? 0 : 1;
}

View file

@ -40,9 +40,11 @@ int main() {
// A prefix with everything crafter-build looks for beside the compiler, // A prefix with everything crafter-build looks for beside the compiler,
// and one with the compiler alone. // and one with the compiler alone.
const fs::path full = scratch / "full"; const fs::path full = scratch / "full";
for (const char* rel : {"bin/clang++", "bin/clang", "bin/llvm-ar", "lib/libclang.so", "include/clang-c/Index.h", "share/libc++/v1/std.cppm", "share/wasi-sysroot/include/.keep"}) { for (const char* rel : {"bin/clang++", "bin/clang", "bin/llvm-ar", "lib/libclang.so", "include/clang-c/Index.h", "include/c++/v1/__config", "share/libc++/v1/std.cppm", "share/wasi-sysroot/include/.keep"}) {
Touch(full / rel); Touch(full / rel);
} }
// The prefix's libc++ headers state its release.
std::ofstream(full / "include/c++/v1/__config") << "# define _LIBCPP_VERSION 230102\n";
const fs::path bare = scratch / "bare"; const fs::path bare = scratch / "bare";
Touch(bare / "bin/clang++"); Touch(bare / "bin/clang++");
@ -56,6 +58,8 @@ int main() {
Check(!tc.libclang.has_value(), "the PATH fallback searches libclang by name"); Check(!tc.libclang.has_value(), "the PATH fallback searches libclang by name");
Check(tc.llvmIncludeDir.empty(), "the PATH fallback adds no include dir"); Check(tc.llvmIncludeDir.empty(), "the PATH fallback adds no include dir");
Check(tc.stdCppm == "/usr/share/libc++/v1/std.cppm", "the system std module source is the default"); Check(tc.stdCppm == "/usr/share/libc++/v1/std.cppm", "the system std module source is the default");
Check(tc.libcxxVersion == LibcxxVersion("/usr/include/c++/v1"), "the system libc++ headers state the release of the system module source");
Check(tc.stdModulesDir.empty(), "no crafter-build prefix, no shipped module sources");
Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "the system wasi sysroot is the default"); Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "the system wasi sysroot is the default");
Check(tc.mingwSysroot == "/usr/x86_64-w64-mingw32" && !tc.mingwSysrootExplicit, "the default mingw dir is not passed as --sysroot"); Check(tc.mingwSysroot == "/usr/x86_64-w64-mingw32" && !tc.mingwSysrootExplicit, "the default mingw dir is not passed as --sysroot");
Check(tc.LibcxxCompileFlags().empty() && tc.LibcxxLinkFlags().empty(), "the PATH fallback adds no libc++ flags"); Check(tc.LibcxxCompileFlags().empty() && tc.LibcxxLinkFlags().empty(), "the PATH fallback adds no libc++ flags");
@ -71,6 +75,8 @@ int main() {
Check(tc.libclang == full / "lib/libclang.so", "libclang beside the compiler is pinned"); Check(tc.libclang == full / "lib/libclang.so", "libclang beside the compiler is pinned");
Check(tc.llvmIncludeDir == full / "include", "clang-c headers beside the compiler are added to the include path"); Check(tc.llvmIncludeDir == full / "include", "clang-c headers beside the compiler are added to the include path");
Check(tc.stdCppm == full / "share/libc++/v1/std.cppm", "the prefix's std module source is used"); Check(tc.stdCppm == full / "share/libc++/v1/std.cppm", "the prefix's std module source is used");
Check(tc.libcxxVersion == 230102, "the prefix's libc++ headers state its release");
Check(tc.stdModulesDir == full / "share/crafter-build/libc++-modules", "module sources for other releases are looked up in crafter-build's own share directory");
Check(tc.wasiSysroot == full / "share/wasi-sysroot", "the prefix's wasi sysroot is used"); Check(tc.wasiSysroot == full / "share/wasi-sysroot", "the prefix's wasi sysroot is used");
Check(tc.LibcxxCompileFlags().empty(), "the compiler finds its own libc++ headers without flags"); Check(tc.LibcxxCompileFlags().empty(), "the compiler finds its own libc++ headers without flags");
Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (full / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (full / "lib").string())), "binaries get -L and an rpath for the prefix's lib dir"); Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (full / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (full / "lib").string())), "binaries get -L and an rpath for the prefix's lib dir");
@ -120,12 +126,15 @@ int main() {
// An explicit libc++ install: headers, library and std module all // An explicit libc++ install: headers, library and std module all
// come from it, whatever the compiler would have used. // come from it, whatever the compiler would have used.
const fs::path libcxx = scratch / "libcxx"; const fs::path libcxx = scratch / "libcxx";
fs::create_directories(libcxx / "include/c++/v1");
std::ofstream(libcxx / "include/c++/v1/__config") << "# define _LIBCPP_VERSION 210108\n";
ToolchainInputs in; ToolchainInputs in;
in.crafterPrefix = full; in.crafterPrefix = full;
in.env["CRAFTER_BUILD_LIBCXX_DIR"] = libcxx.string(); in.env["CRAFTER_BUILD_LIBCXX_DIR"] = libcxx.string();
Toolchain tc = ResolveToolchain(in); Toolchain tc = ResolveToolchain(in);
Check(tc.libcxxDir == libcxx, "CRAFTER_BUILD_LIBCXX_DIR is recorded"); Check(tc.libcxxDir == libcxx, "CRAFTER_BUILD_LIBCXX_DIR is recorded");
Check(tc.stdCppm == libcxx / "share/libc++/v1/std.cppm", "the std module source follows the explicit libc++"); Check(tc.stdCppm == libcxx / "share/libc++/v1/std.cppm", "the std module source follows the explicit libc++");
Check(tc.libcxxVersion == 210108, "the release is read from the explicit libc++, not the compiler's");
Check(Contains(tc.LibcxxCompileFlags(), std::format("-nostdinc++ -isystem {}", (libcxx / "include/c++/v1").string())), "compiles use the explicit libc++ headers"); Check(Contains(tc.LibcxxCompileFlags(), std::format("-nostdinc++ -isystem {}", (libcxx / "include/c++/v1").string())), "compiles use the explicit libc++ headers");
Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (libcxx / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (libcxx / "lib").string())), "links use the explicit libc++ library dir, with an rpath"); Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (libcxx / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (libcxx / "lib").string())), "links use the explicit libc++ library dir, with an rpath");
} }