Catcrafts/Crafter.Build
Watch
2
0
Fork
You've already forked Crafter.Build
 0
Code Issues 5 Pull requests Projects Releases 7 Packages Wiki Activity Actions
Crafter.Build/tests/ToolchainResolve/main.cpp

220 lines
13 KiB
C++
Raw Permalink Normal View History

feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// ResolveToolchain is pure path resolution, so this test needs no compiler:
// it stages fake prefixes under the temp dir and checks which one wins,
// what is picked up beside it, and what the defaults are when nothing is.
// The last block runs the real GetToolchain() once, which is the only part
// that spawns clang.
namespace {
std::int32_t Failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++Failures;
}
}
void Touch(const fs::path& file) {
fs::create_directories(file.parent_path());
std::ofstream(file) << "";
}
bool Contains(std::string_view haystack, std::string_view needle) {
return haystack.find(needle) != std::string_view::npos;
}
}
int main() {
const fs::path scratch = fs::temp_directory_path() / "crafter-build-toolchain-resolve";
std::error_code ec;
fs::remove_all(scratch, ec);
fs::create_directories(scratch);
// A prefix with everything crafter-build looks for beside the compiler,
// and one with the compiler alone.
const fs::path full = scratch / "full";
test(sysroot): cover the std module source choice for a sysroot build SysrootStdModule runs over staged layouts that need no compiler: a sysroot with its own module source, one on a point release of the toolchain's libc++, one on the next major release with a shipped source, one nothing was shipped for, one without libc++ at all, and the two cases where a release cannot be read. The last block drives Build() against a sysroot claiming libc++ 99 and expects the one-line explanation back instead of compiler output, which is the failure the fix replaces. ToolchainResolve additionally checks that the release is read from the libc++ the module source comes from (the prefix's, an explicit libc++ install's, the system's) and where the shipped sources are looked up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
for (const char* rel : {"bin/clang++", "bin/clang", "bin/llvm-ar", "lib/libclang.so", "include/clang-c/Index.h", "include/c++/v1/__config", "share/libc++/v1/std.cppm", "share/wasi-sysroot/include/.keep"}) {
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
Touch(full / rel);
}
test(sysroot): cover the std module source choice for a sysroot build SysrootStdModule runs over staged layouts that need no compiler: a sysroot with its own module source, one on a point release of the toolchain's libc++, one on the next major release with a shipped source, one nothing was shipped for, one without libc++ at all, and the two cases where a release cannot be read. The last block drives Build() against a sysroot claiming libc++ 99 and expects the one-line explanation back instead of compiler output, which is the failure the fix replaces. ToolchainResolve additionally checks that the release is read from the libc++ the module source comes from (the prefix's, an explicit libc++ install's, the system's) and where the shipped sources are looked up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
// The prefix's libc++ headers state its release.
std::ofstream(full / "include/c++/v1/__config") << "# define _LIBCPP_VERSION 230102\n";
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
const fs::path bare = scratch / "bare";
Touch(bare / "bin/clang++");
{
// Nothing set and nothing beside crafter-build: bare names on PATH
// and the system locations.
Toolchain tc = ResolveToolchain({});
Check(tc.error.empty(), "the PATH fallback is not an error");
Check(tc.prefix.empty(), "the PATH fallback has no prefix");
Check(tc.clangxx == "clang++" && tc.clang == "clang" && tc.llvmAr == "llvm-ar", "the PATH fallback uses bare tool names");
Check(!tc.libclang.has_value(), "the PATH fallback searches libclang by name");
Check(tc.llvmIncludeDir.empty(), "the PATH fallback adds no include dir");
Check(tc.stdCppm == "/usr/share/libc++/v1/std.cppm", "the system std module source is the default");
test(sysroot): cover the std module source choice for a sysroot build SysrootStdModule runs over staged layouts that need no compiler: a sysroot with its own module source, one on a point release of the toolchain's libc++, one on the next major release with a shipped source, one nothing was shipped for, one without libc++ at all, and the two cases where a release cannot be read. The last block drives Build() against a sysroot claiming libc++ 99 and expects the one-line explanation back instead of compiler output, which is the failure the fix replaces. ToolchainResolve additionally checks that the release is read from the libc++ the module source comes from (the prefix's, an explicit libc++ install's, the system's) and where the shipped sources are looked up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
Check(tc.libcxxVersion == LibcxxVersion("/usr/include/c++/v1"), "the system libc++ headers state the release of the system module source");
Check(tc.stdModulesDir.empty(), "no crafter-build prefix, no shipped module sources");
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "the system wasi sysroot is the default");
fix: give the mingw target a sysroot that holds mingw-w64 alone Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree that holds mingw-w64 alone, and adds its include/ to the include path and its lib/ to the library path. For /usr those are the Linux system's own: every header mingw-w64 lacks came from glibc, and libstdc++'s <ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read __libc_single_threaded, which nothing on Windows defines. Every program that used one failed to link, among them every Crafter.Graphics Win32 executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak (#43). The sysroot is now MingwSysroot(): a directory in the cache, one per prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/ and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds mingw-w64, gcc's version and libstdc++ exactly as it does through the mingw gcc on PATH, and nothing else. It is made by the first mingw compile, link or CMake configure of a process, so no other command writes to the cache for it and a failure to make it is that build's error. The Toolchain field is now mingwPrefix, since clang never gets the prefix itself, and it is made absolute when the toolchain is resolved, as it ends up in link targets and directory names. Messages keep naming paths under the prefix rather than under the view. The prefix is also part of the mingw target's VariantId and of its std PCM directory. Units do not track the std PCM they import: when the fixed release built in a directory a broken build had left, the std module was rebuilt and the objects compiled against glibc's headers were only relinked, failing as before. As with a compiler update, a new directory is what gets them rebuilt. It also keeps two releases that share a cache, as the Flatpak's branches do, from rebuilding one std PCM under each other's PCMs, and two prefixes apart. Without a prefix both stay as they were: clang's own discovery has not changed. Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32 loses the directory gcc's version names, so <string> is not found, and clang ignores --gcc-toolchain and --gcc-install-dir for this target. Fixes #43 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
Check(tc.mingwPrefix.empty(), "outside a sandbox, clang finds mingw-w64 itself: no prefix");
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
Check(tc.LibcxxCompileFlags().empty() && tc.LibcxxLinkFlags().empty(), "the PATH fallback adds no libc++ flags");
}
{
// A toolchain installed beside crafter-build wins over PATH, and
// everything found beside it is used.
ToolchainInputs in;
in.crafterPrefix = full;
Toolchain tc = ResolveToolchain(in);
Check(tc.prefix == full, "the prefix beside crafter-build is picked");
Check(tc.clangxx == full / "bin/clang++" && tc.clang == full / "bin/clang" && tc.llvmAr == full / "bin/llvm-ar", "the tools come from that prefix");
Check(tc.libclang == full / "lib/libclang.so", "libclang beside the compiler is pinned");
Check(tc.llvmIncludeDir == full / "include", "clang-c headers beside the compiler are added to the include path");
Check(tc.stdCppm == full / "share/libc++/v1/std.cppm", "the prefix's std module source is used");
test(sysroot): cover the std module source choice for a sysroot build SysrootStdModule runs over staged layouts that need no compiler: a sysroot with its own module source, one on a point release of the toolchain's libc++, one on the next major release with a shipped source, one nothing was shipped for, one without libc++ at all, and the two cases where a release cannot be read. The last block drives Build() against a sysroot claiming libc++ 99 and expects the one-line explanation back instead of compiler output, which is the failure the fix replaces. ToolchainResolve additionally checks that the release is read from the libc++ the module source comes from (the prefix's, an explicit libc++ install's, the system's) and where the shipped sources are looked up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
Check(tc.libcxxVersion == 230102, "the prefix's libc++ headers state its release");
Check(tc.stdModulesDir == full / "share/crafter-build/libc++-modules", "module sources for other releases are looked up in crafter-build's own share directory");
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
Check(tc.wasiSysroot == full / "share/wasi-sysroot", "the prefix's wasi sysroot is used");
Check(tc.LibcxxCompileFlags().empty(), "the compiler finds its own libc++ headers without flags");
fix(toolchain): pass -L as well as an rpath for a toolchain outside /usr Inside the Flatpak the bundled toolchain compiled every bootstrap object, then glslang's configure failed its test link with `unable to find library -lc++`. clang searches its per-target runtime directory (lib/<triple>) and the system library paths for libc++, not <prefix>/lib, so with the classic layout a toolchain outside /usr needs the search path spelled out, just as its loader needs the rpath. LibcxxLinkFlags and build.sh now emit both, the same pair an explicit CRAFTER_BUILD_LIBCXX_DIR already got. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 23:18:07 +02:00
Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (full / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (full / "lib").string())), "binaries get -L and an rpath for the prefix's lib dir");
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
}
{
ToolchainInputs in;
in.crafterPrefix = scratch / "nothing-here";
Toolchain tc = ResolveToolchain(in);
Check(tc.prefix.empty() && tc.clangxx == "clang++", "a crafter-build prefix without clang++ falls back to PATH");
}
{
// The environment wins over the prefix beside crafter-build. A
// prefix with nothing but the compiler keeps the system defaults for
// everything else.
ToolchainInputs in;
in.crafterPrefix = full;
in.env["CRAFTER_BUILD_LLVM_DIR"] = bare.string();
Toolchain tc = ResolveToolchain(in);
Check(tc.error.empty(), "an explicit prefix with a clang++ is accepted");
Check(tc.prefix == bare, "CRAFTER_BUILD_LLVM_DIR wins over the prefix beside crafter-build");
Check(!tc.libclang.has_value(), "no libclang.so in the prefix: search by name");
Check(tc.llvmIncludeDir.empty(), "no clang-c headers in the prefix: nothing added");
Check(tc.stdCppm == "/usr/share/libc++/v1/std.cppm", "no std module in the prefix: the system copy");
Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "no wasi sysroot in the prefix: the system one");
fix(toolchain): pass -L as well as an rpath for a toolchain outside /usr Inside the Flatpak the bundled toolchain compiled every bootstrap object, then glslang's configure failed its test link with `unable to find library -lc++`. clang searches its per-target runtime directory (lib/<triple>) and the system library paths for libc++, not <prefix>/lib, so with the classic layout a toolchain outside /usr needs the search path spelled out, just as its loader needs the rpath. LibcxxLinkFlags and build.sh now emit both, the same pair an explicit CRAFTER_BUILD_LIBCXX_DIR already got. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 23:18:07 +02:00
Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (bare / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (bare / "lib").string())), "the link flags follow the explicit prefix");
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
}
{
ToolchainInputs in;
in.env["CRAFTER_BUILD_LLVM_DIR"] = (scratch / "missing").string();
Toolchain tc = ResolveToolchain(in);
Check(!tc.error.empty() && Contains(tc.error, "CRAFTER_BUILD_LLVM_DIR"), "an explicit prefix without clang++ is an error, not a fallback");
}
{
ToolchainInputs in;
in.env["CRAFTER_BUILD_LLVM_DIR"] = "/usr";
Toolchain tc = ResolveToolchain(in);
Check(tc.LibcxxLinkFlags().empty(), "the system prefix needs no rpath");
Check(tc.llvmIncludeDir.empty(), "the system prefix needs no include dir");
}
{
ToolchainInputs in;
in.crafterPrefix = full;
in.env["CRAFTER_BUILD_LIBCLANG"] = "/nonexistent/libclang.so";
Check(ResolveToolchain(in).libclang == fs::path("/nonexistent/libclang.so"), "CRAFTER_BUILD_LIBCLANG is taken verbatim, even when it does not exist");
}
{
// An explicit libc++ install: headers, library and std module all
// come from it, whatever the compiler would have used.
const fs::path libcxx = scratch / "libcxx";
test(sysroot): cover the std module source choice for a sysroot build SysrootStdModule runs over staged layouts that need no compiler: a sysroot with its own module source, one on a point release of the toolchain's libc++, one on the next major release with a shipped source, one nothing was shipped for, one without libc++ at all, and the two cases where a release cannot be read. The last block drives Build() against a sysroot claiming libc++ 99 and expects the one-line explanation back instead of compiler output, which is the failure the fix replaces. ToolchainResolve additionally checks that the release is read from the libc++ the module source comes from (the prefix's, an explicit libc++ install's, the system's) and where the shipped sources are looked up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
fs::create_directories(libcxx / "include/c++/v1");
std::ofstream(libcxx / "include/c++/v1/__config") << "# define _LIBCPP_VERSION 210108\n";
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
ToolchainInputs in;
in.crafterPrefix = full;
in.env["CRAFTER_BUILD_LIBCXX_DIR"] = libcxx.string();
Toolchain tc = ResolveToolchain(in);
Check(tc.libcxxDir == libcxx, "CRAFTER_BUILD_LIBCXX_DIR is recorded");
Check(tc.stdCppm == libcxx / "share/libc++/v1/std.cppm", "the std module source follows the explicit libc++");
test(sysroot): cover the std module source choice for a sysroot build SysrootStdModule runs over staged layouts that need no compiler: a sysroot with its own module source, one on a point release of the toolchain's libc++, one on the next major release with a shipped source, one nothing was shipped for, one without libc++ at all, and the two cases where a release cannot be read. The last block drives Build() against a sysroot claiming libc++ 99 and expects the one-line explanation back instead of compiler output, which is the failure the fix replaces. ToolchainResolve additionally checks that the release is read from the libc++ the module source comes from (the prefix's, an explicit libc++ install's, the system's) and where the shipped sources are looked up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
Check(tc.libcxxVersion == 210108, "the release is read from the explicit libc++, not the compiler's");
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
Check(Contains(tc.LibcxxCompileFlags(), std::format("-nostdinc++ -isystem {}", (libcxx / "include/c++/v1").string())), "compiles use the explicit libc++ headers");
Check(Contains(tc.LibcxxLinkFlags(), std::format("-L{}", (libcxx / "lib").string())) && Contains(tc.LibcxxLinkFlags(), std::format("-Wl,-rpath,{}", (libcxx / "lib").string())), "links use the explicit libc++ library dir, with an rpath");
}
{
ToolchainInputs in;
in.env["CRAFTER_BUILD_WASI_SYSROOT"] = "/opt/wasi";
in.env["CRAFTER_MINGW_DIR"] = "/opt/mingw";
Toolchain tc = ResolveToolchain(in);
Check(tc.wasiSysroot == "/opt/wasi", "CRAFTER_BUILD_WASI_SYSROOT overrides the wasi sysroot");
fix: give the mingw target a sysroot that holds mingw-w64 alone Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree that holds mingw-w64 alone, and adds its include/ to the include path and its lib/ to the library path. For /usr those are the Linux system's own: every header mingw-w64 lacks came from glibc, and libstdc++'s <ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read __libc_single_threaded, which nothing on Windows defines. Every program that used one failed to link, among them every Crafter.Graphics Win32 executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak (#43). The sysroot is now MingwSysroot(): a directory in the cache, one per prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/ and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds mingw-w64, gcc's version and libstdc++ exactly as it does through the mingw gcc on PATH, and nothing else. It is made by the first mingw compile, link or CMake configure of a process, so no other command writes to the cache for it and a failure to make it is that build's error. The Toolchain field is now mingwPrefix, since clang never gets the prefix itself, and it is made absolute when the toolchain is resolved, as it ends up in link targets and directory names. Messages keep naming paths under the prefix rather than under the view. The prefix is also part of the mingw target's VariantId and of its std PCM directory. Units do not track the std PCM they import: when the fixed release built in a directory a broken build had left, the std module was rebuilt and the objects compiled against glibc's headers were only relinked, failing as before. As with a compiler update, a new directory is what gets them rebuilt. It also keeps two releases that share a cache, as the Flatpak's branches do, from rebuilding one std PCM under each other's PCMs, and two prefixes apart. Without a prefix both stay as they were: clang's own discovery has not changed. Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32 loses the directory gcc's version names, so <string> is not found, and clang ignores --gcc-toolchain and --gcc-install-dir for this target. Fixes #43 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
Check(tc.mingwPrefix == "/opt/mingw", "CRAFTER_MINGW_DIR names the mingw prefix");
in.env["CRAFTER_MINGW_DIR"] = "mingw";
Check(ResolveToolchain(in).mingwPrefix == fs::absolute("mingw"), "a relative CRAFTER_MINGW_DIR is resolved against the working directory once, when the toolchain is");
fix(mingw): view only a prefix that holds a Linux system The first fix showed clang every mingw prefix through a view of its mingw-w64 parts. That broke a prefix that holds mingw-w64 alone, where Windows libraries installed with --prefix land in include/ and lib/: with CRAFTER_MINGW_DIR=/opt/mingw, <SDL2/SDL.h> and -lSDL2 were no longer found. Such a prefix is clang's sysroot whole again. Only a prefix that also holds a Linux system gets the view (Toolchain::mingwPrefixIsSystem): the host's /usr inside the Flatpak, whatever its include/ holds, and a prefix whose include/ has a C library's features.h, which every glibc and musl header includes and mingw-w64 has no use for, as CRAFTER_MINGW_DIR=/usr has. MakeMingwSysroot makes the view in the cache, or in $XDG_RUNTIME_DIR/crafter.build when the cache's filesystem takes no symbolic links (vfat, exFAT, CIFS without unix extensions): the XDG spec promises them there, and nothing else in the cache needs them. Its path is absolute, because CMake runs the compiler from directories of its own and a relative cache did not resolve from there. The prefix is resolved once, absolute and lexically normal, so /opt/mingw/ and /opt/mingw are one prefix and not two builds. clang names a header by the path it found it at, so every diagnostic from a viewed mingw-w64 header named a hashed cache directory. RunBuildCommand now writes paths into the view as paths into the prefix, as BuildStdPcm did for its manifest messages already. Also from the review of PR #44: SysrootCompileFlags builds a mingw command, which can make a view, so it redirects the cache like the other tests; the manifest filter's comment names what it still keeps out; the C path asks MingwSysroot() once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:58:30 +01:00
// The prefix names directories, so one directory has one spelling:
// otherwise a trailing slash rebuilds every mingw build for nothing.
for (const char* spelling : {"/opt/mingw/", "/opt//mingw", "/opt/./mingw", "/opt/x/../mingw"}) {
in.env["CRAFTER_MINGW_DIR"] = spelling;
Check(ResolveToolchain(in).mingwPrefix == "/opt/mingw", std::format("{} is spelled /opt/mingw (got '{}')", spelling, ResolveToolchain(in).mingwPrefix.string()));
}
}
{
// A prefix that holds a Linux system's C library, as /usr does, is
// shown to clang through a view; one that holds mingw-w64 alone is
// the sysroot itself, Windows libraries in its include/ and lib/
// included.
const fs::path staged = fs::temp_directory_path() / "crafter-build-toolchain-mingw-prefixes";
fs::remove_all(staged);
Touch(staged / "system" / "include" / "features.h");
Touch(staged / "system" / "x86_64-w64-mingw32" / "include" / "_mingw.h");
Touch(staged / "dedicated" / "include" / "SDL2" / "SDL.h");
Touch(staged / "dedicated" / "x86_64-w64-mingw32" / "include" / "_mingw.h");
ToolchainInputs in;
in.env["CRAFTER_MINGW_DIR"] = (staged / "system").string();
Check(ResolveToolchain(in).mingwPrefixIsSystem, "a prefix with a C library's features.h holds a Linux system");
in.env["CRAFTER_MINGW_DIR"] = (staged / "dedicated").string();
Check(!ResolveToolchain(in).mingwPrefixIsSystem, "a prefix of mingw-w64 and Windows libraries does not");
Check(!ResolveToolchain({}).mingwPrefixIsSystem, "no prefix, nothing to show");
fs::remove_all(staged);
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
}
{
// Sandboxed, the host's /usr stands in for the missing mingw-w64,
// unless CRAFTER_MINGW_DIR says otherwise.
const fs::path host = fs::temp_directory_path() / "crafter-build-toolchain-host";
fs::remove_all(host);
ToolchainInputs in;
in.hostRoot = host;
fix: give the mingw target a sysroot that holds mingw-w64 alone Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree that holds mingw-w64 alone, and adds its include/ to the include path and its lib/ to the library path. For /usr those are the Linux system's own: every header mingw-w64 lacks came from glibc, and libstdc++'s <ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read __libc_single_threaded, which nothing on Windows defines. Every program that used one failed to link, among them every Crafter.Graphics Win32 executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak (#43). The sysroot is now MingwSysroot(): a directory in the cache, one per prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/ and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds mingw-w64, gcc's version and libstdc++ exactly as it does through the mingw gcc on PATH, and nothing else. It is made by the first mingw compile, link or CMake configure of a process, so no other command writes to the cache for it and a failure to make it is that build's error. The Toolchain field is now mingwPrefix, since clang never gets the prefix itself, and it is made absolute when the toolchain is resolved, as it ends up in link targets and directory names. Messages keep naming paths under the prefix rather than under the view. The prefix is also part of the mingw target's VariantId and of its std PCM directory. Units do not track the std PCM they import: when the fixed release built in a directory a broken build had left, the std module was rebuilt and the objects compiled against glibc's headers were only relinked, failing as before. As with a compiler update, a new directory is what gets them rebuilt. It also keeps two releases that share a cache, as the Flatpak's branches do, from rebuilding one std PCM under each other's PCMs, and two prefixes apart. Without a prefix both stay as they were: clang's own discovery has not changed. Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32 loses the directory gcc's version names, so <string> is not found, and clang ignores --gcc-toolchain and --gcc-install-dir for this target. Fixes #43 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
Check(ResolveToolchain(in).mingwPrefix.empty(), "a host without mingw-w64 gives no mingw prefix");
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
fs::create_directories(host / "usr" / "x86_64-w64-mingw32");
fix: give the mingw target a sysroot that holds mingw-w64 alone Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree that holds mingw-w64 alone, and adds its include/ to the include path and its lib/ to the library path. For /usr those are the Linux system's own: every header mingw-w64 lacks came from glibc, and libstdc++'s <ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read __libc_single_threaded, which nothing on Windows defines. Every program that used one failed to link, among them every Crafter.Graphics Win32 executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak (#43). The sysroot is now MingwSysroot(): a directory in the cache, one per prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/ and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds mingw-w64, gcc's version and libstdc++ exactly as it does through the mingw gcc on PATH, and nothing else. It is made by the first mingw compile, link or CMake configure of a process, so no other command writes to the cache for it and a failure to make it is that build's error. The Toolchain field is now mingwPrefix, since clang never gets the prefix itself, and it is made absolute when the toolchain is resolved, as it ends up in link targets and directory names. Messages keep naming paths under the prefix rather than under the view. The prefix is also part of the mingw target's VariantId and of its std PCM directory. Units do not track the std PCM they import: when the fixed release built in a directory a broken build had left, the std module was rebuilt and the objects compiled against glibc's headers were only relinked, failing as before. As with a compiler update, a new directory is what gets them rebuilt. It also keeps two releases that share a cache, as the Flatpak's branches do, from rebuilding one std PCM under each other's PCMs, and two prefixes apart. Without a prefix both stay as they were: clang's own discovery has not changed. Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32 loses the directory gcc's version names, so <string> is not found, and clang ignores --gcc-toolchain and --gcc-install-dir for this target. Fixes #43 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
Check(ResolveToolchain(in).mingwPrefix == host / "usr", "a host with mingw-w64 gives its /usr as the mingw prefix");
fix(mingw): view only a prefix that holds a Linux system The first fix showed clang every mingw prefix through a view of its mingw-w64 parts. That broke a prefix that holds mingw-w64 alone, where Windows libraries installed with --prefix land in include/ and lib/: with CRAFTER_MINGW_DIR=/opt/mingw, <SDL2/SDL.h> and -lSDL2 were no longer found. Such a prefix is clang's sysroot whole again. Only a prefix that also holds a Linux system gets the view (Toolchain::mingwPrefixIsSystem): the host's /usr inside the Flatpak, whatever its include/ holds, and a prefix whose include/ has a C library's features.h, which every glibc and musl header includes and mingw-w64 has no use for, as CRAFTER_MINGW_DIR=/usr has. MakeMingwSysroot makes the view in the cache, or in $XDG_RUNTIME_DIR/crafter.build when the cache's filesystem takes no symbolic links (vfat, exFAT, CIFS without unix extensions): the XDG spec promises them there, and nothing else in the cache needs them. Its path is absolute, because CMake runs the compiler from directories of its own and a relative cache did not resolve from there. The prefix is resolved once, absolute and lexically normal, so /opt/mingw/ and /opt/mingw are one prefix and not two builds. clang names a header by the path it found it at, so every diagnostic from a viewed mingw-w64 header named a hashed cache directory. RunBuildCommand now writes paths into the view as paths into the prefix, as BuildStdPcm did for its manifest messages already. Also from the review of PR #44: SysrootCompileFlags builds a mingw command, which can make a view, so it redirects the cache like the other tests; the manifest filter's comment names what it still keeps out; the C path asks MingwSysroot() once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:58:30 +01:00
Check(ResolveToolchain(in).mingwPrefixIsSystem, "the host's /usr is the host's system, with or without its C library's headers");
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
in.env["CRAFTER_MINGW_DIR"] = "/opt/mingw";
fix: give the mingw target a sysroot that holds mingw-w64 alone Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree that holds mingw-w64 alone, and adds its include/ to the include path and its lib/ to the library path. For /usr those are the Linux system's own: every header mingw-w64 lacks came from glibc, and libstdc++'s <ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read __libc_single_threaded, which nothing on Windows defines. Every program that used one failed to link, among them every Crafter.Graphics Win32 executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak (#43). The sysroot is now MingwSysroot(): a directory in the cache, one per prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/ and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds mingw-w64, gcc's version and libstdc++ exactly as it does through the mingw gcc on PATH, and nothing else. It is made by the first mingw compile, link or CMake configure of a process, so no other command writes to the cache for it and a failure to make it is that build's error. The Toolchain field is now mingwPrefix, since clang never gets the prefix itself, and it is made absolute when the toolchain is resolved, as it ends up in link targets and directory names. Messages keep naming paths under the prefix rather than under the view. The prefix is also part of the mingw target's VariantId and of its std PCM directory. Units do not track the std PCM they import: when the fixed release built in a directory a broken build had left, the std module was rebuilt and the objects compiled against glibc's headers were only relinked, failing as before. As with a compiler update, a new directory is what gets them rebuilt. It also keeps two releases that share a cache, as the Flatpak's branches do, from rebuilding one std PCM under each other's PCMs, and two prefixes apart. Without a prefix both stay as they were: clang's own discovery has not changed. Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32 loses the directory gcc's version names, so <string> is not found, and clang ignores --gcc-toolchain and --gcc-install-dir for this target. Fixes #43 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
Check(ResolveToolchain(in).mingwPrefix == "/opt/mingw", "CRAFTER_MINGW_DIR wins over the host's");
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
fs::remove_all(host);
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
}
{
// The cache key separates toolchains that would build different
// PCMs and nothing else.
Toolchain a = ResolveToolchain({});
a.clangVersion = "clang version 22.1.8";
Toolchain b = a;
b.clangVersion = "Arch Linux clang version 22.1.9";
Toolchain c = a;
c.libcxxDir = "/opt/libcxx";
Toolchain d = ResolveToolchain({});
d.clangVersion = a.clangVersion;
Check(a.Key().starts_with("22.1.8-"), std::format("the key starts with the short clang version (got '{}')", a.Key()));
Check(b.Key().starts_with("22.1.9-"), "the short version is parsed past a distro prefix");
Check(a.Key() != b.Key(), "a different clang version changes the key");
Check(a.Key() != c.Key(), "a different libc++ location changes the key");
Check(a.Key() == d.Key(), "the key is deterministic");
}
{
// The real toolchain on this machine resolves and the probes work;
// HostTarget is the probe's answer.
const Toolchain& real = GetToolchain();
Check(real.error.empty(), std::format("the real toolchain resolves (error: '{}')", real.error));
Check(!real.hostTriple.empty() && real.hostTriple == HostTarget(), "HostTarget comes from the toolchain probe");
Check(!real.clangVersion.empty(), "the clang version was probed");
Check(!real.Key().starts_with("unknown-"), std::format("the real clang version parses (got '{}')", real.Key()));
}
fs::remove_all(scratch, ec);
return Failures == 0 ? 0 : 1;
}
Reference in a new issue Copy permalink