Crafter.Build/project.cpp

145 lines
7.5 KiB
C++
Raw Permalink Normal View History

2026-07-23 01:24:42 +02:00
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
2026-04-27 07:04:42 +02:00
import std;
import Crafter.Build;
#pragma crafter dependency "https://forgejo.catcrafts.net/Catcrafts/Crafter.LintRules.git"
import Crafter.LintRules;
2026-04-27 07:04:42 +02:00
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
2026-05-12 01:16:40 +02:00
std::vector<std::string> depArgs(args.begin(), args.end());
2026-05-27 04:36:11 +02:00
Configuration* math = GitProject({
.source = { "https://forgejo.catcrafts.net/Catcrafts/Crafter.Math.git" },
.args = depArgs,
});
Configuration* asset = GitProject({
.source = { "https://forgejo.catcrafts.net/Catcrafts/Crafter.Asset.git" },
.args = depArgs,
});
2026-05-12 01:16:40 +02:00
2026-07-23 01:24:42 +02:00
static auto CrafterBuildLib = std::make_unique<Configuration>();
CrafterBuildLib->path = "./";
CrafterBuildLib->name = "crafter.build-lib";
CrafterBuildLib->outputName = "crafter-build";
ApplyStandardArgs(*CrafterBuildLib, args);
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
// Monolithic: project.so resolves Crafter:: symbols from the running
// executable, so the library is only ever linked statically into it.
CrafterBuildLib->type = ConfigurationType::LibraryStatic;
2026-07-23 01:24:42 +02:00
CrafterBuildLib->dependencies = { math, asset };
CrafterBuildLib->defines.push_back({"CRAFTER_BUILD_HAS_ASSET", ""});
musl host support: bootstrap and run on Alpine (x86_64-alpine-linux-musl) The Linux host code paths were gated on the x86_64_pc_linux_gnu target define alone, and LoadProject hardcoded --target=x86_64-pc-linux-gnu when compiling project.cpp and the module PCMs. Both break on musl hosts: the glibc release launcher cannot run there at all (no glibc loader; gcompat lacks libgcc_s and the __isoc23_* symbols), and a musl-built launcher would still compile project.cpp for the wrong libc. - The host gates test clang's __linux__ instead of one triple's define: they mean "Linux host", whatever the libc (or, later, the CPU). - LoadProject derives the host triple from HostTarget() (clang's -print-target-triple), which is also more correct on glibc distros whose triple isn't x86_64-pc-linux-gnu. - project.cpp / the cmake libc++ flags treat any *-linux-gnu or *-linux-musl target as Linux. - build.sh targets the host toolchain's triple (CRAFTER_BUILD_TARGET overrides) and derives the per-target define from it. On Arch this is byte-for-byte the previous behaviour. - CI: a release-musl job bootstraps on alpine:edge, runs the tests, and attaches crafter-build-linux-x86_64-musl-v2.tar.gz to the rolling latest release (via the API, so the glibc assets survive). Dynamic against musl libc++: the launcher dlopens project.so, so it cannot be fully static. First consumer: imsd's package CI, which cross-compiles for aarch64 from an Alpine container. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 23:49:00 +02:00
// Crafter.Build-Lint includes libclang's clang-c/Index.h. glibc distros
// keep it in /usr/include; Alpine keeps LLVM under /usr/lib/llvmNN, so ask
// llvm-config and add the dir when it is not already on the default path
// (never /usr/include itself: that breaks libc++'s #include_next).
if (CrafterBuildLib->target.ends_with("-linux-gnu") || CrafterBuildLib->target.ends_with("-linux-musl")) {
auto llvmInc = RunCommandWithTimeout("llvm-config --includedir 2>/dev/null", std::chrono::seconds(10));
std::string incDir = llvmInc.output;
while (!incDir.empty() && (incDir.back() == '\n' || incDir.back() == '\r')) incDir.pop_back();
if (llvmInc.exitCode == 0 && !incDir.empty() && incDir != "/usr/include") {
CrafterBuildLib->compileFlags.push_back(std::format("-I{}", incDir));
}
}
{
2026-07-23 01:24:42 +02:00
std::array<fs::path, 11> interfaces = {
"interfaces/Crafter.Build",
"interfaces/Crafter.Build-Shader",
"interfaces/Crafter.Build-Platform",
"interfaces/Crafter.Build-Interface",
"interfaces/Crafter.Build-Implementation",
"interfaces/Crafter.Build-External",
"interfaces/Crafter.Build-Clang",
"interfaces/Crafter.Build-Test",
2026-07-23 01:24:42 +02:00
"interfaces/Crafter.Build-Lint",
2026-04-29 03:27:11 +02:00
"interfaces/Crafter.Build-Progress",
2026-05-12 01:16:40 +02:00
"interfaces/Crafter.Build-Asset",
};
2026-07-23 01:24:42 +02:00
std::array<fs::path, 10> implementations = {
"implementations/Crafter.Build-Shader",
"implementations/Crafter.Build-Platform",
"implementations/Crafter.Build-Interface",
"implementations/Crafter.Build-Implementation",
"implementations/Crafter.Build-External",
"implementations/Crafter.Build-Clang",
"implementations/Crafter.Build-Test",
2026-07-23 01:24:42 +02:00
"implementations/Crafter.Build-Lint",
2026-04-29 03:27:11 +02:00
"implementations/Crafter.Build-Progress",
2026-05-12 01:16:40 +02:00
"implementations/Crafter.Build-Asset",
};
2026-07-23 01:24:42 +02:00
CrafterBuildLib->GetInterfacesAndImplementations(interfaces, implementations);
}
2026-07-23 01:24:42 +02:00
ExternalDependency& glslang = CrafterBuildLib->externalDependencies.emplace_back();
2026-04-27 07:04:42 +02:00
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "GenericCodeGen", "glslang", "OSDependent", "MachineIndependent", "glslang-default-resource-limits" };
Configuration cfg;
cfg.path = "./";
cfg.name = "crafter.build-exe";
cfg.outputName = "crafter-build";
2026-04-29 18:59:01 +02:00
ApplyStandardArgs(cfg, args);
cfg.type = ConfigurationType::Executable;
2026-07-23 01:24:42 +02:00
cfg.dependencies = { CrafterBuildLib.get() };
{
std::array<fs::path, 0> interfaces = {};
std::array<fs::path, 1> implementations = { "implementations/main" };
cfg.GetInterfacesAndImplementations(interfaces, implementations);
}
musl host support: bootstrap and run on Alpine (x86_64-alpine-linux-musl) The Linux host code paths were gated on the x86_64_pc_linux_gnu target define alone, and LoadProject hardcoded --target=x86_64-pc-linux-gnu when compiling project.cpp and the module PCMs. Both break on musl hosts: the glibc release launcher cannot run there at all (no glibc loader; gcompat lacks libgcc_s and the __isoc23_* symbols), and a musl-built launcher would still compile project.cpp for the wrong libc. - The host gates test clang's __linux__ instead of one triple's define: they mean "Linux host", whatever the libc (or, later, the CPU). - LoadProject derives the host triple from HostTarget() (clang's -print-target-triple), which is also more correct on glibc distros whose triple isn't x86_64-pc-linux-gnu. - project.cpp / the cmake libc++ flags treat any *-linux-gnu or *-linux-musl target as Linux. - build.sh targets the host toolchain's triple (CRAFTER_BUILD_TARGET overrides) and derives the per-target define from it. On Arch this is byte-for-byte the previous behaviour. - CI: a release-musl job bootstraps on alpine:edge, runs the tests, and attaches crafter-build-linux-x86_64-musl-v2.tar.gz to the rolling latest release (via the API, so the glibc assets survive). Dynamic against musl libc++: the launcher dlopens project.so, so it cannot be fully static. First consumer: imsd's package CI, which cross-compiles for aarch64 from an Alpine container. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 23:49:00 +02:00
// Linux hosts, glibc or musl (Alpine): -ldl is a libc stub on musl, harmless.
const bool linuxTarget = cfg.target.ends_with("-linux-gnu") || cfg.target.ends_with("-linux-musl");
if (linuxTarget) {
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
}
2026-05-27 19:45:05 +02:00
// Self-tests link the local crafter-build library and exercise it in
// process. The harness (whichever crafter-build invokes `test`) compiles
// these against the *installed* share/crafter-build .cppm files, then
2026-07-23 01:24:42 +02:00
// links each test exe against CrafterBuildLib built from the local
2026-05-27 19:45:05 +02:00
// sources — so the code under test is whatever's in this checkout.
// Mirrors how downstream consumers link their own libraries into tests.
musl host support: bootstrap and run on Alpine (x86_64-alpine-linux-musl) The Linux host code paths were gated on the x86_64_pc_linux_gnu target define alone, and LoadProject hardcoded --target=x86_64-pc-linux-gnu when compiling project.cpp and the module PCMs. Both break on musl hosts: the glibc release launcher cannot run there at all (no glibc loader; gcompat lacks libgcc_s and the __isoc23_* symbols), and a musl-built launcher would still compile project.cpp for the wrong libc. - The host gates test clang's __linux__ instead of one triple's define: they mean "Linux host", whatever the libc (or, later, the CPU). - LoadProject derives the host triple from HostTarget() (clang's -print-target-triple), which is also more correct on glibc distros whose triple isn't x86_64-pc-linux-gnu. - project.cpp / the cmake libc++ flags treat any *-linux-gnu or *-linux-musl target as Linux. - build.sh targets the host toolchain's triple (CRAFTER_BUILD_TARGET overrides) and derives the per-target define from it. On Arch this is byte-for-byte the previous behaviour. - CI: a release-musl job bootstraps on alpine:edge, runs the tests, and attaches crafter-build-linux-x86_64-musl-v2.tar.gz to the rolling latest release (via the API, so the glibc assets survive). Dynamic against musl libc++: the launcher dlopens project.so, so it cannot be fully static. First consumer: imsd's package CI, which cross-compiles for aarch64 from an Alpine container. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 23:49:00 +02:00
if (linuxTarget) {
2026-07-23 01:24:42 +02:00
cfg.AddTest("HelloWorld").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("StaticLib").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ModuleInterface").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("DependencyLink").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("IncrementalInterfaceChange").Dependencies({ CrafterBuildLib.get() });
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
cfg.AddTest("TransitiveInterfaceChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("IncrementalHeaderChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("CompilerWarning").Dependencies({ CrafterBuildLib.get() });
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
cfg.AddTest("CleanProject").Dependencies({ CrafterBuildLib.get() });
2026-07-23 01:24:42 +02:00
cfg.AddTest("ShaderCompile").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("StandardArgs").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("TestRunnerSpec").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("VariantId").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasiBrowserRuntime").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasmVariants").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("RunSingleTestExit").Dependencies({ CrafterBuildLib.get() });
// LoadProject dlopens the synthesized project.so, which references
// Crafter:: symbols (HostTarget, Configuration ctors) that have to be
// visible from the test exe — same wiring crafter-build itself uses
// for project.so.
2026-07-23 01:24:42 +02:00
cfg.AddTest("ConcurrentCacheRace").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
// Same LoadProject wiring as ConcurrentCacheRace above.
cfg.AddTest("HostCacheSourceStamp").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
cfg.AddTest("ProjectDependency").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
2026-07-23 01:24:42 +02:00
cfg.AddTest("ConcurrentDependencyReset").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("Lint").Dependencies({ CrafterBuildLib.get() });
2026-05-27 19:45:05 +02:00
}
// Dogfood: the house-style rules, imported from their own repo through
// the pragma above. Report rules gate `crafter-build lint`; transform
// rules also auto-fix under `crafter-build format`.
LintRules::AddRules(cfg);
2026-07-23 01:24:42 +02:00
2026-04-27 07:04:42 +02:00
return cfg;
}