Crafter.Build/project.cpp

124 lines
6 KiB
C++
Raw Normal View History

//SPDX-License-Identifier: LGPL-3.0-only
//SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
2026-04-27 07:04:42 +02:00
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
2026-05-12 01:16:40 +02:00
std::vector<std::string> depArgs(args.begin(), args.end());
2026-05-27 04:36:11 +02:00
Configuration* math = GitProject({
.source = { "https://forgejo.catcrafts.net/Catcrafts/Crafter.Math.git" },
.args = depArgs,
});
Configuration* asset = GitProject({
.source = { "https://forgejo.catcrafts.net/Catcrafts/Crafter.Asset.git" },
.args = depArgs,
});
2026-05-12 01:16:40 +02:00
static auto crafterBuildLib = std::make_unique<Configuration>();
crafterBuildLib->path = "./";
crafterBuildLib->name = "crafter.build-lib";
crafterBuildLib->outputName = "crafter-build";
2026-04-29 18:59:01 +02:00
ApplyStandardArgs(*crafterBuildLib, args);
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
// Windows builds (native msvc via build.cmd or cross-compiled mingw from
// Linux) need a DLL + import lib + launcher exe so LoadProject can
// compile project.cpp against a stable ABI boundary. Linux is monolithic.
2026-04-29 18:59:01 +02:00
crafterBuildLib->type = (crafterBuildLib->target == "x86_64-w64-mingw32" || crafterBuildLib->target == "x86_64-pc-windows-msvc")
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
? ConfigurationType::LibraryDynamic
: ConfigurationType::LibraryStatic;
2026-05-27 04:36:11 +02:00
2026-05-12 01:16:40 +02:00
crafterBuildLib->dependencies = { math, asset };
crafterBuildLib->defines.push_back({"CRAFTER_BUILD_HAS_ASSET", ""});
{
2026-05-12 01:16:40 +02:00
std::array<fs::path, 10> interfaces = {
"interfaces/Crafter.Build",
"interfaces/Crafter.Build-Shader",
"interfaces/Crafter.Build-Platform",
"interfaces/Crafter.Build-Interface",
"interfaces/Crafter.Build-Implementation",
"interfaces/Crafter.Build-External",
"interfaces/Crafter.Build-Clang",
"interfaces/Crafter.Build-Test",
2026-04-29 03:27:11 +02:00
"interfaces/Crafter.Build-Progress",
2026-05-12 01:16:40 +02:00
"interfaces/Crafter.Build-Asset",
};
2026-05-12 01:16:40 +02:00
std::array<fs::path, 9> implementations = {
"implementations/Crafter.Build-Shader",
"implementations/Crafter.Build-Platform",
"implementations/Crafter.Build-Interface",
"implementations/Crafter.Build-Implementation",
"implementations/Crafter.Build-External",
"implementations/Crafter.Build-Clang",
"implementations/Crafter.Build-Test",
2026-04-29 03:27:11 +02:00
"implementations/Crafter.Build-Progress",
2026-05-12 01:16:40 +02:00
"implementations/Crafter.Build-Asset",
};
crafterBuildLib->GetInterfacesAndImplementations(interfaces, implementations);
}
ExternalDependency& glslang = crafterBuildLib->externalDependencies.emplace_back();
2026-04-27 07:04:42 +02:00
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
// mingw cross-build: skip the standalone executable. We only consume the
// libraries, and glslang.exe pulls in libgcc_eh which needs pthread that
// mingw-w64 doesn't link by default.
2026-04-29 18:59:01 +02:00
if (crafterBuildLib->target == "x86_64-w64-mingw32") {
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
glslang.options.push_back("-DENABLE_GLSLANG_BINARIES=OFF");
}
2026-04-27 07:04:42 +02:00
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "GenericCodeGen", "glslang", "OSDependent", "MachineIndependent", "glslang-default-resource-limits" };
Configuration cfg;
cfg.path = "./";
cfg.name = "crafter.build-exe";
cfg.outputName = "crafter-build";
2026-04-29 18:59:01 +02:00
ApplyStandardArgs(cfg, args);
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { crafterBuildLib.get() };
{
std::array<fs::path, 0> interfaces = {};
std::array<fs::path, 1> implementations = { "implementations/main" };
cfg.GetInterfacesAndImplementations(interfaces, implementations);
}
2026-04-29 18:59:01 +02:00
if (cfg.target == "x86_64-pc-linux-gnu") {
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
}
2026-05-19 00:50:06 +02:00
if (cfg.target == "x86_64-w64-mingw32" || cfg.target == "x86_64-pc-windows-msvc") {
// winsock for the -r wasm port probe (bind/WSAStartup).
crafterBuildLib->linkFlags.push_back("-lws2_32");
}
2026-05-27 19:45:05 +02:00
// Self-tests link the local crafter-build library and exercise it in
// process. The harness (whichever crafter-build invokes `test`) compiles
// these against the *installed* share/crafter-build .cppm files, then
// links each test exe against crafterBuildLib built from the local
// sources — so the code under test is whatever's in this checkout.
// Mirrors how downstream consumers link their own libraries into tests.
if (cfg.target == "x86_64-pc-linux-gnu") {
cfg.AddTest("HelloWorld").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("StaticLib").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("ModuleInterface").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("DependencyLink").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("ShaderCompile").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("StandardArgs").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("TestRunnerSpec").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("VariantId").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("WasiBrowserRuntime").Dependencies({ crafterBuildLib.get() });
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
cfg.AddTest("WasmVariants").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("RunSingleTestExit").Dependencies({ crafterBuildLib.get() });
// LoadProject dlopens the synthesized project.so, which references
// Crafter:: symbols (HostTarget, Configuration ctors) that have to be
// visible from the test exe — same wiring crafter-build itself uses
// for project.so.
cfg.AddTest("ConcurrentCacheRace").Dependencies({ crafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
fix: scope per-build module-state reset to the config being built Build() resets each Module/ModulePartition's per-build `compiled`/`checked` flags so a reused Configuration re-evaluates mtimes. That reset recursed into cfg.dependencies — but dependency Configurations are shared across the build DAG and each is compiled concurrently by its own Build() call. A parent/sibling's recursive reset could therefore clear a shared dependency's module `compiled` atomic *after* that dependency's module-compile thread had set it true and exited, but before an intra-config waiter (its impl, or a dependent partition) ran compiled.wait(false). The waiter then blocked forever on a flag nothing would re-signal: the build froze mid-compile, idle, with no compiler process alive — exactly the hang in issue #16. Reset only the current configuration's own modules. Every config in the tree already gets its own Build() call (the per-PcmDir builder registered in depResults), which resets its own state at the top of that call, sequenced before its compile threads spawn. Cross-config module state is consulted only via PCM file mtimes and the depResults futures, never via these flags, so the narrower reset is correct and removes the data race entirely. Adds ConcurrentDependencyReset: builds a static-lib dependency fully, then builds a consumer that depends on it while the dependency is already cached in depResults (so it is never rebuilt), and asserts the consumer build leaves the dependency's module `compiled` flag intact. Fails deterministically on the old recursive reset; passes with the fix. Resolves #16 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 11:32:46 +00:00
cfg.AddTest("ConcurrentDependencyReset").Dependencies({ crafterBuildLib.get() });
2026-05-27 19:45:05 +02:00
}
2026-04-27 07:04:42 +02:00
return cfg;
}