Crafter.Build/project.cpp

120 lines
5.9 KiB
C++
Raw Permalink Normal View History

2026-04-27 07:04:42 +02:00
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
2026-05-12 01:16:40 +02:00
std::vector<std::string> depArgs(args.begin(), args.end());
2026-05-27 04:36:11 +02:00
Configuration* math = GitProject({
.source = { "https://forgejo.catcrafts.net/Catcrafts/Crafter.Math.git" },
.args = depArgs,
});
Configuration* asset = GitProject({
.source = { "https://forgejo.catcrafts.net/Catcrafts/Crafter.Asset.git" },
.args = depArgs,
});
2026-05-12 01:16:40 +02:00
static auto crafterBuildLib = std::make_unique<Configuration>();
crafterBuildLib->path = "./";
crafterBuildLib->name = "crafter.build-lib";
crafterBuildLib->outputName = "crafter-build";
2026-04-29 18:59:01 +02:00
ApplyStandardArgs(*crafterBuildLib, args);
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
// Windows builds (native msvc via build.cmd or cross-compiled mingw from
// Linux) need a DLL + import lib + launcher exe so LoadProject can
// compile project.cpp against a stable ABI boundary. Linux is monolithic.
2026-04-29 18:59:01 +02:00
crafterBuildLib->type = (crafterBuildLib->target == "x86_64-w64-mingw32" || crafterBuildLib->target == "x86_64-pc-windows-msvc")
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
? ConfigurationType::LibraryDynamic
: ConfigurationType::LibraryStatic;
2026-05-27 04:36:11 +02:00
2026-05-12 01:16:40 +02:00
crafterBuildLib->dependencies = { math, asset };
crafterBuildLib->defines.push_back({"CRAFTER_BUILD_HAS_ASSET", ""});
{
2026-05-12 01:16:40 +02:00
std::array<fs::path, 10> interfaces = {
"interfaces/Crafter.Build",
"interfaces/Crafter.Build-Shader",
"interfaces/Crafter.Build-Platform",
"interfaces/Crafter.Build-Interface",
"interfaces/Crafter.Build-Implementation",
"interfaces/Crafter.Build-External",
"interfaces/Crafter.Build-Clang",
"interfaces/Crafter.Build-Test",
2026-04-29 03:27:11 +02:00
"interfaces/Crafter.Build-Progress",
2026-05-12 01:16:40 +02:00
"interfaces/Crafter.Build-Asset",
};
2026-05-12 01:16:40 +02:00
std::array<fs::path, 9> implementations = {
"implementations/Crafter.Build-Shader",
"implementations/Crafter.Build-Platform",
"implementations/Crafter.Build-Interface",
"implementations/Crafter.Build-Implementation",
"implementations/Crafter.Build-External",
"implementations/Crafter.Build-Clang",
"implementations/Crafter.Build-Test",
2026-04-29 03:27:11 +02:00
"implementations/Crafter.Build-Progress",
2026-05-12 01:16:40 +02:00
"implementations/Crafter.Build-Asset",
};
crafterBuildLib->GetInterfacesAndImplementations(interfaces, implementations);
}
ExternalDependency& glslang = crafterBuildLib->externalDependencies.emplace_back();
2026-04-27 07:04:42 +02:00
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
// mingw cross-build: skip the standalone executable. We only consume the
// libraries, and glslang.exe pulls in libgcc_eh which needs pthread that
// mingw-w64 doesn't link by default.
2026-04-29 18:59:01 +02:00
if (crafterBuildLib->target == "x86_64-w64-mingw32") {
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
glslang.options.push_back("-DENABLE_GLSLANG_BINARIES=OFF");
}
2026-04-27 07:04:42 +02:00
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "GenericCodeGen", "glslang", "OSDependent", "MachineIndependent", "glslang-default-resource-limits" };
Configuration cfg;
cfg.path = "./";
cfg.name = "crafter.build-exe";
cfg.outputName = "crafter-build";
2026-04-29 18:59:01 +02:00
ApplyStandardArgs(cfg, args);
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { crafterBuildLib.get() };
{
std::array<fs::path, 0> interfaces = {};
std::array<fs::path, 1> implementations = { "implementations/main" };
cfg.GetInterfacesAndImplementations(interfaces, implementations);
}
2026-04-29 18:59:01 +02:00
if (cfg.target == "x86_64-pc-linux-gnu") {
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
}
2026-05-19 00:50:06 +02:00
if (cfg.target == "x86_64-w64-mingw32" || cfg.target == "x86_64-pc-windows-msvc") {
// winsock for the -r wasm port probe (bind/WSAStartup).
crafterBuildLib->linkFlags.push_back("-lws2_32");
}
2026-05-27 19:45:05 +02:00
// Self-tests link the local crafter-build library and exercise it in
// process. The harness (whichever crafter-build invokes `test`) compiles
// these against the *installed* share/crafter-build .cppm files, then
// links each test exe against crafterBuildLib built from the local
// sources — so the code under test is whatever's in this checkout.
// Mirrors how downstream consumers link their own libraries into tests.
if (cfg.target == "x86_64-pc-linux-gnu") {
cfg.AddTest("HelloWorld").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("StaticLib").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("ModuleInterface").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("DependencyLink").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("ShaderCompile").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("StandardArgs").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("TestRunnerSpec").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("VariantId").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("WasiBrowserRuntime").Dependencies({ crafterBuildLib.get() });
cfg.AddTest("RunSingleTestExit").Dependencies({ crafterBuildLib.get() });
// LoadProject dlopens the synthesized project.so, which references
// Crafter:: symbols (HostTarget, Configuration ctors) that have to be
// visible from the test exe — same wiring crafter-build itself uses
// for project.so.
cfg.AddTest("ConcurrentCacheRace").Dependencies({ crafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
fix: scope per-build module-state reset to the config being built Build() resets each Module/ModulePartition's per-build `compiled`/`checked` flags so a reused Configuration re-evaluates mtimes. That reset recursed into cfg.dependencies — but dependency Configurations are shared across the build DAG and each is compiled concurrently by its own Build() call. A parent/sibling's recursive reset could therefore clear a shared dependency's module `compiled` atomic *after* that dependency's module-compile thread had set it true and exited, but before an intra-config waiter (its impl, or a dependent partition) ran compiled.wait(false). The waiter then blocked forever on a flag nothing would re-signal: the build froze mid-compile, idle, with no compiler process alive — exactly the hang in issue #16. Reset only the current configuration's own modules. Every config in the tree already gets its own Build() call (the per-PcmDir builder registered in depResults), which resets its own state at the top of that call, sequenced before its compile threads spawn. Cross-config module state is consulted only via PCM file mtimes and the depResults futures, never via these flags, so the narrower reset is correct and removes the data race entirely. Adds ConcurrentDependencyReset: builds a static-lib dependency fully, then builds a consumer that depends on it while the dependency is already cached in depResults (so it is never rebuilt), and asserts the consumer build leaves the dependency's module `compiled` flag intact. Fails deterministically on the old recursive reset; passes with the fix. Resolves #16 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 11:32:46 +00:00
cfg.AddTest("ConcurrentDependencyReset").Dependencies({ crafterBuildLib.get() });
2026-05-27 19:45:05 +02:00
}
2026-04-27 07:04:42 +02:00
return cfg;
}