Two crafter-build invocations sharing XDG_CACHE_HOME used to clobber each
other's writes to <cache>/<target>-<march>/std.pcm and the
Crafter.Build-*.pcm modules: each LoadProject path wrote directly to the
final path, so a reader could see a half-written file and die with
"malformed or corrupted precompiled file: 'can't skip to bit X from Y'"
(issue #14). Every BuildStdPcm / EnsureCrafterBuildPcms write now goes via
<final>.tmp.<pid>.<seq> and atomic-renames into place; concurrent writers
always see either the old or the new file, never torn bytes. The mingw-on-
Linux std.cppm copy is per-PID for the same reason. Adds a regression test
(ConcurrentCacheRace) that races four LoadProject() calls against a cold
scratch cache — reproduces the race 5/5 without the fix and passes 5/5
with it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The suite had only HelloWorld, which built and exited an empty exe. Add
in-process tests covering each public surface area users actually touch:
- StaticLib / ModuleInterface / DependencyLink — Build() against
fixtures for libraries, project-local module interfaces, and
cross-config module deps with link verification (runs the built exe).
- ShaderCompile — drives Shader::Compile directly, validates SPIR-V
magic + Check() idempotency.
- StandardArgs — covers --debug, --target=, --march=, --mtune=,
--lib/--shared promotions, and ArgQuery::Has / Get.
- TestRunnerSpec — FromSpec parse rules, ForTarget routing for host,
wasm32-wasip1, aarch64-linux-gnu (+ sysroot QEMU_LD_PREFIX),
i686 → qemu-i386 rewrite, mingw → wine on Linux hosts, FromEnv.
- VariantId — confirms type / debug / sysroot / defines / compileFlags /
target / march all perturb the cache key, plus PcmDir routing.
- WasiBrowserRuntime — calls EnableWasiBrowserRuntime, asserts the
three cfg.files entries get registered and index.html had its
template placeholder substituted.
- RunSingleTestExit — drives RunSingleTest against tiny sh scripts and
pins the documented exit-code mapping (0/77/non-zero) and the
Cmd-prefix runner path.
Closes#12.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
With test.toml + ForTarget covering the cross-arch + Windows-on-Linux
cases, the env-var-driven transport runners are dead weight. This commit
removes them and the retired tests that exercised the env-var plumbing:
- TestRunner::Ssh / SshWin / Wsl factories and their copy/exec/cleanup
template machinery.
- TestRunner::Shell enum (Host/Sh/Cmd) and the ShellQuoteSh helper —
only Host shell quoting is needed once the remote shells are gone.
- TestRunner::copy / cleanup / remoteDir / argsShell fields.
- WindowsPathToWsl and the {remote_bundle}/{bin_win}/{bundle_wsl}
placeholder substitution in RunSingleTest's transport branch.
- ParseRunnerSpec narrowed from {local, cmd, ssh, sshwin, wsl} to
{local, cmd} — the override hatch is preserved, just simpler.
- tests/SshRunner, tests/WindowsViaSsh, tests/QemuUser: these tested
the CRAFTER_BUILD_RUNNER_<target> → runner plumbing that has been
replaced by ForTarget. The runner derivation is exercised every
time CrossArchAarch64 / Wasi / WindowsViaWine runs.
- tests/UnitLib: ssh/sshwin spec assertions become "throws on bogus
spec" assertions.
Refs issue #8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collapses three outer-driver tests into declarative top-level fixtures:
- CrossArchAarch64: outer + inner pair becomes main.cpp + test.toml. The
'is this really an ARM aarch64 ELF?' artifact-introspection check is
dropped — qemu-aarch64 refuses to run wrong-arch ELFs anyway, so a
silent host-arch fallback would still fail the run.
- Wasi: outer + inner pair becomes main.cpp + test.toml. The WASM
magic-byte check is dropped on the same logic (wasmtime refuses
non-WASM input).
- Defines: simple defines-propagation smoke test becomes test.toml with
[defines] CRAFTER_TEST_FOO = "42".
Adds WindowsViaWine to replace the (forthcoming-deletion) WindowsViaSsh:
declarative target=x86_64-w64-mingw32 + requires=[tool:wine,
tool:x86_64-w64-mingw32-g++]. Exercises the new Wine runner and the
ForTarget derivation end-to-end.
Diamond and CrossProjectModule had speculative --target= reading that
made them attempt cross-compilation under the multi-target sweep. They
have no sysroot/toolchain plumbing, so those cross-builds always fail.
Hardcoded them to HostTarget(); cross-arch tests live in their own
test.toml from now on.
Refs issue #8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Vendors toml++ v3.4.0 as lib/toml.hpp and wires it into Crafter.Build-Test
to parse a declarative test.toml manifest (target/march/mtune/sysroot/
requires/timeout/args/defines). Test discovery now treats project.cpp and
test.toml as mutually exclusive: project.cpp stays the escape hatch for
outer-driver tests, test.toml gives downstream test authors a no-boilerplate
path.
Adds:
- TestRunner::Wine() and TestRunner::ForTarget(cfg) — runner is now derived
from cfg.target (Local for host, Wine for Windows-on-Linux, wasmtime for
WASI, qemu-<arch> with QEMU_LD_PREFIX for non-host Linux). The env-var
override CRAFTER_BUILD_RUNNER_<target> still wins as a power-user escape
hatch via FromEnv.
- Declarative preconditions: tool:<name>, file:<path>, env:<VAR> are
evaluated before the build; missing preconditions Skip without paying
the compile cost.
- Hard-fail-unless-declared: when a derived runner's tool is missing AND
the test didn't declare 'tool:<that>' in requires, the missing runner
is a Fail instead of a silent Skip. Surfaces broken cross-arch CI
config that previously hid as "skipped".
- Multi-target sweep: bare `crafter-build test` (no --target=) now
iterates every distinct test.toml-declared target plus the host, so
cross-arch tests run by default without the user needing to know which
targets exist. `--target=X` bypasses the sweep.
Test struct gains a `requires_` vector so project.cpp users can declare
preconditions too (matching what test.toml writes there).
Existing tests, factories (Ssh/SshWin/Wsl/Cmd), and CRAFTER_BUILD_RUNNER_*
machinery remain intact — this commit only adds; migration and deletion
follow in subsequent commits.
Refs issue #8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two Windows-only compile errors blocked the mingw cross-compile:
* `<winsock2.h>` transitively includes `<rpc.h>`, which defines
`#define interface struct`. The file uses `interface` as a loop
variable name in three for-loops, so on mingw it expanded into
`for(... struct : ...)` and cascaded into a wall of unrelated parse
errors. Undefine the macro right after the Windows headers in the
global module fragment.
* `static_cast<uint16_t>` in the port-probe helper failed because
mingw's `<stdint.h>` typedefs are in the global module fragment and
the C-namespace `::uint16_t` isn't anchored into the module purview
on this toolchain. `import std;` does export `std::uint16_t`, so
qualify the cast.
Verified by running `crafter-build --target=x86_64-w64-mingw32` end to
end and the full test suite (13 passed, 5 environment-skipped).
PRs now stop after bootstrap + tests. The per-march variant builds,
packaging, and upload-artifact steps reuse the same guard the rolling
'latest' tag/release steps already had, so artifacts are only produced
on push to master (or manual workflow_dispatch).
Two crafter-build invocations resolving to the same external dep both
cloned into <cache>/<name>-<hash>.tmp, corrupting each other's pack
tempfiles. Use a random per-invocation tmp suffix and treat a failed
rename whose destination now exists as the loser-of-the-race case —
discard the local clone and reuse the winner's.
Before LoadLibraryA on the project.dll, point Windows's loader at the
directory we already know holds the runtime DLLs the dll depends on
(libstdc++/libgcc/libwinpthread for mingw-host, c++.dll for msvc-host).
Lets a user run crafter-build.exe straight out of the release zip
without having to prepend C:\msys64\ucrt64\bin (or the libc++ bin
dir) to PATH first.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In-class inline methods on a module-exported class get the @<module>
linkage attachment, and clang does not emit their bodies into
consumers; the resulting external reference fails to resolve when a
project.dll on Windows tries to call ArgQuery::Has after consuming
ApplyStandardArgs's return value. Move the bodies to the implementation
unit and dllexport them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
cmd.exe treats single quotes as literal characters, so the existing
single-quote wrapping broke git/cmake invocations on a Windows host
("could not create leading directories of '<path>'").
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the single Linux + single Windows tarball with six self-
contained variants — three µarch levels × two OSes. End users on the
Releases page see them as six individually-downloadable archives:
crafter-build-{linux,windows}-x86_64-{v2,v3,v4}.{tar.gz,zip}
Pick the v-level your CPU supports:
v2: SSE4.2 baseline (every x86_64 since ~2011)
v3: AVX2 + FMA + BMI (Intel Haswell+, AMD Excavator+, ~2013+)
v4: AVX-512 (Skylake-X+, recent server-class)
How: the bootstrap binary stays at v2 because the CI SBC (N5105 / Tremont,
no AVX) can't execute v3 or v4 instructions and would crash before it
could rebuild itself for higher levels. Once bootstrap's done, the same
v2 binary is re-invoked with CRAFTER_BUILD_MARCH overridden per variant
to produce v2/v3/v4 outputs. Same again with --target=x86_64-w64-mingw32
for the Windows variants. ExternalDependency cache keys on (url, target,
march) already so glslang gets built per variant; subsequent CI runs hit
the cache.
Also drops the wrapper zip from workflow artifacts: instead of one
upload-artifact call producing crafter-build.zip containing six archives,
there's now one call per archive. The PR / run page shows six small
downloads named for what they are.
forgejo-release continues to upload release-dir/* as individual assets,
so the Releases page already has the right shape — this just brings the
workflow artifact UX in line.
Bumped the glslang cache key to v2 (one-time invalidation) since the
cache layout now needs space for six glslang builds rather than two.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Linux→mingw cross-compile now produces the same architectural shape as
build.cmd (DLL + import lib + launcher exe) instead of a single static
binary. The CI Windows artifact becomes a first-class drop-in: a user
on Windows can run crafter-build.exe against any project.cpp and have
it produce real Windows binaries — for either mingw or MSVC ABI.
What changed:
project.cpp: when target=mingw or target=msvc, crafter.build-lib is
built as LibraryDynamic instead of LibraryStatic so the link emits a
DLL + import lib (matching what build.cmd produces natively).
Crafter.Build-Clang.cpp Build():
- LibraryDynamic now branches per target — mingw emits <name>.dll +
lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll +
<name>.lib via /IMPLIB; unix unchanged.
- expectedOutputFor returns .dll for Windows-target dynamic libs.
- Executable on Windows host now branches per target: mingw target
uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps
the existing path. Both auto-copy LibraryDynamic dep DLLs + import
libs alongside the launcher exe (Windows resolves DLLs from the exe's
own directory at load time).
- Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so
CRAFTER_API resolves to dllimport in their PCMs.
- mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic
-lpthread so produced .exe/.dll don't depend on a particular
libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH
(avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole).
Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which
is now dead weight.
- -r flag resolves to an absolute path before std::system, otherwise
cmd.exe rejects "./bin/..." with "'.' is not recognized...".
Crafter.Build-Platform.cpp:
- Split the Windows-host block into shared shell helpers (#if MSVC ||
MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject /
EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm.
- Mingw-host LoadProject compiles project.cpp with --target=mingw,
--sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR),
-femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept
/EXPORT:NAME), and links against libcrafter-build.dll.a from the
launcher's directory.
- Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target
so a mingw-host crafter-build can also build msvc-target outputs
(uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when
the user sets cfg.target = "x86_64-pc-windows-msvc".
README adds a Quick start (Windows) section covering both build paths
(native MSVC via build.cmd and the cross-compiled mingw artifact),
documenting the msys2 UCRT toolchain prerequisite.
Verified end-to-end on the winvm:
- mingw target: cross-compiled crafter-build.exe builds hello-world's
project.cpp, compiles main.cpp, links a hello.exe that runs without
any custom PATH (only Windows system DLLs needed).
- msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe
linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Setting CRAFTER_BUILD_MARCH=x86-64-v2 made crafter-build emit the
mingw cross-compile to bin/crafter.build-exe-x86_64-w64-mingw32-x86-64-v2/
but the package step still hardcoded -native, causing cp to fail.
Use ${CRAFTER_BUILD_MARCH} in the path so workflow env and packaging
stay in sync.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two CI fixes from run #276 (got all the way through tests + mingw cross-
compile + packaging, only failed on artifact upload):
march: workflow now sets CRAFTER_BUILD_MARCH=x86-64-v2 / MTUNE=generic;
project.cpp reads both and applies them to the lib + exe Configurations
so the self-rebuild and mingw cross-compile honor the same baseline.
v3 is unusable on the runner — Intel N5105 (Tremont) has no AVX2, so
a v3 bootstrap binary wouldn't even start. v2 (SSE4.2) runs on the SBC
and on every x86_64 CPU since ~2011.
upload-artifact: pinned to v3. v4+ uses a GHES-only API that Forgejo
Actions doesn't implement; the v3 action stays on the older API that
Forgejo supports.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
archlinux:latest slim image has no local pacman master key and an
unpopulated upstream keyring, so:
- the archlinux-keyring upgrade fails with "no secret key available to
sign with" because pacman can't sign the keyring it's rewriting
- falling through to -Syu hits the original "unknown trust" errors on
libseccomp and zip
Run pacman-key --init then --populate archlinux before any pacman -S.
This is the documented bootstrap for slim Arch CI containers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
archlinux:latest container ships a snapshot keyring; packages signed by
keys added after the snapshot date fail PGP verification (zip-3.0-13 hit
this with a "signature from Robin Candau is unknown trust" error). Update
the keyring first via pacman -Sy archlinux-keyring, then -Syu the rest.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
WASI / wasm32 target support
- Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32")
- Skip -march/-mtune for wasm (clang rejects them)
- Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj
-D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync)
- .wasm output extension in expectedOutputFor and link command
- EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html +
runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL
set in the templated index.html so a single shim handles any output name
-r run flag in the CLI: build then exec the artifact (host targets only;
rejects libraries; auto .exe/.wasm extension handling)
CI pipeline (.forgejo/workflows/ci.yaml)
- Triggers: PR/push to master + manual dispatch
- Single arch-latest container job: install deps, bootstrap, self-rebuild,
run tests, cross-compile mingw, package both archives, upload artifacts
- Rolling 'latest' release published only on push/dispatch to master
mingw cross-compile from Linux now works end-to-end:
- ExternalDependency cache key includes target so per-target glslang builds
don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends
'd' to lib names and breaks linking); cross-compile cmake flags
(CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...)
- project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic
and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh
link pulls pthread which mingw doesn't link by default)
- mingw compile uses -femulated-tls so std::__once_callable etc reference
the same emutls symbols libstdc++ provides
- mingw link auto-adds -lstdc++exp -lpthread
GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux
+ Windows) now both use it instead of duplicating the resolution.
Examples reorg: hello-world, library, with-module, wasi, tests — each with
its own README. Tests reorg: per-test directory with inner/ fixture, no
shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
