Crafter Build

A C++26 modules build system. Project descriptions are written in C++ — no JSON, no Lua, no embedded DSL. You write a project.cpp that constructs a Configuration and returns it; Crafter Build compiles, loads, and executes it.

Status

• Linux: working end-to-end, packageable as a distro package. Verified on Arch Linux via PKGBUILD in a fresh container.
• Windows: working end-to-end. Bootstrap (build.cmd) produces crafter-build.exe + crafter-build.dll + crafter-build.lib; user project.cpp links against the import lib via the host-controlled DLL boundary.

Quick start (Linux)

Bootstrap requires clang, cmake, git, lld, and libc++.

./build.sh                                                              # produces bin/crafter-build
CRAFTER_BUILD_HOME=$PWD/build ./bin/crafter-build                       # rebuild via project.cpp

For distro-packaged installs, crafter-build finds its modules at <prefix>/share/crafter-build/ automatically — no env var required.

To build the system as a distro package on Arch:

makepkg -si        # uses CRAFTER_BUILD_MARCH=x86-64-v3 by default for portability

Writing a project.cpp

Crafter Build loads a project.cpp from the current directory. The file exports one function that returns a populated Configuration:

import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;

extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
    Configuration cfg;
    cfg.path = "./";
    cfg.name = "myapp";
    cfg.outputName = "myapp";
    cfg.target = "x86_64-pc-linux-gnu";
    cfg.type = ConfigurationType::Executable;

    std::array<fs::path, 1> ifaces = { "interfaces/Hello" };
    std::array<fs::path, 2> impls  = { "implementations/Hello", "implementations/main" };
    cfg.GetInterfacesAndImplementations(ifaces, impls);

    return cfg;
}

Run crafter-build in that directory; outputs land at bin/myapp-<target>-<march>/myapp and intermediates at build/myapp-<target>-<march>/.

Dependencies

Three kinds, all fetched/built incrementally:

Cross-project Crafter sub-projects — point cfg.dependencies at another Configuration*. Cross-project module imports (import OtherProject;) are tracked per-import: only the modules that actually import a changed dep rebuild.

External git+cmake deps — for things like glslang. Declare in cfg.externalDependencies:

ExternalDependency& glslang = cfg.externalDependencies.emplace_back();
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "glslang", "OSDependent" };

Clones live at ~/.cache/crafter.build/external/<name>-<hash>/, keyed by (url, branch, commit, options) so different projects sharing the same dep with the same spec reuse the clone and cmake build, while projects with diverging specs each get their own cache entry. The clone is resilient to partial-clone recovery and CMake reconfigure is triggered automatically on options changes.

Header-only git deps — same as cmake deps but builder = ExternalBuilder::None. Just clones and propagates -I flags.

Install layout

<prefix>/bin/crafter-build               # the executable
<prefix>/lib/libcrafter-build.a          # static archive for downstream consumers
<prefix>/share/crafter-build/*.cppm      # module sources (rebuilt on user machine on first run)
~/.cache/crafter.build/<target>-<march>/ # std.pcm + Crafter.Build*.pcm, built locally

Sources ship instead of PCMs because libc++ ABI varies between machines — the user's machine builds its own PCMs against its own libc++ on first run (~6-8s one-time cost), making the install resilient to libc++ version differences across distros.

The static archive is for downstream projects that want to use Crafter.Build as a library (construct Configuration objects, call Build(), etc., directly in C++) without invoking the CLI. Link with -lcrafter-build plus the project's external dep libs (-lSPIRV -lglslang -lOSDependent -lMachineIndependent -lglslang-default-resource-limits -lGenericCodeGen) and -ldl (for LoadProject's dlopen path on Linux). On Windows the equivalent is bin/crafter-build-static.lib.

Build incrementality

Per-import precise tracking for both within-project and cross-project module dependencies:

• Touch lib/Hello.cppm → only consumers of Hello rebuild.
• Touch lib/Other.cppm → only consumers of Other rebuild.
• External CMake dep produces fresh .a files → whole project rebuilds (deliberately coarse — cmake-dep changes are rare).

Diamond deps (A → {B, C}; B → X; C → X) build X exactly once via a std::shared_future<BuildResult> cache.

Tests

Tests are normal executables — any int main(), exit code 0 = pass, nonzero = fail, captured stdout+stderr is the failure message. There's no Crafter-specific entry-point convention, so a GoogleTest/Catch2/doctest binary works as-is.

Declare tests by pushing onto cfg.tests in your project.cpp:

Test t;
t.config.path = "./";
t.config.name = "VectorMath";
t.config.outputName = "VectorMath";
t.config.target = "x86_64-pc-linux-gnu";
t.config.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls  = { "tests/VectorMath" };
t.config.GetInterfacesAndImplementations(ifaces, impls);
cfg.tests.push_back(std::move(t));

Each test runs in its own subprocess so a segfault doesn't take the runner down. Default timeout is 60 s per test (--timeout=N overrides). Tests run in parallel up to hardware_concurrency(); pass --jobs=N to override. Per-test stdout+stderr is buffered and printed atomically on completion so parallel output stays readable, and is also written to build/test-logs/<name>.log when a test fails, crashes, or times out.

crafter-build test                  # build + run all tests
crafter-build test 'Vector*'        # glob-filter test names
crafter-build test --list           # enumerate without running
crafter-build test --jobs=1         # serial mode
crafter-build test --timeout=5      # override per-test timeout

Outcomes are reported as ✅ pass, ❌ fail, 💥 crash (with the signal name), or ⏱ timeout. The runner exits 0 only if every test passed.

Cross-target test runners

Test::runner runs the test binary somewhere other than this machine. Three factories ship today (Linux host only):

t.runner = TestRunner::Local();                                  // default
t.runner = TestRunner::QemuUser("qemu-aarch64");                 // cross-arch via qemu user-mode
t.runner = TestRunner::Ssh("winvm", "/tmp/crafter-test");        // cross-OS via scp + ssh

TestRunner::Ssh(host, remoteDir) ships the test's whole bin/<name>-<target>-<march>/ output dir via scp -r, runs the binary over ssh, then cleans up the remote temp dir. Each test invocation gets a unique remote subdir to keep parallel runs isolated. Assumes Unix-like shell on the remote (mkdir -p, rm -rf); for Windows remotes (e.g. cmd.exe) construct TestRunner directly with your own command templates — copy, exec, cleanup are public string fields with {bundle}, {remote_bundle}, {bin}, {bin_name}, {args} placeholders.

TestRunner::QemuUser(qemuBin) simply wraps <qemuBin> {bin} {args} — same machine, just a command prefix. Pair it with t.config.target = "aarch64-linux-gnu" (plus an aarch64 sysroot installed on the host) for cross-arch Linux testing.

For cross-arch builds, set cfg.sysroot on the Configuration to point at a sysroot for the target. BuildStdPcm and the compile/link commands both consume it: --sysroot=<path> is passed to clang, and <sysroot>/usr/share/libc++/v1/std.cppm is precompiled instead of the host's.

For aarch64 specifically, the cheapest sysroot is an extracted Arch Linux ARM rootfs plus libc++ + libc++abi + gcc installed into it via pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy … (point pacman at ALARM's mirrors with an explicit config to avoid pulling host-arch packages). Then cfg.sysroot = "<rootfs>"; plus a manual TestRunner whose exec template prepends QEMU_LD_PREFIX=<rootfs> qemu-aarch64 … so qemu-user finds ld-linux-aarch64.so.1. See tests/fixtures/cross-arch-aarch64/ for a working example.

For t.config.target = "x86_64-w64-mingw32" (Windows from Linux) — V2's existing build pipeline cross-compiles on Linux as long as mingw-w64 is installed. Combine with TestRunner::Ssh("winvm", ...) for end-to-end Windows-target testing from a Linux host. Full MSVC ABI (x86_64-pc-windows-msvc) cross-compile from Linux is not yet wired; use MinGW for now.

Global config: TestRunner::FromEnv

To avoid hardcoding hosts in every project's project.cpp, use TestRunner::FromEnv(target):

t.runner = TestRunner::FromEnv(t.config.target);

This reads the env var CRAFTER_BUILD_RUNNER_<NORMALIZED_TARGET> (target triple with - and . replaced by _). Set it once per shell or in CI:

export CRAFTER_BUILD_RUNNER_x86_64_w64_mingw32=ssh:winvm:/tmp/crafter-test
export CRAFTER_BUILD_RUNNER_aarch64_linux_gnu=qemu:qemu-aarch64

Spec grammar: local, qemu:<qemu-bin>, ssh:<host>, ssh:<host>:<remoteDir>, sshwin:<host>, sshwin:<host>:<remoteDir> (cmd.exe-shell variant for Windows remotes). Unset → falls back to the second arg of FromEnv (defaults to Local()).

For anything more elaborate (rsync, ssh ControlMaster, custom flags, env vars on the spawn, QEMU_LD_PREFIX for sysroots) — construct TestRunner manually with your own templates. The factories are sugar over the same fields.

Not yet supported: WSL2 / Windows-host execution (the runner currently requires a Linux host because RunCommandWithTimeout is Linux-only), MSVC ABI cross-compile from Linux.

Skipping unreachable runners

TestRunner carries a probe command that runs once per RunTests invocation (cached by runner.name); if it exits non-zero, every Test using that runner reports as ⏭ Skipped: runner '<name>' not available and the suite continues with everything else. The factories set sensible defaults: Ssh/SshWin probe via ssh -q -o BatchMode=yes -o ConnectTimeout=5 <host> …, QemuUser via which <qemu-bin>, Local is unconditionally available.

Effect: if you don't have a Windows VM at all, you don't have to do anything — the Windows variants in the in-tree suite skip cleanly and the Linux variants pass. If you do have one, set up an SSH host alias matching what the project.cpp expects (in this repo: winvm in ~/.ssh/config) and the Windows variants run automatically. Skipped tests count toward the summary but do not fail the suite (AllPassed() returns true on skip-only runs).

Per-(target, runner) tests

The 5 simple tests HelloWorld, WithModule, Defines, CrossProjectModule, Diamond are declared directly in the project's project.cpp, one entry per (target, runner) pair:

std::vector<TargetRunner> targets = {
    { "x86_64-pc-linux-gnu", TestRunner::Local(), {} },
    { "x86_64-w64-mingw32",  TestRunner::SshWin("winvm", "C:/temp/crafter-tests"), {"-lstdc++exp"} },
};

So ./bin/crafter-build test (no env vars) spawns one Test per (target, runner) pair, all in one report. With the winvm SSH alias reachable:

✅ HelloWorld (5ms)
✅ HelloWorld (sshwin:winvm) (2348ms)
✅ WithModule (3ms)
✅ WithModule (sshwin:winvm) (2204ms)
…
18 passed

Without winvm reachable, the SshWin variants skip:

✅ HelloWorld (5ms)
⏭  HelloWorld (sshwin:winvm) skipped: runner 'sshwin:winvm' not available
…

Local runner emits no parenthetical (existing default for the common case); cross-target runners show their name (e.g. sshwin:winvm, qemu:qemu-aarch64) so you see at a glance which environment ran each test.

The 8 outer-driver tests (Incremental, BuildError, RunnerClassification, QemuUser, SshRunner, WindowsViaSsh, CrossArchAarch64, Libraries) stay one-line in the report because they're meta-tests — the test driver is the test, validating host-side build behavior or runner integration. They're not multi-target.

Architecture

• Modules: Crafter.Build:Shader / :Platform / :Interface / :Implementation / :External / :Clang / :Test partitions, re-exported by the Crafter.Build umbrella.
• Build process: parallel — interface PCMs, implementation .o files, external dep clones, dep-config recursive builds, and shader compilations all spawn threads, sync at well-defined join points.
• LoadProject compiles project.cpp to a shared object (.so on Linux, .dll on Windows) and loads it. On Linux the host exe is linked with -Wl,--export-dynamic so the project's undefined symbols resolve back into the running binary. On Windows the host is split into crafter-build.dll (everything) + a thin crafter-build.exe launcher; the public Crafter::* API is annotated with a CRAFTER_API macro (__declspec(dllexport) when the host is built, __declspec(dllimport) when project.dll consumes it via the rebuilt user-cache PCMs), and the host passes -Wl,/EXPORT:CrafterBuildProject when linking project.dll so user project.cpp files don't need platform-gated annotations.

Compatibility / portability

• Linux: x86_64. Tested on Arch in a clean container.
• Windows: x86_64 MSVC ABI. build.cmd produces bin/crafter-build.{exe,dll,lib}; userprojs verified end-to-end (project.cpp → project.dll loaded by the host → userapp.exe built and run).

The CRAFTER_BUILD_MARCH and CRAFTER_BUILD_MTUNE env vars override the default -march=native / -mtune=native for distro CI builds:

CRAFTER_BUILD_MARCH=x86-64-v3 CRAFTER_BUILD_MTUNE=generic ./build.sh

Roadmap

• Target-triple auto-detection (clang -print-target-triple)
• Prefer system glslang when available, fall back to git+cmake
• Order-preserving deduplication of BuildResult.libs (current unordered_set doesn't preserve link order — only matters for cyclic static libs)

License

LGPL-3.0-only. See LICENSE.