Crafter.Build/README.md

Crafter Build

A C++26 modules build system. Project descriptions are written in C++ — no JSON, no Lua, no embedded DSL. You write a project.cpp that constructs a Configuration and returns it; Crafter Build compiles, loads, and executes it.

Status

• Linux: working end-to-end, packageable as a distro package. Verified on Arch Linux via PKGBUILD in a fresh container.
• Windows: working end-to-end, including the test runner. Bootstrap (build.cmd) produces crafter-build.exe + crafter-build.dll + crafter-build.lib; user project.cpp links against the import lib via the host-controlled DLL boundary. crafter-build test runs natively on Windows hosts; cross-OS runs back to Linux via WSL2 (--runner=wsl).

Quick start (Linux)

Bootstrap requires clang, cmake, git, lld, and libc++.

./build.sh                                                              # produces bin/crafter-build
CRAFTER_BUILD_HOME=$PWD/build ./bin/crafter-build                       # rebuild via project.cpp

For distro-packaged installs, crafter-build finds its modules at <prefix>/share/crafter-build/ automatically — no env var required.

To build the system as a distro package on Arch:

makepkg -si        # uses CRAFTER_BUILD_MARCH=x86-64-v3 by default for portability

Writing a project.cpp

Crafter Build loads a project.cpp from the current directory. The file exports one function that returns a populated Configuration:

import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;

extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
    Configuration cfg;
    cfg.path = "./";
    cfg.name = "myapp";
    cfg.outputName = "myapp";
    cfg.target = "x86_64-pc-linux-gnu";
    cfg.type = ConfigurationType::Executable;

    std::array<fs::path, 1> ifaces = { "interfaces/Hello" };
    std::array<fs::path, 2> impls  = { "implementations/Hello", "implementations/main" };
    cfg.GetInterfacesAndImplementations(ifaces, impls);

    return cfg;
}

Run crafter-build in that directory; outputs land at bin/myapp-<target>-<march>/myapp and intermediates at build/myapp-<target>-<march>/. Add -r to run the produced executable straight after a successful build (host targets only — for cross targets use crafter-build test --runner=...).

Dependencies

Three kinds, all fetched/built incrementally:

Cross-project Crafter sub-projects — point cfg.dependencies at another Configuration*. Cross-project module imports (import OtherProject;) are tracked per-import: only the modules that actually import a changed dep rebuild.

External git+cmake deps — for things like glslang. Declare in cfg.externalDependencies:

ExternalDependency& glslang = cfg.externalDependencies.emplace_back();
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "glslang", "OSDependent" };

Clones live at ~/.cache/crafter.build/external/<name>-<hash>/, keyed by (url, branch, commit, options) so different projects sharing the same dep with the same spec reuse the clone and cmake build, while projects with diverging specs each get their own cache entry. The clone is resilient to partial-clone recovery and CMake reconfigure is triggered automatically on options changes.

Header-only git deps — same as cmake deps but builder = ExternalBuilder::None. Just clones and propagates -I flags.

Install layout

<prefix>/bin/crafter-build               # the executable
<prefix>/lib/libcrafter-build.a          # static archive for downstream consumers
<prefix>/share/crafter-build/*.cppm      # module sources (rebuilt on user machine on first run)
~/.cache/crafter.build/<target>-<march>/ # std.pcm + Crafter.Build*.pcm, built locally

Sources ship instead of PCMs because libc++ ABI varies between machines — the user's machine builds its own PCMs against its own libc++ on first run (~6-8s one-time cost), making the install resilient to libc++ version differences across distros.

The static archive is for downstream projects that want to use Crafter.Build as a library (construct Configuration objects, call Build(), etc., directly in C++) without invoking the CLI. Link with -lcrafter-build plus the project's external dep libs (-lSPIRV -lglslang -lOSDependent -lMachineIndependent -lglslang-default-resource-limits -lGenericCodeGen) and -ldl (for LoadProject's dlopen path on Linux). On Windows the equivalent is bin/crafter-build-static.lib.

Build incrementality

Per-import precise tracking for both within-project and cross-project module dependencies:

• Touch lib/Hello.cppm → only consumers of Hello rebuild.
• Touch lib/Other.cppm → only consumers of Other rebuild.
• External CMake dep produces fresh .a files → whole project rebuilds (deliberately coarse — cmake-dep changes are rare).

Diamond deps (A → {B, C}; B → X; C → X) build X exactly once via a std::shared_future<BuildResult> cache.

Tests

Tests live under tests/<Name>/. The simplest case is a single C++ file:

tests/Smoke/main.cpp        # int main() — exit 0 = pass, nonzero = fail, 77 = skipped

Run them all:

crafter-build test

Auto-discovery walks tests/ one level deep. Three escalation layers:

1. tests/<Name>/main.cpp with no project.cpp — a Configuration is synthesized: top-level *.cpp files become implementations, interfaces/*.cppm become module interfaces, target = the run's --target= (host triple by default). Most tests need nothing more.
2. tests/<Name>/project.cpp — full control. Use this for tests with defines, dependencies, or a non-default target.
3. Folders starting with _ or . are skipped, so tests/_shared/ can hold cross-test code without becoming a test itself.

Test conventions:

• Exit codes: 0 = ✅ pass, nonzero = ❌ fail, 77 = ⏭ skipped (autoconf convention; use it for runtime opt-outs like "tool not on PATH").
• Each test runs in its own subprocess — a segfault doesn't take the runner down.
• Default timeout is 60 s per test; runs in parallel up to hardware_concurrency().
• Stdout+stderr is captured and printed atomically on completion. Failed/crashed/timed-out tests also write to build/test-logs/<name>.log.
• A broken fixture project.cpp is reported as a single failed test, not a fatal error — other tests still run.

CLI:

crafter-build test                                              # run all
crafter-build test 'Unit*'                                      # glob-filter test names
crafter-build test --list                                       # enumerate without running
crafter-build test --target=x86_64-w64-mingw32                  # only tests for this target
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine
crafter-build test --jobs=1                                     # serial mode
crafter-build test --timeout=5                                  # override per-test timeout

Outcomes: ✅ pass, ❌ fail, 💥 crash (with signal name), ⏱ timeout, ⏭ skipped. The runner exits 0 only if every non-skipped test passed.

--target= filters the run to tests whose cfg.target matches; default is the host triple. Per-target tests parse --target=... from their own args span and build for it; tests with hardcoded targets (e.g. host-only meta-tests) are filtered out under non-matching --target=.

Cross-target test runners

Test::runner runs the test binary somewhere other than this machine. Four factories:

TestRunner::Local()                                  // run on this machine (default)
TestRunner::Cmd("wine")                              // local exec via prefix command
TestRunner::Ssh("hostname")                          // remote linux via scp + ssh
TestRunner::SshWin("winhost")                        // remote windows via scp + ssh-cmd.exe
TestRunner::Wsl()                                    // Linux binaries via WSL2 (Windows host)

Cmd(cmd) is a generic <cmd> <bin> <args> prefix runner — covers wine, qemu-user, valgrind, gdbserver, anything with that shape. Probes via which <cmd> (or where on Windows hosts). The Ssh variants scp -r the test's whole bin dir, run the binary remotely, then clean up; each invocation uses a unique remote subdir so parallel runs don't collide. Wsl() mirrors the SSH model but copies the bundle into WSL's native filesystem (faster than executing in-place from /mnt/c) — useful when a Windows host wants to exercise Linux test binaries.

For elaborate setups (rsync, ssh ControlMaster, env vars on the spawn, QEMU_LD_PREFIX for sysroots), construct TestRunner directly — copy/exec/cleanup/probe are public string fields with {bundle}, {remote_bundle}, {bin}, {bin_name}, {args} placeholders.

For cross-arch builds, set cfg.sysroot on the Configuration to point at a sysroot for the target. BuildStdPcm and the compile/link commands both consume it: --sysroot=<path> is passed to clang, and <sysroot>/usr/share/libc++/v1/std.cppm is precompiled instead of the host's. The cheapest aarch64 sysroot is an extracted Arch Linux ARM rootfs plus libc++ + libc++abi + gcc installed into it via pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy … (point pacman at ALARM's mirrors via explicit config to avoid pulling host-arch packages). See tests/CrossArchAarch64/inner/ for a working sysroot+qemu example.

For --target=x86_64-w64-mingw32 (Windows from Linux), V2's pipeline cross-compiles as long as mingw-w64 is installed. Pair it with --runner=cmd:wine (no VM needed) or --runner=sshwin:winhost (real Windows). Full MSVC ABI (x86_64-pc-windows-msvc) cross-compile from Linux is not yet wired; use MinGW for now. -fuse-ld=lld is auto-injected on every link, and -lstdc++exp is auto-linked for the mingw target so user projects don't need to declare those flags.

Configuring runners

Two layers, CLI wins over env, both override the per-test default of Local():

Env var (persistent, per shell or in CI):

export CRAFTER_BUILD_RUNNER_x86_64_w64_mingw32=cmd:wine
export CRAFTER_BUILD_RUNNER_aarch64_linux_gnu=cmd:qemu-aarch64
crafter-build test --target=x86_64-w64-mingw32           # uses cmd:wine

The variable name is CRAFTER_BUILD_RUNNER_<NORMALIZED_TARGET> (target triple with - and . replaced by _).

CLI (per invocation, scoped to --target=):

crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine

Spec grammar:

local
cmd:<command>                  # cmd:wine, cmd:qemu-aarch64, cmd:valgrind, ...
ssh:<host>[:<remoteDir>]
sshwin:<host>[:<remoteDir>]    # cmd.exe-shell variant for Windows remotes
wsl[:<remoteDir>]              # Windows host -> Linux binaries via WSL2

Each runner carries a probe command (e.g. which wine, ssh -BatchMode=yes <host> true); if it fails, every test using that runner is reported as ⏭ skipped and the rest of the suite continues. Skipped tests count toward the summary but don't fail the suite (AllPassed() returns true on skip-only runs).

Linking the parent project's library

A test fixture can depend on the project's own library so it can import and exercise its API:

// tests/UnitFoo/project.cpp
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
    Configuration cfg;
    cfg.path = "tests/UnitFoo/";
    cfg.name = "UnitFoo";
    cfg.outputName = "UnitFoo";
    cfg.target = "x86_64-pc-linux-gnu";
    cfg.type = ConfigurationType::Executable;
    cfg.dependencies = { ParentLib("MyLib") };       // <-- the magic line

    std::array<fs::path, 0> ifaces = {};
    std::array<fs::path, 1> impls = { "main" };
    cfg.GetInterfacesAndImplementations(ifaces, impls);
    return cfg;
}

ParentLib(name) walks the parent project's dependency graph (and the parent itself) to find a Configuration* by Configuration::name. The build links it into the test exe, so import MyLib; resolves naturally. See examples/tests/ for the complete worked example.

Examples

examples/ contains progressively larger self-contained projects:

Example	Shows
hello-world	Smallest possible project: project.cpp + main.cpp
with-module	Adding a C++ module interface
library	Library + executable, linked via cfg.dependencies
tests	Auto-discovered tests + tests linking the parent library

Each builds standalone: cd examples/<name> && crafter-build.

Architecture

• Modules: Crafter.Build:Shader / :Platform / :Interface / :Implementation / :External / :Clang / :Test partitions, re-exported by the Crafter.Build umbrella.
• Build process: parallel — interface PCMs, implementation .o files, external dep clones, dep-config recursive builds, and shader compilations all spawn threads, sync at well-defined join points.
• LoadProject compiles project.cpp to a shared object (.so on Linux, .dll on Windows) and loads it. On Linux the host exe is linked with -Wl,--export-dynamic so the project's undefined symbols resolve back into the running binary. On Windows the host is split into crafter-build.dll (everything) + a thin crafter-build.exe launcher; the public Crafter::* API is annotated with a CRAFTER_API macro (__declspec(dllexport) when the host is built, __declspec(dllimport) when project.dll consumes it via the rebuilt user-cache PCMs), and the host passes -Wl,/EXPORT:CrafterBuildProject when linking project.dll so user project.cpp files don't need platform-gated annotations.

Compatibility / portability

• Linux: x86_64. Tested on Arch in a clean container.
• Windows: x86_64 MSVC ABI. build.cmd produces bin/crafter-build.{exe,dll,lib}; userprojs verified end-to-end (project.cpp → project.dll loaded by the host → userapp.exe built and run).

The CRAFTER_BUILD_MARCH and CRAFTER_BUILD_MTUNE env vars override the default -march=native / -mtune=native for distro CI builds:

CRAFTER_BUILD_MARCH=x86-64-v3 CRAFTER_BUILD_MTUNE=generic ./build.sh

Roadmap

• Target-triple auto-detection (clang -print-target-triple)
• Prefer system glslang when available, fall back to git+cmake
• Order-preserving deduplication of BuildResult.libs (current unordered_set doesn't preserve link order — only matters for cyclic static libs)

License

LGPL-3.0-only. See LICENSE.