Crafter.Build/README.md

# Crafter Build

A C++26 modules build system. Project descriptions are written in C++ with no JSON, no Lua, no embedded DSL. You write a `project.cpp` that constructs a `Configuration` and returns it. Crafter Build compiles, loads, and executes it.

## Install

crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler:

```bash
flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo
flatpak install catcrafts net.catcrafts.CrafterBuild
alias crafter-build='flatpak run net.catcrafts.CrafterBuild'
```

Pre-release builds of `master` are the `beta` branch (`flatpak install catcrafts net.catcrafts.CrafterBuild//beta`), and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-<version>.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host.

## Writing a project.cpp

Crafter Build loads a `project.cpp` from the current directory. The file exports one function that returns a populated `Configuration`:

```cpp
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;

extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
    Configuration cfg;
    cfg.path = "./";
    cfg.name = "myapp";
    cfg.outputName = "myapp";
    cfg.type = ConfigurationType::Executable;

    std::array<fs::path, 1> ifaces = { "interfaces/Hello" };
    std::array<fs::path, 2> impls  = { "implementations/Hello", "implementations/main" };
    cfg.GetInterfacesAndImplementations(ifaces, impls);

    return cfg;
}
```

Run `crafter-build` in that directory, outputs land at `bin/myapp-<target>-<march>-<mtune>-<hash>/myapp` and intermediates at `build/myapp-<target>-<march>-<mtune>-<hash>/`. The hash covers everything that changes what is compiled (build type, defines, flags, sysroot, the compiler itself), so two such builds never share a directory. After a compiler update, a new crafter-build release included, the first build starts a fresh directory and recompiles. `crafter-build clean` removes the ones left behind. Add `-r` to run the produced executable straight after a successful build.

## Dependencies

Three kinds, all fetched/built incrementally:

**Cross-project Crafter sub-projects** — point `cfg.dependencies` at another `Configuration*`. Cross-project module imports (`import OtherProject;`) are tracked per-import: only the modules that actually import a changed dep rebuild.

**External git+cmake deps** — for things like glslang. Declare in `cfg.externalDependencies`:

```cpp
ExternalDependency& glslang = cfg.externalDependencies.emplace_back();
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "glslang", "OSDependent" };
```

Clones live at `~/.cache/crafter.build/external/<name>-<hash>/`, keyed by `(url, branch, commit, options, target)` so different projects sharing the same dep with the same spec reuse the clone and cmake build, while projects with diverging specs each get their own cache entry. Without a `commit`, the clone is `git pull`ed once per build: every configuration of the build that declares the dependency, such as each test of a project whose tests copy its `externalDependencies`, shares that one fetch and cmake build. The clone is resilient to partial-clone recovery and CMake reconfigure is triggered automatically on `options` changes. Submodules are checked out at the revisions the fetched commit records, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules and not the default branch's.

**Header-only git deps** — same as cmake deps but `builder = ExternalBuilder::None`. Just clones and propagates `-I` flags.

**Sibling crafter-build projects** — `GitProject` / `LocalProject` load another project's `project.cpp` in-process and return a `Configuration*` for `cfg.dependencies`:

```cpp
Configuration* math = GitProject({
    .source = { .url = "https://forgejo.catcrafts.net/Catcrafts/Crafter.Math.git" },  // .branch or .commit to pin
});
Configuration* sibling = LocalProject({ .projectFile = "../Crafter.Math/project.cpp" });
cfg.dependencies = { math };
```

The sibling's `CrafterBuildProject` gets the args this `project.cpp` was loaded with (`ProjectArgs()`), so `--target=`, `--march=`, `--debug` and the project's own flags reach every sibling down the chain. That is all of them: `cfg.projectArgs` holds only what `ApplyStandardArgs` did not interpret, and passing it on would drop the target and the build type. Set `.args` to send something else: `{}` for nothing, or a list built from the project's args, such as one without a flag only this project reads. Build such a list once and give it to every sibling. A project that two paths load with different lists would end up in the program twice, so the build stops with an error that names both paths. Releases up to 2.2.0 forward nothing when `.args` is left out.

The default is read when the spec is made, so make it inside `CrafterBuildProject`, on the thread that calls it. A spec made at namespace scope, on a thread of your own or in a script forwards nothing.

Clones share the external cache and identical specs share one `Configuration`. An unpinned source is `git pull`ed on every invocation, set `.commit` for reproducible, offline-after-first-fetch builds.

A project loaded this way runs its `CrafterBuildProject` as a dependency, and `IsTopLevelProject()` is `false` while it does (`true` in the project crafter-build was pointed at, and outside any project file). Wire test-only dependencies behind it: a library whose test links crafter-build itself would otherwise load crafter-build's `project.cpp` from inside crafter-build and recurse.

### Dependencies of project.cpp itself

`project.cpp` can import modules from another crafter-build project. The file has to compile before it can say anything, so the dependency is a directive next to the `import` it enables:

```cpp
import std;
import Crafter.Build;
#pragma crafter dependency "https://forgejo.catcrafts.net/Catcrafts/Crafter.LintRules.git"
import Crafter.LintRules;
```

```
#pragma crafter dependency "<source>" [branch "<name>" | commit "<sha>"] [project "<file>"]
```

Such a dependency is built for the host, whatever the project targets, and its `project.cpp` gets none of the project's args.

### Dependency notes for agents

An agent working on a project reads its `AGENTS.md`, which says nothing about the libraries the project builds against unless someone copied that in by hand. A library ships it instead: an `AGENTS.consumer.md` beside its `project.cpp`, written for whoever uses the library rather than whoever works on it. When a consumer has an `AGENTS.md` beside its own `project.cpp`, `crafter-build` and `crafter-build test` keep its dependencies' notes in a section of it:

```markdown
<!-- crafter-build: dependency notes begin -->
<!-- Written by crafter-build from the AGENTS.consumer.md each dependency ships, and rewritten on every build. Edit those files, not this section; cfg.dependencyNotes in project.cpp picks which dependencies are included. -->

<!-- from Crafter.Math -->
## Crafter.Math

...

<!-- crafter-build: dependency notes end -->
```

- Each file goes in as written, under a line naming the dependency: its repository's name, or a `LocalProject`'s directory name. That reads the same on every machine, and for a `--local` checkout and its git clone alike. The file lands among the consumer's own headings, so start it with a `##` heading that names the library.
- The section is appended the first time, after a blank line, and rewritten in place after that, wherever it has been moved to. Edits inside it are lost on the next build; the text around it is left as it is. The file is written only when the section changes, in place, so hard links, ownership and a read-only `AGENTS.md` are respected, and an `AGENTS.md` is never created.
- The update runs once `project.cpp` has loaded and before the project compiles, so a build that fails still has the notes. A `project.cpp` that fails to load gets no update. `test --list`, `lint`, `format`, `run` and `clean` leave the file alone.
- The section follows what the build loaded: a dependency that only some flags pull in (a `--target=`, a flag of the project's own) comes and goes with them. An unpinned dependency's notes follow its default branch, as its code does, so the section can change on any build. Pin a `commit` to decide when.
- A library reached through two specs, pinned at one revision here and another further down, is described once, as the nearer one has it.
- When there are notes to write, unbalanced markers leave the file as it is, with a warning. A notes file that would break the section (a code block it never closes, a marker line of its own) is left out, also with a warning. Markers quoted inside a fenced code block don't count.

`cfg.dependencyNotes` picks the dependencies:

| Value | Notes from |
|---|---|
| `DependencyNotes::Transitive` (default) | everything the project builds against, all the way down: its `dependencies` and its own `#pragma crafter dependency` tooling |
| `DependencyNotes::Direct` | only the projects the project's own configurations and `#pragma crafter dependency` lines name |
| `DependencyNotes::None` | none, and a section from an earlier build is removed |

Three things never count: the project's own `AGENTS.consumer.md`; its tests' dependencies, since a plain build never looks at the tests; and a dependency's `#pragma crafter dependency` tooling, which is about how the dependency is built, not how it is used.

## Tests

Tests live under `tests/<Name>/`. The simplest case is a single C++ file:

```
tests/Smoke/main.cpp        # int main() — exit 0 = pass, nonzero = fail, 77 = skipped
```

Run them all:

```bash
crafter-build test
```

Test conventions:

- **Exit codes**: `0` = ✅ pass, nonzero = ❌ fail, **`77`** = ⏭ skipped (autoconf convention; use it for runtime opt-outs like "tool not on PATH").
- Each test runs in its own subprocess — a segfault doesn't take the runner down.
- Default timeout is 60 s per test, runs in parallel up to `hardware_concurrency()`.
- Stdout+stderr is captured and printed atomically on completion. Failed/crashed/timed-out tests also write to `build/test-logs/<name>.log`.
- A broken fixture `project.cpp` is reported as a single failed test, not a fatal error — other tests still run.

CLI:

```bash
crafter-build test                                              # run all
crafter-build test 'Unit*'                                      # glob-filter test names
crafter-build test --list                                       # enumerate without running
crafter-build test --target=x86_64-w64-mingw32                  # only tests for this target
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine
crafter-build test --jobs=1                                     # serial mode
crafter-build test --timeout=5                                  # override per-test timeout
```

Outcomes: ✅ pass, ❌ fail, 💥 crash (with signal name), ⏱ timeout, ⏭ skipped. The runner exits 0 only if every non-skipped test passed.

`--target=` filters the run to tests whose `cfg.target` matches, default is the host triple. Per-target tests parse `--target=...` from their own args span and build for it, tests with hardcoded targets (e.g. host-only meta-tests) are filtered out under non-matching `--target=`.

### Cross-target test runners

`Test::runner` runs the test binary somewhere other than this machine. Four factories:

```cpp
TestRunner::Local()                                  // run on this machine (default)
TestRunner::Cmd("wine")                              // local exec via prefix command
```

`Cmd(cmd)` is a generic `<cmd> <bin> <args>` prefix runner — covers wine, qemu-user, valgrind, gdbserver, anything with that shape. Probes via `which <cmd>`.

For elaborate setups (rsync, ssh ControlMaster, env vars on the spawn, `QEMU_LD_PREFIX` for sysroots), construct `TestRunner` directly — `copy`/`exec`/`cleanup`/`probe` are public string fields with `{bundle}`, `{remote_bundle}`, `{bin}`, `{bin_name}`, `{args}` placeholders.

For cross-arch builds, set `cfg.sysroot` on the `Configuration` to point at a sysroot for the target. `BuildStdPcm` and the compile/link commands both consume it: `--sysroot=<path>` is passed to clang, and the std module is precompiled against the sysroot's libc++ headers. Ordinary compiles are pointed at the same headers (`-nostdinc++ -isystem <sysroot>/usr/include/c++/v1`), so a unit that `#include`s a standard header gets the sysroot's libc++ and not the host toolchain's. The module source has to come from the same libc++ release as those headers, so it is taken from `<sysroot>/usr/share/libc++/v1/std.cppm` when the sysroot ships it, else from the toolchain when both are the same major release, else from the sources crafter-build carries for other releases (`share/crafter-build/libc++-modules/<major>/`, `crafter-build --version` lists them). Alpine is the common case for the last two: its `libc++-dev` has the headers and no module source. When nothing matches, the build stops with one line naming both releases, installing that release's `share/libc++/v1` directory into the sysroot resolves it. The cheapest aarch64 sysroot is an extracted [Arch Linux ARM rootfs](http://os.archlinuxarm.org/) plus `libc++` + `libc++abi` + `gcc` installed into it via `pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy …` (point pacman at ALARM's mirrors via explicit config to avoid pulling host-arch packages).

For `--target=x86_64-w64-mingw32` (Windows from Linux), the pipeline cross-compiles as long as `mingw-w64` is installed, with libstdc++ 15 or newer for its std module. clang finds it by the `x86_64-w64-mingw32-gcc` on `PATH`; `CRAFTER_MINGW_DIR` names another prefix (the directory holding `x86_64-w64-mingw32/` and `lib/gcc/`), and inside the Flatpak the host's `/usr` is used when it has mingw-w64. Without one the build stops with a message saying where it looked. Pair it with `--runner=cmd:wine` to run the result. Only the MinGW ABI is supported, there is no MSVC target and no Windows host. `-fuse-ld=lld` is auto-injected on every link, and `-lstdc++exp` is auto-linked for the mingw target so user projects don't need to declare those flags.

### Configuring runners

Two layers, CLI wins over env, both override the per-test default of `Local()`:

**Env var** (persistent, per shell or in CI):

```bash
export CRAFTER_BUILD_RUNNER_x86_64_w64_mingw32=cmd:wine
export CRAFTER_BUILD_RUNNER_aarch64_linux_gnu=cmd:qemu-aarch64
crafter-build test --target=x86_64-w64-mingw32           # uses cmd:wine
```

The variable name is `CRAFTER_BUILD_RUNNER_<NORMALIZED_TARGET>` (target triple with `-` and `.` replaced by `_`).

**CLI** (per invocation, scoped to `--target=`):

```bash
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine
```

Spec grammar:

```
local
cmd:<command>                  # cmd:wine, cmd:qemu-aarch64, cmd:valgrind, ...
```

Each runner carries a `probe` command (e.g. `which wine`), if it fails, every test using that runner is reported as ⏭ skipped and the rest of the suite continues. Skipped tests count toward the summary but don't fail the suite (`AllPassed()` returns true on skip-only runs).

### Linking the parent project's library

A test fixture can depend on the project's own library so it can `import` and exercise its API:

```cpp
// tests/UnitFoo/project.cpp
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
    Configuration cfg;
    cfg.path = "tests/UnitFoo/";
    cfg.name = "UnitFoo";
    cfg.outputName = "UnitFoo";
    cfg.type = ConfigurationType::Executable;
    cfg.dependencies = { ParentLib("MyLib") };       // <-- the magic line

    std::array<fs::path, 0> ifaces = {};
    std::array<fs::path, 1> impls = { "main" };
    cfg.GetInterfacesAndImplementations(ifaces, impls);
    return cfg;
}
```

`ParentLib(name)` walks the parent project's dependency graph (and the parent itself) to find a `Configuration*` by `Configuration::name`. The build links it into the test exe, so `import MyLib;` resolves naturally. See [examples/tests/](examples/tests/) for the complete worked example.

## Linting & formatting

Lint rules are C++ in `project.cpp` with no `.lintrc`, no YAML, and no built-in rules. A rule is a named callback that runs once per source file, register it on the `Configuration` before returning:

```cpp
cfg.AddLintRule("no-tabs", [](Crafter::LintContext& ctx) {
    if (ctx.Extension() != ".cpp" && ctx.Extension() != ".cppm") return;
    for (std::size_t n = 1; n <= ctx.lines.size(); ++n) {
        if (ctx.Line(n).contains('\t')) ctx.Report(n, "tab character (use spaces)");
    }
});
```

A rule that calls `ctx.SetContent(newContent)` is a **transform** which is defined once, it powers both verbs: `lint` reports where the transform would change the file (dry, never writes), and `format` applies it to disk:

```cpp
cfg.AddLintRule("final-newline", [](Crafter::LintContext& ctx) {
    if (!ctx.content.empty() && !ctx.content.ends_with('\n')) {
        ctx.SetContent(ctx.content + '\n');
    }
});
```

```bash
crafter-build lint            # full dry gate: Report() findings + would-reformat, exit 1 on any
crafter-build lint 'spdx*'    # rule-name glob filter
crafter-build lint --list     # enumerate rules without running them
crafter-build format          # apply transforms, rewrite changed files in place (exit 0)
crafter-build format --check  # CI gate: list files that would change, exit 1 if any
```

Lint findings print compiler-style — `file:line: warning: message [rule]` — so editors and CI pick them up. `format` rewrites files in place (gofmt convention) and only touches files whose bytes actually change; report-only findings are `lint`'s business and don't affect `format`'s exit code. A project with no rules registered exits 1 with a message showing how to add one.

`LintContext` gives each rule:

| Member | Meaning |
|---|---|
| `file` | absolute path of the file under lint |
| `content` | the whole file |
| `lines` | pre-split line views (report 1-based via `Line(n)`) |
| `Extension()` | file extension, e.g. `".cppm"` |
| `CommentStripped()` | `content` with comments and string/char literal bodies blanked to spaces — newlines kept, so line numbers stay valid (raw strings not recognized yet) |
| `Report(line, msg)` | record a finding (`line == 0` for whole-file) |
| `SetContent(str)` | replace the file's content — makes the rule a transform, `lines` re-splits, `CommentStripped()` re-derives. Build the new string, call once at the end, don't assign `content` directly |

Suppression is comment-based :

```cpp
// lint-disable-next-line no-char-pointer      — one rule, the following line
// lint-disable-next-line naming, wrap-join    — several rules (space- or comma-separated)
// lint-disable-next-line                      — all rules, the following line
// lint-disable-file fixed-width-types         — one rule, whole file (any position)
// lint-disable-file                           — all rules, whole file
```

## Scripts

A project has chores that are not part of building it: packaging, installing, publishing, generating documentation. A script gives one a name. Scripts are C++ in `project.cpp` like everything else, a name and a callback pushed onto `cfg.scripts` before returning the `Configuration`:

```cpp
cfg.scripts.push_back({"docs", [](Crafter::ScriptContext& ctx) {
    return ctx.Run("python3 tools/gen-docs.py");
}});
```

```bash
crafter-build run                  # list the project's scripts
crafter-build run docs             # run one
crafter-build run docs --open api  # everything after the name is handed to the script
```

The callback returns the exit status, and `crafter-build run` exits with it. `ctx.Run(command)` runs a shell command with the terminal attached, so its output appears as it is produced and it can ask questions, and returns the command's exit code (128 plus the signal number when it was killed). A script with several steps stops at the first failure by returning that code:

```cpp
cfg.scripts.push_back({"package", [](Crafter::ScriptContext& ctx) {
    const std::string dist = Crafter::ShellQuote((ctx.projectDir / "dist").string());
    if (const std::int32_t code = ctx.Run(std::format("mkdir -p {}", dist)); code != 0) return code;
    return ctx.Run(std::format("tar -czf {}/myapp.tar.gz -C {} .", dist, Crafter::ShellQuote(ctx.project.BinDir().string())));
}});
```

`ScriptContext` gives each script:

| Member | Meaning |
|---|---|
| `project` | the `Configuration` that `project.cpp` returned. Read the project through this; a `cfg` captured by reference is gone by the time a script runs |
| `projectDir` | absolute directory of the project file |
| `args` | everything typed after the script's name, untouched |
| `Run(command)` | run a shell command with the terminal attached and return its exit code |

Things to know:

- Everything after the script's name belongs to the script, including what looks like an option of crafter-build (`--help`, `-v`). Options and project args for crafter-build go before the name: `crafter-build --debug run package`.
- `run` loads `project.cpp` but does not build the project. A script that needs the build output runs after `crafter-build`.
- Commands run in the directory crafter-build was started from. Build paths from `ctx.projectDir`, and quote anything that is not a literal with `Crafter::ShellQuote`.
- Only the scripts of the project you run crafter-build in are available. A dependency's scripts are not.
- Inside the Flatpak, commands run in its sandbox: the host's files are visible, but the tools on `PATH` are the ones the Flatpak ships (`git`, `cmake`, `python3`, `tar`, ...), not everything installed on the host.
- A return value outside 0 to 255 is not an exit status and becomes 1. So does a script that throws, with the exception's message printed.

## Examples

[examples/](examples/) contains progressively larger self-contained projects:

| Example | Shows |
|---|---|
| [hello-world](examples/hello-world/) | Smallest possible project: `project.cpp` + `main.cpp` |
| [with-module](examples/with-module/) | Adding a C++ module interface |
| [library](examples/library/) | Library + executable, linked via `cfg.dependencies` |
| [tests](examples/tests/) | tests linking the parent library |

Each builds standalone: `cd examples/<name> && crafter-build`.

## License

This project is licensed under the GNU Lesser General Public License, version 3 only (LGPL-3.0-only). See [LICENSE](LICENSE) for the license text. The LGPL supplements the GNU General Public License version 3, a copy of which is included as [LICENSE.GPL](LICENSE.GPL).

The example code under [examples/](examples/) is licensed under the MIT license (see [examples/LICENSE](examples/LICENSE)), so it can be freely copied into your own projects.

The [lib/](lib/) directory contains vendored third-party code (nlohmann/json, glslang's DirStackFileIncluder) that remains under its own permissive licenses, see the headers of those files.

## Copyright

Copyright (C) 2026 Catcrafts®
catcrafts.net