Crafter.Build/AGENTS.md

# AGENTS.md

Guidance for coding agents and contributors working on crafter-build itself. The [README](README.md) is the user-facing manual; this file holds what you need to change the build system. When the two disagree, the code wins: check the interface in `interfaces/` and the `--help` output of the built binary.

## Project overview

crafter-build is a C++26 modules build system. A project is described by a `project.cpp` that returns a `Crafter::Configuration`; the tool compiles that file to a shared object, loads it in-process and builds what it describes. This repository is self-hosting: its own `project.cpp` builds crafter-build with crafter-build, and its tests link the library and drive `Build()` in-process.

| Path | What lives there |
|---|---|
| `interfaces/Crafter.Build-<Partition>.cppm` | Module interface partitions. `Crafter.Build.cppm` is the umbrella that re-exports them. `Crafter.Build-Api.h` defines `CRAFTER_API`. |
| `implementations/Crafter.Build-<Partition>.cpp` | Implementation units, one per partition, named `Crafter.Build:<Partition>_impl`. `main.cpp` is a thin wrapper around `Crafter::Run`. |
| `project.cpp` | The self-host build description: a `crafter.build-lib` library, a `crafter.build-exe` executable, the test declarations and the lint rules. |
| `tests/<Name>/main.cpp` | One self-test per directory; `tests/<Name>/fixture/` holds the sample project it builds. |
| `examples/` | Standalone user-facing projects, MIT licensed. |
| `build.sh`, `build.cmd` | Bootstrap scripts for Linux and Windows. They compile everything by hand, without crafter-build. |
| `wasi-runtime/` | `runtime.js` and `index.html.in`, shipped for browser targets. |
| `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. |
| `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. |
| `PKGBUILD`, `.forgejo/workflows/ci.yaml` | Arch package and the Forgejo Actions CI. |

Partitions and what they own:

- `:Clang` holds `Configuration`, `Build()`, `VariantId()`, `ApplyStandardArgs`, `HostTarget`, the CLI entry point `Crafter::Run`, and the lint and test data types.
- `:Platform` runs commands, implements `LoadProject` (compile and load `project.cpp`), the host PCM cache, `#pragma crafter dependency`, and all OS-specific code.
- `:Interface` and `:Implementation` model module units, their staleness checks and their compiles.
- `:External` handles git + cmake dependencies, `GitProject` and `LocalProject`.
- `:Test` holds `AddTest`, `RunTests` and `TestRunner`.
- `:Lint` is the lint/format driver plus the libclang token and AST layer.
- `:Progress` is console output. `:Shader` compiles shaders through glslang. `:Asset` is the hook into Crafter.Asset.

## Dev environment

Prerequisites on Linux: `clang` (with `libclang`, which the AST lint rules load at runtime), `lld`, `libc++` with `/usr/share/libc++/v1/std.cppm`, `cmake`, `git`. Network access to `forgejo.catcrafts.net` and `github.com` is needed: loading `project.cpp` fetches Crafter.Math, Crafter.Asset and Crafter.LintRules into `~/.cache/crafter.build/external/`, and the bootstrap clones glslang. Those sources are unpinned, so every invocation runs `git pull` on them.

```bash
./build.sh              # bootstrap: bin/crafter-build, lib/libcrafter-build.a, share/crafter-build/
./bin/crafter-build     # self-host build through project.cpp
```

Three things to know about this loop:

- `bin/crafter-build` is the bootstrap binary and the harness for every command below. The self-host build writes to `bin/crafter.build-exe-<variant>/crafter-build` and does not replace it.
- Changes under `implementations/` are picked up by the self-host build and by the tests, which link the local library. Changes under `interfaces/` are not: the harness compiles against the copies in `share/crafter-build/` and the PCM cache built from them, so a new or changed declaration is invisible until you run `./build.sh` again. The symptom is `use of undeclared identifier` for the symbol you just added.
- A new `CRAFTER_API` symbol that a `project.cpp` reaches on its default path also needs `./build.sh`: the old binary does not export it, and `LoadProject` fails with an undefined symbol.

`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`.

Windows builds use `build.cmd` with `LIBCXX_DIR` pointing at a libc++ install; the Linux-to-mingw cross-compile is a separate artifact with a different ABI. See the README. Windows branches of recent features are wired but often unverified: say so in the PR when you touch one and cannot test it.

## Build, test and style commands

```bash
./bin/crafter-build                          # build (self-host)
./bin/crafter-build test                     # build and run every self-test
./bin/crafter-build test 'Incremental*'      # name glob
./bin/crafter-build test --list              # enumerate without running
./bin/crafter-build test --jobs=1 --timeout=120
./bin/crafter-build lint                     # dry style gate, exit 1 on any finding
./bin/crafter-build format                   # apply the transform rules in place
./bin/crafter-build format --check           # CI-style dry run
./bin/crafter-build clean
```

Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` on every PR; its lint step is currently commented out, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/<name>.log`.

## Testing instructions

Tests are declared, not discovered. Each one is a line in `project.cpp`:

```cpp
cfg.AddTest("IncrementalHeaderChange").Dependencies({ CrafterBuildLib.get() });
```

The source is `tests/<Name>/main.cpp`. The test inherits target, march and the rest from the root configuration and links the local `crafter.build-lib`, so `import Crafter.Build;` resolves to the code in this checkout. A test that calls `LoadProject` also needs `.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl")`, as the existing ones show. The declarations sit in an `if (linuxTarget)` block that matches any `*-linux-gnu` or `*-linux-musl` target: the suite is Linux-host only today.

A test is an ordinary `int main()`. Exit 0 passes, nonzero fails, 77 skips (for a missing tool, never for a failed assertion). Each runs in its own subprocess, in parallel, with a 60 s default timeout.

Conventions the existing tests follow:

- Open `main.cpp` with a comment that states the behaviour under test and the failure it guards against. `tests/CompilerWarning/main.cpp` shows the shape.
- Collect failures with a `Check(cond, msg)` helper that prints `FAIL: ...` to stderr and counts, then return nonzero at the end. Do not stop at the first failure unless later steps are meaningless without it.
- Drive the engine in-process: `Build(cfg, depResults, depMutex)`, `RunTests`, the lint driver. Do not shell out to `bin/crafter-build` and grep its stdout.
- Fixtures under `tests/<Name>/fixture/` are inputs. A test that edits or touches files first stages a copy under `fs::temp_directory_path() / "crafter-build-<name>"` and removes it on the next run. Variant sources are checked in as `*.in` files and copied over the original inside the staged copy.
- Use a fresh `depResults` map per build pass: the map memoises each configuration's result for one pass, so reusing it skips rebuilds you meant to observe. `Build()` itself is safe to call repeatedly on the same `Configuration`.
- Set `cfg.path` before calling `GetInterfacesAndImplementations`, and spell `cFiles` as absolute paths: both resolve against the working directory, not the configuration.

A bug fix ships with a test that fails before the fix and passes after it, in its own `test:` commit when it documents the behaviour on its own.

Cross-target runs use `test --target=<triple>` with `--runner=local` or `--runner=cmd:<wrapper>` (wine, qemu-user, valgrind), or `CRAFTER_BUILD_RUNNER_<TRIPLE>` as the persistent form. A failed runner probe skips its tests rather than failing the suite.

## Code style

Style is enforced by the 17 rules in [Crafter.LintRules](https://forgejo.catcrafts.net/Catcrafts/Crafter.LintRules), which `project.cpp` pulls in through `#pragma crafter dependency` and applies with `crafter-build format` and `lint`. Most rules are transforms, so run `format` and let it restyle rather than hand-formatting. In short:

- The first three lines of every source: `// SPDX-License-Identifier: LGPL-3.0-only`, `// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®`, blank. Examples use MIT.
- Four-space indentation, no tabs, no trailing whitespace, final newline.
- `PascalCase` for types, functions, statics and constants; `camelCase` for variables; a trailing `_` is allowed.
- `enum class` only. `std::println` instead of iostream. `std::format` instead of string concatenation. `std::int32_t` and friends instead of `int` or `long` (exempt: `main`, `argc`/`argv`, `extern "C"` and foreign API signatures). No `char*` outside C interop.
- One declaration per statement. K&R braces. `if (x) return;` on one line when it fits. The formatter joins wrapped statements that fit on one line, so do not hand-wrap short ones. `if (` and `for (` with a space; no space before a call's `(`.
- `const` on scalar locals that are never written; `constexpr` on constants with constant initialisers.

Suppress one finding with `// lint-disable-next-line <rule>` or a whole file with `// lint-disable-file <rule>`. Both are rare in this tree and need a reason.

Beyond the rules:

- `import std;` everywhere; standard headers are never `#include`d. C headers (`clang-c/Index.h`, `dlfcn.h`, `windows.h`) go in the global module fragment after `module;`.
- Mark every function or method that `project.cpp` or a downstream consumer calls with `CRAFTER_API` in the interface. It is the Windows DLL boundary.
- Comments explain why: the failure mode, the constraint, the rejected alternative. The codebase carries long comments on purpose. Do not strip them, and write them for new non-obvious code. Commit bodies follow the same rule.
- Platform splits live in `:Platform`, guarded by `CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc` and `_x86_64_w64_mingw32`. In `project.cpp` and tests, branch on `cfg.target` strings instead of preprocessor macros.
- A build step fails when its command exits nonzero, never because it printed. Use `RunBuildCommand` or `RunCommandChecked`; warnings go to `Progress::Diagnostic`.
- Every output path goes through `Configuration::BinDir()`, `BuildDir()` and `PcmDir()`. Hand-rolled `path / "bin" / ...` clobbered variants before; do not add new ones.
- Anything that changes what gets compiled or bundled belongs in `VariantId()`. It already hashes type, debug, sysroot, defines, compile flags, wasm variant flags and uninterpreted project args.

## Design rules

These were decided deliberately. Do not undo them without the maintainer's say-so.

1. **Configuration is C++.** No JSON, YAML, TOML or DSL for project, runner or lint configuration. Lint rules are C++ callbacks; there is no clang-format or clang-tidy integration and no separate "format rule" type. A rule that calls `ctx.SetContent` is a transform, and that is the whole mechanism.
2. **Per-project artifacts go in `<project>/build` and `<project>/bin`. Only cross-project things go in `~/.cache/crafter.build/`:** the std PCM and Crafter.Build PCMs per target and march, and external clones keyed by their spec hash.
3. **Sources ship, PCMs build locally.** Packages install `*.cppm` under `share/crafter-build/`. Nothing target-baked ships except the executable itself, and that defaults to a portable `-march` in `PKGBUILD` and CI.
4. **Linux is a monolithic executable with `--export-dynamic`; Windows is a DLL plus a launcher.** The asymmetry is intentional.
5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point.
6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch.
7. **The house-style rules stay in their own repo.** They are not library API of crafter-build and are not forced on downstream projects.
8. **Fix warts inline; do not stack workarounds.** The previous version of this tool died of accumulated hacks. If a change needs a shortcut, say so in the PR rather than hiding it.
9. **`project.cpp` keeps the library/executable split.** `crafter.build-lib` owns the sources, dependencies and glslang; `crafter.build-exe` is `main.cpp` plus link flags.
10. **Host-tooling builds** (the target of a `#pragma crafter dependency`) get `-fPIC` and the host triple scoped to that build. Keep it scoped.

## Commit and PR guidelines

- Branch from `master` and open the PR against `master` on the project's Forgejo at `forgejo.catcrafts.net/Catcrafts/Crafter.Build`. Issue work uses `claude/issue-<n>` branch names. Pushes go over SSH; HTTPS push is disabled on the server.
- Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice.
- Separate commits for the fix, its test and the README change when each stands on its own.
- Update the README in the same PR when user-visible behaviour changes.
- `format`, `lint` and `test` green before pushing. CI runs the bootstrap and the test suite on PRs and publishes release archives on `master`.
- Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`.
- Keep CI's `CRAFTER_BUILD_MARCH=x86-64-v2` for the bootstrap: the runner's CPU cannot execute v3 code, and the v3 and v4 artifacts are produced from the v2 binary.

## Security considerations

- `project.cpp` is compiled and loaded into the crafter-build process, and every `GitProject`, `#pragma crafter dependency` and `ExternalDependency` clones a repository and runs its build on the host. Treat a dependency URL as code you are choosing to run. This repo's own dependencies are sibling Catcrafts repos tracking their default branch on purpose; pin with `commit` when adding anything else, and whenever reproducibility matters.
- Unpinned dependencies `git pull` on every invocation, so builds and tests need network. A missing or private repo surfaces as a clone failure before anything is built.
- `--runner=cmd:<command>` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests.
- The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library.
- CI force-pushes the rolling `latest` tag and publishes a prerelease on every push to `master` with the workflow's own token. Do not add secrets to the workflow or echo the environment in it.
- Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way.

## Gotchas

- A `#pragma crafter dependency` target is rebuilt on every load on purpose: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the depfile check is what catches a regenerated host cache. Do not optimise the rebuild away.
- A host-tooling library such as Crafter.LintRules may `import Crafter.Build;` without declaring it as a dependency, because `LoadProject` fills the cache it resolves from. Listing Crafter.Build in its `cfg.dependencies` would clone and build all of crafter-build for every consumer.
- `crafter-build clean` leaves a pragma dependency's build alone; it lives with the clone in the cache.
- The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop.
- A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone.
- `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash.
- When driving `build.cmd` from a Windows `cmd` shell over SSH: write `set VAR=value&&next` without a space before `&&`, and `if exist X rmdir /S /Q X` instead of chaining `rmdir` with `&&`.