Crafter.Build/AGENTS.template.md

206 lines
32 KiB
Markdown
Raw Permalink Normal View History

2026-10-02 20:35:38 +02:00
# AGENTS.md
Guidance for coding agents and contributors working on crafter-build itself. The [README](README.md) is the user-facing manual; this file holds what you need to change the build system. When the two disagree, the code wins: check the interface in `interfaces/` and the `--help` output of the built binary.
## Project overview
crafter-build is a C++26 modules build system. A project is described by a `project.cpp` that returns a `Crafter::Configuration`; the tool compiles that file to a shared object, loads it in-process and builds what it describes. This repository is self-hosting: its own `project.cpp` builds crafter-build with crafter-build, and its tests link the library and drive `Build()` in-process.
| Path | What lives there |
|---|---|
| `interfaces/Crafter.Build-<Partition>.cppm` | Module interface partitions. `Crafter.Build.cppm` is the umbrella that re-exports them. |
2026-10-02 20:35:38 +02:00
| `implementations/Crafter.Build-<Partition>.cpp` | Implementation units, one per partition, named `Crafter.Build:<Partition>_impl`. `main.cpp` is a thin wrapper around `Crafter::Run`. |
feat(packaging): `run flatpak` and `run install` for a local Flatpak The local Flatpak build was one long flatpak-builder line in AGENTS.md, with its state wherever the caller happened to stand, and installing the result was not written down anywhere. It is not a rare chore: the development box compiles with the distro's clang and the Flatpak with its bundled LLVM, a major version apart at the moment, so anything toolchain-sensitive has to be tried in the Flatpak too. Both are now scripts in project.cpp, the first users of cfg.scripts. `run flatpak` builds the manifest as it is, from the working tree, and ends with `crafter-build --version` from inside the result. Its state is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in the sandbox, so it does not depend on the host variant, and a variant-keyed directory would strand the LLVM cache on every host compiler update. The manifest already skips build/ when it copies the sources. flatpak-builder keys its cache on the manifest's path, so the manifest is never rewritten into a copy the way CI stamps the version in, and a local build reports 0.0.0-dev. The only argument accepted is --jobs=<N>: anything that ends the run without a build would leave the smoke test checking the previous one. `run install` installs the last export as branch `local` from a user remote of its own and makes it current, leaving the published branch installed. It does not build first: flatpak-builder never caches a `type: dir` module, so every build reruns build.sh and the suite. remote-add --if-not-exists is followed by remote-modify --url, because the remote may exist and point at another checkout. Both refuse to run from the Flatpak build of crafter-build, which has no flatpak to drive. The smoke test goes through `flatpak build` and not `<builder> --run`, which is what CI uses. flatpak-builder 1.4.9 run as the org.flatpak.Builder app exits 1 when the command it ran succeeded and 0 when it failed (`true` gives 1, `exit 3` gives 0), so the script reported a good build as a failure. Measured on the 56-core box with LLVM and the WASI sysroot cached: 260 seconds, the crafter-build module only, 29 tests passing in the sandbox. A second `run install` after a rebuild moved the installed commit. `clean` now also drops that state, about 2 GB and a cold LLVM build. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:29:32 +02:00
| `project.cpp` | The self-host build description: a `crafter.build-lib` library, a `crafter.build-exe` executable, the test declarations, the lint rules and the `flatpak` and `install` scripts. |
| `AGENTS.template.md` | The source of AGENTS.md. Every build and test run writes AGENTS.md from it, with the notes the dependencies ship; AGENTS.md is gitignored, so changes go here. |
2026-10-02 20:35:38 +02:00
| `tests/<Name>/main.cpp` | One self-test per directory; `tests/<Name>/fixture/` holds the sample project it builds. |
| `examples/` | Standalone user-facing projects, MIT licensed. |
| `build.sh` | Bootstrap script. It compiles everything by hand, without crafter-build. |
2026-10-02 20:35:38 +02:00
| `wasi-runtime/` | `runtime.js` and `index.html.in`, shipped for browser targets. |
| `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. |
| `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. |
| `packaging/flatpak/`, `.forgejo/workflows/ci.yaml` | The Flatpak manifest with its metainfo, repo and ref files, and the Forgejo Actions CI that builds and publishes it. |
feat(review): set up Crafter.Review's pre-push review Crafter.Review reviews a frozen commit range before it is pushed: the approach, a static proof of the code's bugs, and every claim in the messages and in the documentation the range changes, one claude process per lens and unit, or per whole-range pass, and the owner approves the exact frozen hash before the push. This sets it up for this repository. .review-gates/config.json turns on three lenses in the commits mode, at xhigh: repo-better-solution, repo-bug-prover and repo-facts. The functional lens is off: it exercises a web application through a browser, and this repository's product is a command-line tool. The security lens is off by the owner's ruling: crafter-build stores no secret of its own, and while its Flatpak runs with the user's SSH agent (--socket=ssh-auth) and the whole host filesystem (--filesystem=host), everything it runs is code its user chose to run. The rules CI keeps around its token and signing key go to the approach lens's section instead. gate0 lists the commands run by hand before a freeze. build.sh comes first, because bin/crafter-build compiles project.cpp against the interfaces it was bootstrapped with, and lint comes after the self-host build, because lint's AST rules parse with the module PCMs a build leaves and lint builds only the std PCM itself. .review-gates/lenses/ holds this repository's section of each lens's charter, appended to the text Crafter.Review ships: that AGENTS.md is absent from the frozen tree, the layers and the consumers of interfaces/, the execution contexts a bug can hide in, where libc++ and clang are readable from the review's sandbox, and what a "passes the suite" claim traces to. The /prepush skill that crafter-review init writes stays untracked. It is generated from the installed Crafter.Review and refreshed by every init, so a committed copy would lag the binary it describes. One init per checkout restores it, which AGENTS.template.md now says, together with the rules freeze holds every commit of a range to: signed (requireSigned), with the owner's address as its author and committer (authors), and no AI attribution trailer (aiTrailers forbid).
2026-11-01 02:55:03 +01:00
| `.review-gates/` | The pre-push review's `config.json` and this repository's lens sections (`lenses/`), read by Crafter.Review. Its `push/` and `trees/` are generated and gitignored. |
2026-10-02 20:35:38 +02:00
Partitions and what they own:
- `:Toolchain` resolves the compiler once per process (`GetToolchain`): clang++, clang, llvm-ar, libclang, libc++'s std module source and the default sysroots, from `CRAFTER_BUILD_LLVM_DIR`, else a toolchain installed beside crafter-build, else PATH. It also holds the version baked in at build time. `SysrootStdModule` picks the std module source for a sysroot build by the sysroot's libc++ release. `MingwSysroot` settles the mingw target's `--sysroot`: the mingw-w64 prefix itself, or, for a prefix that also holds a Linux system (the host's `/usr`, anything whose `include/` has `features.h`), a view in the cache of its mingw-w64 directories, since clang would search its `include/` and `lib/` too (issue #43). `MingwViewToPrefix` turns paths into the view back into paths into the prefix for messages.
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
- `:Clang` holds `Configuration`, `Build()`, `VariantId()`, `ApplyStandardArgs`, `HostTarget`, the CLI entry point `Crafter::Run`, and the lint, test and script data types.
2026-10-02 20:35:38 +02:00
- `:Platform` runs commands, implements `LoadProject` (compile and load `project.cpp`), the host PCM cache, `#pragma crafter dependency`, and all OS-specific code.
- `:Interface` and `:Implementation` model module units, their staleness checks and their compiles.
- `:External` handles git + cmake dependencies, `GitProject` and `LocalProject`.
- `:Test` holds `AddTest`, `RunTests` and `TestRunner`.
- `:Lint` is the lint/format driver plus the libclang token and AST layer.
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
- `:Script` is the driver behind `crafter-build run`: it looks a name up in `cfg.scripts`, calls the callback and turns what it returns into the exit code.
feat(agents): write AGENTS.md from a tracked AGENTS.template.md The dependency notes were kept in a section of the AGENTS.md beside project.cpp, the file a project commits, so a build could change a tracked file at any time: a dependency editing its AGENTS.consumer.md, an unpinned one moving on, a flag pulling a dependency in or out, each showed up in the consumer's diff as a change nobody made. Users reported it as the feature clobbering their diffs. A project now commits its instructions as AGENTS.template.md, and a build and a test run write AGENTS.md from it: a first line saying the file is generated and where to edit, the template's text as written, then the notes section, after it or where the template's marker lines put it. AGENTS.md is build output and goes into .gitignore. It is created when missing, since a fresh clone has none, and otherwise written as before: only when its text changes, in place and under a lock, so links, ownership and a read-only file are respected. The first line is also how a build tells its own AGENTS.md from one someone wrote. Beside a template the second is easily there, as a committed AGENTS.md nobody has removed yet; it is never replaced, and each build warns until it is deleted. An empty file counts as a build's, since that is what a write cut short leaves of a new one. An AGENTS.md linked to the template is refused: writing it would write the committed file again. A template the notes cannot be placed in leaves no empty AGENTS.md behind. The in-place mode is removed rather than kept as the fallback for a project without a template. It is the default that put generated text into tracked files unasked, the moment any dependency started shipping notes. Without a template a build no longer touches AGENTS.md. One that still holds a section from 3.0.0 would go stale silently, under a preamble promising that every build rewrites it, so it gets a warning naming the move instead. MergeDependencyNotes keeps its rules and now runs on the template. The test changes in this commit rather than its own, because it pinned the in-place behaviour this removes: it now covers the generated file's shape, edits and markers in the template, the ownership check, a CRLF template, a link to the template and the cases without one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 16:53:42 +01:00
- `:Agents` writes a project's AGENTS.md from its AGENTS.template.md: it walks the dependencies `cfg.dependencyNotes` selects, reads each one's `AGENTS.consumer.md` and merges them into the template's dependency notes section. Which Configurations belong to another project, and where it lives, comes from the loader (`LoadedProjectOf` in `:External`, `PragmaDependencies` in `:Platform`), because only the Configuration `GitProject` and `LocalProject` return has its `path` rebased.
2026-10-02 20:35:38 +02:00
- `:Progress` is console output. `:Shader` compiles shaders through glslang. `:Asset` is the hook into Crafter.Asset.
## Dev environment
The shipped artifact is the Flatpak (see "CI and publishing"); the loop below is the native development loop.
Prerequisites on Linux: `clang` (with `libclang`, which the AST lint rules load at runtime), `lld`, `libc++` with its std module source (`share/libc++/v1/std.cppm`), `cmake`, `git`. `crafter-build --version` prints which compiler was resolved; `CRAFTER_BUILD_LLVM_DIR` points it at another LLVM prefix, and `build.sh` honours the same variable. Network access to `forgejo.catcrafts.net` and `github.com` is needed: loading `project.cpp` fetches Crafter.Math, Crafter.Asset and Crafter.LintRules into `~/.cache/crafter.build/external/`, and the bootstrap clones glslang. Those sources are unpinned, so every invocation runs `git pull` on them.
2026-10-02 20:35:38 +02:00
```bash
./build.sh # bootstrap: bin/crafter-build, lib/libcrafter-build.a, share/crafter-build/
./bin/crafter-build # self-host build through project.cpp
```
Three things to know about this loop:
- `bin/crafter-build` is the bootstrap binary and the harness for every command below. The self-host build writes to `bin/crafter.build-exe-<variant>/crafter-build` and does not replace it.
- Changes under `implementations/` are picked up by the self-host build and by the tests, which link the local library. Changes under `interfaces/` are not: the harness compiles against the copies in `share/crafter-build/` and the PCM cache built from them, so a new or changed declaration is invisible until you run `./build.sh` again. The symptom is `use of undeclared identifier` for the symbol you just added. A changed default member initialiser has no symptom at all: `project.cpp` compiles against the old one and quietly gets the old default, as `project.cpp`'s sibling loads do with a harness that predates `GitProjectSpec::args = ProjectArgs()`.
- A new exported symbol that a `project.cpp` reaches on its default path also needs `./build.sh`: the old binary does not export it, and `LoadProject` fails with an undefined symbol.
2026-10-02 20:35:38 +02:00
`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. That takes `bin/crafter-build` itself with it, and `build/flatpak`, the state of the local Flatpak build (see "CI and publishing"): about 4 GB after a first run, and the next `run flatpak` rebuilds LLVM. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`.
2026-10-02 20:35:38 +02:00
## Build, test and style commands
```bash
./bin/crafter-build # build (self-host)
./bin/crafter-build test # build and run every self-test
./bin/crafter-build test 'Incremental*' # name glob
./bin/crafter-build test --list # enumerate without running
./bin/crafter-build test --jobs=1 --timeout=120
./bin/crafter-build lint # dry style gate, exit 1 on any finding
./bin/crafter-build format # apply the transform rules in place
./bin/crafter-build format --check # CI-style dry run
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
./bin/crafter-build run # list the scripts project.cpp defines
./bin/crafter-build run <name> [args...] # run one; everything after the name is the script's
2026-10-02 20:35:38 +02:00
./bin/crafter-build clean
```
Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` for every release tag, every pull request into `master` and every manual run, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/<name>.log`.
2026-10-02 20:35:38 +02:00
## Testing instructions
Tests are declared, not discovered. Each one is a line in `project.cpp`:
```cpp
cfg.AddTest("IncrementalHeaderChange").Dependencies({ CrafterBuildLib.get() });
```
The source is `tests/<Name>/main.cpp`. The test inherits target, march and the rest from the root configuration and links the local `crafter.build-lib`, so `import Crafter.Build;` resolves to the code in this checkout. A test that calls `LoadProject` also needs `.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl")`, as the existing ones show. The declarations sit in an `if (linuxTarget)` block that matches any `*-linux-gnu` or `*-linux-musl` target: the suite is Linux-host only today.
2026-10-02 20:35:38 +02:00
A test is an ordinary `int main()`. Exit 0 passes, nonzero fails, 77 skips (for a missing tool, never for a failed assertion). Each runs in its own subprocess, in parallel, with a 60 s default timeout.
Conventions the existing tests follow:
- Open `main.cpp` with a comment that states the behaviour under test and the failure it guards against. `tests/CompilerWarning/main.cpp` shows the shape.
- Collect failures with a `Check(cond, msg)` helper that prints `FAIL: ...` to stderr and counts, then return nonzero at the end. Do not stop at the first failure unless later steps are meaningless without it.
- Drive the engine in-process: `Build(cfg, depResults, depMutex)`, `RunTests`, the lint driver. Do not shell out to `bin/crafter-build` and grep its stdout.
- Fixtures under `tests/<Name>/fixture/` are inputs. A test that edits or touches files first stages a copy under `fs::temp_directory_path() / "crafter-build-<name>"` and removes it on the next run. Variant sources are checked in as `*.in` files and copied over the original inside the staged copy.
fix(external): fetch and build each clone once per build pass `crafter-build test` builds a project's tests side by side, and a project whose tests each copy its externalDependencies hands every one of them the same unpinned dependency. Each test's Build() pulled the shared clone. Before the clone had a lock, those pulls ran at the same time and broke its .git/FETCH_HEAD, which failed every test of Crafter.Graphics (issue #34). Since 653dd0b they wait for each other, but every one is still a network round trip: 26 tests cost 26 pulls of each dependency, one after another, and for a CMake dependency as many configure checks and builds. depResults already records one build pass, keyed by the PcmDir of every dependency configuration built in it. An external dependency now goes in the same map, keyed by the directory its build leaves (ExternalBuildDir: the clone, or for CMake its build directory). The first configuration that needs it fetches and builds it; the others wait for that and take its outcome. Keying on the build directory rather than the clone keeps a CMake declaration from taking the outcome of a header-only declaration of the same clone, which only fetched it. A fresh depResults is a fresh pass, so the next build still pulls, and every caller of Build() gets this: RunTests, a plain build, and the pragma dependencies LoadProject builds. To make that possible BuildExternal is split. BuildExternal is now the shared work, lock, fetch, configure and build, and returns the error. ExternalOutputs computes what one declaration compiles and links with, because two declarations of one clone can differ in includeDirs, libDirs and libs; ExternalBuildResult loses its error field. The work no longer stops between steps when the configuration that started it is cancelled: others may be waiting on it, and a run cut short would leave them a clone nobody fetched. A dependency configuration's Build() already runs to completion for the same reason. The promise and future the dependency threads used to build or wait for a configuration are now one helper, OncePerPass, which both kinds use. An exception thrown in BuildExternal used to escape its thread into std::terminate; it is now a build error, as for a dependency. With eight tests that share a local dependency, a test run fetched it eight times before this and fetches it once now (SharedExternalFetch, next commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 02:09:24 +01:00
- Use a fresh `depResults` map per build pass: the map memoises each configuration's result, and each external dependency's fetch and build, for one pass, so reusing it skips rebuilds and pulls you meant to observe. `Build()` itself is safe to call repeatedly on the same `Configuration`.
2026-10-02 20:35:38 +02:00
- Set `cfg.path` before calling `GetInterfacesAndImplementations`, and spell `cFiles` as absolute paths: both resolve against the working directory, not the configuration.
A bug fix ships with a test that fails before the fix and passes after it, in its own `test:` commit when it documents the behaviour on its own.
Cross-target runs use `test --target=<triple>` with `--runner=local` or `--runner=cmd:<wrapper>` (wine, qemu-user, valgrind), or `CRAFTER_BUILD_RUNNER_<TRIPLE>` as the persistent form. A failed runner probe skips its tests rather than failing the suite.
## Code style
`project.cpp` pulls in [Crafter.LintRules](https://forgejo.catcrafts.net/Catcrafts/Crafter.LintRules) through `#pragma crafter dependency` and applies its lite set, `LintRules::AddLiteRules`: the 8 rules that ask for different code, without the ones that only decide layout, whitespace and naming. `lint` reports all 8; `format` applies the two transforms among them, `format-concat` and `fixed-width-types`, and leaves everything else as written. Enforced:
2026-10-02 20:35:38 +02:00
- The first three lines of every source: `// SPDX-License-Identifier: LGPL-3.0-only`, `// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®`, blank. Examples use MIT.
- `enum class` only. `std::println` instead of iostream. `std::format` instead of string concatenation. `std::int32_t` and friends instead of `int` or `long` (exempt: `main`, `argc`/`argv`, `extern "C"` and foreign API signatures). No `char*` outside C interop.
- `const` on scalar locals that are never written; `constexpr` on constants with constant initialisers.
Not checked, so match the surrounding code:
- Four-space indentation, no tabs, no trailing whitespace, final newline.
- `PascalCase` for types, functions, statics and constants; `camelCase` for variables; a trailing `_` is allowed.
- One declaration per statement. K&R braces. `if (x) return;` on one line when it fits, and no hand-wrapping of a statement that fits on one line. `if (` and `for (` with a space; no space before a call's `(`.
2026-10-02 20:35:38 +02:00
Suppress one finding with `// lint-disable-next-line <rule>` or a whole file with `// lint-disable-file <rule>`. Both are rare in this tree and need a reason.
Beyond the rules:
- `import std;` everywhere; standard headers are never `#include`d. C headers (`clang-c/Index.h`, `dlfcn.h`) go in the global module fragment after `module;`.
2026-10-02 20:35:38 +02:00
- Comments explain why: the failure mode, the constraint, the rejected alternative. The codebase carries long comments on purpose. Do not strip them, and write them for new non-obvious code. Commit bodies follow the same rule.
- There is no host-platform preprocessor split: the host is Linux. Target differences (mingw, wasm) branch on `cfg.target` strings at runtime, in the library as well as in `project.cpp` and tests.
2026-10-02 20:35:38 +02:00
- A build step fails when its command exits nonzero, never because it printed. Use `RunBuildCommand` or `RunCommandChecked`; warnings go to `Progress::Diagnostic`.
- Every output path goes through `Configuration::BinDir()`, `BuildDir()` and `PcmDir()`. Hand-rolled `path / "bin" / ...` clobbered variants before; do not add new ones.
- Anything that changes what gets compiled or bundled belongs in `VariantId()`. It already hashes type, debug, sysroot, defines, compile flags, the wasm baseline and variant flags, the std module the units import (`StdPcmKey`), uninterpreted project args and the toolchain (`Toolchain::Key()`).
2026-10-02 20:35:38 +02:00
## Design rules
These were decided deliberately. Do not undo them without the maintainer's say-so.
1. **Configuration is C++.** No JSON, YAML, TOML or DSL for project, runner or lint configuration. Lint rules are C++ callbacks; there is no clang-format or clang-tidy integration and no separate "format rule" type. A rule that calls `ctx.SetContent` is a transform, and that is the whole mechanism.
2. **Per-project artifacts go in `<project>/build` and `<project>/bin`. Only cross-project things go in `~/.cache/crafter.build/`:** the std PCM and Crafter.Build PCMs per target, march, toolchain and std module flags, external clones keyed by their spec hash, and the mingw sysroot views keyed by prefix.
3. **Sources ship, PCMs build locally.** The Flatpak installs `*.cppm` under `/app/share/crafter-build/`. Nothing target-baked ships except the executable and the toolchain beside it, and the executable is built for a portable `-march`.
4. **crafter-build is a monolithic executable with `--export-dynamic`.** `project.so` resolves `Crafter::` symbols from the running binary at dlopen time. There is no Windows host; `x86_64-w64-mingw32` is a cross-compile target for user projects only.
2026-10-02 20:35:38 +02:00
5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point.
6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch.
7. **The house-style rules stay in their own repo.** They are not library API of crafter-build and are not forced on downstream projects.
8. **Fix warts inline; do not stack workarounds.** The previous version of this tool died of accumulated hacks. If a change needs a shortcut, say so in its commit message rather than hiding it.
2026-10-02 20:35:38 +02:00
9. **`project.cpp` keeps the library/executable split.** `crafter.build-lib` owns the sources, dependencies and glslang; `crafter.build-exe` is `main.cpp` plus link flags.
10. **Host-tooling builds** (the target of a `#pragma crafter dependency`) get `-fPIC` and the host triple scoped to that build. Keep it scoped.
## Commit and push guidelines
2026-10-02 20:35:38 +02:00
- Branch from `master`; issue work uses `claude/issue-<n>` branch names. Once the owner approves its frozen hash, a range goes straight to `master` on the project's Forgejo at `forgejo.catcrafts.net/Catcrafts/Crafter.Build`, as a fast-forward push with no merge commit and no pull request; open a pull request against `master` only when the owner asks for one. If `master` has moved since the freeze, rebase onto it, then freeze and review the rebased range again. Pushes go over SSH; HTTPS push is disabled on the server.
2026-10-02 20:35:38 +02:00
- Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice.
- Separate commits for the fix, its test and the README change when each stands on its own.
- Update the README in the same push when user-visible behaviour changes.
- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it for every release tag, every pull request into `master` and every manual run. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push.
- Every push to `master` is a release, made by the signed tag pushed with it (see "CI and publishing"). Every pushed range adds a `<release>` entry for its version at the top of `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, in a commit of its own (`docs(packaging): add the X.Y.Z release entry`), dated the day of the push in UTC. Its notes have up to three sections, `<p>New</p>`, `<p>Fixes</p>` and `<p>Breaking</p>` in that order, each followed by a `<ul>` of its items, the most important first, and nothing after the last section. The version follows from them: a Breaking section makes it a major, New items a minor, and Fixes alone a patch. Once the owner has approved the frozen hash, the commit `master` is to receive gets the signed tag of that version, made locally, and both go out in one push: `git tag -s X.Y.Z -m 'crafter-build X.Y.Z'`, then `git push --atomic origin HEAD:master X.Y.Z`. That commit is the range's last one, or, when the owner asks for a pull request instead, the merge of the reviewed branch into `master`.
- Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits: a range reaches `master` as a fast-forward, and when the owner asks for a pull request instead, the merge of the reviewed branch into `master` is pushed without a freeze of its own.
feat(review): set up Crafter.Review's pre-push review Crafter.Review reviews a frozen commit range before it is pushed: the approach, a static proof of the code's bugs, and every claim in the messages and in the documentation the range changes, one claude process per lens and unit, or per whole-range pass, and the owner approves the exact frozen hash before the push. This sets it up for this repository. .review-gates/config.json turns on three lenses in the commits mode, at xhigh: repo-better-solution, repo-bug-prover and repo-facts. The functional lens is off: it exercises a web application through a browser, and this repository's product is a command-line tool. The security lens is off by the owner's ruling: crafter-build stores no secret of its own, and while its Flatpak runs with the user's SSH agent (--socket=ssh-auth) and the whole host filesystem (--filesystem=host), everything it runs is code its user chose to run. The rules CI keeps around its token and signing key go to the approach lens's section instead. gate0 lists the commands run by hand before a freeze. build.sh comes first, because bin/crafter-build compiles project.cpp against the interfaces it was bootstrapped with, and lint comes after the self-host build, because lint's AST rules parse with the module PCMs a build leaves and lint builds only the std PCM itself. .review-gates/lenses/ holds this repository's section of each lens's charter, appended to the text Crafter.Review ships: that AGENTS.md is absent from the frozen tree, the layers and the consumers of interfaces/, the execution contexts a bug can hide in, where libc++ and clang are readable from the review's sandbox, and what a "passes the suite" claim traces to. The /prepush skill that crafter-review init writes stays untracked. It is generated from the installed Crafter.Review and refreshed by every init, so a committed copy would lag the binary it describes. One init per checkout restores it, which AGENTS.template.md now says, together with the rules freeze holds every commit of a range to: signed (requireSigned), with the owner's address as its author and committer (authors), and no AI attribution trailer (aiTrailers forbid).
2026-11-01 02:55:03 +01:00
- Every commit of a pushed range is GPG-signed, has `jorijnvdgraaf@catcrafts.net` as its author and committer, and carries no AI attribution trailer in its message, neither `Co-Authored-By:` nor `Assisted-by:`. `freeze` refuses a range with a commit that breaks any of these (`requireSigned`, `authors` and `aiTrailers` in `.review-gates/config.json`).
- Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a`, `AGENTS.md` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`.
- Keep `CRAFTER_BUILD_MARCH=x86-64-v2` in the manifest for the shipped binary: it has to run on any x86_64 machine. User projects still default to `native`.
2026-10-02 20:35:38 +02:00
## Security considerations
- `project.cpp` is compiled and loaded into the crafter-build process, and every `GitProject`, `#pragma crafter dependency` and `ExternalDependency` clones a repository and runs its build on the host. Treat a dependency URL as code you are choosing to run. This repo's own dependencies are sibling Catcrafts repos tracking their default branch on purpose; pin with `commit` when adding anything else, and whenever reproducibility matters.
- Unpinned dependencies `git pull` on every invocation, so builds and tests need network. A missing or private repo surfaces as a clone failure before anything is built.
- `--runner=cmd:<command>` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests.
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
- `crafter-build run <name>` executes whatever the script does, with your permissions. Only the scripts of the project crafter-build was pointed at can be run, never a dependency's; keep it that way.
2026-10-02 20:35:38 +02:00
- The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library.
- CI creates a release for each release tag with the workflow's own token, pushes nothing to git, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it.
2026-10-02 20:35:38 +02:00
- Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way.
## CI and publishing
`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and no other repository's job runs on it. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The web root and the key are not this runner's alone: the organisation-wide `flatpak` runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository, mounts both for Crafter.Review's publish job, and the volumes a runner allows (`valid_volumes` in its config) are open to every job it runs, so a job of any Catcrafts repository that runs there can mount them as well. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. Crafter.Review's workflow publishes into the same repo, and each workflow holds an flock on `/var/www/flatpak/.publish.lock` while it commits and updates the summary.
fix(ci): run on a label only the Crafter.Build runner carries Master's last two runs (#368 and #370) failed in the Build step with The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir (/workspace/Catcrafts/Crafter.Build) while #369, the 2.2.0 tag run on the same commit with the same flatpak-builder, passed. The difference was the runner. Since 14 October the Hetzner box has a second privileged runner, cx33-org-flatpak, which serves every Catcrafts repository under the label `flatpak` so their package jobs can run crafter-build as a Flatpak. It mounts no host volumes, deliberately, since any repository in the organisation can send it jobs. Whenever it was idle first it took this job too, logged our four volumes as "not a valid volume, will be ignored", and left the flatpak-builder state on the container's overlay while the workspace sat on a Docker volume. Even past that check it would have had no build cache, no web root and no signing key. The org runner's label is right for the jobs it was made for, and giving it our volumes would hand the signing key to every repository in the organisation. So the job asks for a label only its own runner carries: `crafter-flatpak`, after the volumes and key directory that runner alone allows. The runner on the host is relabelled to match (/etc/forgejo-runner/config-flatpak.yaml, previous copy kept as config-flatpak.yaml.bak-2026-10-31-label), so until this reaches master a push there with the old `runs-on: flatpak` goes only to the org runner and fails the same way. AGENTS.md now says the label must stay unique and quotes the error a job on the wrong runner stops with, so the next collision is found from the log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 01:25:44 +01:00
Keep the label unique to that runner. Plain `flatpak` belongs to the organisation-wide runner. When this job still asked for `flatpak` and that runner took it (runs #368 and #370, in October 2026), the runner allowed none of the job's volumes, and flatpak-builder stopped with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of such a job's log shows which runner took it and lists every volume it ignored.
The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing:
```bash
install -d -m 700 /etc/crafter-flatpak/gnupg && export GNUPGHOME=/etc/crafter-flatpak/gnupg
gpg --batch --pinentry-mode loopback --passphrase '' --quick-generate-key 'Catcrafts Flatpak Repository <flatpak@catcrafts.net>' ed25519 sign never
KEYID=$(gpg --list-secret-keys --with-colons | awk -F: '/^sec/{print $5; exit}')
gpg --batch --pinentry-mode loopback --passphrase '' --export-secret-keys --armor "$KEYID" > /etc/crafter-flatpak/signing-key.asc
printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id
```
A release is a tag `X.Y.Z`, annotated and signed, pushed with the `master` it releases; a push to `master` without one runs nothing. Before building, every run renders the release notes from the newest `<release>` entry in `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, and stops when the metainfo does not parse or the notes come out empty. A tag run also checks that Forgejo verifies the tag as signed by the address the workflow's `RELEASE_SIGNER` names, that the tag's commit is on `master`, that its version has the form `X.Y.Z` and is newer than every release the forge has published, its own included, and that the newest entry is that version. Once the build and the suite inside it pass, the tag run publishes `stable` and creates the release, with the notes, the bundle, the `.flatpakref` and `SHA256SUMS` attached. Release runs queue one behind another, and a failed one can be re-run while no newer release exists, once a release it left half made is deleted; when the fix needs a commit of its own, that push is the next version's release. Pull requests and manual runs publish nothing. The `beta` branch, which every push to `master` used to go to, gets no more builds: the publish step marks its last build end-of-life, once, and the next `flatpak update` of a beta installation prints the reason.
feat(packaging): `run flatpak` and `run install` for a local Flatpak The local Flatpak build was one long flatpak-builder line in AGENTS.md, with its state wherever the caller happened to stand, and installing the result was not written down anywhere. It is not a rare chore: the development box compiles with the distro's clang and the Flatpak with its bundled LLVM, a major version apart at the moment, so anything toolchain-sensitive has to be tried in the Flatpak too. Both are now scripts in project.cpp, the first users of cfg.scripts. `run flatpak` builds the manifest as it is, from the working tree, and ends with `crafter-build --version` from inside the result. Its state is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in the sandbox, so it does not depend on the host variant, and a variant-keyed directory would strand the LLVM cache on every host compiler update. The manifest already skips build/ when it copies the sources. flatpak-builder keys its cache on the manifest's path, so the manifest is never rewritten into a copy the way CI stamps the version in, and a local build reports 0.0.0-dev. The only argument accepted is --jobs=<N>: anything that ends the run without a build would leave the smoke test checking the previous one. `run install` installs the last export as branch `local` from a user remote of its own and makes it current, leaving the published branch installed. It does not build first: flatpak-builder never caches a `type: dir` module, so every build reruns build.sh and the suite. remote-add --if-not-exists is followed by remote-modify --url, because the remote may exist and point at another checkout. Both refuse to run from the Flatpak build of crafter-build, which has no flatpak to drive. The smoke test goes through `flatpak build` and not `<builder> --run`, which is what CI uses. flatpak-builder 1.4.9 run as the org.flatpak.Builder app exits 1 when the command it ran succeeded and 0 when it failed (`true` gives 1, `exit 3` gives 0), so the script reported a good build as a failure. Measured on the 56-core box with LLVM and the WASI sysroot cached: 260 seconds, the crafter-build module only, 29 tests passing in the sandbox. A second `run install` after a rebuild moved the installed commit. `clean` now also drops that state, about 2 GB and a cold LLVM build. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:29:32 +02:00
Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive.
```bash
./bin/crafter-build run flatpak [--jobs=<N>] # build the Flatpak, then print its version from inside the sandbox
./bin/crafter-build run install # install that build as branch `local` and make it current
```
`run flatpak` needs `flatpak-builder` or the `org.flatpak.Builder` app, and the Sdk (`flatpak install flathub org.freedesktop.Sdk//26.08`). It builds the working tree, uncommitted changes included, runs the whole suite inside the sandbox and reports version `0.0.0-dev`. Its state is `build/flatpak/` (`state/`, `repo/`, `build-dir/`). The first run builds every module, LLVM with lldb, the WASI sysroot and the Vulkan validation layer among them, in about 30 minutes with 56 CPU threads; later runs rebuild only the crafter-build module while the manifest is unchanged, a few minutes, because flatpak-builder never caches a `type: dir` source. The checkout must not be under `/tmp`, which `org.flatpak.Builder` cannot see. Use it for anything toolchain-sensitive: the native loop compiles with the distro's clang and the Flatpak with its bundled LLVM, and the two can be a major version apart.
feat(packaging): `run flatpak` and `run install` for a local Flatpak The local Flatpak build was one long flatpak-builder line in AGENTS.md, with its state wherever the caller happened to stand, and installing the result was not written down anywhere. It is not a rare chore: the development box compiles with the distro's clang and the Flatpak with its bundled LLVM, a major version apart at the moment, so anything toolchain-sensitive has to be tried in the Flatpak too. Both are now scripts in project.cpp, the first users of cfg.scripts. `run flatpak` builds the manifest as it is, from the working tree, and ends with `crafter-build --version` from inside the result. Its state is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in the sandbox, so it does not depend on the host variant, and a variant-keyed directory would strand the LLVM cache on every host compiler update. The manifest already skips build/ when it copies the sources. flatpak-builder keys its cache on the manifest's path, so the manifest is never rewritten into a copy the way CI stamps the version in, and a local build reports 0.0.0-dev. The only argument accepted is --jobs=<N>: anything that ends the run without a build would leave the smoke test checking the previous one. `run install` installs the last export as branch `local` from a user remote of its own and makes it current, leaving the published branch installed. It does not build first: flatpak-builder never caches a `type: dir` module, so every build reruns build.sh and the suite. remote-add --if-not-exists is followed by remote-modify --url, because the remote may exist and point at another checkout. Both refuse to run from the Flatpak build of crafter-build, which has no flatpak to drive. The smoke test goes through `flatpak build` and not `<builder> --run`, which is what CI uses. flatpak-builder 1.4.9 run as the org.flatpak.Builder app exits 1 when the command it ran succeeded and 0 when it failed (`true` gives 1, `exit 3` gives 0), so the script reported a good build as a failure. Measured on the 56-core box with LLVM and the WASI sysroot cached: 260 seconds, the crafter-build module only, 29 tests passing in the sandbox. A second `run install` after a rebuild moved the installed commit. `clean` now also drops that state, about 2 GB and a cold LLVM build. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:29:32 +02:00
`run install` does not build; it installs what `run flatpak` last exported. It points the user remote `crafter-build-local` at `build/flatpak/repo`, installs `net.catcrafts.CrafterBuild//local` from it next to whichever published branch is installed, and makes it the branch `flatpak run net.catcrafts.CrafterBuild` starts. `flatpak make-current --user net.catcrafts.CrafterBuild <branch>` switches back. Before `crafter-build clean`, or before deleting the checkout, run `flatpak uninstall --user net.catcrafts.CrafterBuild//local` and `flatpak remote-delete --user crafter-build-local`; otherwise the remote is left pointing at a repo that is gone.
2026-10-02 20:35:38 +02:00
## Gotchas
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
- A new partition is listed in four places, in import order: `PARTITIONS` in `build.sh`, `CrafterBuildModules` in `Crafter.Build-Platform.cpp`, both arrays in `project.cpp`, and the umbrella `Crafter.Build.cppm`. Then `./build.sh`.
2026-10-02 20:35:38 +02:00
- A `#pragma crafter dependency` target is rebuilt on every load on purpose: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the depfile check is what catches a regenerated host cache. Do not optimise the rebuild away.
- A host-tooling library such as Crafter.LintRules may `import Crafter.Build;` without declaring it as a dependency, because `LoadProject` fills the cache it resolves from. Listing Crafter.Build in its `cfg.dependencies` would clone and build all of crafter-build for every consumer. Its test does link crafter-build, so that test is wired behind `IsTopLevelProject()`: without the guard, crafter-build loading the library as a dependency would load its own `project.cpp` again through the test's `GitProject` and hit the cycle guard.
2026-10-02 20:35:38 +02:00
- `crafter-build clean` leaves a pragma dependency's build alone; it lives with the clone in the cache.
- Units see a std module rebuilt in place, because their depfiles list the BMIs they import (`-fmodule-file-deps`), but not one that moved to another directory: their old depfiles still name the old one. So anything that changes how the std module is compiled goes into `StdPcmFlags`, which `BuildStdPcm` compiles with and `StdPcmKey` hashes. `VariantId()` hashes `StdPcmKey`, so the build moves with it. A flag added to the command anywhere else leaves both directories where they are.
- A CMake external whose recorded configure inputs (`<build>/.crafter-options`) differ from this build's is built again from an empty build directory, not reconfigured in place: CMake keeps what an old compiler or `--sysroot` compiled, and the check results it cached.
2026-10-02 20:35:38 +02:00
- The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop.
- A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone.
- A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules/<major>/`, one `libcxx-<major>-modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot.
- The Flatpak carries the debugging tools for what it builds, because those programs run in its sandbox: lldb is a project of the `llvm` module, with `LLDB_ENABLE_PYTHON`, `LIBEDIT`, `CURSES`, `LIBXML2` and `LZMA` set ON so that a missing dependency fails the configure step (SWIG through Python, which needs it), and the `swig` and `libedit` modules come before it. The Khronos validation layer is the `vulkan-validation-layers` module plus four modules before it, build-only except for SPIRV-Tools' programs (`spirv-val` and the rest), all after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. The layer follows the Sdk's Vulkan loader: when `runtime-version` moves to a runtime with a newer loader, move the layer to the Vulkan SDK release of that loader and take the other four revisions from that release's `scripts/known_good.json`.
- The wasm compile flags live in one place, `WasmBaselineFlags` in `Crafter.Build-Clang.cppm`: the std module and every unit have to agree on them, and `VariantId()` and the std PCM directory hash them, so a change rebuilds instead of mixing old and new BMIs. They enable C++ exceptions, so the link takes the exception-enabled libc++ and libunwind from the sysroot's `lib/<triple>/eh/` (wasi-sdk 33 and later). The Flatpak's sysroot has them and a distro's may not; where they are missing the wasm tests skip, so run those through the Flatpak. A browser page checks the engine with the `WasmBaselineProbes` named in `variants.json`.
- `WasiBrowserRuntimeExit` runs `wasi-runtime/runtime.js` under node and skips where there is none. The Flatpak has no node, so CI never runs it: after touching `runtime.js`, run the suite in the native loop with node on `PATH`.
- `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both.
- AGENTS.md is build output: the self-host build and `test` write it from `AGENTS.template.md`, with the `AGENTS.consumer.md` of each dependency that ships one (Crafter.Math, today) merged in, and it is gitignored. Edit the template; an edit made in AGENTS.md is gone after the next build. A fresh clone or worktree has no AGENTS.md until its first `./bin/crafter-build` or `test`, since `./build.sh` does not write it. A harness from before the template (3.0.0 and older) never creates one and keeps its section in one it finds; the next build with a current harness writes it over.