All checks were successful
Flatpak / flatpak (push) Successful in 30m58s
AGENTS.template.md had every change go through a pull request against master. The owner has made pushing straight to master the default. The commit and push guidelines say so: an approved range is pushed to master as a fast-forward, with no merge commit, and a pull request is opened only when the owner asks for one; when master has moved since the freeze, the range is rebased, frozen and reviewed again. What follows from that: the README changes in the same push; without a pull request CI does not run on a range before it reaches master unless someone starts it by hand, so the review's gate0 is the check before the push; and the freeze exception for a merge commit applies only to a pull request. Design rule 8, the testing note and the release steps lose their pull-request wording, and the testing note no longer speaks of a commented-out lint step: ci.yaml has no lint step at all.
204 lines
30 KiB
Markdown
204 lines
30 KiB
Markdown
# AGENTS.md
|
|
|
|
Guidance for coding agents and contributors working on crafter-build itself. The [README](README.md) is the user-facing manual; this file holds what you need to change the build system. When the two disagree, the code wins: check the interface in `interfaces/` and the `--help` output of the built binary.
|
|
|
|
## Project overview
|
|
|
|
crafter-build is a C++26 modules build system. A project is described by a `project.cpp` that returns a `Crafter::Configuration`; the tool compiles that file to a shared object, loads it in-process and builds what it describes. This repository is self-hosting: its own `project.cpp` builds crafter-build with crafter-build, and its tests link the library and drive `Build()` in-process.
|
|
|
|
| Path | What lives there |
|
|
|---|---|
|
|
| `interfaces/Crafter.Build-<Partition>.cppm` | Module interface partitions. `Crafter.Build.cppm` is the umbrella that re-exports them. |
|
|
| `implementations/Crafter.Build-<Partition>.cpp` | Implementation units, one per partition, named `Crafter.Build:<Partition>_impl`. `main.cpp` is a thin wrapper around `Crafter::Run`. |
|
|
| `project.cpp` | The self-host build description: a `crafter.build-lib` library, a `crafter.build-exe` executable, the test declarations, the lint rules and the `flatpak` and `install` scripts. |
|
|
| `AGENTS.template.md` | The source of AGENTS.md. Every build and test run writes AGENTS.md from it, with the notes the dependencies ship; AGENTS.md is gitignored, so changes go here. |
|
|
| `tests/<Name>/main.cpp` | One self-test per directory; `tests/<Name>/fixture/` holds the sample project it builds. |
|
|
| `examples/` | Standalone user-facing projects, MIT licensed. |
|
|
| `build.sh` | Bootstrap script. It compiles everything by hand, without crafter-build. |
|
|
| `wasi-runtime/` | `runtime.js` and `index.html.in`, shipped for browser targets. |
|
|
| `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. |
|
|
| `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. |
|
|
| `packaging/flatpak/`, `.forgejo/workflows/ci.yaml` | The Flatpak manifest with its metainfo, repo and ref files, and the Forgejo Actions CI that builds and publishes it. |
|
|
| `.review-gates/` | The pre-push review's `config.json` and this repository's lens sections (`lenses/`), read by Crafter.Review. Its `push/` and `trees/` are generated and gitignored. |
|
|
|
|
Partitions and what they own:
|
|
|
|
- `:Toolchain` resolves the compiler once per process (`GetToolchain`): clang++, clang, llvm-ar, libclang, libc++'s std module source and the default sysroots, from `CRAFTER_BUILD_LLVM_DIR`, else a toolchain installed beside crafter-build, else PATH. It also holds the version baked in at build time. `SysrootStdModule` picks the std module source for a sysroot build by the sysroot's libc++ release. `MingwSysroot` settles the mingw target's `--sysroot`: the mingw-w64 prefix itself, or, for a prefix that also holds a Linux system (the host's `/usr`, anything whose `include/` has `features.h`), a view in the cache of its mingw-w64 directories, since clang would search its `include/` and `lib/` too (issue #43). `MingwViewToPrefix` turns paths into the view back into paths into the prefix for messages.
|
|
- `:Clang` holds `Configuration`, `Build()`, `VariantId()`, `ApplyStandardArgs`, `HostTarget`, the CLI entry point `Crafter::Run`, and the lint, test and script data types.
|
|
- `:Platform` runs commands, implements `LoadProject` (compile and load `project.cpp`), the host PCM cache, `#pragma crafter dependency`, and all OS-specific code.
|
|
- `:Interface` and `:Implementation` model module units, their staleness checks and their compiles.
|
|
- `:External` handles git + cmake dependencies, `GitProject` and `LocalProject`.
|
|
- `:Test` holds `AddTest`, `RunTests` and `TestRunner`.
|
|
- `:Lint` is the lint/format driver plus the libclang token and AST layer.
|
|
- `:Script` is the driver behind `crafter-build run`: it looks a name up in `cfg.scripts`, calls the callback and turns what it returns into the exit code.
|
|
- `:Agents` writes a project's AGENTS.md from its AGENTS.template.md: it walks the dependencies `cfg.dependencyNotes` selects, reads each one's `AGENTS.consumer.md` and merges them into the template's dependency notes section. Which Configurations belong to another project, and where it lives, comes from the loader (`LoadedProjectOf` in `:External`, `PragmaDependencies` in `:Platform`), because only the Configuration `GitProject` and `LocalProject` return has its `path` rebased.
|
|
- `:Progress` is console output. `:Shader` compiles shaders through glslang. `:Asset` is the hook into Crafter.Asset.
|
|
|
|
## Dev environment
|
|
|
|
The shipped artifact is the Flatpak (see "CI and publishing"); the loop below is the native development loop.
|
|
|
|
Prerequisites on Linux: `clang` (with `libclang`, which the AST lint rules load at runtime), `lld`, `libc++` with its std module source (`share/libc++/v1/std.cppm`), `cmake`, `git`. `crafter-build --version` prints which compiler was resolved; `CRAFTER_BUILD_LLVM_DIR` points it at another LLVM prefix, and `build.sh` honours the same variable. Network access to `forgejo.catcrafts.net` and `github.com` is needed: loading `project.cpp` fetches Crafter.Math, Crafter.Asset and Crafter.LintRules into `~/.cache/crafter.build/external/`, and the bootstrap clones glslang. Those sources are unpinned, so every invocation runs `git pull` on them.
|
|
|
|
```bash
|
|
./build.sh # bootstrap: bin/crafter-build, lib/libcrafter-build.a, share/crafter-build/
|
|
./bin/crafter-build # self-host build through project.cpp
|
|
```
|
|
|
|
Three things to know about this loop:
|
|
|
|
- `bin/crafter-build` is the bootstrap binary and the harness for every command below. The self-host build writes to `bin/crafter.build-exe-<variant>/crafter-build` and does not replace it.
|
|
- Changes under `implementations/` are picked up by the self-host build and by the tests, which link the local library. Changes under `interfaces/` are not: the harness compiles against the copies in `share/crafter-build/` and the PCM cache built from them, so a new or changed declaration is invisible until you run `./build.sh` again. The symptom is `use of undeclared identifier` for the symbol you just added. A changed default member initialiser has no symptom at all: `project.cpp` compiles against the old one and quietly gets the old default, as `project.cpp`'s sibling loads do with a harness that predates `GitProjectSpec::args = ProjectArgs()`.
|
|
- A new exported symbol that a `project.cpp` reaches on its default path also needs `./build.sh`: the old binary does not export it, and `LoadProject` fails with an undefined symbol.
|
|
|
|
`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. That takes `bin/crafter-build` itself with it, and `build/flatpak`, the state of the local Flatpak build (see "CI and publishing"): about 4 GB after a first run, and the next `run flatpak` rebuilds LLVM. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`.
|
|
|
|
## Build, test and style commands
|
|
|
|
```bash
|
|
./bin/crafter-build # build (self-host)
|
|
./bin/crafter-build test # build and run every self-test
|
|
./bin/crafter-build test 'Incremental*' # name glob
|
|
./bin/crafter-build test --list # enumerate without running
|
|
./bin/crafter-build test --jobs=1 --timeout=120
|
|
./bin/crafter-build lint # dry style gate, exit 1 on any finding
|
|
./bin/crafter-build format # apply the transform rules in place
|
|
./bin/crafter-build format --check # CI-style dry run
|
|
./bin/crafter-build run # list the scripts project.cpp defines
|
|
./bin/crafter-build run <name> [args...] # run one; everything after the name is the script's
|
|
./bin/crafter-build clean
|
|
```
|
|
|
|
Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` on every push to `master` and on pull requests, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/<name>.log`.
|
|
|
|
## Testing instructions
|
|
|
|
Tests are declared, not discovered. Each one is a line in `project.cpp`:
|
|
|
|
```cpp
|
|
cfg.AddTest("IncrementalHeaderChange").Dependencies({ CrafterBuildLib.get() });
|
|
```
|
|
|
|
The source is `tests/<Name>/main.cpp`. The test inherits target, march and the rest from the root configuration and links the local `crafter.build-lib`, so `import Crafter.Build;` resolves to the code in this checkout. A test that calls `LoadProject` also needs `.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl")`, as the existing ones show. The declarations sit in an `if (linuxTarget)` block that matches any `*-linux-gnu` or `*-linux-musl` target: the suite is Linux-host only today.
|
|
|
|
A test is an ordinary `int main()`. Exit 0 passes, nonzero fails, 77 skips (for a missing tool, never for a failed assertion). Each runs in its own subprocess, in parallel, with a 60 s default timeout.
|
|
|
|
Conventions the existing tests follow:
|
|
|
|
- Open `main.cpp` with a comment that states the behaviour under test and the failure it guards against. `tests/CompilerWarning/main.cpp` shows the shape.
|
|
- Collect failures with a `Check(cond, msg)` helper that prints `FAIL: ...` to stderr and counts, then return nonzero at the end. Do not stop at the first failure unless later steps are meaningless without it.
|
|
- Drive the engine in-process: `Build(cfg, depResults, depMutex)`, `RunTests`, the lint driver. Do not shell out to `bin/crafter-build` and grep its stdout.
|
|
- Fixtures under `tests/<Name>/fixture/` are inputs. A test that edits or touches files first stages a copy under `fs::temp_directory_path() / "crafter-build-<name>"` and removes it on the next run. Variant sources are checked in as `*.in` files and copied over the original inside the staged copy.
|
|
- Use a fresh `depResults` map per build pass: the map memoises each configuration's result, and each external dependency's fetch and build, for one pass, so reusing it skips rebuilds and pulls you meant to observe. `Build()` itself is safe to call repeatedly on the same `Configuration`.
|
|
- Set `cfg.path` before calling `GetInterfacesAndImplementations`, and spell `cFiles` as absolute paths: both resolve against the working directory, not the configuration.
|
|
|
|
A bug fix ships with a test that fails before the fix and passes after it, in its own `test:` commit when it documents the behaviour on its own.
|
|
|
|
Cross-target runs use `test --target=<triple>` with `--runner=local` or `--runner=cmd:<wrapper>` (wine, qemu-user, valgrind), or `CRAFTER_BUILD_RUNNER_<TRIPLE>` as the persistent form. A failed runner probe skips its tests rather than failing the suite.
|
|
|
|
## Code style
|
|
|
|
`project.cpp` pulls in [Crafter.LintRules](https://forgejo.catcrafts.net/Catcrafts/Crafter.LintRules) through `#pragma crafter dependency` and applies its lite set, `LintRules::AddLiteRules`: the 8 rules that ask for different code, without the ones that only decide layout, whitespace and naming. `lint` reports all 8; `format` applies the two transforms among them, `format-concat` and `fixed-width-types`, and leaves everything else as written. Enforced:
|
|
|
|
- The first three lines of every source: `// SPDX-License-Identifier: LGPL-3.0-only`, `// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®`, blank. Examples use MIT.
|
|
- `enum class` only. `std::println` instead of iostream. `std::format` instead of string concatenation. `std::int32_t` and friends instead of `int` or `long` (exempt: `main`, `argc`/`argv`, `extern "C"` and foreign API signatures). No `char*` outside C interop.
|
|
- `const` on scalar locals that are never written; `constexpr` on constants with constant initialisers.
|
|
|
|
Not checked, so match the surrounding code:
|
|
|
|
- Four-space indentation, no tabs, no trailing whitespace, final newline.
|
|
- `PascalCase` for types, functions, statics and constants; `camelCase` for variables; a trailing `_` is allowed.
|
|
- One declaration per statement. K&R braces. `if (x) return;` on one line when it fits, and no hand-wrapping of a statement that fits on one line. `if (` and `for (` with a space; no space before a call's `(`.
|
|
|
|
Suppress one finding with `// lint-disable-next-line <rule>` or a whole file with `// lint-disable-file <rule>`. Both are rare in this tree and need a reason.
|
|
|
|
Beyond the rules:
|
|
|
|
- `import std;` everywhere; standard headers are never `#include`d. C headers (`clang-c/Index.h`, `dlfcn.h`) go in the global module fragment after `module;`.
|
|
- Comments explain why: the failure mode, the constraint, the rejected alternative. The codebase carries long comments on purpose. Do not strip them, and write them for new non-obvious code. Commit bodies follow the same rule.
|
|
- There is no host-platform preprocessor split: the host is Linux. Target differences (mingw, wasm) branch on `cfg.target` strings at runtime, in the library as well as in `project.cpp` and tests.
|
|
- A build step fails when its command exits nonzero, never because it printed. Use `RunBuildCommand` or `RunCommandChecked`; warnings go to `Progress::Diagnostic`.
|
|
- Every output path goes through `Configuration::BinDir()`, `BuildDir()` and `PcmDir()`. Hand-rolled `path / "bin" / ...` clobbered variants before; do not add new ones.
|
|
- Anything that changes what gets compiled or bundled belongs in `VariantId()`. It already hashes type, debug, sysroot, defines, compile flags, the wasm baseline and variant flags, the std module the units import (`StdPcmKey`), uninterpreted project args and the toolchain (`Toolchain::Key()`).
|
|
|
|
## Design rules
|
|
|
|
These were decided deliberately. Do not undo them without the maintainer's say-so.
|
|
|
|
1. **Configuration is C++.** No JSON, YAML, TOML or DSL for project, runner or lint configuration. Lint rules are C++ callbacks; there is no clang-format or clang-tidy integration and no separate "format rule" type. A rule that calls `ctx.SetContent` is a transform, and that is the whole mechanism.
|
|
2. **Per-project artifacts go in `<project>/build` and `<project>/bin`. Only cross-project things go in `~/.cache/crafter.build/`:** the std PCM and Crafter.Build PCMs per target, march, toolchain and std module flags, external clones keyed by their spec hash, and the mingw sysroot views keyed by prefix.
|
|
3. **Sources ship, PCMs build locally.** The Flatpak installs `*.cppm` under `/app/share/crafter-build/`. Nothing target-baked ships except the executable and the toolchain beside it, and the executable is built for a portable `-march`.
|
|
4. **crafter-build is a monolithic executable with `--export-dynamic`.** `project.so` resolves `Crafter::` symbols from the running binary at dlopen time. There is no Windows host; `x86_64-w64-mingw32` is a cross-compile target for user projects only.
|
|
5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point.
|
|
6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch.
|
|
7. **The house-style rules stay in their own repo.** They are not library API of crafter-build and are not forced on downstream projects.
|
|
8. **Fix warts inline; do not stack workarounds.** The previous version of this tool died of accumulated hacks. If a change needs a shortcut, say so in its commit message rather than hiding it.
|
|
9. **`project.cpp` keeps the library/executable split.** `crafter.build-lib` owns the sources, dependencies and glslang; `crafter.build-exe` is `main.cpp` plus link flags.
|
|
10. **Host-tooling builds** (the target of a `#pragma crafter dependency`) get `-fPIC` and the host triple scoped to that build. Keep it scoped.
|
|
|
|
## Commit and push guidelines
|
|
|
|
- Branch from `master`; issue work uses `claude/issue-<n>` branch names. Once the owner approves its frozen hash, a range goes straight to `master` on the project's Forgejo at `forgejo.catcrafts.net/Catcrafts/Crafter.Build`, as a fast-forward push with no merge commit and no pull request; open a pull request against `master` only when the owner asks for one. If `master` has moved since the freeze, rebase onto it, then freeze and review the rebased range again. Pushes go over SSH; HTTPS push is disabled on the server.
|
|
- Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice.
|
|
- Separate commits for the fix, its test and the README change when each stands on its own.
|
|
- Update the README in the same push when user-visible behaviour changes.
|
|
- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on every push to `master` and on pull requests; a push to `master` then publishes the `beta` branch of the Catcrafts repo, and an `X.Y.Z` tag the `stable` branch plus a release. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push.
|
|
- Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits: a range reaches `master` as a fast-forward, and when the owner asks for a pull request instead, the merge of the reviewed branch into `master` is pushed without a freeze of its own.
|
|
- Every commit of a pushed range is GPG-signed, has `jorijnvdgraaf@catcrafts.net` as its author and committer, and carries no AI attribution trailer in its message, neither `Co-Authored-By:` nor `Assisted-by:`. `freeze` refuses a range with a commit that breaks any of these (`requireSigned`, `authors` and `aiTrailers` in `.review-gates/config.json`).
|
|
- Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a`, `AGENTS.md` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`.
|
|
- Keep `CRAFTER_BUILD_MARCH=x86-64-v2` in the manifest for the shipped binary: it has to run on any x86_64 machine. User projects still default to `native`.
|
|
|
|
## Security considerations
|
|
|
|
- `project.cpp` is compiled and loaded into the crafter-build process, and every `GitProject`, `#pragma crafter dependency` and `ExternalDependency` clones a repository and runs its build on the host. Treat a dependency URL as code you are choosing to run. This repo's own dependencies are sibling Catcrafts repos tracking their default branch on purpose; pin with `commit` when adding anything else, and whenever reproducibility matters.
|
|
- Unpinned dependencies `git pull` on every invocation, so builds and tests need network. A missing or private repo surfaces as a clone failure before anything is built.
|
|
- `--runner=cmd:<command>` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests.
|
|
- `crafter-build run <name>` executes whatever the script does, with your permissions. Only the scripts of the project crafter-build was pointed at can be run, never a dependency's; keep it that way.
|
|
- The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library.
|
|
- CI force-pushes the rolling `latest` tag and publishes a prerelease on every push to `master` with the workflow's own token, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it.
|
|
- Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way.
|
|
|
|
## CI and publishing
|
|
|
|
`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it.
|
|
|
|
Keep the label unique to that runner. Plain `flatpak` also belongs to the organisation-wide runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository and on purpose allows no volumes. Whenever this job lands there, flatpak-builder stops with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of the job log shows which runner took it and lists every volume it ignored.
|
|
|
|
The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing:
|
|
|
|
```bash
|
|
install -d -m 700 /etc/crafter-flatpak/gnupg && export GNUPGHOME=/etc/crafter-flatpak/gnupg
|
|
gpg --batch --pinentry-mode loopback --passphrase '' --quick-generate-key 'Catcrafts Flatpak Repository <flatpak@catcrafts.net>' ed25519 sign never
|
|
KEYID=$(gpg --list-secret-keys --with-colons | awk -F: '/^sec/{print $5; exit}')
|
|
gpg --batch --pinentry-mode loopback --passphrase '' --export-secret-keys --armor "$KEYID" > /etc/crafter-flatpak/signing-key.asc
|
|
printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id
|
|
```
|
|
|
|
To release: add a `<release>` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), push it to `master`, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry.
|
|
Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive.
|
|
|
|
```bash
|
|
./bin/crafter-build run flatpak [--jobs=<N>] # build the Flatpak, then print its version from inside the sandbox
|
|
./bin/crafter-build run install # install that build as branch `local` and make it current
|
|
```
|
|
|
|
`run flatpak` needs `flatpak-builder` or the `org.flatpak.Builder` app, and the Sdk (`flatpak install flathub org.freedesktop.Sdk//26.08`). It builds the working tree, uncommitted changes included, runs the whole suite inside the sandbox and reports version `0.0.0-dev`. Its state is `build/flatpak/` (`state/`, `repo/`, `build-dir/`). The first run builds every module, LLVM with lldb, the WASI sysroot and the Vulkan validation layer among them, in about 30 minutes with 56 CPU threads; later runs rebuild only the crafter-build module while the manifest is unchanged, a few minutes, because flatpak-builder never caches a `type: dir` source. The checkout must not be under `/tmp`, which `org.flatpak.Builder` cannot see. Use it for anything toolchain-sensitive: the native loop compiles with the distro's clang and the Flatpak with its bundled LLVM, and the two can be a major version apart.
|
|
|
|
`run install` does not build; it installs what `run flatpak` last exported. It points the user remote `crafter-build-local` at `build/flatpak/repo`, installs `net.catcrafts.CrafterBuild//local` from it next to whichever published branch is installed, and makes it the branch `flatpak run net.catcrafts.CrafterBuild` starts. `flatpak make-current --user net.catcrafts.CrafterBuild <branch>` switches back. Before `crafter-build clean`, or before deleting the checkout, run `flatpak uninstall --user net.catcrafts.CrafterBuild//local` and `flatpak remote-delete --user crafter-build-local`; otherwise the remote is left pointing at a repo that is gone.
|
|
|
|
## Gotchas
|
|
|
|
- A new partition is listed in four places, in import order: `PARTITIONS` in `build.sh`, `CrafterBuildModules` in `Crafter.Build-Platform.cpp`, both arrays in `project.cpp`, and the umbrella `Crafter.Build.cppm`. Then `./build.sh`.
|
|
- A `#pragma crafter dependency` target is rebuilt on every load on purpose: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the depfile check is what catches a regenerated host cache. Do not optimise the rebuild away.
|
|
- A host-tooling library such as Crafter.LintRules may `import Crafter.Build;` without declaring it as a dependency, because `LoadProject` fills the cache it resolves from. Listing Crafter.Build in its `cfg.dependencies` would clone and build all of crafter-build for every consumer. Its test does link crafter-build, so that test is wired behind `IsTopLevelProject()`: without the guard, crafter-build loading the library as a dependency would load its own `project.cpp` again through the test's `GitProject` and hit the cycle guard.
|
|
- `crafter-build clean` leaves a pragma dependency's build alone; it lives with the clone in the cache.
|
|
- Units see a std module rebuilt in place, because their depfiles list the BMIs they import (`-fmodule-file-deps`), but not one that moved to another directory: their old depfiles still name the old one. So anything that changes how the std module is compiled goes into `StdPcmFlags`, which `BuildStdPcm` compiles with and `StdPcmKey` hashes. `VariantId()` hashes `StdPcmKey`, so the build moves with it. A flag added to the command anywhere else leaves both directories where they are.
|
|
- A CMake external whose recorded configure inputs (`<build>/.crafter-options`) differ from this build's is built again from an empty build directory, not reconfigured in place: CMake keeps what an old compiler or `--sysroot` compiled, and the check results it cached.
|
|
- The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop.
|
|
- A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone.
|
|
- A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules/<major>/`, one `libcxx-<major>-modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot.
|
|
- The Flatpak carries the debugging tools for what it builds, because those programs run in its sandbox: lldb is a project of the `llvm` module, with `LLDB_ENABLE_PYTHON`, `LIBEDIT`, `CURSES`, `LIBXML2` and `LZMA` set ON so that a missing dependency fails the configure step (SWIG through Python, which needs it), and the `swig` and `libedit` modules come before it. The Khronos validation layer is the `vulkan-validation-layers` module plus four modules before it, build-only except for SPIRV-Tools' programs (`spirv-val` and the rest), all after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. The layer follows the Sdk's Vulkan loader: when `runtime-version` moves to a runtime with a newer loader, move the layer to the Vulkan SDK release of that loader and take the other four revisions from that release's `scripts/known_good.json`.
|
|
- The wasm compile flags live in one place, `WasmBaselineFlags` in `Crafter.Build-Clang.cppm`: the std module and every unit have to agree on them, and `VariantId()` and the std PCM directory hash them, so a change rebuilds instead of mixing old and new BMIs. They enable C++ exceptions, so the link takes the exception-enabled libc++ and libunwind from the sysroot's `lib/<triple>/eh/` (wasi-sdk 33 and later). The Flatpak's sysroot has them and a distro's may not; where they are missing the wasm tests skip, so run those through the Flatpak. A browser page checks the engine with the `WasmBaselineProbes` named in `variants.json`.
|
|
- `WasiBrowserRuntimeExit` runs `wasi-runtime/runtime.js` under node and skips where there is none. The Flatpak has no node, so CI never runs it: after touching `runtime.js`, run the suite in the native loop with node on `PATH`.
|
|
- `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both.
|
|
- AGENTS.md is build output: the self-host build and `test` write it from `AGENTS.template.md`, with the `AGENTS.consumer.md` of each dependency that ships one (Crafter.Math, today) merged in, and it is gitignored. Edit the template; an edit made in AGENTS.md is gone after the next build. A fresh clone or worktree has no AGENTS.md until its first `./bin/crafter-build` or `test`, since `./build.sh` does not write it. A harness from before the template (3.0.0 and older) never creates one and keeps its section in one it finds; the next build with a current harness writes it over.
|