Merge branch 'claude/project-dependencies': project.cpp dependencies, AGENTS.md
Brings in `#pragma crafter dependency`, the switch to Crafter.LintRules for the house style, and the contributor guide in AGENTS.md. The branch predates the musl host work on master, and both rewrote the command that compiles project.so, so the merge reconciles them by hand: - LoadProject compiles project.so for hostConfig.target (HostTarget(), from master) with the pre-dependency module path and archives from the branch. -fuse-ld=lld appears once, with both reasons in the comment. - ForceHostTooling added -fPIC only for the literal x86_64-pc-linux-gnu triple. hostConfig now comes from HostTarget(), so on Alpine the host is x86_64-alpine-linux-musl and the dependency archive would not have been position-independent. It now matches any *-linux-gnu or *-linux-musl host, as project.cpp and the cmake flags do since the musl change, and its comment no longer argues against HostTarget(). - The ProjectDependency test compared the dependency's target against the glibc triple, which the musl CI job would fail; it compares against HostTarget(). - README keeps master's paragraph on the bootstrap triple. AGENTS.md describes the test block as `if (linuxTarget)`, which is what project.cpp has now. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
commit
c3d97be59e
GPG key ID:
2937E59CDCC1BCFB
18 changed files with 784 additions and 1635 deletions
134
README.md
134
README.md
|
|
@ -1,23 +1,16 @@
|
|||
# Crafter Build
|
||||
|
||||
A C++26 modules build system. Project descriptions are written in C++ — no JSON, no Lua, no embedded DSL. You write a `project.cpp` that constructs a `Configuration` and returns it; Crafter Build compiles, loads, and executes it.
|
||||
|
||||
## Status
|
||||
|
||||
- **Linux**: working end-to-end, packageable as a distro package. Verified on Arch Linux via [PKGBUILD](PKGBUILD) in a fresh container.
|
||||
- **Windows**: working end-to-end, including the test runner. Bootstrap (`build.cmd`) produces `crafter-build.exe` + `crafter-build.dll` + `crafter-build.lib`; user `project.cpp` links against the import lib via the host-controlled DLL boundary. `crafter-build test` runs natively on Windows hosts; cross-OS runs back to Linux via WSL2 (`--runner=wsl`).
|
||||
A C++26 modules build system. Project descriptions are written in C++ with no JSON, no Lua, no embedded DSL. You write a `project.cpp` that constructs a `Configuration` and returns it; Crafter Build compiles, loads, and executes it.
|
||||
|
||||
## Quick start (Linux)
|
||||
|
||||
Bootstrap requires `clang`, `cmake`, `git`, `lld`, and `libc++`.
|
||||
|
||||
```bash
|
||||
./build.sh # produces bin/crafter-build
|
||||
./build.sh # produces bin/crafter-build
|
||||
./bin/crafter-build # rebuild via project.cpp
|
||||
```
|
||||
|
||||
For distro-packaged installs, `crafter-build` finds its modules at `<prefix>/share/crafter-build/` automatically — no env var required.
|
||||
|
||||
The bootstrap targets the host toolchain's own triple (`clang++ -print-target-triple`): `x86_64-pc-linux-gnu` on glibc distros, `x86_64-alpine-linux-musl` on Alpine/postmarketOS (`apk add clang clang-dev lld libc++-dev llvm-libunwind-dev llvm-runtimes llvm llvm-dev cmake build-base python3 git`). Override with `CRAFTER_BUILD_TARGET=`. CI ships both: `crafter-build-linux-x86_64-v{2,3,4}.tar.gz` (glibc) and `crafter-build-linux-x86_64-musl-v2.tar.gz` (musl, needs `libc++` at runtime — the launcher dlopens `project.so`, so it cannot be static).
|
||||
|
||||
To build the system as a distro package on Arch:
|
||||
|
|
@ -26,33 +19,6 @@ To build the system as a distro package on Arch:
|
|||
makepkg -si # uses CRAFTER_BUILD_MARCH=x86-64-v3 by default for portability
|
||||
```
|
||||
|
||||
## Quick start (Windows)
|
||||
|
||||
Two ways to get a working `crafter-build` on Windows: native build via `build.cmd` (MSVC ABI), or download the cross-compiled mingw artifact from CI (mingw ABI). They produce different binaries with different ABI requirements; pick one and stick with it.
|
||||
|
||||
**Native MSVC build** — what `build.cmd` does, what CI doesn't ship.
|
||||
|
||||
Need clang+lld targeting `x86_64-pc-windows-msvc` and a Windows libc++ install pointed to by `LIBCXX_DIR`. Then `build.cmd` produces `bin/{crafter-build.exe, crafter-build.dll, crafter-build.lib}`.
|
||||
|
||||
**Cross-compiled mingw artifact** — what CI ships from the Linux runner.
|
||||
|
||||
The CI's Windows zip contains `crafter-build.exe` + `crafter-build.dll` + `libcrafter-build.dll.a`. The DLL is **statically linked against libstdc++/libgcc/libwinpthread**, so it doesn't depend on a particular runtime DLL being on the consumer's PATH. To use it for builds the consumer needs the mingw-w64 sysroot installed, since `crafter-build.exe` invokes the user's clang to compile their `project.cpp`:
|
||||
|
||||
```powershell
|
||||
# one-time setup
|
||||
winget install MSYS2.MSYS2
|
||||
C:\msys64\usr\bin\pacman.exe -S --noconfirm mingw-w64-ucrt-x86_64-toolchain
|
||||
# add C:\msys64\ucrt64\bin to PATH (so user-built executables can find msys2 clang)
|
||||
|
||||
# then any project with a project.cpp:
|
||||
cd <project-dir>
|
||||
crafter-build.exe -r
|
||||
```
|
||||
|
||||
Match the toolchain flavor to the artifact: **UCRT mingw** (`mingw-w64-ucrt-x86_64-toolchain`, sysroot at `C:\msys64\ucrt64`) — the artifact is cross-compiled with UCRT, the older MSVCRT mingw won't ABI-match. Override the auto-detected sysroot path with `CRAFTER_MINGW_DIR=...` if you have mingw-w64 installed somewhere else.
|
||||
|
||||
A project.cpp built on Windows by this artifact gets compiled with `--target=x86_64-w64-mingw32` by default, producing self-contained mingw exes (libstdc++/libgcc/libwinpthread statically linked, no runtime DLLs alongside). For MSVC-ABI output set `cfg.target = "x86_64-pc-windows-msvc"` in your `project.cpp` *and* point `LIBCXX_DIR` at a Windows libc++ install — same prerequisite as the native build.
|
||||
|
||||
## Writing a project.cpp
|
||||
|
||||
Crafter Build loads a `project.cpp` from the current directory. The file exports one function that returns a populated `Configuration`:
|
||||
|
|
@ -79,7 +45,7 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
|
|||
}
|
||||
```
|
||||
|
||||
Run `crafter-build` in that directory; outputs land at `bin/myapp-<target>-<march>/myapp` and intermediates at `build/myapp-<target>-<march>/`. Add `-r` to run the produced executable straight after a successful build (host targets only — for cross targets use `crafter-build test --runner=...`).
|
||||
Run `crafter-build` in that directory; outputs land at `bin/myapp-<target>-<march>/myapp` and intermediates at `build/myapp-<target>-<march>/`. Add `-r` to run the produced executable straight after a successful build.
|
||||
|
||||
## Dependencies
|
||||
|
||||
|
|
@ -104,6 +70,34 @@ Clones live at `~/.cache/crafter.build/external/<name>-<hash>/`, keyed by `(url,
|
|||
|
||||
**Header-only git deps** — same as cmake deps but `builder = ExternalBuilder::None`. Just clones and propagates `-I` flags.
|
||||
|
||||
**Sibling crafter-build projects** — `GitProject` / `LocalProject` load another project's `project.cpp` in-process and return a `Configuration*` for `cfg.dependencies`:
|
||||
|
||||
```cpp
|
||||
Configuration* math = GitProject({
|
||||
.source = { .url = "https://forgejo.catcrafts.net/Catcrafts/Crafter.Math.git" }, // .branch or .commit to pin
|
||||
.args = depArgs, // forwarded to its CrafterBuildProject
|
||||
});
|
||||
Configuration* sibling = LocalProject({ .projectFile = "../Crafter.Math/project.cpp", .args = depArgs });
|
||||
cfg.dependencies = { math };
|
||||
```
|
||||
|
||||
Clones share the external cache and identical specs share one `Configuration`. An unpinned source is `git pull`ed on every invocation; set `.commit` for reproducible, offline-after-first-fetch builds.
|
||||
|
||||
### Dependencies of project.cpp itself
|
||||
|
||||
`project.cpp` can import modules from another crafter-build project. The file has to compile before it can say anything, so the dependency is a directive next to the `import` it enables:
|
||||
|
||||
```cpp
|
||||
import std;
|
||||
import Crafter.Build;
|
||||
#pragma crafter dependency "https://forgejo.catcrafts.net/Catcrafts/Crafter.LintRules.git"
|
||||
import Crafter.LintRules;
|
||||
```
|
||||
|
||||
```
|
||||
#pragma crafter dependency "<source>" [branch "<name>" | commit "<sha>"] [project "<file>"]
|
||||
```
|
||||
|
||||
## Install layout
|
||||
|
||||
```
|
||||
|
|
@ -113,31 +107,6 @@ Clones live at `~/.cache/crafter.build/external/<name>-<hash>/`, keyed by `(url,
|
|||
~/.cache/crafter.build/<target>-<march>/ # std.pcm + Crafter.Build*.pcm, built locally
|
||||
```
|
||||
|
||||
Sources ship instead of PCMs because libc++ ABI varies between machines — the user's machine builds its own PCMs against its own libc++ on first run (~6-8s one-time cost), making the install resilient to libc++ version differences across distros.
|
||||
|
||||
The static archive is for downstream projects that want to use Crafter.Build *as a library* (construct `Configuration` objects, call `Build()`, etc., directly in C++) without invoking the CLI. Link with `-lcrafter-build` plus the project's external dep libs (`-lSPIRV -lglslang -lOSDependent -lMachineIndependent -lglslang-default-resource-limits -lGenericCodeGen`) and `-ldl` (for `LoadProject`'s dlopen path on Linux). On Windows the equivalent is `bin/crafter-build-static.lib`.
|
||||
|
||||
## Build incrementality
|
||||
|
||||
Per-import precise tracking for both within-project and cross-project module dependencies:
|
||||
|
||||
- Touch `lib/Hello.cppm` → only consumers of `Hello` rebuild.
|
||||
- Touch `lib/Other.cppm` → only consumers of `Other` rebuild.
|
||||
- Touch a header → only the units that `#include` it (directly or transitively) rebuild.
|
||||
- External CMake dep produces fresh `.a` files → whole project rebuilds (deliberately coarse — cmake-dep changes are rare).
|
||||
|
||||
Diamond deps (`A → {B, C}; B → X; C → X`) build `X` exactly once via a `std::shared_future<BuildResult>` cache.
|
||||
|
||||
Tracking is derived from each translation unit's `import` statements, which are scanned when the sources are declared. Every kind of unit is scanned — primary module interfaces, partitions and implementation units alike — and an interface that imports a sibling module in the same `Configuration` also gets its compile ordered behind it. `cfg.dependencies` is often assigned *afterwards* — `AddTest` works that way — so `Build()` re-resolves any import that matched nothing at scan time before it compares mtimes. Without all of that, a consumer of a dependency's module carried no edge to it at all: adding a data member to that dependency's interface rebuilt the library, relinked the consumer, and left the consumer's object compiled against the old class layout. Nothing fails to link when a member is added, so the result was a working build and a crash later.
|
||||
|
||||
`#include`s are tracked separately, because scanning can't see them: a header pulls in more headers, and which ones are open depends on the preprocessor state at the point of inclusion. So every compile is asked to report what it actually opened — `clang -MD -MF <artifact>.d`, written next to the object or BMI it belongs to — and the next staleness check reads that file back and compares every prerequisite's mtime against the artifact. C sources are covered the same way, and there they are the *only* dependency record. An artifact whose depfile is missing, or that names a prerequisite which no longer exists, is rebuilt once: neither state is evidence of freshness. Before this, a header edit changed no `.cppm` or `.cpp` mtime at all, so a build reported nothing to do and left objects compiled against the old contents — the same mixed-layout failure as above, reached through `#include` instead of `import`.
|
||||
|
||||
Everything that changes what gets built belongs in the variant id, since it names the `bin/` and `build/` directory. That includes project args crafter-build itself doesn't interpret: `crafter-build` and `crafter-build -- --no-webgpu` get separate directories rather than interleaving their outputs in one. The cached host PCMs under `<cache>/crafter.build/<target>-<march>/` are shared by every crafter-build on the machine, so they are invalidated by a hash of the module sources rather than by mtime — an mtime can't tell a newer PCM from one built by a different install.
|
||||
|
||||
`crafter-build clean` removes the project's `bin/` and `build/` trees. It doesn't load `project.cpp`, so it still works when the project no longer compiles.
|
||||
|
||||
A build step fails when its command exits non-zero — never because it printed something. Warnings are printed to stderr and the build carries on; nothing here implies `-Werror`, so add it to `cfg.compileFlags` if you want warnings to be fatal, and they will then be fatal every time. Note that a warning only appears when the unit it belongs to is actually recompiled, so a build with nothing to do is silent.
|
||||
|
||||
## Tests
|
||||
|
||||
Tests live under `tests/<Name>/`. The simplest case is a single C++ file:
|
||||
|
|
@ -152,12 +121,6 @@ Run them all:
|
|||
crafter-build test
|
||||
```
|
||||
|
||||
Auto-discovery walks `tests/` one level deep. Three escalation layers:
|
||||
|
||||
1. **`tests/<Name>/main.cpp`** with no `project.cpp` — a Configuration is synthesized: top-level `*.cpp` files become implementations, `interfaces/*.cppm` become module interfaces, target = the run's `--target=` (host triple by default). Most tests need nothing more.
|
||||
2. **`tests/<Name>/project.cpp`** — full control. Use this for tests with defines, dependencies, or a non-default target.
|
||||
3. Folders starting with `_` or `.` are skipped, so `tests/_shared/` can hold cross-test code without becoming a test itself.
|
||||
|
||||
Test conventions:
|
||||
|
||||
- **Exit codes**: `0` = ✅ pass, nonzero = ❌ fail, **`77`** = ⏭ skipped (autoconf convention; use it for runtime opt-outs like "tool not on PATH").
|
||||
|
|
@ -260,7 +223,7 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>)
|
|||
|
||||
## Linting & formatting
|
||||
|
||||
Lint rules are C++ in `project.cpp` — no `.lintrc`, no YAML, and no built-in rules. A rule is a named callback that runs once per source file; register it on the `Configuration` before returning:
|
||||
Lint rules are C++ in `project.cpp` with no `.lintrc`, no YAML, and no built-in rules. A rule is a named callback that runs once per source file; register it on the `Configuration` before returning:
|
||||
|
||||
```cpp
|
||||
cfg.AddLintRule("no-tabs", [](Crafter::LintContext& ctx) {
|
||||
|
|
@ -271,7 +234,7 @@ cfg.AddLintRule("no-tabs", [](Crafter::LintContext& ctx) {
|
|||
});
|
||||
```
|
||||
|
||||
A rule that calls `ctx.SetContent(newContent)` is a **transform** — defined once, it powers both verbs: `lint` reports where the transform would change the file (dry, never writes), and `format` applies it to disk:
|
||||
A rule that calls `ctx.SetContent(newContent)` is a **transform** which is defined once, it powers both verbs: `lint` reports where the transform would change the file (dry, never writes), and `format` applies it to disk:
|
||||
|
||||
```cpp
|
||||
cfg.AddLintRule("final-newline", [](Crafter::LintContext& ctx) {
|
||||
|
|
@ -313,12 +276,6 @@ Suppression is comment-based :
|
|||
// lint-disable-file — all rules, whole file
|
||||
```
|
||||
|
||||
Directives silence findings **and** stop `format` from rewriting the suppressed lines. The driver handles this automatically for reports and line-preserving transforms; a custom transform that merges or splits lines must ask `ctx.Suppressed(rule, line)` for the lines it touches (the built-in house rules do).
|
||||
|
||||
The linted file set is the project's own sources: module interfaces (+ partitions), implementations, `cFiles`, `cuda`, shader sources, declared tests' sources, and `project.cpp` itself — for the root `Configuration` plus every transitive dependency whose path lies inside the project root. `GitProject` / external dependencies are foreign code and are skipped, as are `files`/`buildFiles`/`assets` (data, not source). Rules registered on any in-root config are collected and deduplicated by name, so with a lib/exe split you can attach them to either. Rules run in registration order, each seeing the previous transform's output; a rule that throws is reverted and becomes a finding instead of aborting the run. Files are read and written as bytes — line views carry a trailing `\r` on CRLF files, so transforms that should be CRLF-safe must handle it (see the `trim-trailing-ws` dogfood rule).
|
||||
|
||||
This repo dogfoods the feature with its full house style — 15 rules in [lint-rules.h](lint-rules.h) (naming conventions, K&R braces, `enum class`, `std::println`, fixed-width integer types, `std::format` over concatenation, wrap-joining, whitespace hygiene), registered by [project.cpp](project.cpp) via one `ProjectLint::AddProjectLintRules(cfg)` call. The header is deliberately project-local, not part of the crafter-build library — copy it into a sibling repo to adopt the same style there; `tests/HouseRules/` validates every transform. Most rules are transforms: `crafter-build format` mechanically restyled this entire codebase (~150 fixes), with only expression rewrites the operand scanner can't prove safe (ternaries, raw-string lines) reported for hand-fixing.
|
||||
|
||||
## Examples
|
||||
|
||||
[examples/](examples/) contains progressively larger self-contained projects:
|
||||
|
|
@ -328,33 +285,10 @@ This repo dogfoods the feature with its full house style — 15 rules in [lint-r
|
|||
| [hello-world](examples/hello-world/) | Smallest possible project: `project.cpp` + `main.cpp` |
|
||||
| [with-module](examples/with-module/) | Adding a C++ module interface |
|
||||
| [library](examples/library/) | Library + executable, linked via `cfg.dependencies` |
|
||||
| [tests](examples/tests/) | Auto-discovered tests + tests linking the parent library |
|
||||
| [tests](examples/tests/) | tests linking the parent library |
|
||||
|
||||
Each builds standalone: `cd examples/<name> && crafter-build`.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **Modules**: `Crafter.Build:Shader` / `:Platform` / `:Interface` / `:Implementation` / `:External` / `:Clang` / `:Test` / `:Lint` / `:Progress` / `:Asset` partitions, re-exported by the `Crafter.Build` umbrella.
|
||||
- **Build process**: parallel — interface PCMs, implementation `.o` files, external dep clones, dep-config recursive builds, and shader compilations all spawn threads, sync at well-defined join points.
|
||||
- **`LoadProject`** compiles `project.cpp` to a shared object (`.so` on Linux, `.dll` on Windows) and loads it. On Linux the host exe is linked with `-Wl,--export-dynamic` so the project's undefined symbols resolve back into the running binary. On Windows the host is split into `crafter-build.dll` (everything) + a thin `crafter-build.exe` launcher; the public `Crafter::*` API is annotated with a `CRAFTER_API` macro (`__declspec(dllexport)` when the host is built, `__declspec(dllimport)` when project.dll consumes it via the rebuilt user-cache PCMs), and the host passes `-Wl,/EXPORT:CrafterBuildProject` when linking project.dll so user `project.cpp` files don't need platform-gated annotations.
|
||||
|
||||
## Compatibility / portability
|
||||
|
||||
- Linux: x86_64. Tested on Arch in a clean container.
|
||||
- Windows: x86_64 MSVC ABI. `build.cmd` produces `bin/crafter-build.{exe,dll,lib}`; userprojs verified end-to-end (`project.cpp` → `project.dll` loaded by the host → `userapp.exe` built and run).
|
||||
|
||||
The `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` env vars override the default `-march=native` / `-mtune=native` for distro CI builds:
|
||||
|
||||
```bash
|
||||
CRAFTER_BUILD_MARCH=x86-64-v3 CRAFTER_BUILD_MTUNE=generic ./build.sh
|
||||
```
|
||||
|
||||
## Roadmap
|
||||
|
||||
- [ ] Target-triple auto-detection (`clang -print-target-triple`)
|
||||
- [ ] Prefer system glslang when available, fall back to git+cmake
|
||||
- [ ] Order-preserving deduplication of `BuildResult.libs` (current `unordered_set` doesn't preserve link order — only matters for cyclic static libs)
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the GNU Lesser General Public License, version 3 only (LGPL-3.0-only). See [LICENSE](LICENSE) for the license text. The LGPL supplements the GNU General Public License version 3, a copy of which is included as [LICENSE.GPL](LICENSE.GPL).
|
||||
|
|
|
|||
Loading…
Reference in a new issue