V2: WASI, -r flag, CI pipeline, examples & tests cleanup
Some checks failed
CI / build-test-release (pull_request) Failing after 44s
Some checks failed
CI / build-test-release (pull_request) Failing after 44s
WASI / wasm32 target support
- Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32")
- Skip -march/-mtune for wasm (clang rejects them)
- Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj
-D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync)
- .wasm output extension in expectedOutputFor and link command
- EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html +
runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL
set in the templated index.html so a single shim handles any output name
-r run flag in the CLI: build then exec the artifact (host targets only;
rejects libraries; auto .exe/.wasm extension handling)
CI pipeline (.forgejo/workflows/ci.yaml)
- Triggers: PR/push to master + manual dispatch
- Single arch-latest container job: install deps, bootstrap, self-rebuild,
run tests, cross-compile mingw, package both archives, upload artifacts
- Rolling 'latest' release published only on push/dispatch to master
mingw cross-compile from Linux now works end-to-end:
- ExternalDependency cache key includes target so per-target glslang builds
don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends
'd' to lib names and breaks linking); cross-compile cmake flags
(CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...)
- project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic
and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh
link pulls pthread which mingw doesn't link by default)
- mingw compile uses -femulated-tls so std::__once_callable etc reference
the same emutls symbols libstdc++ provides
- mingw link auto-adds -lstdc++exp -lpthread
GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux
+ Windows) now both use it instead of duplicating the resolution.
Examples reorg: hello-world, library, with-module, wasi, tests — each with
its own README. Tests reorg: per-test directory with inner/ fixture, no
shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
cdfdb976c8
commit
eaee502e8c
102 changed files with 2211 additions and 686 deletions
165
README.md
165
README.md
|
|
@ -5,7 +5,7 @@ A C++26 modules build system. Project descriptions are written in C++ — no JSO
|
|||
## Status
|
||||
|
||||
- **Linux**: working end-to-end, packageable as a distro package. Verified on Arch Linux via [PKGBUILD](PKGBUILD) in a fresh container.
|
||||
- **Windows**: working end-to-end. Bootstrap (`build.cmd`) produces `crafter-build.exe` + `crafter-build.dll` + `crafter-build.lib`; user `project.cpp` links against the import lib via the host-controlled DLL boundary.
|
||||
- **Windows**: working end-to-end, including the test runner. Bootstrap (`build.cmd`) produces `crafter-build.exe` + `crafter-build.dll` + `crafter-build.lib`; user `project.cpp` links against the import lib via the host-controlled DLL boundary. `crafter-build test` runs natively on Windows hosts; cross-OS runs back to Linux via WSL2 (`--runner=wsl`).
|
||||
|
||||
## Quick start (Linux)
|
||||
|
||||
|
|
@ -50,7 +50,7 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
|
|||
}
|
||||
```
|
||||
|
||||
Run `crafter-build` in that directory; outputs land at `bin/myapp-<target>-<march>/myapp` and intermediates at `build/myapp-<target>-<march>/`.
|
||||
Run `crafter-build` in that directory; outputs land at `bin/myapp-<target>-<march>/myapp` and intermediates at `build/myapp-<target>-<march>/`. Add `-r` to run the produced executable straight after a successful build (host targets only — for cross targets use `crafter-build test --runner=...`).
|
||||
|
||||
## Dependencies
|
||||
|
||||
|
|
@ -100,115 +100,136 @@ Diamond deps (`A → {B, C}; B → X; C → X`) build `X` exactly once via a `st
|
|||
|
||||
## Tests
|
||||
|
||||
Tests are normal executables — any `int main()`, exit code `0` = pass, nonzero = fail, captured stdout+stderr is the failure message. There's no Crafter-specific entry-point convention, so a GoogleTest/Catch2/doctest binary works as-is.
|
||||
Tests live under `tests/<Name>/`. The simplest case is a single C++ file:
|
||||
|
||||
Declare tests by pushing onto `cfg.tests` in your `project.cpp`:
|
||||
|
||||
```cpp
|
||||
Test t;
|
||||
t.config.path = "./";
|
||||
t.config.name = "VectorMath";
|
||||
t.config.outputName = "VectorMath";
|
||||
t.config.target = "x86_64-pc-linux-gnu";
|
||||
t.config.type = ConfigurationType::Executable;
|
||||
std::array<fs::path, 0> ifaces = {};
|
||||
std::array<fs::path, 1> impls = { "tests/VectorMath" };
|
||||
t.config.GetInterfacesAndImplementations(ifaces, impls);
|
||||
cfg.tests.push_back(std::move(t));
|
||||
```
|
||||
tests/Smoke/main.cpp # int main() — exit 0 = pass, nonzero = fail, 77 = skipped
|
||||
```
|
||||
|
||||
Each test runs in its own subprocess so a segfault doesn't take the runner down. Default timeout is 60 s per test (`--timeout=N` overrides). Tests run in parallel up to `hardware_concurrency()`; pass `--jobs=N` to override. Per-test stdout+stderr is buffered and printed atomically on completion so parallel output stays readable, and is also written to `build/test-logs/<name>.log` when a test fails, crashes, or times out.
|
||||
Run them all:
|
||||
|
||||
```bash
|
||||
crafter-build test # build + run all tests
|
||||
crafter-build test 'Vector*' # glob-filter test names
|
||||
crafter-build test --list # enumerate without running
|
||||
crafter-build test --jobs=1 # serial mode
|
||||
crafter-build test --timeout=5 # override per-test timeout
|
||||
crafter-build test
|
||||
```
|
||||
|
||||
Outcomes are reported as ✅ pass, ❌ fail, 💥 crash (with the signal name), or ⏱ timeout. The runner exits 0 only if every test passed.
|
||||
Auto-discovery walks `tests/` one level deep. Three escalation layers:
|
||||
|
||||
1. **`tests/<Name>/main.cpp`** with no `project.cpp` — a Configuration is synthesized: top-level `*.cpp` files become implementations, `interfaces/*.cppm` become module interfaces, target = the run's `--target=` (host triple by default). Most tests need nothing more.
|
||||
2. **`tests/<Name>/project.cpp`** — full control. Use this for tests with defines, dependencies, or a non-default target.
|
||||
3. Folders starting with `_` or `.` are skipped, so `tests/_shared/` can hold cross-test code without becoming a test itself.
|
||||
|
||||
Test conventions:
|
||||
|
||||
- **Exit codes**: `0` = ✅ pass, nonzero = ❌ fail, **`77`** = ⏭ skipped (autoconf convention; use it for runtime opt-outs like "tool not on PATH").
|
||||
- Each test runs in its own subprocess — a segfault doesn't take the runner down.
|
||||
- Default timeout is 60 s per test; runs in parallel up to `hardware_concurrency()`.
|
||||
- Stdout+stderr is captured and printed atomically on completion. Failed/crashed/timed-out tests also write to `build/test-logs/<name>.log`.
|
||||
- A broken fixture `project.cpp` is reported as a single failed test, not a fatal error — other tests still run.
|
||||
|
||||
CLI:
|
||||
|
||||
```bash
|
||||
crafter-build test # run all
|
||||
crafter-build test 'Unit*' # glob-filter test names
|
||||
crafter-build test --list # enumerate without running
|
||||
crafter-build test --target=x86_64-w64-mingw32 # only tests for this target
|
||||
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine
|
||||
crafter-build test --jobs=1 # serial mode
|
||||
crafter-build test --timeout=5 # override per-test timeout
|
||||
```
|
||||
|
||||
Outcomes: ✅ pass, ❌ fail, 💥 crash (with signal name), ⏱ timeout, ⏭ skipped. The runner exits 0 only if every non-skipped test passed.
|
||||
|
||||
`--target=` filters the run to tests whose `cfg.target` matches; default is the host triple. Per-target tests parse `--target=...` from their own args span and build for it; tests with hardcoded targets (e.g. host-only meta-tests) are filtered out under non-matching `--target=`.
|
||||
|
||||
### Cross-target test runners
|
||||
|
||||
`Test::runner` runs the test binary somewhere other than this machine. Three factories ship today (Linux host only):
|
||||
`Test::runner` runs the test binary somewhere other than this machine. Four factories:
|
||||
|
||||
```cpp
|
||||
t.runner = TestRunner::Local(); // default
|
||||
t.runner = TestRunner::QemuUser("qemu-aarch64"); // cross-arch via qemu user-mode
|
||||
t.runner = TestRunner::Ssh("winvm", "/tmp/crafter-test"); // cross-OS via scp + ssh
|
||||
TestRunner::Local() // run on this machine (default)
|
||||
TestRunner::Cmd("wine") // local exec via prefix command
|
||||
TestRunner::Ssh("hostname") // remote linux via scp + ssh
|
||||
TestRunner::SshWin("winhost") // remote windows via scp + ssh-cmd.exe
|
||||
TestRunner::Wsl() // Linux binaries via WSL2 (Windows host)
|
||||
```
|
||||
|
||||
`TestRunner::Ssh(host, remoteDir)` ships the test's whole `bin/<name>-<target>-<march>/` output dir via `scp -r`, runs the binary over `ssh`, then cleans up the remote temp dir. Each test invocation gets a unique remote subdir to keep parallel runs isolated. Assumes Unix-like shell on the remote (`mkdir -p`, `rm -rf`); for Windows remotes (e.g. `cmd.exe`) construct `TestRunner` directly with your own command templates — `copy`, `exec`, `cleanup` are public string fields with `{bundle}`, `{remote_bundle}`, `{bin}`, `{bin_name}`, `{args}` placeholders.
|
||||
`Cmd(cmd)` is a generic `<cmd> <bin> <args>` prefix runner — covers wine, qemu-user, valgrind, gdbserver, anything with that shape. Probes via `which <cmd>` (or `where` on Windows hosts). The `Ssh` variants `scp -r` the test's whole bin dir, run the binary remotely, then clean up; each invocation uses a unique remote subdir so parallel runs don't collide. `Wsl()` mirrors the SSH model but copies the bundle into WSL's native filesystem (faster than executing in-place from `/mnt/c`) — useful when a Windows host wants to exercise Linux test binaries.
|
||||
|
||||
`TestRunner::QemuUser(qemuBin)` simply wraps `<qemuBin> {bin} {args}` — same machine, just a command prefix. Pair it with `t.config.target = "aarch64-linux-gnu"` (plus an aarch64 sysroot installed on the host) for cross-arch Linux testing.
|
||||
For elaborate setups (rsync, ssh ControlMaster, env vars on the spawn, `QEMU_LD_PREFIX` for sysroots), construct `TestRunner` directly — `copy`/`exec`/`cleanup`/`probe` are public string fields with `{bundle}`, `{remote_bundle}`, `{bin}`, `{bin_name}`, `{args}` placeholders.
|
||||
|
||||
For cross-arch builds, set `cfg.sysroot` on the `Configuration` to point at a sysroot for the target. `BuildStdPcm` and the compile/link commands both consume it: `--sysroot=<path>` is passed to clang, and `<sysroot>/usr/share/libc++/v1/std.cppm` is precompiled instead of the host's.
|
||||
For cross-arch builds, set `cfg.sysroot` on the `Configuration` to point at a sysroot for the target. `BuildStdPcm` and the compile/link commands both consume it: `--sysroot=<path>` is passed to clang, and `<sysroot>/usr/share/libc++/v1/std.cppm` is precompiled instead of the host's. The cheapest aarch64 sysroot is an extracted [Arch Linux ARM rootfs](http://os.archlinuxarm.org/) plus `libc++` + `libc++abi` + `gcc` installed into it via `pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy …` (point pacman at ALARM's mirrors via explicit config to avoid pulling host-arch packages). See `tests/CrossArchAarch64/inner/` for a working sysroot+qemu example.
|
||||
|
||||
For aarch64 specifically, the cheapest sysroot is an extracted [Arch Linux ARM rootfs](http://os.archlinuxarm.org/) plus `libc++` + `libc++abi` + `gcc` installed into it via `pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy …` (point pacman at ALARM's mirrors with an explicit config to avoid pulling host-arch packages). Then `cfg.sysroot = "<rootfs>";` plus a manual `TestRunner` whose exec template prepends `QEMU_LD_PREFIX=<rootfs> qemu-aarch64 …` so qemu-user finds `ld-linux-aarch64.so.1`. See `tests/fixtures/cross-arch-aarch64/` for a working example.
|
||||
For `--target=x86_64-w64-mingw32` (Windows from Linux), V2's pipeline cross-compiles as long as `mingw-w64` is installed. Pair it with `--runner=cmd:wine` (no VM needed) or `--runner=sshwin:winhost` (real Windows). Full MSVC ABI (`x86_64-pc-windows-msvc`) cross-compile from Linux is not yet wired; use MinGW for now. `-fuse-ld=lld` is auto-injected on every link, and `-lstdc++exp` is auto-linked for the mingw target so user projects don't need to declare those flags.
|
||||
|
||||
For `t.config.target = "x86_64-w64-mingw32"` (Windows from Linux) — V2's existing build pipeline cross-compiles on Linux as long as `mingw-w64` is installed. Combine with `TestRunner::Ssh("winvm", ...)` for end-to-end Windows-target testing from a Linux host. Full MSVC ABI (`x86_64-pc-windows-msvc`) cross-compile from Linux is not yet wired; use MinGW for now.
|
||||
### Configuring runners
|
||||
|
||||
#### Global config: `TestRunner::FromEnv`
|
||||
Two layers, CLI wins over env, both override the per-test default of `Local()`:
|
||||
|
||||
To avoid hardcoding hosts in every project's `project.cpp`, use `TestRunner::FromEnv(target)`:
|
||||
|
||||
```cpp
|
||||
t.runner = TestRunner::FromEnv(t.config.target);
|
||||
```
|
||||
|
||||
This reads the env var `CRAFTER_BUILD_RUNNER_<NORMALIZED_TARGET>` (target triple with `-` and `.` replaced by `_`). Set it once per shell or in CI:
|
||||
**Env var** (persistent, per shell or in CI):
|
||||
|
||||
```bash
|
||||
export CRAFTER_BUILD_RUNNER_x86_64_w64_mingw32=ssh:winvm:/tmp/crafter-test
|
||||
export CRAFTER_BUILD_RUNNER_aarch64_linux_gnu=qemu:qemu-aarch64
|
||||
export CRAFTER_BUILD_RUNNER_x86_64_w64_mingw32=cmd:wine
|
||||
export CRAFTER_BUILD_RUNNER_aarch64_linux_gnu=cmd:qemu-aarch64
|
||||
crafter-build test --target=x86_64-w64-mingw32 # uses cmd:wine
|
||||
```
|
||||
|
||||
Spec grammar: `local`, `qemu:<qemu-bin>`, `ssh:<host>`, `ssh:<host>:<remoteDir>`, `sshwin:<host>`, `sshwin:<host>:<remoteDir>` (cmd.exe-shell variant for Windows remotes). Unset → falls back to the second arg of `FromEnv` (defaults to `Local()`).
|
||||
The variable name is `CRAFTER_BUILD_RUNNER_<NORMALIZED_TARGET>` (target triple with `-` and `.` replaced by `_`).
|
||||
|
||||
For anything more elaborate (rsync, ssh ControlMaster, custom flags, env vars on the spawn, `QEMU_LD_PREFIX` for sysroots) — construct `TestRunner` manually with your own templates. The factories are sugar over the same fields.
|
||||
**CLI** (per invocation, scoped to `--target=`):
|
||||
|
||||
Not yet supported: WSL2 / Windows-host execution (the runner currently requires a Linux host because `RunCommandWithTimeout` is Linux-only), MSVC ABI cross-compile from Linux.
|
||||
```bash
|
||||
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine
|
||||
```
|
||||
|
||||
#### Skipping unreachable runners
|
||||
Spec grammar:
|
||||
|
||||
`TestRunner` carries a `probe` command that runs once per `RunTests` invocation (cached by `runner.name`); if it exits non-zero, every Test using that runner reports as `⏭ Skipped: runner '<name>' not available` and the suite continues with everything else. The factories set sensible defaults: `Ssh`/`SshWin` probe via `ssh -q -o BatchMode=yes -o ConnectTimeout=5 <host> …`, `QemuUser` via `which <qemu-bin>`, `Local` is unconditionally available.
|
||||
```
|
||||
local
|
||||
cmd:<command> # cmd:wine, cmd:qemu-aarch64, cmd:valgrind, ...
|
||||
ssh:<host>[:<remoteDir>]
|
||||
sshwin:<host>[:<remoteDir>] # cmd.exe-shell variant for Windows remotes
|
||||
wsl[:<remoteDir>] # Windows host -> Linux binaries via WSL2
|
||||
```
|
||||
|
||||
Effect: if you don't have a Windows VM at all, you don't have to do anything — the Windows variants in the in-tree suite skip cleanly and the Linux variants pass. If you do have one, set up an SSH host alias matching what the project.cpp expects (in this repo: `winvm` in `~/.ssh/config`) and the Windows variants run automatically. Skipped tests count toward the summary but do not fail the suite (`AllPassed()` returns true on skip-only runs).
|
||||
Each runner carries a `probe` command (e.g. `which wine`, `ssh -BatchMode=yes <host> true`); if it fails, every test using that runner is reported as ⏭ skipped and the rest of the suite continues. Skipped tests count toward the summary but don't fail the suite (`AllPassed()` returns true on skip-only runs).
|
||||
|
||||
#### Per-(target, runner) tests
|
||||
### Linking the parent project's library
|
||||
|
||||
The 5 simple tests `HelloWorld`, `WithModule`, `Defines`, `CrossProjectModule`, `Diamond` are declared **directly** in the project's `project.cpp`, one entry per `(target, runner)` pair:
|
||||
A test fixture can depend on the project's own library so it can `import` and exercise its API:
|
||||
|
||||
```cpp
|
||||
std::vector<TargetRunner> targets = {
|
||||
{ "x86_64-pc-linux-gnu", TestRunner::Local(), {} },
|
||||
{ "x86_64-w64-mingw32", TestRunner::SshWin("winvm", "C:/temp/crafter-tests"), {"-lstdc++exp"} },
|
||||
};
|
||||
// tests/UnitFoo/project.cpp
|
||||
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
|
||||
Configuration cfg;
|
||||
cfg.path = "tests/UnitFoo/";
|
||||
cfg.name = "UnitFoo";
|
||||
cfg.outputName = "UnitFoo";
|
||||
cfg.target = "x86_64-pc-linux-gnu";
|
||||
cfg.type = ConfigurationType::Executable;
|
||||
cfg.dependencies = { ParentLib("MyLib") }; // <-- the magic line
|
||||
|
||||
std::array<fs::path, 0> ifaces = {};
|
||||
std::array<fs::path, 1> impls = { "main" };
|
||||
cfg.GetInterfacesAndImplementations(ifaces, impls);
|
||||
return cfg;
|
||||
}
|
||||
```
|
||||
|
||||
So `./bin/crafter-build test` (no env vars) spawns one Test per `(target, runner)` pair, all in one report. With the `winvm` SSH alias reachable:
|
||||
`ParentLib(name)` walks the parent project's dependency graph (and the parent itself) to find a `Configuration*` by `Configuration::name`. The build links it into the test exe, so `import MyLib;` resolves naturally. See [examples/tests/](examples/tests/) for the complete worked example.
|
||||
|
||||
```
|
||||
✅ HelloWorld (5ms)
|
||||
✅ HelloWorld (sshwin:winvm) (2348ms)
|
||||
✅ WithModule (3ms)
|
||||
✅ WithModule (sshwin:winvm) (2204ms)
|
||||
…
|
||||
18 passed
|
||||
```
|
||||
## Examples
|
||||
|
||||
Without `winvm` reachable, the SshWin variants skip:
|
||||
[examples/](examples/) contains progressively larger self-contained projects:
|
||||
|
||||
```
|
||||
✅ HelloWorld (5ms)
|
||||
⏭ HelloWorld (sshwin:winvm) skipped: runner 'sshwin:winvm' not available
|
||||
…
|
||||
```
|
||||
| Example | Shows |
|
||||
|---|---|
|
||||
| [hello-world](examples/hello-world/) | Smallest possible project: `project.cpp` + `main.cpp` |
|
||||
| [with-module](examples/with-module/) | Adding a C++ module interface |
|
||||
| [library](examples/library/) | Library + executable, linked via `cfg.dependencies` |
|
||||
| [tests](examples/tests/) | Auto-discovered tests + tests linking the parent library |
|
||||
|
||||
Local runner emits no parenthetical (existing default for the common case); cross-target runners show their name (e.g. `sshwin:winvm`, `qemu:qemu-aarch64`) so you see at a glance which environment ran each test.
|
||||
|
||||
The 8 outer-driver tests (Incremental, BuildError, RunnerClassification, QemuUser, SshRunner, WindowsViaSsh, CrossArchAarch64, Libraries) stay one-line in the report because they're meta-tests — the test driver is the test, validating host-side build behavior or runner integration. They're not multi-target.
|
||||
Each builds standalone: `cd examples/<name> && crafter-build`.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue