Catcrafts/Crafter.Build
2
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

new tests
All checks were successful
CI / build-test-release (push) Successful in 1h4m52s
Details

Browse source
This commit is contained in:
Jorijn van der Graaf 2026-05-27 19:45:05 +02:00
commit 603840879d
11 changed files with 283 additions and 18235 deletions
Download patch file Download diff file Expand all files Collapse all files

80
examples/tests/README.md
View file

@ -1,6 +1,7 @@
# tests # tests
Two ways to write tests, in one project. Tests are declared in `project.cpp` via `cfg.AddTest(name)`. One line per
test; deps and other options chain off the returned builder.
```sh ```sh
cd examples/tests cd examples/tests
@ -11,20 +12,31 @@ Layout:
``` ```
mylib/MyMath.cppm # the library being tested mylib/MyMath.cppm # the library being tested
project.cpp # declares the library project.cpp # declares the library and its tests
tests/Smoke/main.cpp # zero-config test (no project.cpp) tests/Smoke/main.cpp # zero-config test
tests/UnitMyMath/main.cpp # test that links MyMath and exercises it tests/UnitMyMath/main.cpp # test that links MyMath
tests/UnitMyMath/project.cpp # required for tests with deps
``` ```
## Auto-discovery `project.cpp` does:
Each `tests/<Name>/` directory becomes a test. Three layers, escalate only as needed: ```cpp
cfg.AddTest("Smoke"); // exit 0 = pass
cfg.AddTest("UnitMyMath").Dependencies({ &cfg }); // imports MyMath
```
1. **`tests/<Name>/main.cpp`** with no `project.cpp` — discovery synthesizes a Configuration. Top-level `*.cpp` files become implementations, `interfaces/*.cppm` become module interfaces. `Smoke` is this case. `AddTest(name)` defaults the source to `tests/<name>/main.cpp` (resolved at
2. **`tests/<Name>/project.cpp`** — full control. Use this when you need defines, dependencies, or non-default targets. `UnitMyMath` is this case (it depends on `MyMath`). the project root) and the target/march/mtune to the parent project's. The
3. Folders starting with `_` or `.` are skipped (e.g. `tests/_shared/` for cross-test helpers). builder methods overlay overrides:
- `.Target(triple)` / `.March(...)` / `.Mtune(...)` / `.Sysroot(path)`
- `.Define(name, value)` / `.Debug()`
- `.Timeout(seconds)`
- `.Args(vec)` &mdash; runtime args
- `.Requires("tool:foo")` / `.Requires("file:/path")` / `.Requires("env:VAR")`
- `.Dependencies({ &otherCfg, ... })`
- `.Path(p)` &mdash; rebase the test's source-resolution path
- `.LinkFlag(s)` / `.CompileFlag(s)`
## Test conventions ## Test conventions
@ -35,20 +47,46 @@ Each `tests/<Name>/` directory becomes a test. Three layers, escalate only as ne
## Linking the parent project ## Linking the parent project
`UnitMyMath/project.cpp` shows how a test links the project's own library: `UnitMyMath` depends on the library via `.Dependencies({ &cfg })` &mdash; the
test imports `MyMath` and the build engine links `cfg`'s output into the
```cpp test exe.
cfg.dependencies = { ParentLib("MyMath") };
```
`ParentLib("name")` looks up a `Configuration*` in the parent project (the root project's own config + its dependency graph) by `Configuration::name`. The fixture's project.cpp can omit `cfg.path`, `cfg.name`, etc. — the discovery loop fills folder-derived defaults.
## Cross-target test runs ## Cross-target test runs
Tests parse `--target=...` from the project args you pass on the command line: `AddTest` inherits the parent project's target; for per-test cross-arch
runs, override via the builder:
```sh ```cpp
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine cfg.AddTest("CrossArchAarch64")
.Target("aarch64-linux-gnu")
.Sysroot("/opt/aarch64-rootfs")
.Requires("tool:qemu-aarch64");
``` ```
`--runner=<spec>` overrides the per-target runner for this invocation. Useful specs: `local`, `cmd:<command>` (prefix-exec, e.g. `cmd:wine`, `cmd:qemu-aarch64`), `ssh:<host>[:<remoteDir>]`, `sshwin:<host>[:<remoteDir>]`. Or persist via env var: `CRAFTER_BUILD_RUNNER_<normalized_target>=<spec>`. `crafter-build test` sweeps every distinct target declared across the
project's tests plus the host triple, so cross-arch tests run by default.
`--target=<triple>` restricts the run to that target.
`--runner=<spec>` overrides the per-target runner for one invocation.
Useful specs: `local`, `cmd:<command>` (e.g. `cmd:wine`, `cmd:qemu-aarch64`).
Persistent override via env: `CRAFTER_BUILD_RUNNER_<normalized_target>=<spec>`.
## SIMD march fan-out
For libraries with per-march SIMD codegen (e.g. Crafter.Math), use
`AddMarchVariants` to produce one Test per tier sharing the same source
and interface set:
```cpp
constexpr MarchTier tiers[] = {
{"sapphirerapids", "native"},
{"x86-64-v4", "generic"},
{"x86-64-v3", "generic"},
};
cfg.AddMarchVariants("Vector", libInterfaces, tiers);
```
Each tier becomes a Test named `Vector-<march>`, with the library's
interfaces rebuilt under that tier's `-march`/`-mtune` (so codegen actually
varies). Per-arch gating is just an `if (target.starts_with("x86_64"))`
guard around the call &mdash; no special toml syntax.

7
examples/tests/project.cpp
View file

@ -4,7 +4,8 @@ namespace fs = std::filesystem;
using namespace Crafter; using namespace Crafter;
// A pure library project: the root Configuration is the lib itself. Tests // A pure library project: the root Configuration is the lib itself. Tests
// under tests/ are auto-discovered (see tests/Smoke and tests/UnitMyMath). // are declared with cfg.AddTest — one line per test, with deps and other
// options applied via the returned builder.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) { extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg; Configuration cfg;
cfg.path = "./mylib/"; cfg.path = "./mylib/";
@ -16,5 +17,9 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>)
std::array<fs::path, 1> ifaces = { "MyMath" }; std::array<fs::path, 1> ifaces = { "MyMath" };
std::array<fs::path, 0> impls = {}; std::array<fs::path, 0> impls = {};
cfg.GetInterfacesAndImplementations(ifaces, impls); cfg.GetInterfacesAndImplementations(ifaces, impls);
cfg.AddTest("Smoke");
cfg.AddTest("UnitMyMath").Dependencies({ &cfg });
return cfg; return cfg;
} }

22
examples/tests/tests/UnitMyMath/project.cpp
View file

@ -1,22 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// Test that links the parent project's library so it can `import MyMath;`
// and call its functions. ParentLib(name) walks the parent project's
// dependency graph (and the parent itself) to find a Configuration by name.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "tests/UnitMyMath/";
cfg.name = "UnitMyMath";
cfg.outputName = "UnitMyMath";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { ParentLib("MyMath") };
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

9
implementations/Crafter.Build-Clang.cpp
View file

@ -1318,12 +1318,9 @@ Test options (after the `test` subcommand):
--runner=<spec> Override the test runner for this run. Specs: --runner=<spec> Override the test runner for this run. Specs:
local local
cmd:<command> (e.g. cmd:wine) cmd:<command> (e.g. cmd:wine)
ssh:<host>[:<remoteDir>] --target=<triple> Filter to tests whose cfg.target matches. Default:
sshwin:<host>[:<remoteDir>] sweep across every distinct target declared by the
wsl[:<remoteDir>] project's tests plus the host triple.
--target=<triple> Filter to tests whose cfg.target matches; this is
also forwarded to project-args so per-target tests
build for that triple. Default: host triple.
<glob> One or more name globs to filter tests (e.g. 'Unit*'). <glob> One or more name globs to filter tests (e.g. 'Unit*').
Project args: Project args:

385
implementations/Crafter.Build-Test.cpp
View file

@ -18,11 +18,6 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/ */
module; module;
// toml++ is consumed as a translation-unit-private dependency in the GMF: the
// parser is only needed for test.toml discovery here, so keeping it out of
// `import std`-using module purviews avoids dragging it through the module
// graph (and through every PCM consumer of Crafter.Build).
#include "../lib/toml.hpp"
export module Crafter.Build:Test_impl; export module Crafter.Build:Test_impl;
import std; import std;
import :Test; import :Test;
@ -387,74 +382,6 @@ TestResult Crafter::RunSingleTest(const Test& test, const fs::path& binary, std:
} }
namespace { namespace {
// Declarative test metadata loaded from tests/<Name>/test.toml. Lets a test
// ship just main.cpp + a few lines of config instead of a whole project.cpp
// when its needs are "pick a target, gate on prerequisites, run with these
// args". project.cpp stays the escape hatch for outer-driver tests that
// call Build() / inspect intermediate state.
struct TestManifest {
std::optional<std::string> target;
std::optional<std::string> march;
std::optional<std::string> mtune;
std::optional<std::string> sysroot;
std::vector<std::string> requires_;
std::optional<int> timeoutSeconds;
std::vector<std::string> args;
std::vector<std::pair<std::string, std::string>> defines;
};
TestManifest ParseTestManifest(const fs::path& path) {
// toml++ builds with exceptions enabled by default; parse_file throws
// toml::parse_error on malformed input. Rethrow with the path attached
// so the discovery loop's catch can surface "where the error came from"
// alongside toml++'s "what was wrong".
toml::table t;
try {
t = toml::parse_file(path.string());
} catch (const toml::parse_error& e) {
throw std::runtime_error(std::format(
"test.toml parse error in {}: {}",
path.string(),
std::string_view(e.description())));
}
TestManifest m;
if (auto v = t["target"].value<std::string>()) m.target = *v;
if (auto v = t["march"].value<std::string>()) m.march = *v;
if (auto v = t["mtune"].value<std::string>()) m.mtune = *v;
if (auto v = t["sysroot"].value<std::string>()) m.sysroot = *v;
if (auto v = t["timeout"].value<int64_t>()) m.timeoutSeconds = static_cast<int>(*v);
if (auto arr = t["requires"].as_array()) {
for (auto& el : *arr) {
if (auto s = el.value<std::string>()) m.requires_.push_back(*s);
}
}
if (auto arr = t["args"].as_array()) {
for (auto& el : *arr) {
if (auto s = el.value<std::string>()) m.args.push_back(*s);
}
}
if (auto tbl = t["defines"].as_table()) {
for (auto&& [k, v] : *tbl) {
if (auto s = v.value<std::string>()) {
m.defines.emplace_back(std::string(k.str()), *s);
}
}
}
return m;
}
// Apply manifest overlay onto a Configuration synthesized from the test
// folder. Target overrides come last so a manifest's `target = "..."`
// wins over the synth default (= run's targetFilter). Defines accumulate;
// they don't replace pre-existing ones.
void ApplyManifest(Configuration& cfg, const TestManifest& m) {
if (m.target) cfg.target = *m.target;
if (m.march) cfg.march = *m.march;
if (m.mtune) cfg.mtune = *m.mtune;
if (m.sysroot) cfg.sysroot = *m.sysroot;
for (auto& [k, v] : m.defines) cfg.defines.push_back({k, v});
}
bool ToolOnPath(std::string_view name) { bool ToolOnPath(std::string_view name) {
#ifdef _WIN32 #ifdef _WIN32
std::string cmd = std::format("where {} > nul 2>&1", name); std::string cmd = std::format("where {} > nul 2>&1", name);
@ -527,99 +454,103 @@ namespace {
} }
} }
namespace { TestBuilder Configuration::AddTest(std::string_view name) {
// Synthesize a Configuration for tests/<Name>/ folders that don't contain // Empty-interfaces trampoline. The span we hand to the (name, interfaces)
// a project.cpp. Convention: cfg.path = the folder, cfg.name/outputName = // overload wraps a stack array; it's only consulted inside that call, so
// folder basename, cfg.target = host (overridable via test.toml `target`), // its lifetime is fine for the duration.
// cfg.type = exe. Sources: top-level *.cpp (excluding project.cpp) become std::array<fs::path, 0> noIfaces = {};
// implementations, interfaces/*.cppm become module interfaces. Tests with return AddTest(name, noIfaces);
// deeper layouts or dependencies still need an explicit project.cpp. }
//
// Why host-default instead of targetFilter-default: under the multi-target
// sweep, an arch-agnostic test (no test.toml target) should run at the
// host iteration only — not get rebuilt against every cross-target the
// suite happens to declare. Cross-targeting is an opt-in via test.toml.
Configuration SynthesizeTest(const fs::path& dir) {
Configuration cfg;
cfg.path = dir;
cfg.name = dir.filename().string();
cfg.outputName = cfg.name;
cfg.target = HostTarget();
cfg.type = ConfigurationType::Executable;
std::vector<fs::path> impls; TestBuilder Configuration::AddTest(std::string_view name, std::span<fs::path> interfaces) {
for (auto& e : fs::directory_iterator(dir)) { Test t;
if (!e.is_regular_file()) continue; // Default to "./" so tests/<name>/main.cpp resolves at the project root
auto p = e.path(); // (where crafter-build sets cwd from project.cpp's location). Projects
if (p.extension() != ".cpp") continue; // whose lib lives in a subdir (e.g. cfg.path = "./mylib/") get this
if (p.filename() == "project.cpp") continue; // right by default; if the test layout is unusual, override via .Path().
impls.push_back(p.stem()); t.config.path = "./";
} t.config.name = std::string(name);
std::ranges::sort(impls); t.config.outputName = std::string(name);
t.config.target = this->target;
t.config.march = this->march;
t.config.mtune = this->mtune;
t.config.debug = this->debug;
t.config.type = ConfigurationType::Executable;
std::vector<fs::path> ifaces; // Default source layout: tests/<name>/main.cpp resolved against the
fs::path interfacesDir = dir / "interfaces"; // parent Configuration's path. cfg.path is typically "./" (project
if (fs::exists(interfacesDir) && fs::is_directory(interfacesDir)) { // root), which puts the test sources at <repo>/tests/<name>/main.cpp.
for (auto& e : fs::directory_iterator(interfacesDir)) { fs::path mainSource = fs::path("tests") / std::string(name) / "main";
if (!e.is_regular_file()) continue; std::array<fs::path, 1> impls = { mainSource };
auto p = e.path(); t.config.GetInterfacesAndImplementations(interfaces, impls);
if (p.extension() != ".cppm") continue;
ifaces.push_back(fs::path("interfaces") / p.stem());
}
std::ranges::sort(ifaces);
}
if (impls.empty() && ifaces.empty()) { tests.push_back(std::move(t));
throw std::runtime_error(std::format( return TestBuilder{this, tests.size() - 1};
"no .cpp or interfaces/*.cppm files found in {}", dir.string())); }
}
cfg.GetInterfacesAndImplementations(ifaces, impls); void Configuration::AddMarchVariants(std::string_view name,
return cfg; std::span<fs::path> interfaces,
std::span<const MarchTier> tiers) {
for (const auto& tier : tiers) {
Test t;
t.config.path = "./";
t.config.name = std::format("{}-{}", name, tier.march);
t.config.outputName = t.config.name;
t.config.target = this->target;
t.config.march = tier.march;
t.config.mtune = tier.mtune;
t.config.debug = this->debug;
t.config.type = ConfigurationType::Executable;
fs::path mainSource = fs::path("tests") / std::string(name) / "main";
std::array<fs::path, 1> impls = { mainSource };
// Interfaces stamped directly into each variant's Configuration —
// NOT via cfg.dependencies. The dependency machinery caches PCMs by
// dep->VariantId(), which would compile the parent's interfaces
// once with the parent's march; for SIMD-sensitive code each
// variant needs its own compile. Listing the interfaces as the
// test's own forces per-variant recompilation.
t.config.GetInterfacesAndImplementations(interfaces, impls);
tests.push_back(std::move(t));
} }
} }
TestBuilder& TestBuilder::Path(fs::path p) { test().config.path = std::move(p); return *this; }
TestBuilder& TestBuilder::Target(std::string t) { test().config.target = std::move(t); return *this; }
TestBuilder& TestBuilder::March(std::string m) { test().config.march = std::move(m); return *this; }
TestBuilder& TestBuilder::Mtune(std::string m) { test().config.mtune = std::move(m); return *this; }
TestBuilder& TestBuilder::Sysroot(fs::path s) { test().config.sysroot = s.string(); return *this; }
TestBuilder& TestBuilder::Debug(bool d) { test().config.debug = d; return *this; }
TestBuilder& TestBuilder::Define(std::string n, std::string v) {
test().config.defines.push_back({std::move(n), std::move(v)});
return *this;
}
TestBuilder& TestBuilder::Timeout(std::chrono::seconds s) { test().timeout = s; return *this; }
TestBuilder& TestBuilder::Args(std::vector<std::string> a) { test().args = std::move(a); return *this; }
TestBuilder& TestBuilder::Requires(std::string r) { test().requires_.push_back(std::move(r)); return *this; }
TestBuilder& TestBuilder::Dependencies(std::vector<Configuration*> d) {
test().config.dependencies = std::move(d);
return *this;
}
TestBuilder& TestBuilder::LinkFlag(std::string f) { test().config.linkFlags.push_back(std::move(f)); return *this; }
TestBuilder& TestBuilder::CompileFlag(std::string f) { test().config.compileFlags.push_back(std::move(f)); return *this; }
TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions& opts, std::span<const std::string_view> projectArgs) { TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions& opts, std::span<const std::string_view> projectArgs) {
// Multi-target sweep: when no --target= was given, the run covers every // Multi-target sweep: when no --target= was given, the run covers every
// distinct target a test.toml declares plus the host target. Lets a bare // distinct target declared across projectCfg.tests plus the host target.
// `crafter-build test` exercise cross-arch tests without the user having // Lets a bare `crafter-build test` exercise cross-arch tests without the
// to know which targets exist in this project. An explicit --target=X // user having to know which targets exist. An explicit --target=X
// bypasses the sweep and runs that target only. // bypasses the sweep and runs that target only.
if (opts.targetFilter.empty()) { if (opts.targetFilter.empty()) {
std::set<std::string> sweep; std::set<std::string> sweep;
sweep.insert(HostTarget()); sweep.insert(HostTarget());
fs::path testsDir = fs::current_path() / "tests"; for (const Test& t : projectCfg.tests) {
if (fs::exists(testsDir) && fs::is_directory(testsDir)) { if (!t.config.target.empty()) sweep.insert(t.config.target);
for (auto& e : fs::directory_iterator(testsDir)) {
if (!e.is_directory()) continue;
auto stem = e.path().filename().string();
if (stem.empty() || stem[0] == '_' || stem[0] == '.') continue;
fs::path tomlPath = e.path() / "test.toml";
if (!fs::exists(tomlPath)) continue;
try {
TestManifest m = ParseTestManifest(tomlPath);
if (m.target) sweep.insert(*m.target);
} catch (...) {
// Parse failures surface as discovery failures during the
// actual run; the sweep phase just collects targets.
}
}
} }
TestSummary aggregate; TestSummary aggregate;
// Inline tests pushed by the caller (fixture-driven inner RunTests
// calls, e.g. RunnerClassification) must survive each sweep
// iteration. Configuration isn't copyable so we can't snapshot+restore;
// instead, remember the inline count and erase only the entries
// appended by the previous iteration's auto-discovery.
size_t inlineCount = projectCfg.tests.size();
for (const auto& target : sweep) { for (const auto& target : sweep) {
RunTestsOptions perTarget = opts; RunTestsOptions perTarget = opts;
perTarget.targetFilter = target; perTarget.targetFilter = target;
if (projectCfg.tests.size() > inlineCount) {
projectCfg.tests.erase(
projectCfg.tests.begin() + inlineCount,
projectCfg.tests.end());
}
if (sweep.size() > 1) { if (sweep.size() > 1) {
Progress::Clear(); Progress::Clear();
std::println("\n=== target: {} ===", target); std::println("\n=== target: {} ===", target);
@ -636,152 +567,37 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
} }
TestSummary summary; TestSummary summary;
std::vector<TestResult> discoveryFailures;
// Auto-discover tests one layer deep: each <project>/tests/<Name>/ folder
// is a test. If it contains project.cpp, that's loaded for full control;
// otherwise the Configuration is synthesized from the folder contents.
// Folders whose name starts with '_' or '.' are skipped (so tests/_shared/
// holds cross-test code without becoming a test). Each project.cpp receives
// the same args the root project did, so --target=... propagates through.
//
// Discovery is keyed off cwd (= the project root, since crafter-build loads
// ./project.cpp), not projectCfg.path: tests live at the project root even
// when projectCfg.path points at a subdirectory like "./src/" or "./lib/".
fs::path testsDir = fs::current_path() / "tests";
if (fs::exists(testsDir) && fs::is_directory(testsDir)) {
// A discovered fixture is one of:
// - project.cpp present → outer-driver test (LoadProject)
// - test.toml present → declarative synth + manifest
// - neither, just *.cpp → bare synth (host-target)
// - both project.cpp and test.toml → XOR violation, discovery Fail
struct TestEntry {
fs::path dir;
fs::path pcpp; // outer-driver path
std::optional<TestManifest> manifest;
};
std::vector<TestEntry> entries;
for (auto& entry : fs::directory_iterator(testsDir)) {
if (!entry.is_directory()) continue;
auto stem = entry.path().filename().string();
if (stem.empty() || stem[0] == '_' || stem[0] == '.') continue;
TestEntry te;
te.dir = entry.path();
auto pcpp = te.dir / "project.cpp";
auto tomlPath = te.dir / "test.toml";
bool hasPcpp = fs::exists(pcpp);
bool hasToml = fs::exists(tomlPath);
if (hasPcpp && hasToml) {
TestResult r;
r.name = stem;
r.outcome = TestOutcome::Fail;
r.exitCode = -1;
r.output = "both project.cpp and test.toml present — they're "
"mutually exclusive (delete one to disambiguate "
"outer-driver vs declarative test)";
discoveryFailures.push_back(std::move(r));
continue;
}
if (hasPcpp) te.pcpp = pcpp;
if (hasToml) {
try {
te.manifest = ParseTestManifest(tomlPath);
} catch (const std::exception& e) {
TestResult r;
r.name = stem;
r.outcome = TestOutcome::Fail;
r.exitCode = -1;
r.output = e.what();
discoveryFailures.push_back(std::move(r));
continue;
}
}
entries.push_back(std::move(te));
}
std::ranges::sort(entries, [](auto& a, auto& b) { return a.dir < b.dir; });
// Inject --target=<filter> into the args we hand each fixture so its
// CrafterBuildProject can default to the run's target. The CLI's own
// --target=... propagates through projectArgs already; this only
// appends when missing so an explicit user choice wins.
std::string targetArg = std::format("--target={}", opts.targetFilter);
std::vector<std::string_view> fixtureArgs(projectArgs.begin(), projectArgs.end());
bool hasTarget = std::ranges::any_of(fixtureArgs, [](std::string_view a) {
return a.starts_with("--target=");
});
if (!hasTarget) fixtureArgs.push_back(targetArg);
for (auto& te : entries) {
Test t;
try {
if (!te.pcpp.empty()) {
t.config = LoadProject(te.pcpp, fixtureArgs);
} else {
t.config = SynthesizeTest(te.dir);
if (te.manifest) {
ApplyManifest(t.config, *te.manifest);
if (te.manifest->timeoutSeconds) {
t.timeout = std::chrono::seconds(*te.manifest->timeoutSeconds);
}
t.args = te.manifest->args;
t.requires_ = te.manifest->requires_;
}
}
} catch (const std::exception& e) {
// A broken fixture shouldn't kill the whole run. Surface as a
// Fail and let other tests proceed.
TestResult r;
r.name = te.dir.filename().string();
r.outcome = TestOutcome::Fail;
r.exitCode = -1;
r.output = !te.pcpp.empty()
? std::format("project.cpp failed to load: {}", e.what())
: std::format("test discovery failed: {}", e.what());
discoveryFailures.push_back(std::move(r));
continue;
}
if (t.config.target != opts.targetFilter) continue;
t.runner = TestRunner::FromEnv(t.config.target, TestRunner::ForTarget(t.config));
if (opts.runnerOverride) {
if (auto r = TestRunner::FromSpec(*opts.runnerOverride)) {
t.runner = std::move(*r);
}
}
projectCfg.tests.push_back(std::move(t));
}
}
// Filter by target + glob, derive each Test's runner just-in-time. The
// runner is recomputed each sweep iteration (rather than once at AddTest
// time) because runnerOverride and FromEnv resolution depend on the
// run's options, which the project.cpp doesn't know about.
std::vector<Test*> filtered; std::vector<Test*> filtered;
filtered.reserve(projectCfg.tests.size()); filtered.reserve(projectCfg.tests.size());
for (auto& test : projectCfg.tests) { for (auto& test : projectCfg.tests) {
if (MatchAny(opts.globs, test.config.name)) { if (test.config.target != opts.targetFilter) continue;
filtered.push_back(&test); if (!MatchAny(opts.globs, test.config.name)) continue;
test.runner = TestRunner::FromEnv(test.config.target, TestRunner::ForTarget(test.config));
if (opts.runnerOverride) {
if (auto r = TestRunner::FromSpec(*opts.runnerOverride)) {
test.runner = std::move(*r);
}
} }
filtered.push_back(&test);
} }
if (opts.listOnly) { if (opts.listOnly) {
for (auto& r : discoveryFailures) {
std::println("{} (project.cpp broken)", r.name);
}
for (auto* t : filtered) { for (auto* t : filtered) {
std::println("{}", t->config.name); std::println("{}", t->config.name);
} }
return summary; return summary;
} }
if (filtered.empty() && discoveryFailures.empty()) { if (filtered.empty()) {
std::println("No tests matched."); std::println("No tests matched.");
return summary; return summary;
} }
// Render discovery failures upfront so they appear before parallel test
// results. They're already-determined Fails — no need to put them through
// the worker pool.
for (auto& r : discoveryFailures) {
PrintResult(r, "");
WriteLog(projectCfg.path, r.name, r.output);
}
int jobs = opts.jobs > 0 int jobs = opts.jobs > 0
? opts.jobs ? opts.jobs
: std::max(1u, std::thread::hardware_concurrency()); : std::max(1u, std::thread::hardware_concurrency());
@ -816,12 +632,11 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
TestResult r; TestResult r;
r.name = t.config.name; r.name = t.config.name;
// Declarative preconditions (test.toml requires = [...] or Test.requires_ // Declarative preconditions set via TestBuilder::Requires. Evaluated
// set in project.cpp). Evaluated before the build so a missing tool/file/env // before the build so a missing tool/file/env turns into a Skip without
// turns into a Skip without paying the compile cost. Reports the first // paying the compile cost. Reports the first failure only — once one
// failure only — once one precondition is unmet the test couldn't run // precondition is unmet the test couldn't run anyway, and a wall of
// anyway, and a wall of "also missing X, also missing Y" buries the // "also missing X, also missing Y" buries the actionable root cause.
// actionable root cause.
if (auto req = EvaluateRequires(t.requires_); !req.ok) { if (auto req = EvaluateRequires(t.requires_); !req.ok) {
r.outcome = TestOutcome::Skipped; r.outcome = TestOutcome::Skipped;
r.output = req.reason; r.output = req.reason;
@ -846,7 +661,7 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
r.exitCode = -1; r.exitCode = -1;
r.output = std::format( r.output = std::format(
"runner '{}' unavailable and not declared in requires " "runner '{}' unavailable and not declared in requires "
"(add 'tool:{}' to test.toml requires to permit skipping)", "(add .Requires(\"tool:{}\") to permit skipping)",
t.runner.name, tool); t.runner.name, tool);
} else { } else {
r.outcome = TestOutcome::Skipped; r.outcome = TestOutcome::Skipped;
@ -916,10 +731,6 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
} }
threads.clear(); // joins all jthreads threads.clear(); // joins all jthreads
for (auto& r : discoveryFailures) {
results.push_back(std::move(r));
}
for (auto& r : results) { for (auto& r : results) {
switch (r.outcome) { switch (r.outcome) {
case TestOutcome::Pass: summary.passed++; break; case TestOutcome::Pass: summary.passed++; break;

18
interfaces/Crafter.Build-Clang.cppm
View file

@ -159,6 +159,24 @@ export namespace Crafter {
std::vector<std::string> linkFlags; std::vector<std::string> linkFlags;
std::vector<Test> tests; std::vector<Test> tests;
CRAFTER_API void GetInterfacesAndImplementations(std::span<fs::path> interfaces, std::span<fs::path> implementations); CRAFTER_API void GetInterfacesAndImplementations(std::span<fs::path> interfaces, std::span<fs::path> implementations);
// Declare a test. Sources default to `tests/<name>/main.cpp` resolved
// against this Configuration's path; target/march/mtune/debug are
// inherited from this Configuration so cross-arch projects don't have
// to re-specify. Returns a builder for chaining defines, deps, etc.
// Defined in Crafter.Build:Test.
CRAFTER_API struct TestBuilder AddTest(std::string_view name);
// Same as AddTest, but compiles the parent's `interfaces` directly
// into this test's Configuration (rather than going through a dep).
// Use when the test must rebuild those interfaces with its own
// compile flags — typically per-march SIMD codegen.
CRAFTER_API struct TestBuilder AddTest(std::string_view name, std::span<fs::path> interfaces);
// Math-style fan-out: one Test per MarchTier, all sharing the same
// `tests/<name>/main.cpp` source and the same interface set, each
// compiled with the tier's `-march`/`-mtune`. Test names are
// `<name>-<march>`.
CRAFTER_API void AddMarchVariants(std::string_view name,
std::span<fs::path> interfaces,
std::span<const struct MarchTier> tiers);
// Suffix that uniquely identifies this Configuration's compile state. // Suffix that uniquely identifies this Configuration's compile state.
// target+march+mtune are spelled out for readability; the rest // target+march+mtune are spelled out for readability; the rest
// (type, debug, sysroot, defines, compileFlags) collapse into a short // (type, debug, sysroot, defines, compileFlags) collapse into a short

49
interfaces/Crafter.Build-Test.cppm
View file

@ -24,6 +24,55 @@ import std;
import :Clang; import :Clang;
export namespace Crafter { export namespace Crafter {
// One row in a SIMD-march fan-out: a single Test variant compiles the
// shared source with `-march=<march> -mtune=<mtune>`. Passed by span to
// Configuration::AddMarchVariants — see Crafter.Math/project.cpp.
struct MarchTier {
std::string march;
std::string mtune;
};
// Fluent helper returned by Configuration::AddTest. Holds a back-pointer
// to the parent Configuration and the index of the just-pushed Test, so
// mutations survive any vector reallocation that subsequent AddTest
// calls might trigger. Each setter returns `*this` to support chaining.
//
// Note: source files (interfaces + implementations) are fixed at
// AddTest-time — pass them via the AddTest overload that takes
// interfaces. The builder only mutates compile state, deps, runtime
// args, and requires; it deliberately does NOT expose Sources() to
// avoid the double-`GetInterfacesAndImplementations` footgun (calling
// it twice would re-parse the same .cppm files into duplicate Module
// entries).
struct CRAFTER_API TestBuilder {
Configuration* parent;
std::size_t index;
Test& test() const { return parent->tests[index]; }
// Override the path the test's sources resolve against. Defaults to
// "./" (project root, where tests/<name>/main.cpp lives). Override
// only when the test source layout sits under a subdir.
TestBuilder& Path(std::filesystem::path p);
TestBuilder& Target(std::string t);
TestBuilder& March(std::string m);
TestBuilder& Mtune(std::string m);
TestBuilder& Sysroot(std::filesystem::path s);
TestBuilder& Debug(bool d = true);
TestBuilder& Define(std::string name, std::string value = "");
TestBuilder& Timeout(std::chrono::seconds s);
// Replaces any previously-set args.
TestBuilder& Args(std::vector<std::string> a);
// Appends one require entry ("tool:foo" / "file:..." / "env:VAR").
TestBuilder& Requires(std::string r);
// Replaces the dependency list. Pointers must outlive the build.
TestBuilder& Dependencies(std::vector<Configuration*> deps);
// Append a single linker flag (-l..., -L..., etc.).
TestBuilder& LinkFlag(std::string f);
// Append a single compile flag.
TestBuilder& CompileFlag(std::string f);
};
struct RunTestsOptions { struct RunTestsOptions {
std::vector<std::string> globs; std::vector<std::string> globs;
int jobs = 0; int jobs = 0;

17899
lib/toml.hpp
View file

File diff suppressed because it is too large Load diff

10
project.cpp
View file

@ -90,5 +90,15 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
crafterBuildLib->linkFlags.push_back("-lws2_32"); crafterBuildLib->linkFlags.push_back("-lws2_32");
} }
// Self-tests link the local crafter-build library and exercise it in
// process. The harness (whichever crafter-build invokes `test`) compiles
// these against the *installed* share/crafter-build .cppm files, then
// links each test exe against crafterBuildLib built from the local
// sources — so the code under test is whatever's in this checkout.
// Mirrors how downstream consumers link their own libraries into tests.
if (cfg.target == "x86_64-pc-linux-gnu") {
cfg.AddTest("HelloWorld").Dependencies({ crafterBuildLib.get() });
}
return cfg; return cfg;
} }

2
tests/HelloWorld/fixture/main.cpp Normal file
View file

@ -0,0 +1,2 @@
import std;
int main() { return 0; }

39
tests/HelloWorld/main.cpp Normal file
View file

@ -0,0 +1,39 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// Smoke test for the local Crafter.Build library: construct a Configuration
// for the fixture/ subdir, call Build() in-process, and assert the binary
// was produced. The build engine being exercised here is whatever this
// test exe was statically linked against — which, via the parent
// project.cpp's .Dependencies({ crafterBuildLib.get() }), is the local
// checkout's library form. Edits to implementations/*.cpp show up here on
// the next test run with no install step required.
int main() {
Configuration cfg;
cfg.path = fs::current_path() / "tests" / "HelloWorld" / "fixture";
cfg.name = "hello";
cfg.outputName = "hello";
cfg.target = HostTarget();
cfg.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
BuildResult r = Build(cfg, depResults, depMutex);
if (!r.result.empty()) {
std::println(std::cerr, "build failed: {}", r.result);
return 1;
}
fs::path bin = cfg.BinDir() / "hello";
if (!fs::exists(bin)) {
std::println(std::cerr, "binary not produced at {}", bin.string());
return 1;
}
return 0;
}