Compare commits

..

3 commits

Author SHA1 Message Date
8de93aaf06 test: drop transport runners (ssh/sshwin/wsl) and the Shell-quoting enum
All checks were successful
CI / build-test-release (pull_request) Successful in 9m56s
With test.toml + ForTarget covering the cross-arch + Windows-on-Linux
cases, the env-var-driven transport runners are dead weight. This commit
removes them and the retired tests that exercised the env-var plumbing:

  - TestRunner::Ssh / SshWin / Wsl factories and their copy/exec/cleanup
    template machinery.
  - TestRunner::Shell enum (Host/Sh/Cmd) and the ShellQuoteSh helper —
    only Host shell quoting is needed once the remote shells are gone.
  - TestRunner::copy / cleanup / remoteDir / argsShell fields.
  - WindowsPathToWsl and the {remote_bundle}/{bin_win}/{bundle_wsl}
    placeholder substitution in RunSingleTest's transport branch.
  - ParseRunnerSpec narrowed from {local, cmd, ssh, sshwin, wsl} to
    {local, cmd} — the override hatch is preserved, just simpler.
  - tests/SshRunner, tests/WindowsViaSsh, tests/QemuUser: these tested
    the CRAFTER_BUILD_RUNNER_<target> → runner plumbing that has been
    replaced by ForTarget. The runner derivation is exercised every
    time CrossArchAarch64 / Wasi / WindowsViaWine runs.
  - tests/UnitLib: ssh/sshwin spec assertions become "throws on bogus
    spec" assertions.

Refs issue #8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 18:07:33 +02:00
406a406420 test: migrate cross-arch tests to declarative test.toml; add WindowsViaWine
Collapses three outer-driver tests into declarative top-level fixtures:

  - CrossArchAarch64: outer + inner pair becomes main.cpp + test.toml. The
    'is this really an ARM aarch64 ELF?' artifact-introspection check is
    dropped — qemu-aarch64 refuses to run wrong-arch ELFs anyway, so a
    silent host-arch fallback would still fail the run.
  - Wasi: outer + inner pair becomes main.cpp + test.toml. The WASM
    magic-byte check is dropped on the same logic (wasmtime refuses
    non-WASM input).
  - Defines: simple defines-propagation smoke test becomes test.toml with
    [defines] CRAFTER_TEST_FOO = "42".

Adds WindowsViaWine to replace the (forthcoming-deletion) WindowsViaSsh:
declarative target=x86_64-w64-mingw32 + requires=[tool:wine,
tool:x86_64-w64-mingw32-g++]. Exercises the new Wine runner and the
ForTarget derivation end-to-end.

Diamond and CrossProjectModule had speculative --target= reading that
made them attempt cross-compilation under the multi-target sweep. They
have no sysroot/toolchain plumbing, so those cross-builds always fail.
Hardcoded them to HostTarget(); cross-arch tests live in their own
test.toml from now on.

Refs issue #8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 18:01:49 +02:00
dc27c5c204 test: introduce test.toml + target-derived runners alongside existing machinery
Vendors toml++ v3.4.0 as lib/toml.hpp and wires it into Crafter.Build-Test
to parse a declarative test.toml manifest (target/march/mtune/sysroot/
requires/timeout/args/defines). Test discovery now treats project.cpp and
test.toml as mutually exclusive: project.cpp stays the escape hatch for
outer-driver tests, test.toml gives downstream test authors a no-boilerplate
path.

Adds:
- TestRunner::Wine() and TestRunner::ForTarget(cfg) — runner is now derived
  from cfg.target (Local for host, Wine for Windows-on-Linux, wasmtime for
  WASI, qemu-<arch> with QEMU_LD_PREFIX for non-host Linux). The env-var
  override CRAFTER_BUILD_RUNNER_<target> still wins as a power-user escape
  hatch via FromEnv.
- Declarative preconditions: tool:<name>, file:<path>, env:<VAR> are
  evaluated before the build; missing preconditions Skip without paying
  the compile cost.
- Hard-fail-unless-declared: when a derived runner's tool is missing AND
  the test didn't declare 'tool:<that>' in requires, the missing runner
  is a Fail instead of a silent Skip. Surfaces broken cross-arch CI
  config that previously hid as "skipped".
- Multi-target sweep: bare `crafter-build test` (no --target=) now
  iterates every distinct test.toml-declared target plus the host, so
  cross-arch tests run by default without the user needing to know which
  targets exist. `--target=X` bypasses the sweep.

Test struct gains a `requires_` vector so project.cpp users can declare
preconditions too (matching what test.toml writes there).

Existing tests, factories (Ssh/SshWin/Wsl/Cmd), and CRAFTER_BUILD_RUNNER_*
machinery remain intact — this commit only adds; migration and deletion
follow in subsequent commits.

Refs issue #8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 17:57:47 +02:00
33 changed files with 18370 additions and 775 deletions

View file

@ -17,6 +17,12 @@ License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/ */
module;
// toml++ is consumed as a translation-unit-private dependency in the GMF: the
// parser is only needed for test.toml discovery here, so keeping it out of
// `import std`-using module purviews avoids dragging it through the module
// graph (and through every PCM consumer of Crafter.Build).
#include "../lib/toml.hpp"
export module Crafter.Build:Test_impl; export module Crafter.Build:Test_impl;
import std; import std;
import :Test; import :Test;
@ -103,15 +109,11 @@ namespace {
#endif #endif
} }
std::string JoinAndQuoteArgs(std::span<const std::string> args, TestRunner::Shell shell) { std::string JoinAndQuoteArgs(std::span<const std::string> args) {
std::string out; std::string out;
for (const auto& a : args) { for (const auto& a : args) {
if (!out.empty()) out.push_back(' '); if (!out.empty()) out.push_back(' ');
switch (shell) { out += ShellQuoteHost(a);
case TestRunner::Shell::Sh: out += ShellQuoteSh(a); break;
case TestRunner::Shell::Cmd: out += ShellQuoteCmd(a); break;
case TestRunner::Shell::Host: out += ShellQuoteHost(a); break;
}
} }
return out; return out;
} }
@ -235,69 +237,12 @@ Configuration* Crafter::ParentLib(std::string_view name) {
TestRunner TestRunner::Local() { TestRunner TestRunner::Local() {
TestRunner r; TestRunner r;
r.name = "local"; r.name = "local";
r.argsShell = Shell::Host;
return r;
}
TestRunner TestRunner::Ssh(std::string host, std::string remoteDir) {
TestRunner r;
r.name = std::format("ssh:{}", host);
r.remoteDir = std::move(remoteDir);
r.argsShell = Shell::Sh;
// Outer "..." (not '...') so the wrapper survives both sh (Linux host)
// and cmd (Windows host). cmd doesn't honor single quotes, which would
// cause it to split on the inner '&&'. Args inside use POSIX quoting
// because they reach a remote bash regardless of which host issued them.
r.copy = std::format("ssh -q {0} \"mkdir -p {{remote_bundle}}\" && scp -r -q {{bundle}}/. {0}:{{remote_bundle}}/", host);
r.exec = std::format("ssh -q {} \"cd {{remote_bundle}} && ./{{bin_name}} {{args}}\"", host);
r.cleanup = std::format("ssh -q {} \"rm -rf {{remote_bundle}}\"", host);
// RunCommandChecked captures stdout+stderr internally — no shell redirect
// needed in the probe spec, which means it works the same way under cmd
// (Windows host) and sh (Linux host) since `> /dev/null` doesn't translate.
r.probe = std::format("ssh -q -o BatchMode=yes -o ConnectTimeout=5 {} true", host);
return r;
}
TestRunner TestRunner::SshWin(std::string host, std::string remoteDir) {
TestRunner r;
r.name = std::format("sshwin:{}", host);
r.remoteDir = std::move(remoteDir);
r.argsShell = Shell::Cmd;
// cmd.exe-friendly templates. Use backslash variants of remote paths because
// cmd's mkdir won't auto-create intermediate directories with forward
// slashes. scp tolerates either, so we keep forward slashes for the scp
// dest (which `{remote_bundle}` provides). 2>nul + `& rem ok` swallows
// mkdir's "already exists" error and forces exit code 0.
r.copy = std::format("ssh -q {0} \"mkdir {{remote_bundle_win}} 2>nul & rem ok\" && scp -r -q {{bundle}}/. {0}:{{remote_bundle}}/", host);
r.exec = std::format("ssh -q {} \"{{bin_win}} {{args}}\"", host);
r.cleanup = std::format("ssh -q {} \"rd /s /q {{remote_bundle_win}} 2>nul & rem ok\"", host);
// No shell redirect in the probe — see TestRunner::Ssh for rationale.
r.probe = std::format("ssh -q -o BatchMode=yes -o ConnectTimeout=5 {} \"ver\"", host);
return r;
}
TestRunner TestRunner::Wsl(std::string remoteDir) {
TestRunner r;
r.name = "wsl";
r.argsShell = Shell::Sh;
r.remoteDir = std::move(remoteDir);
// Transport runner: stage the test bundle into WSL's native filesystem
// (faster than executing in-place from /mnt/c) then run via bash. {bundle_wsl}
// is the local Windows path translated to /mnt/<drive>/... form so wsl cp
// can read it. Args are POSIX-quoted because they reach a Linux shell.
r.copy = "wsl mkdir -p {remote_bundle} && wsl cp -r {bundle_wsl}/. {remote_bundle}/";
r.exec = "wsl bash -c \"cd {remote_bundle} && ./{bin_name} {args}\"";
r.cleanup = "wsl rm -rf {remote_bundle}";
// `where wsl` matches the host-shell convention; on a non-Windows host
// it fails (no `where` command), which correctly skips this runner.
r.probe = "where wsl";
return r; return r;
} }
TestRunner TestRunner::Cmd(std::string command) { TestRunner TestRunner::Cmd(std::string command) {
TestRunner r; TestRunner r;
r.name = std::format("cmd:{}", command); r.name = std::format("cmd:{}", command);
r.argsShell = Shell::Host;
r.exec = std::format("{} {{bin}} {{args}}", command); r.exec = std::format("{} {{bin}} {{args}}", command);
#ifdef _WIN32 #ifdef _WIN32
r.probe = std::format("where {}", command); r.probe = std::format("where {}", command);
@ -307,6 +252,68 @@ TestRunner TestRunner::Cmd(std::string command) {
return r; return r;
} }
TestRunner TestRunner::Wine() {
TestRunner r;
r.name = "wine";
r.exec = "wine {bin} {args}";
#ifdef _WIN32
r.probe = "where wine";
#else
r.probe = "which wine";
#endif
return r;
}
TestRunner TestRunner::ForTarget(const Configuration& cfg) {
const std::string& target = cfg.target;
// Same triple as the host → run the binary directly. Covers the common
// case (cfg.target defaulted to HostTarget()) without any wrapper.
if (target == HostTarget()) return Local();
// Windows targets: native on a Windows host, Wine on Linux. We don't
// distinguish mingw vs msvc here — the produced .exe runs the same way.
if (TargetIsWindows(target)) {
return TargetIsWindows(HostTarget()) ? Local() : Wine();
}
// WASI: a .wasm file isn't directly executable; wasmtime is the canonical
// runtime. wasi-cli also works but the upstream Bytecode Alliance name is
// wasmtime, so we standardize on that.
if (target.starts_with("wasm32-wasi") || target.starts_with("wasm64-wasi")) {
return Cmd("wasmtime");
}
// Non-host Linux triple: extract the architecture and route through
// qemu-user. Triple is <arch>-<vendor>-<os>-<env> (or sometimes 3 parts);
// qemu-user's binary names mostly follow the arch field, with two known
// mismatches handled below. cfg.sysroot, when set, becomes QEMU_LD_PREFIX
// so the target's dynamic linker / shared libs are reachable — without
// it qemu-user crashes on dynamic ELFs with "could not open /lib/ld...".
if (target.find("-linux-") != std::string::npos) {
auto dash = target.find('-');
std::string arch = target.substr(0, dash);
// i686-linux-gnu → qemu-i386; arm-* already matches qemu-arm; aarch64,
// riscv64, ppc64le, mips, mips64, s390x all match their qemu names.
if (arch == "i686") arch = "i386";
TestRunner r = Cmd(std::format("qemu-{}", arch));
if (!cfg.sysroot.empty()) {
// Use `env VAR=value cmd` rather than the shell's `VAR=value cmd`
// prefix syntax: RunCommandWithTimeout pipes through GNU `timeout`,
// which execvp's its argument list directly without going through a
// shell. A bare VAR=value would be exec'd as a command path and
// fail with "No such file or directory".
r.exec = std::format("env QEMU_LD_PREFIX={} {}", cfg.sysroot, r.exec);
}
return r;
}
// Unknown / bare-metal / freestanding targets: fall back to Local. The
// caller's runner-availability probe (or absence of the binary) surfaces
// the problem rather than us inventing a wrong wrapper here.
return Local();
}
namespace { namespace {
std::string NormalizeTriple(std::string_view target) { std::string NormalizeTriple(std::string_view target) {
std::string out(target); std::string out(target);
@ -316,56 +323,18 @@ namespace {
return out; return out;
} }
// Spec grammar: "local" | "cmd:<binary>". Used by CRAFTER_BUILD_RUNNER_*
// env vars and --runner=. The set used to include ssh/sshwin/wsl variants;
// those were removed when transport-style runners were dropped (issue #8).
std::optional<TestRunner> ParseRunnerSpec(std::string_view spec) { std::optional<TestRunner> ParseRunnerSpec(std::string_view spec) {
if (spec.empty()) return std::nullopt; if (spec.empty()) return std::nullopt;
std::vector<std::string> parts; if (spec == "local") return TestRunner::Local();
for (auto piece : std::views::split(spec, ':')) { if (spec.starts_with("cmd:") && spec.size() > 4) {
parts.emplace_back(std::string_view(piece.begin(), piece.end())); return TestRunner::Cmd(std::string(spec.substr(4)));
}
if (parts.empty()) return std::nullopt;
if (parts[0] == "local") return TestRunner::Local();
if (parts[0] == "cmd" && parts.size() == 2) return TestRunner::Cmd(parts[1]);
if (parts[0] == "ssh" && parts.size() == 2) return TestRunner::Ssh(parts[1]);
if (parts[0] == "ssh" && parts.size() >= 3) {
std::string remote;
for (std::size_t i = 2; i < parts.size(); ++i) {
if (i > 2) remote.push_back(':');
remote += parts[i];
}
return TestRunner::Ssh(parts[1], remote);
}
if (parts[0] == "sshwin" && parts.size() == 2) return TestRunner::SshWin(parts[1]);
if (parts[0] == "sshwin" && parts.size() >= 3) {
std::string remote;
for (std::size_t i = 2; i < parts.size(); ++i) {
if (i > 2) remote.push_back(':');
remote += parts[i];
}
return TestRunner::SshWin(parts[1], remote);
}
if (parts[0] == "wsl" && parts.size() == 1) return TestRunner::Wsl();
if (parts[0] == "wsl" && parts.size() >= 2) {
std::string remote;
for (std::size_t i = 1; i < parts.size(); ++i) {
if (i > 1) remote.push_back(':');
remote += parts[i];
}
return TestRunner::Wsl(remote);
} }
throw std::runtime_error(std::format( throw std::runtime_error(std::format(
"TestRunner::FromSpec: unrecognized runner spec '{}'", spec)); "TestRunner::FromSpec: unrecognized runner spec '{}' "
} "(expected 'local' or 'cmd:<binary>')", spec));
// C:\Users\jorij -> /mnt/c/Users/jorij. Idempotent on paths that don't
// start with a drive letter, so callers can compute it unconditionally.
std::string WindowsPathToWsl(std::string_view p) {
if (p.size() < 2 || p[1] != ':') return std::string(p);
char drive = static_cast<char>(std::tolower(static_cast<unsigned char>(p[0])));
std::string out = std::format("/mnt/{}", drive);
for (std::size_t i = 2; i < p.size(); ++i) {
out.push_back(p[i] == '\\' ? '/' : p[i]);
}
return out;
} }
} }
@ -382,60 +351,24 @@ TestRunner TestRunner::FromEnv(std::string_view target, TestRunner fallback) {
} }
TestResult Crafter::RunSingleTest(const Test& test, const fs::path& binary, std::chrono::seconds timeout) { TestResult Crafter::RunSingleTest(const Test& test, const fs::path& binary, std::chrono::seconds timeout) {
using namespace std::chrono_literals;
TestResult result; TestResult result;
result.name = test.config.name; result.name = test.config.name;
std::map<std::string, std::string> ph; std::map<std::string, std::string> ph;
ph["{args}"] = JoinAndQuoteArgs(test.args, test.runner.argsShell); ph["{args}"] = JoinAndQuoteArgs(test.args);
ph["{bin_name}"] = binary.filename().string();
ph["{bundle}"] = binary.parent_path().string();
auto start = std::chrono::steady_clock::now(); auto start = std::chrono::steady_clock::now();
CommandResult r; CommandResult r;
if (test.runner.exec.empty()) { if (test.runner.exec.empty()) {
// Pure-local runner: spawn the binary directly through the host shell. // Local runner: spawn the binary directly through the host shell.
std::string cmd = std::format("{} {}", ShellQuoteHost(binary.string()), ph["{args}"]); std::string cmd = std::format("{} {}", ShellQuoteHost(binary.string()), ph["{args}"]);
r = RunCommandWithTimeout(cmd, timeout); r = RunCommandWithTimeout(cmd, timeout);
} else if (test.runner.copy.empty()) { } else {
// Prefix runner (qemu-user, wsl, ...): templated exec wraps a local binary. // Prefix runner (qemu-user, wasmtime, wine, ...): templated exec
// wraps the local binary.
ph["{bin}"] = binary.string(); ph["{bin}"] = binary.string();
r = RunCommandWithTimeout(Substitute(test.runner.exec, ph), timeout); r = RunCommandWithTimeout(Substitute(test.runner.exec, ph), timeout);
} else {
// Transport runner (ssh, ...): copy bundle → exec remotely → cleanup.
std::string unique = std::format("{}-{}-{}",
test.config.name,
std::hash<std::thread::id>{}(std::this_thread::get_id()),
std::chrono::steady_clock::now().time_since_epoch().count());
ph["{remote_bundle}"] = std::format("{}/{}", test.runner.remoteDir, unique);
ph["{bin}"] = std::format("{}/{}", ph["{remote_bundle}"], ph["{bin_name}"]);
// Backslash variants for cmd.exe-shell remotes (cmd's mkdir won't
// auto-create intermediate directories when path uses forward slashes).
ph["{remote_bundle_win}"] = ph["{remote_bundle}"];
std::replace(ph["{remote_bundle_win}"].begin(), ph["{remote_bundle_win}"].end(), '/', '\\');
ph["{bin_win}"] = ph["{bin}"];
std::replace(ph["{bin_win}"].begin(), ph["{bin_win}"].end(), '/', '\\');
// WSL form of the local bundle path: C:\foo -> /mnt/c/foo. Used by
// the Wsl runner's `wsl cp -r` step to read the test bundle out of
// the Windows host's filesystem.
ph["{bundle_wsl}"] = WindowsPathToWsl(ph["{bundle}"]);
CommandResult cp = RunCommandWithTimeout(Substitute(test.runner.copy, ph), 5min);
if (cp.exitCode != 0) {
auto end = std::chrono::steady_clock::now();
result.duration = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
result.outcome = TestOutcome::Fail;
result.exitCode = cp.exitCode;
result.output = std::format("copy step failed (exit {}):\n{}", cp.exitCode, cp.output);
return result;
}
r = RunCommandWithTimeout(Substitute(test.runner.exec, ph), timeout);
if (!test.runner.cleanup.empty()) {
std::system(Substitute(test.runner.cleanup, ph).c_str());
}
} }
auto end = std::chrono::steady_clock::now(); auto end = std::chrono::steady_clock::now();
@ -453,20 +386,165 @@ TestResult Crafter::RunSingleTest(const Test& test, const fs::path& binary, std:
return result; return result;
} }
namespace {
// Declarative test metadata loaded from tests/<Name>/test.toml. Lets a test
// ship just main.cpp + a few lines of config instead of a whole project.cpp
// when its needs are "pick a target, gate on prerequisites, run with these
// args". project.cpp stays the escape hatch for outer-driver tests that
// call Build() / inspect intermediate state.
struct TestManifest {
std::optional<std::string> target;
std::optional<std::string> march;
std::optional<std::string> mtune;
std::optional<std::string> sysroot;
std::vector<std::string> requires_;
std::optional<int> timeoutSeconds;
std::vector<std::string> args;
std::vector<std::pair<std::string, std::string>> defines;
};
TestManifest ParseTestManifest(const fs::path& path) {
// toml++ builds with exceptions enabled by default; parse_file throws
// toml::parse_error on malformed input. Rethrow with the path attached
// so the discovery loop's catch can surface "where the error came from"
// alongside toml++'s "what was wrong".
toml::table t;
try {
t = toml::parse_file(path.string());
} catch (const toml::parse_error& e) {
throw std::runtime_error(std::format(
"test.toml parse error in {}: {}",
path.string(),
std::string_view(e.description())));
}
TestManifest m;
if (auto v = t["target"].value<std::string>()) m.target = *v;
if (auto v = t["march"].value<std::string>()) m.march = *v;
if (auto v = t["mtune"].value<std::string>()) m.mtune = *v;
if (auto v = t["sysroot"].value<std::string>()) m.sysroot = *v;
if (auto v = t["timeout"].value<int64_t>()) m.timeoutSeconds = static_cast<int>(*v);
if (auto arr = t["requires"].as_array()) {
for (auto& el : *arr) {
if (auto s = el.value<std::string>()) m.requires_.push_back(*s);
}
}
if (auto arr = t["args"].as_array()) {
for (auto& el : *arr) {
if (auto s = el.value<std::string>()) m.args.push_back(*s);
}
}
if (auto tbl = t["defines"].as_table()) {
for (auto&& [k, v] : *tbl) {
if (auto s = v.value<std::string>()) {
m.defines.emplace_back(std::string(k.str()), *s);
}
}
}
return m;
}
// Apply manifest overlay onto a Configuration synthesized from the test
// folder. Target overrides come last so a manifest's `target = "..."`
// wins over the synth default (= run's targetFilter). Defines accumulate;
// they don't replace pre-existing ones.
void ApplyManifest(Configuration& cfg, const TestManifest& m) {
if (m.target) cfg.target = *m.target;
if (m.march) cfg.march = *m.march;
if (m.mtune) cfg.mtune = *m.mtune;
if (m.sysroot) cfg.sysroot = *m.sysroot;
for (auto& [k, v] : m.defines) cfg.defines.push_back({k, v});
}
bool ToolOnPath(std::string_view name) {
#ifdef _WIN32
std::string cmd = std::format("where {} > nul 2>&1", name);
#else
std::string cmd = std::format("which {} > /dev/null 2>&1", name);
#endif
return std::system(cmd.c_str()) == 0;
}
struct RequireResult {
bool ok;
std::string reason; // human-readable when !ok
};
// Evaluate each `<kind>:<arg>` precondition. Returns the first failure
// (short-circuit; reporting one missing dep at a time is enough to act on
// and keeps the test log uncluttered).
RequireResult EvaluateRequires(std::span<const std::string> reqs) {
for (const auto& r : reqs) {
auto sep = r.find(':');
if (sep == std::string::npos || sep == 0 || sep == r.size() - 1) {
return {false, std::format("malformed require '{}' (expected kind:arg)", r)};
}
std::string_view kind(r.data(), sep);
std::string_view arg(r.data() + sep + 1, r.size() - sep - 1);
if (kind == "tool") {
if (!ToolOnPath(arg)) {
return {false, std::format("tool '{}' not on PATH", arg)};
}
} else if (kind == "file") {
if (!fs::exists(std::string(arg))) {
return {false, std::format("file '{}' missing", arg)};
}
} else if (kind == "env") {
const char* v = std::getenv(std::string(arg).c_str());
if (!v || !*v) {
return {false, std::format("env '{}' unset", arg)};
}
} else {
return {false, std::format(
"unknown require kind '{}' (expected tool/file/env)", kind)};
}
}
return {true, ""};
}
// Match a runner's tool dependency against the test's declared
// requirements. Used to decide between Skip (declared, may legitimately
// be missing) and Fail (runner unavailable but test didn't declare it —
// a silent skip would mask broken cross-arch CI configuration).
bool RequiresMentionsTool(std::span<const std::string> reqs, std::string_view tool) {
std::string needle = std::format("tool:{}", tool);
return std::ranges::any_of(reqs, [&](const std::string& s) { return s == needle; });
}
// Best-effort extraction of the runner-tool name from a TestRunner so the
// hard-fail-unless-declared check can match it against `requires`. For
// Cmd("foo"), the name is "cmd:foo"; for Wine, it's "wine". Anything else
// (Local, transport runners) returns empty — those don't trigger the
// declared/undeclared gate.
std::string RunnerToolName(const TestRunner& runner) {
if (runner.name == "wine") return "wine";
if (runner.name.starts_with("cmd:")) {
std::string tool = runner.name.substr(4);
// QEMU_LD_PREFIX prefix may be glued onto exec but the runner's
// `name` field already isolates the command, so no extra parsing.
return tool;
}
return "";
}
}
namespace { namespace {
// Synthesize a Configuration for tests/<Name>/ folders that don't contain // Synthesize a Configuration for tests/<Name>/ folders that don't contain
// a project.cpp. Convention: cfg.path = the folder, cfg.name/outputName = // a project.cpp. Convention: cfg.path = the folder, cfg.name/outputName =
// folder basename, cfg.target = the run's targetFilter, cfg.type = exe. // folder basename, cfg.target = host (overridable via test.toml `target`),
// Sources: top-level *.cpp (excluding project.cpp) become implementations, // cfg.type = exe. Sources: top-level *.cpp (excluding project.cpp) become
// interfaces/*.cppm become module interfaces (matching the layout used // implementations, interfaces/*.cppm become module interfaces. Tests with
// elsewhere in this codebase). Tests with deeper layouts, defines, or // deeper layouts or dependencies still need an explicit project.cpp.
// dependencies still need an explicit project.cpp. //
Configuration SynthesizeTest(const fs::path& dir, std::string_view target) { // Why host-default instead of targetFilter-default: under the multi-target
// sweep, an arch-agnostic test (no test.toml target) should run at the
// host iteration only — not get rebuilt against every cross-target the
// suite happens to declare. Cross-targeting is an opt-in via test.toml.
Configuration SynthesizeTest(const fs::path& dir) {
Configuration cfg; Configuration cfg;
cfg.path = dir; cfg.path = dir;
cfg.name = dir.filename().string(); cfg.name = dir.filename().string();
cfg.outputName = cfg.name; cfg.outputName = cfg.name;
cfg.target = std::string(target); cfg.target = HostTarget();
cfg.type = ConfigurationType::Executable; cfg.type = ConfigurationType::Executable;
std::vector<fs::path> impls; std::vector<fs::path> impls;
@ -502,6 +580,61 @@ namespace {
} }
TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions& opts, std::span<const std::string_view> projectArgs) { TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions& opts, std::span<const std::string_view> projectArgs) {
// Multi-target sweep: when no --target= was given, the run covers every
// distinct target a test.toml declares plus the host target. Lets a bare
// `crafter-build test` exercise cross-arch tests without the user having
// to know which targets exist in this project. An explicit --target=X
// bypasses the sweep and runs that target only.
if (opts.targetFilter.empty()) {
std::set<std::string> sweep;
sweep.insert(HostTarget());
fs::path testsDir = fs::current_path() / "tests";
if (fs::exists(testsDir) && fs::is_directory(testsDir)) {
for (auto& e : fs::directory_iterator(testsDir)) {
if (!e.is_directory()) continue;
auto stem = e.path().filename().string();
if (stem.empty() || stem[0] == '_' || stem[0] == '.') continue;
fs::path tomlPath = e.path() / "test.toml";
if (!fs::exists(tomlPath)) continue;
try {
TestManifest m = ParseTestManifest(tomlPath);
if (m.target) sweep.insert(*m.target);
} catch (...) {
// Parse failures surface as discovery failures during the
// actual run; the sweep phase just collects targets.
}
}
}
TestSummary aggregate;
// Inline tests pushed by the caller (fixture-driven inner RunTests
// calls, e.g. RunnerClassification) must survive each sweep
// iteration. Configuration isn't copyable so we can't snapshot+restore;
// instead, remember the inline count and erase only the entries
// appended by the previous iteration's auto-discovery.
size_t inlineCount = projectCfg.tests.size();
for (const auto& target : sweep) {
RunTestsOptions perTarget = opts;
perTarget.targetFilter = target;
if (projectCfg.tests.size() > inlineCount) {
projectCfg.tests.erase(
projectCfg.tests.begin() + inlineCount,
projectCfg.tests.end());
}
if (sweep.size() > 1) {
Progress::Clear();
std::println("\n=== target: {} ===", target);
}
TestSummary s = RunTests(projectCfg, perTarget, projectArgs);
aggregate.passed += s.passed;
aggregate.failed += s.failed;
aggregate.crashed += s.crashed;
aggregate.timedOut += s.timedOut;
aggregate.skipped += s.skipped;
for (auto& r : s.results) aggregate.results.push_back(std::move(r));
}
return aggregate;
}
TestSummary summary; TestSummary summary;
std::vector<TestResult> discoveryFailures; std::vector<TestResult> discoveryFailures;
@ -517,7 +650,16 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
// when projectCfg.path points at a subdirectory like "./src/" or "./lib/". // when projectCfg.path points at a subdirectory like "./src/" or "./lib/".
fs::path testsDir = fs::current_path() / "tests"; fs::path testsDir = fs::current_path() / "tests";
if (fs::exists(testsDir) && fs::is_directory(testsDir)) { if (fs::exists(testsDir) && fs::is_directory(testsDir)) {
struct TestEntry { fs::path dir; fs::path pcpp; }; // pcpp empty = synth // A discovered fixture is one of:
// - project.cpp present → outer-driver test (LoadProject)
// - test.toml present → declarative synth + manifest
// - neither, just *.cpp → bare synth (host-target)
// - both project.cpp and test.toml → XOR violation, discovery Fail
struct TestEntry {
fs::path dir;
fs::path pcpp; // outer-driver path
std::optional<TestManifest> manifest;
};
std::vector<TestEntry> entries; std::vector<TestEntry> entries;
for (auto& entry : fs::directory_iterator(testsDir)) { for (auto& entry : fs::directory_iterator(testsDir)) {
if (!entry.is_directory()) continue; if (!entry.is_directory()) continue;
@ -526,7 +668,34 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
TestEntry te; TestEntry te;
te.dir = entry.path(); te.dir = entry.path();
auto pcpp = te.dir / "project.cpp"; auto pcpp = te.dir / "project.cpp";
if (fs::exists(pcpp)) te.pcpp = pcpp; auto tomlPath = te.dir / "test.toml";
bool hasPcpp = fs::exists(pcpp);
bool hasToml = fs::exists(tomlPath);
if (hasPcpp && hasToml) {
TestResult r;
r.name = stem;
r.outcome = TestOutcome::Fail;
r.exitCode = -1;
r.output = "both project.cpp and test.toml present — they're "
"mutually exclusive (delete one to disambiguate "
"outer-driver vs declarative test)";
discoveryFailures.push_back(std::move(r));
continue;
}
if (hasPcpp) te.pcpp = pcpp;
if (hasToml) {
try {
te.manifest = ParseTestManifest(tomlPath);
} catch (const std::exception& e) {
TestResult r;
r.name = stem;
r.outcome = TestOutcome::Fail;
r.exitCode = -1;
r.output = e.what();
discoveryFailures.push_back(std::move(r));
continue;
}
}
entries.push_back(std::move(te)); entries.push_back(std::move(te));
} }
std::ranges::sort(entries, [](auto& a, auto& b) { return a.dir < b.dir; }); std::ranges::sort(entries, [](auto& a, auto& b) { return a.dir < b.dir; });
@ -548,7 +717,15 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
if (!te.pcpp.empty()) { if (!te.pcpp.empty()) {
t.config = LoadProject(te.pcpp, fixtureArgs); t.config = LoadProject(te.pcpp, fixtureArgs);
} else { } else {
t.config = SynthesizeTest(te.dir, opts.targetFilter); t.config = SynthesizeTest(te.dir);
if (te.manifest) {
ApplyManifest(t.config, *te.manifest);
if (te.manifest->timeoutSeconds) {
t.timeout = std::chrono::seconds(*te.manifest->timeoutSeconds);
}
t.args = te.manifest->args;
t.requires_ = te.manifest->requires_;
}
} }
} catch (const std::exception& e) { } catch (const std::exception& e) {
// A broken fixture shouldn't kill the whole run. Surface as a // A broken fixture shouldn't kill the whole run. Surface as a
@ -564,7 +741,7 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
continue; continue;
} }
if (t.config.target != opts.targetFilter) continue; if (t.config.target != opts.targetFilter) continue;
t.runner = TestRunner::FromEnv(t.config.target, TestRunner::Local()); t.runner = TestRunner::FromEnv(t.config.target, TestRunner::ForTarget(t.config));
if (opts.runnerOverride) { if (opts.runnerOverride) {
if (auto r = TestRunner::FromSpec(*opts.runnerOverride)) { if (auto r = TestRunner::FromSpec(*opts.runnerOverride)) {
t.runner = std::move(*r); t.runner = std::move(*r);
@ -639,9 +816,42 @@ TestSummary Crafter::RunTests(Configuration& projectCfg, const RunTestsOptions&
TestResult r; TestResult r;
r.name = t.config.name; r.name = t.config.name;
if (!runnerAvailable(t.runner)) { // Declarative preconditions (test.toml requires = [...] or Test.requires_
// set in project.cpp). Evaluated before the build so a missing tool/file/env
// turns into a Skip without paying the compile cost. Reports the first
// failure only — once one precondition is unmet the test couldn't run
// anyway, and a wall of "also missing X, also missing Y" buries the
// actionable root cause.
if (auto req = EvaluateRequires(t.requires_); !req.ok) {
r.outcome = TestOutcome::Skipped; r.outcome = TestOutcome::Skipped;
r.output = std::format("runner '{}' not available", t.runner.name); r.output = req.reason;
{
std::lock_guard lk(printMutex);
PrintResult(r, t.runner.name);
}
results[i] = std::move(r);
continue;
}
if (!runnerAvailable(t.runner)) {
// Hard-fail-unless-declared: if the runner depends on a tool
// (qemu-aarch64, wasmtime, wine, ...) and the test didn't say
// "tool:<that>" in requires, the missing runner is a Fail. The
// intent is to surface broken cross-arch CI configuration
// instead of letting it masquerade as a Skip; tests that
// legitimately may run without their runner have to opt in.
std::string tool = RunnerToolName(t.runner);
if (!tool.empty() && !RequiresMentionsTool(t.requires_, tool)) {
r.outcome = TestOutcome::Fail;
r.exitCode = -1;
r.output = std::format(
"runner '{}' unavailable and not declared in requires "
"(add 'tool:{}' to test.toml requires to permit skipping)",
t.runner.name, tool);
} else {
r.outcome = TestOutcome::Skipped;
r.output = std::format("runner '{}' not available", t.runner.name);
}
{ {
std::lock_guard lk(printMutex); std::lock_guard lk(printMutex);
PrintResult(r, t.runner.name); PrintResult(r, t.runner.name);

View file

@ -53,32 +53,43 @@ export namespace Crafter {
struct Test; struct Test;
struct TestRunner { struct TestRunner {
// Quoting style for {args} when this runner is used. Args reach // Command template the harness executes to run the test binary.
// whichever shell the runner ultimately delivers them to: Host for // Local runners leave this empty; prefix runners (Cmd, Wine) set it to
// Local + Cmd-prefix (host shell parses), Sh for Ssh (remote bash), // a template like "wine {bin} {args}" or "qemu-aarch64 {bin} {args}".
// Cmd for SshWin (remote cmd.exe).
enum class Shell { Host, Sh, Cmd };
std::string copy;
std::string exec; std::string exec;
std::string cleanup; // Display name; also the cache key for the availability probe.
std::string remoteDir;
std::string name; std::string name;
// Runs once per RunTests invocation (cached by `name`). Exit 0 = runner // Runs once per RunTests invocation (cached by `name`). Exit 0 = runner
// is available; non-zero = skip every Test using this runner with a // is available; non-zero = skip every Test using this runner with a
// "runner unavailable" message. Empty = always available (e.g., Local). // "runner unavailable" message. Empty = always available (e.g., Local).
std::string probe; std::string probe;
Shell argsShell = Shell::Host;
bool IsLocal() const { return exec.empty(); } bool IsLocal() const { return exec.empty(); }
static CRAFTER_API TestRunner Local(); static CRAFTER_API TestRunner Local();
static CRAFTER_API TestRunner Ssh(std::string host, std::string remoteDir = "/tmp/crafter-tests"); // Prefix runner: wraps the local binary in `<command> {bin} {args}`.
static CRAFTER_API TestRunner SshWin(std::string host, std::string remoteDir = "C:/temp/crafter-tests"); // Used for qemu-user, wasmtime, and similar single-binary wrappers.
static CRAFTER_API TestRunner Wsl(std::string remoteDir = "/tmp/crafter-tests-wsl");
static CRAFTER_API TestRunner Cmd(std::string command); static CRAFTER_API TestRunner Cmd(std::string command);
// Run a Windows .exe through Wine. Probes `wine` on PATH; on a Windows
// host the wine wrapper is pointless, so callers should route to Local
// before reaching here.
static CRAFTER_API TestRunner Wine();
// Parse a `<kind>[:<arg>]` spec used by CRAFTER_BUILD_RUNNER_<target>
// and --runner=. Supported: "local", "cmd:<binary>". Returns nullopt
// for an empty string; throws on a non-empty unrecognized spec.
static CRAFTER_API std::optional<TestRunner> FromSpec(std::string_view spec); static CRAFTER_API std::optional<TestRunner> FromSpec(std::string_view spec);
// Honor CRAFTER_BUILD_RUNNER_<target> (power-user override). Triple
// dashes/dots become underscores so they're valid in env-var names.
// Returns `fallback` when the env var is unset.
static CRAFTER_API TestRunner FromEnv(std::string_view target, TestRunner fallback = Local()); static CRAFTER_API TestRunner FromEnv(std::string_view target, TestRunner fallback = Local());
// Derive a runner from a Configuration's target triple + sysroot.
// Returns Local() when target equals the host, Wine() for Windows
// targets on a non-Windows host, `qemu-<arch>` (with QEMU_LD_PREFIX
// set when cfg.sysroot is non-empty) for non-host -linux- triples,
// `wasmtime` for wasm32-wasi/wasm64-wasi, and Local() as a last
// resort. CRAFTER_BUILD_RUNNER_<target> still wins as an override
// upstream of this — see FromEnv.
static CRAFTER_API TestRunner ForTarget(const struct Configuration& cfg);
}; };
enum class TestOutcome { Pass, Fail, Crash, Timeout, Skipped }; enum class TestOutcome { Pass, Fail, Crash, Timeout, Skipped };
@ -185,6 +196,15 @@ export namespace Crafter {
TestRunner runner; TestRunner runner;
std::chrono::seconds timeout{60}; std::chrono::seconds timeout{60};
std::vector<std::string> args; std::vector<std::string> args;
// Declarative preconditions. Each entry is "tool:<name>",
// "file:<path>", or "env:<VAR>". Evaluated before the test runs; any
// unmet require turns the test into a Skip with a derived reason.
// Also doubles as the "I know this runner might not be here" opt-in:
// when the test's derived runner needs a tool (e.g. qemu-aarch64,
// wasmtime, wine) and the matching tool: entry isn't present, an
// unavailable runner becomes a Fail instead of a silent Skip — the
// dependency has to be declared to be allowed to be missing.
std::vector<std::string> requires_;
}; };
CRAFTER_API BuildResult Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex); CRAFTER_API BuildResult Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex);

View file

@ -29,15 +29,13 @@ export namespace Crafter {
int jobs = 0; int jobs = 0;
std::optional<std::chrono::seconds> timeoutOverride; std::optional<std::chrono::seconds> timeoutOverride;
bool listOnly = false; bool listOnly = false;
// Only tests whose Configuration::target equals targetFilter are run. // Single-target run: only tests whose Configuration::target matches
// Set from --target=... (host triple if unspecified). Tests for other // are included. Empty (default) = run every distinct target declared
// targets are silently excluded so e.g. `--target=mingw` doesn't drag // across discovered tests, plus the host target. Set from --target=...
// in host-only outer-driver tests. // (when omitted, the harness sweeps all declared targets so cross-arch
#ifdef _WIN32 // tests run by default without the user having to know which targets
std::string targetFilter = "x86_64-pc-windows-msvc"; // exist).
#else std::string targetFilter;
std::string targetFilter = "x86_64-pc-linux-gnu";
#endif
// CLI override for --runner=<spec>: applies to every test in the run. // CLI override for --runner=<spec>: applies to every test in the run.
// Target scoping is unnecessary because targetFilter ensures the run // Target scoping is unnecessary because targetFilter ensures the run
// contains only one target's tests. // contains only one target's tests.

17899
lib/toml.hpp Normal file

File diff suppressed because it is too large Load diff

View file

@ -1,78 +0,0 @@
/*
Crafter® Build
Copyright (C) 2026 Catcrafts®
Catcrafts.net
LGPL-3.0-only.
End-to-end cross-arch build through the V2 pipeline:
the inner project.cpp targets aarch64-linux-gnu with cfg.sysroot pointing at
the Arch Linux ARM rootfs at /opt/aarch64-rootfs. crafter.build-lib
cross-compiles the C++ source (with the libc++ std module from the sysroot),
produces a real aarch64 ELF, and qemu-aarch64 (with QEMU_LD_PREFIX pointing at
the sysroot so it can find ld-linux-aarch64.so.1) executes it.
*/
import std;
import Crafter.Build;
#include "../_shared/TestUtil.h"
namespace fs = std::filesystem;
using namespace TestUtil;
using namespace Crafter;
namespace {
bool ToolPresent(std::string_view name) {
return std::system(std::format("which {} > /dev/null 2>&1", name).c_str()) == 0;
}
}
int main() {
try {
const fs::path sysroot = "/opt/aarch64-rootfs";
if (!fs::exists(sysroot / "usr/share/libc++/v1/std.cppm")) {
Skip(std::format("aarch64 sysroot missing at {} — see README", sysroot.string()));
}
if (!ToolPresent("qemu-aarch64")) Skip("qemu-aarch64 not on PATH");
fs::path src = fs::current_path() / "tests" / "CrossArchAarch64" / "inner";
Configuration cfg = LoadFixture("CrossArchAarch64", src);
fs::path work = fs::current_path();
auto br = BuildOnce(cfg);
if (!br.result.empty()) {
std::println(std::cerr, "build failed:\n{}", br.result);
return 1;
}
fs::path artifact = cfg.BinDir() / "aarch-hello";
if (!fs::exists(artifact)) {
std::println(std::cerr, "expected artifact missing at {}", artifact.string());
return 1;
}
// Defend against silent host-arch fallback.
auto probe = RunInDir(work, std::format("file '{}'", artifact.string()));
if (probe.output.find("ARM aarch64") == std::string::npos) {
std::println(std::cerr, "artifact is not ARM aarch64 ELF:\n{}", probe.output);
return 1;
}
// Run via qemu-aarch64. QEMU_LD_PREFIX tells qemu where the target's
// dynamic linker (ld-linux-aarch64.so.1) and shared libs live.
auto run = RunInDir(work, std::format(
"QEMU_LD_PREFIX={} qemu-aarch64 '{}'",
sysroot.string(), artifact.string()));
if (run.exitCode != 0) {
std::println(std::cerr, "qemu-aarch64 run failed (rc={}):\n{}", run.exitCode, run.output);
return 1;
}
if (run.output != "hi from 64-bit aarch64\n") {
std::println(std::cerr, "output mismatch:\n expected: \"hi from 64-bit aarch64\\n\"\n got: {:?}", run.output);
return 1;
}
return 0;
} catch (const std::exception& e) {
std::println(std::cerr, "test exception: {}", e.what());
return 1;
}
}

View file

@ -1,23 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "aarch-hello";
cfg.outputName = "aarch-hello";
cfg.target = "aarch64-linux-gnu";
cfg.march = "armv8-a";
cfg.mtune = "generic";
cfg.sysroot = "/opt/aarch64-rootfs";
cfg.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

View file

@ -1,20 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "tests/CrossArchAarch64/";
cfg.name = "CrossArchAarch64";
cfg.outputName = "CrossArchAarch64";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { ParentLib("crafter.build-lib") };
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "CrossArchAarch64" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

View file

@ -0,0 +1,13 @@
# End-to-end cross-arch build:
# crafter-build cross-compiles main.cpp for aarch64-linux-gnu using the
# Arch Linux ARM sysroot, and the runner derivation (TestRunner::ForTarget)
# wraps the produced ELF in `qemu-aarch64`. cfg.sysroot is forwarded to
# QEMU_LD_PREFIX automatically so the target's dynamic linker is reachable.
target = "aarch64-linux-gnu"
march = "armv8-a"
mtune = "generic"
sysroot = "/opt/aarch64-rootfs"
requires = [
"tool:qemu-aarch64",
"file:/opt/aarch64-rootfs/usr/share/libc++/v1/std.cppm",
]

View file

@ -3,11 +3,10 @@ import Crafter.Build;
namespace fs = std::filesystem; namespace fs = std::filesystem;
using namespace Crafter; using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) { extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
std::string target = "x86_64-pc-linux-gnu"; // Host-only: verifies cross-project module imports work, doesn't exercise
for (auto a : args) { // any cross-compile path. See tests/Diamond for the same rationale.
if (a.starts_with("--target=")) target = std::string(a.substr(9)); std::string target = HostTarget();
}
static auto fooLib = std::make_unique<Configuration>(); static auto fooLib = std::make_unique<Configuration>();
fooLib->path = "tests/CrossProjectModule/lib/"; fooLib->path = "tests/CrossProjectModule/lib/";

View file

@ -1,24 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
std::string target = "x86_64-pc-linux-gnu";
for (auto a : args) {
if (a.starts_with("--target=")) target = std::string(a.substr(9));
}
Configuration cfg;
cfg.path = "tests/Defines/";
cfg.name = "Defines";
cfg.outputName = "defines-app";
cfg.target = target;
cfg.type = ConfigurationType::Executable;
cfg.defines.push_back({"CRAFTER_TEST_FOO", "42"});
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

5
tests/Defines/test.toml Normal file
View file

@ -0,0 +1,5 @@
# Smoke test for cfg.defines propagation. main.cpp consumes CRAFTER_TEST_FOO
# both at compile time (static_assert) and at runtime; if the define doesn't
# reach the compile, the test won't even build.
[defines]
CRAFTER_TEST_FOO = "42"

View file

@ -21,11 +21,12 @@ namespace {
} }
} }
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) { extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
std::string target = "x86_64-pc-linux-gnu"; // Diamond exercises the build engine's dependency wiring at the host
for (auto a : args) { // target. There's no cross-compile plumbing (no sysroot, no per-target
if (a.starts_with("--target=")) target = std::string(a.substr(9)); // toolchain), so we don't honor --target=. A real cross-arch test would
} // live in its own tests/<Name>/test.toml.
std::string target = HostTarget();
static std::unique_ptr<Configuration> X, B, C; static std::unique_ptr<Configuration> X, B, C;
X = MakeLib("X", "X", target, {}); X = MakeLib("X", "X", target, {});

View file

@ -1,58 +0,0 @@
/*
Crafter® Build
Copyright (C) 2026 Catcrafts®
Catcrafts.net
LGPL-3.0-only.
*/
import std;
import Crafter.Build;
#include "../_shared/TestUtil.h"
namespace fs = std::filesystem;
using namespace TestUtil;
using namespace Crafter;
namespace {
std::string PickQemu() {
if (const char* v = std::getenv("CRAFTER_TEST_QEMU"); v && *v) return v;
return "qemu-x86_64";
}
bool QemuPresent(const std::string& qemu) {
return std::system(std::format("which {} > /dev/null 2>&1", qemu).c_str()) == 0;
}
}
int main() {
try {
std::string qemu = PickQemu();
if (!QemuPresent(qemu)) Skip(std::format("{} not on PATH", qemu));
std::string spec = std::format("cmd:{}", qemu);
::setenv("CRAFTER_BUILD_RUNNER_x86_64_pc_linux_gnu", spec.c_str(), 1);
// Verify env-var translation independently of RunTests.
auto runner = TestRunner::FromEnv("x86_64-pc-linux-gnu", TestRunner::Local());
if (runner.name != spec) {
std::println(std::cerr, "FromEnv produced '{}', expected '{}'", runner.name, spec);
return 1;
}
fs::path src = fs::current_path() / "tests" / "QemuUser" / "inner";
Configuration cfg = LoadFixture("QemuUser", src);
RunTestsOptions opts;
TestSummary summary = RunTests(cfg, opts);
if (summary.passed != 1 || summary.failed != 0 || summary.crashed != 0 ||
summary.timedOut != 0 || summary.skipped != 0) {
std::println(std::cerr,
"outcome counts mismatch: passed={} failed={} crashed={} timedOut={} skipped={}",
summary.passed, summary.failed, summary.crashed, summary.timedOut, summary.skipped);
return 1;
}
return 0;
} catch (const std::exception& e) {
std::println(std::cerr, "test exception: {}", e.what());
return 1;
}
}

View file

@ -1,27 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "qemu-meta";
cfg.outputName = "qemu-meta";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
Test t;
t.config.path = "./";
t.config.name = "Hello";
t.config.outputName = "Hello";
t.config.target = "x86_64-pc-linux-gnu";
t.config.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "tests/Hello" };
t.config.GetInterfacesAndImplementations(ifaces, impls);
t.runner = TestRunner::FromEnv(t.config.target);
cfg.tests.push_back(std::move(t));
return cfg;
}

View file

@ -1,6 +0,0 @@
import std;
int main() {
std::println("hello-from-qemu");
return 0;
}

View file

@ -1,20 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "tests/QemuUser/";
cfg.name = "QemuUser";
cfg.outputName = "QemuUser";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { ParentLib("crafter.build-lib") };
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "QemuUser" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

View file

@ -1,52 +0,0 @@
/*
Crafter® Build
Copyright (C) 2026 Catcrafts®
Catcrafts.net
LGPL-3.0-only.
*/
import std;
import Crafter.Build;
#include "../_shared/TestUtil.h"
namespace fs = std::filesystem;
using namespace TestUtil;
using namespace Crafter;
int main() {
try {
const char* hostEnv = std::getenv("CRAFTER_TEST_SSH_HOST");
if (!hostEnv || !*hostEnv) Skip("set CRAFTER_TEST_SSH_HOST to enable");
std::string host = hostEnv;
std::string probe = std::format("ssh -o BatchMode=yes -o ConnectTimeout=5 {} true > /dev/null 2>&1", host);
if (std::system(probe.c_str()) != 0) Skip(std::format("ssh {} not reachable", host));
std::string remoteDir = "/tmp/crafter-test-ssh-runner";
std::string spec = std::format("ssh:{}:{}", host, remoteDir);
::setenv("CRAFTER_BUILD_RUNNER_x86_64_pc_linux_gnu", spec.c_str(), 1);
auto runner = TestRunner::FromEnv("x86_64-pc-linux-gnu", TestRunner::Local());
if (runner.name != std::format("ssh:{}", host)) {
std::println(std::cerr, "FromEnv produced '{}', expected 'ssh:{}'", runner.name, host);
return 1;
}
fs::path src = fs::current_path() / "tests" / "SshRunner" / "inner";
Configuration cfg = LoadFixture("SshRunner", src);
RunTestsOptions opts;
TestSummary summary = RunTests(cfg, opts);
if (summary.passed != 1 || summary.failed != 0 || summary.crashed != 0 ||
summary.timedOut != 0 || summary.skipped != 0) {
std::println(std::cerr,
"outcome counts mismatch: passed={} failed={} crashed={} timedOut={} skipped={}",
summary.passed, summary.failed, summary.crashed, summary.timedOut, summary.skipped);
return 1;
}
return 0;
} catch (const std::exception& e) {
std::println(std::cerr, "test exception: {}", e.what());
return 1;
}
}

View file

@ -1,27 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "ssh-meta";
cfg.outputName = "ssh-meta";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
Test t;
t.config.path = "./";
t.config.name = "Hello";
t.config.outputName = "Hello";
t.config.target = "x86_64-pc-linux-gnu";
t.config.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "tests/Hello" };
t.config.GetInterfacesAndImplementations(ifaces, impls);
t.runner = TestRunner::FromEnv(t.config.target);
cfg.tests.push_back(std::move(t));
return cfg;
}

View file

@ -1,6 +0,0 @@
import std;
int main() {
std::println("hello-from-ssh");
return 0;
}

View file

@ -1,20 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "tests/SshRunner/";
cfg.name = "SshRunner";
cfg.outputName = "SshRunner";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { ParentLib("crafter.build-lib") };
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "SshRunner" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

View file

@ -13,21 +13,15 @@ int main() {
auto local = TestRunner::FromSpec("local"); auto local = TestRunner::FromSpec("local");
if (!local || local->name != "local") return 1; if (!local || local->name != "local") return 1;
auto ssh = TestRunner::FromSpec("ssh:somehost"); // Empty input returns nullopt; anything else unrecognized throws.
if (!ssh || ssh->name != "ssh:somehost") return 1; // ssh/sshwin/wsl used to be valid specs; they're now bogus (issue #8).
auto sshWithDir = TestRunner::FromSpec("ssh:somehost:/var/tmp/x");
if (!sshWithDir || sshWithDir->remoteDir != "/var/tmp/x") return 1;
auto sshWin = TestRunner::FromSpec("sshwin:winhost");
if (!sshWin || sshWin->name != "sshwin:winhost") return 1;
// Empty input returns nullopt; bogus prefix throws.
if (TestRunner::FromSpec("")) return 1; if (TestRunner::FromSpec("")) return 1;
try { for (auto bogus : { "nonsense:thing", "ssh:somehost", "sshwin:winhost", "wsl" }) {
TestRunner::FromSpec("nonsense:thing"); try {
return 1; TestRunner::FromSpec(bogus);
} catch (const std::exception&) {} return 1;
} catch (const std::exception&) {}
}
return 0; return 0;
} }

View file

@ -1,44 +0,0 @@
import std;
import Crafter.Build;
#include "../_shared/TestUtil.h"
namespace fs = std::filesystem;
using namespace TestUtil;
using namespace Crafter;
int main() {
try {
if (!fs::exists("/usr/share/wasi-sysroot/share/libc++/v1/std.cppm")) {
Skip("WASI sysroot/libc++ missing — install wasi-libc, wasi-libc++, wasi-libc++abi");
}
fs::path src = fs::current_path() / "tests" / "Wasi" / "inner";
Configuration cfg = LoadFixture("Wasi", src);
fs::path work = fs::current_path();
auto br = BuildOnce(cfg);
if (!br.result.empty()) {
std::println(std::cerr, "build failed:\n{}", br.result);
return 1;
}
fs::path artifact = cfg.BinDir() / "wasi-hello.wasm";
if (!fs::exists(artifact)) {
std::println(std::cerr, "expected artifact missing at {}", artifact.string());
return 1;
}
// Verify WASM magic bytes: \0asm
std::ifstream f(artifact, std::ios::binary);
char magic[4] = {};
f.read(magic, 4);
if (magic[0] != '\0' || magic[1] != 'a' || magic[2] != 's' || magic[3] != 'm') {
std::println(std::cerr, "artifact is not a valid WASM file (bad magic bytes)");
return 1;
}
return 0;
} catch (const std::exception& e) {
std::println(std::cerr, "test exception: {}", e.what());
return 1;
}
}

View file

@ -1,19 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "wasi-hello";
cfg.outputName = "wasi-hello";
cfg.target = "wasm32-wasip1";
cfg.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

View file

@ -1,20 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "tests/Wasi/";
cfg.name = "Wasi";
cfg.outputName = "Wasi";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { ParentLib("crafter.build-lib") };
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "Wasi" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

7
tests/Wasi/test.toml Normal file
View file

@ -0,0 +1,7 @@
# WASI build via the WASI SDK's libc++ + the wasm32-wasip1 target. The
# runner derivation routes the .wasm artifact through `wasmtime`.
target = "wasm32-wasip1"
requires = [
"tool:wasmtime",
"file:/usr/share/wasi-sysroot/share/libc++/v1/std.cppm",
]

View file

@ -1,68 +0,0 @@
/*
Crafter® Build
Copyright (C) 2026 Catcrafts®
Catcrafts.net
LGPL-3.0-only.
End-to-end LinuxWindows via SSH:
the inner fixture cross-compiles main.cpp for x86_64-w64-mingw32, the runner
specified via CRAFTER_BUILD_RUNNER_x86_64_w64_mingw32 scp's it to a Windows
host (winvm by default) and runs the .exe under cmd.exe via ssh. Gated on:
- mingw cross-toolchain installed (x86_64-w64-mingw32-g++)
- CRAFTER_TEST_WIN_SSH_HOST env var set
- the host reachable via ssh
*/
import std;
import Crafter.Build;
#include "../_shared/TestUtil.h"
namespace fs = std::filesystem;
using namespace TestUtil;
using namespace Crafter;
namespace {
bool ToolPresent(std::string_view name) {
return std::system(std::format("which {} > /dev/null 2>&1", name).c_str()) == 0;
}
}
int main() {
try {
const char* hostEnv = std::getenv("CRAFTER_TEST_WIN_SSH_HOST");
if (!hostEnv || !*hostEnv) Skip("set CRAFTER_TEST_WIN_SSH_HOST to enable, e.g. winvm");
std::string host = hostEnv;
if (!ToolPresent("x86_64-w64-mingw32-g++")) Skip("mingw cross-toolchain not on PATH");
std::string probe = std::format("ssh -o BatchMode=yes -o ConnectTimeout=5 {} \"ver\" > /dev/null 2>&1", host);
if (std::system(probe.c_str()) != 0) Skip(std::format("ssh {} not reachable", host));
std::string remoteDir = "C:/temp/crafter-test-winhello";
std::string spec = std::format("sshwin:{}:{}", host, remoteDir);
::setenv("CRAFTER_BUILD_RUNNER_x86_64_w64_mingw32", spec.c_str(), 1);
auto runner = TestRunner::FromEnv("x86_64-w64-mingw32", TestRunner::Local());
if (runner.name != std::format("sshwin:{}", host)) {
std::println(std::cerr, "FromEnv produced '{}', expected 'sshwin:{}'", runner.name, host);
return 1;
}
fs::path src = fs::current_path() / "tests" / "WindowsViaSsh" / "inner";
Configuration cfg = LoadFixture("WindowsViaSsh", src);
RunTestsOptions opts;
TestSummary summary = RunTests(cfg, opts);
if (summary.passed != 1 || summary.failed != 0 || summary.crashed != 0 ||
summary.timedOut != 0 || summary.skipped != 0) {
std::println(std::cerr,
"outcome counts mismatch: passed={} failed={} crashed={} timedOut={} skipped={}",
summary.passed, summary.failed, summary.crashed, summary.timedOut, summary.skipped);
return 1;
}
return 0;
} catch (const std::exception& e) {
std::println(std::cerr, "test exception: {}", e.what());
return 1;
}
}

View file

@ -1,6 +0,0 @@
import std;
int main() {
std::println("hi from windows");
return 0;
}

View file

@ -1,28 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "winhello-meta";
cfg.outputName = "winhello-meta";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
Test t;
t.config.path = "./";
t.config.name = "winhello";
t.config.outputName = "winhello";
t.config.target = "x86_64-w64-mingw32";
t.config.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
t.config.GetInterfacesAndImplementations(ifaces, impls);
t.runner = TestRunner::FromEnv(t.config.target);
cfg.tests.push_back(std::move(t));
return cfg;
}

View file

@ -1,20 +0,0 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "tests/WindowsViaSsh/";
cfg.name = "WindowsViaSsh";
cfg.outputName = "WindowsViaSsh";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
cfg.dependencies = { ParentLib("crafter.build-lib") };
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "WindowsViaSsh" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}

View file

@ -0,0 +1,6 @@
import std;
int main() {
std::println("hi from win{}", sizeof(void*) * 8);
return 0;
}

View file

@ -0,0 +1,9 @@
# Linux→Windows end-to-end:
# crafter-build cross-compiles main.cpp to x86_64-w64-mingw32 (.exe) and the
# runner derivation wraps it in `wine`. Replaces the old WindowsViaSsh test,
# which required a reachable Windows VM and an ssh + cmd.exe shell chain.
target = "x86_64-w64-mingw32"
requires = [
"tool:wine",
"tool:x86_64-w64-mingw32-g++",
]