Crafter.Build/implementations/Crafter.Build-Clang.cpp

2246 lines
117 KiB
C++
Raw Normal View History

2026-07-23 01:24:42 +02:00
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
2026-04-23 01:57:25 +02:00
2026-05-19 00:50:06 +02:00
module;
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
2026-04-23 01:57:25 +02:00
export module Crafter.Build:Clang_impl;
import std;
import :Clang;
2026-04-27 07:04:42 +02:00
import :Platform;
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
import :Toolchain;
import :Test;
2026-07-23 01:24:42 +02:00
import :Lint;
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
import :Script;
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md An agent working on a crafter-build project reads its AGENTS.md, and learns nothing there about the libraries the project builds against unless someone copied that in by hand. This repository shows the cost: the code style section of its AGENTS.md restates what Crafter.LintRules enforces, and goes stale whenever the rules change. A library now ships those notes itself, as AGENTS.consumer.md beside its project.cpp; nothing in its project.cpp changes. When a consumer has an AGENTS.md beside its own project.cpp, `crafter-build` and `crafter-build test` keep a section in it, between two marker comments, holding its dependencies' notes. The section is appended the first time and rewritten in place after that. The file is written only when the section changes, and an AGENTS.md is never created. lint, format, run and clean leave it alone. The update runs after loading, which has just fetched every dependency, and before compiling, so a build that fails still has the notes. cfg.dependencyNotes picks the dependencies. Transitive, the default, follows `dependencies` all the way down from the project's own configurations, its tests and its `#pragma crafter dependency` tooling; Direct stops at the first loaded project on each path; None removes the section. A dependency's own pragma tooling never counts: it is about how that dependency is built, not how it is used, and being no `dependencies` edge it is never reached. Nor are a dependency's tests: nothing builds them for the consumer, and their configurations can point at a local their project file has since moved out of. The walk has to know which Configurations belong to another project, and where that project lives, and cfg.path cannot say: only the Configuration GitProject and LocalProject return is rebased, so a dependency's internal configurations keep "./", which resolves to the consumer. The loader records it instead, LoadedProjectOf beside the project cache, and keeps what the root's pragmas resolved to past the frame LoadProject pops, PragmaDependencies. Neither is exported. The markers count only as whole lines outside fenced code blocks, so documentation can quote them. Unbalanced markers and a document ending inside a code block leave the file alone with a warning: guessing which part of someone's file is generated would overwrite their text, and missing the section would add another one on every build. A note that would break the section when it is read back, through a marker line of its own or a code block it never closes, is left out with a warning. The file is replaced through a temporary renamed onto the symlink's target, so a linked AGENTS.md stays a link and a write cut short leaves the old file rather than part of the new one. The code is a new partition, Crafter.Build:Agents, listed in build.sh, CrafterBuildModules, project.cpp and the umbrella. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
import :Agents;
2026-04-29 03:27:11 +02:00
import :Progress;
2026-05-12 01:16:40 +02:00
import :Asset;
2026-04-23 01:57:25 +02:00
namespace fs = std::filesystem;
2026-04-27 07:04:42 +02:00
using namespace Crafter;
2026-04-23 01:57:25 +02:00
2026-04-27 07:04:42 +02:00
namespace {
// Map one `import X;` name onto either a module this Configuration owns or
// one exported by a Configuration reachable through its dependency DAG,
// appending the matching staleness edge. False means nothing in reach
// provides X — either it's supplied from outside the graph (`std`) or the
// dependency isn't wired up *yet*, which is why callers remember the name.
bool ResolveImportName(Configuration& cfg, const std::string& importName, std::vector<Module*>& localDeps, std::vector<std::pair<Module*, fs::path>>& externalDeps) {
for(const std::unique_ptr<Module>& interface : cfg.interfaces) {
2026-04-27 07:04:42 +02:00
if(interface->name == importName) {
localDeps.push_back(interface.get());
return true;
}
}
std::unordered_set<Configuration*> seen;
std::function<bool(Configuration*)> walk = [&](Configuration* depCfg) -> bool {
if (!seen.insert(depCfg).second) return false;
2026-04-27 07:04:42 +02:00
for(const std::unique_ptr<Module>& depInterface : depCfg->interfaces) {
if(depInterface->name == importName) {
2026-07-23 01:24:42 +02:00
fs::path depPcmPath = std::format("{}.pcm", (depCfg->PcmDir() / depInterface->path.filename()).string());
2026-04-27 07:04:42 +02:00
externalDeps.emplace_back(depInterface.get(), std::move(depPcmPath));
return true;
}
}
for(Configuration* sub : depCfg->dependencies) {
if (walk(sub)) return true;
}
return false;
};
for(Configuration* depCfg : cfg.dependencies) {
if (walk(depCfg)) return true;
2026-04-27 07:04:42 +02:00
}
return false;
}
}
void Configuration::ResolvePendingImports() {
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
// Same resolution the scan used, and against both dependency kinds — a
// second GetInterfacesAndImplementations call can add interfaces that an
// earlier batch's import was looking for, so a pending name may land on a
// local module and not just an external one.
auto sweep = [this](std::vector<std::string>& pending, std::vector<Module*>& localDeps, std::vector<std::pair<Module*, fs::path>>& externalDeps) {
std::erase_if(pending, [&](const std::string& name) {
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
return ResolveImportName(*this, name, localDeps, externalDeps);
});
};
for(const std::unique_ptr<Module>& interface : interfaces) {
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
sweep(interface->pendingImports, interface->moduleDependencies, interface->externalModuleDependencies);
for(const std::unique_ptr<ModulePartition>& partition : interface->partitions) {
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
sweep(partition->pendingImports, partition->moduleDependencies, partition->externalModuleDependencies);
}
}
for(Implementation& implementation : implementations) {
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
sweep(implementation.pendingImports, implementation.moduleDependencies, implementation.externalModuleDependencies);
}
}
void Configuration::GetInterfacesAndImplementations(std::span<fs::path> interfaces, std::span<fs::path> implementations) {
auto resolveImport = [this](const std::string& importName, std::vector<Module*>& localDeps, std::vector<std::pair<Module*, fs::path>>& externalDeps, std::vector<std::string>& pending) {
if (!ResolveImportName(*this, importName, localDeps, externalDeps)) {
pending.push_back(importName);
}
2026-04-27 07:04:42 +02:00
};
std::vector<std::tuple<fs::path, std::string, ModulePartition*, Module*>> tempModulePaths = std::vector<std::tuple<fs::path, std::string, ModulePartition*, Module*>>(interfaces.size());
for(std::uint16_t i = 0; i < interfaces.size(); i++){
2026-04-30 02:20:19 +02:00
// Resolve to absolute now so the stored path survives cwd changes
// (matters for GitProject deps loaded from a different working dir).
fs::path file = fs::absolute(path / interfaces[i]).lexically_normal();
2026-04-27 07:04:42 +02:00
file += ".cppm";
std::ifstream t(file);
std::stringstream buffer;
buffer << t.rdbuf();
std::string fileContent = buffer.str();
fileContent = std::regex_replace(fileContent, std::regex(R"(//[^\n]*)"), "");
fileContent = std::regex_replace(fileContent, std::regex(R"(/\*.*?\*/)"), "");
tempModulePaths[i] = {file, fileContent, nullptr, nullptr};
}
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
// Primary interface units first, so the partition pass below can find their
// parent Module. They stay in tempModulePaths — a primary unit's own
// `import X;` lines are layout dependencies exactly like a partition's, and
// dropping the entry here is what left them unrecorded (issue #26). Marked
// by a null partition slot with the Module slot filled in.
for(std::tuple<fs::path, std::string, ModulePartition*, Module*>& file : tempModulePaths) {
2026-04-27 07:04:42 +02:00
std::smatch match;
if (std::regex_search(std::get<1>(file), match, std::regex(R"(export module ([a-zA-Z0-9_\.\-]+);)"))) {
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
fs::path pthCpy = std::get<0>(file);
pthCpy.replace_extension("");
this->interfaces.push_back(std::make_unique<Module>(std::move(match[1].str()), std::move(pthCpy)));
std::get<3>(file) = this->interfaces.back().get();
2026-04-27 07:04:42 +02:00
}
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
}
2026-04-27 07:04:42 +02:00
for(std::uint16_t i = 0; i < tempModulePaths.size(); i++) {
std::smatch match;
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
if (std::get<3>(tempModulePaths[i]) != nullptr) {
continue;
}
2026-04-27 07:04:42 +02:00
if (std::regex_search(std::get<1>(tempModulePaths[i]), match, std::regex(R"(export module ([a-zA-Z_0-9\.\-]+):([a-zA-Z_0-9\.\-]+);)"))) {
for(const std::unique_ptr<Module>& modulee : this->interfaces) {
if(modulee->name == match[1]) {
std::string name = match[2].str();
fs::path pthCpy = std::get<0>(tempModulePaths[i]);
pthCpy.replace_extension("");
std::unique_ptr<ModulePartition> partition = std::make_unique<ModulePartition>(std::move(name), std::move(pthCpy));
std::get<2>(tempModulePaths[i]) = partition.get();
modulee->partitions.push_back(std::move(partition));
std::get<3>(tempModulePaths[i]) = modulee.get();
goto next;
}
}
2026-07-23 01:24:42 +02:00
throw std::runtime_error(std::format("Module {} not found, referenced in {}", match[1].str(), std::get<0>(tempModulePaths[i]).string()));
2026-04-27 07:04:42 +02:00
} else {
2026-07-23 01:24:42 +02:00
throw std::runtime_error(std::format("No module declaration found in {}", std::get<0>(tempModulePaths[i]).string()));
2026-04-27 07:04:42 +02:00
}
next:;
}
for(std::tuple<fs::path, std::string, ModulePartition*, Module*>& file : tempModulePaths) {
ModulePartition* partition = std::get<2>(file);
Module* parentModule = std::get<3>(file);
const std::string& fileContent = std::get<1>(file);
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
// Primary interface unit: record only its module imports. Its
// `[export] import :Part;` lines need no edge — Module::Check walks
// every partition it owns and Module::Compile builds them all before
// precompiling itself, so partitions are covered wholesale.
if (partition == nullptr) {
std::regex primaryPattern(R"(import ([a-zA-Z_0-9\.\-]+);)");
std::sregex_iterator primaryCurrent(fileContent.begin(), fileContent.end(), primaryPattern);
std::sregex_iterator primaryLast;
while (primaryCurrent != primaryLast) {
std::smatch match = *primaryCurrent;
resolveImport(match[1].str(), parentModule->moduleDependencies, parentModule->externalModuleDependencies, parentModule->pendingImports);
++primaryCurrent;
}
continue;
}
2026-04-27 07:04:42 +02:00
std::regex partitionPattern(R"(import :([a-zA-Z_\-0-9\.]+);)");
std::sregex_iterator currentMatch(fileContent.begin(), fileContent.end(), partitionPattern);
std::sregex_iterator lastMatch;
while (currentMatch != lastMatch) {
std::smatch match = *currentMatch;
for(std::unique_ptr<ModulePartition>& sibling : parentModule->partitions) {
if(sibling->name == match[1]) {
partition->partitionDependencies.push_back(sibling.get());
goto next2;
}
}
throw std::runtime_error(std::format("imported partition {}:{} not found, referenced in {}", parentModule->name, match[1].str(), std::get<0>(file).string()));
next2: ++currentMatch;
}
std::regex modulePattern(R"(import ([a-zA-Z_0-9\.\-]+);)");
std::sregex_iterator modCurrent(fileContent.begin(), fileContent.end(), modulePattern);
while (modCurrent != lastMatch) {
std::smatch match = *modCurrent;
resolveImport(match[1].str(), partition->moduleDependencies, partition->externalModuleDependencies, partition->pendingImports);
2026-04-27 07:04:42 +02:00
++modCurrent;
}
}
for(const fs::path& tempFile : implementations) {
2026-04-30 02:20:19 +02:00
fs::path file = fs::absolute(path / tempFile).lexically_normal();
2026-04-27 07:04:42 +02:00
file += ".cpp";
std::ifstream t(file);
std::stringstream buffer;
buffer << t.rdbuf();
std::string fileContent = buffer.str();
fileContent = std::regex_replace(fileContent, std::regex(R"(//[^\n]*)"), "");
fileContent = std::regex_replace(fileContent, std::regex(R"(/\*.*?\*/)"), "");
std::smatch match;
fs::path fileCopy = file;
fileCopy.replace_extension("");
Implementation& implementation = this->implementations.emplace_back(std::move(fileCopy));
if (std::regex_search(fileContent, match, std::regex(R"(module ([a-zA-Z0-9_\.\-]+)(:[a-zA-Z0-9_\.\-]+)?\s*;)"))) {
feat(lint): const-local and constexpr-constant rules Two rules the AST makes possible, plus the mutation analysis behind them. const-local reports a local that is never written. It is restricted to SCALARS — integers, bools, enums, floating types — and that restriction is what makes the answer exact rather than a guess: a scalar has no member functions, so the only ways to write one are assignment, ++/--, having its address taken, or binding to a non-const reference. All four are now tracked in the walk: - assignment and compound assignment visit their LEFT operand in a write context, the right one normally; - ++/-- and & write their operand; - a call argument is checked against the callee's parameter type, so passing to `const int&` or by value is a read while `int&` is a write; - initialising a non-const reference writes what it binds to. For a class type a non-const method call could mutate it, and deciding that is the whole-program analysis clang-tidy does, so those are simply out of scope rather than guessed at. constexpr-constant promotes a const constant whose initialiser is made only of literals and operators, so `const int A = 1 << 4;` qualifies and `const int B = Compute();` does not. On this repository const-local found 103 candidates, which was too many to be useful, and the reason was informative: most were range-for bindings and pointer locals. `for (T* const x : …)` and `T* const p` are not spellings anybody writes, and the useful constness for a pointer is on the pointee, which this rule cannot advise on. Excluding both leaves 36, all plain bool or enum locals worth fixing — isWasm, isPe, exists, writes, isC and so on. Those 36 are fixed in this commit; the compiler verified every one. Both rules are report-only. The analysis is exact, but adding const is a judgement about intent as much as mechanics, and a wrong suggestion should cost a glance rather than a build. const-local also deliberately does not become a transform: inserting `const` before a shared type would apply it to every declarator in a multi-declarator statement, including any that IS written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 00:50:48 +02:00
const bool isPartitionImpl = match[2].length() > 0;
2026-04-27 07:04:42 +02:00
for(const std::unique_ptr<Module>& interface : this->interfaces) {
if(interface->name == match[1]) {
if (!isPartitionImpl) {
implementation.moduleDependencies.push_back(interface.get());
}
std::regex partitionPattern(R"(import :([a-zA-Z_\-0-9\.]+);)");
std::sregex_iterator currentMatch(fileContent.begin(), fileContent.end(), partitionPattern);
std::sregex_iterator lastMatch;
while (currentMatch != lastMatch) {
std::smatch match2 = *currentMatch;
for(const std::unique_ptr<ModulePartition>& partition : interface->partitions) {
if(partition->name == match2[1]) {
implementation.partitionDependencies.push_back(partition.get());
goto next3;
}
}
throw std::runtime_error(std::format("imported partition {}:{} not found, referenced in {}", match[1].str(), match2[1].str(), file.string()));
next3: ++currentMatch;
}
std::regex modulePattern(R"(import ([a-zA-Z_0-9\.\-]+);)");
std::sregex_iterator modCurrent(fileContent.begin(), fileContent.end(), modulePattern);
while (modCurrent != lastMatch) {
std::smatch match2 = *modCurrent;
if (match2[1] != match[1]) {
resolveImport(match2[1].str(), implementation.moduleDependencies, implementation.externalModuleDependencies, implementation.pendingImports);
2026-04-27 07:04:42 +02:00
}
++modCurrent;
}
goto next4;
}
}
throw std::runtime_error(std::format("Module {} not found not found, referenced in {}", match[1].str(), file.string()));
next4:;
} else {
std::regex pattern(R"(import ([a-zA-Z_\-0-9\.]+);)");
std::sregex_iterator currentMatch(fileContent.begin(), fileContent.end(), pattern);
std::sregex_iterator lastMatch;
while (currentMatch != lastMatch) {
std::smatch match2 = *currentMatch;
resolveImport(match2[1].str(), implementation.moduleDependencies, implementation.externalModuleDependencies, implementation.pendingImports);
2026-04-27 07:04:42 +02:00
++currentMatch;
}
}
}
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
fs::path Crafter::StdPcmDir(const Configuration& config) {
fix: key PCM and project.so freshness on content, not installed mtimes std.pcm was rebuilt when it was older than std.cppm, and project.so when it was older than the crafter-build executable. Both inputs are installed files, and an OSTree deployment (the Flatpak) gives every installed file mtime 0: std.cppm and the executable always looked older than the cache, so a clang bump would never rebuild std.pcm and a crafter-build update would never recompile project.so. clang crashes on a PCM from another compiler instead of reporting it, so the failure would have been a backtrace on the user's first build after an update. Each artifact now has a one-line stamp beside it. std.pcm's covers the precompile command and the module source's bytes, which also catches a libc++ update under the same clang. project.so's covers the crafter-build version, the toolchain key, the module source stamp and the compile command; project.cpp's and the pre-dependency artifacts' mtimes remain inputs, since those live next to it with real timestamps. The module source stamp stands in for the executable: project.so resolves Crafter:: symbols from the running binary, so only declarations matter, and an implementation-only rebuild of crafter-build no longer recompiles it. The std.pcm stamp folds into the module cache stamp, so a rebuilt std.pcm rebuilds the PCMs that import it. The cache directory carries the toolchain key (<target>-<march>-<clang version>-<hash>), so a compiler bump starts cold instead of handing the new clang PCMs it will crash on. LoadProject and Build() now derive the directory from the same StdPcmDir. Old <target>-<march> directories are left behind; `clean` never touched the cache and still does not. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:32:35 +02:00
// The std PCM is cached per target+march+toolchain: a PCM is only valid
// for the exact clang that wrote it, so a compiler bump starts a cold
// directory instead of handing the new clang a PCM it will crash on. A
// wasm variant pass compiles it with extra codegen flags (e.g.
// -mrelaxed-simd) — its BMI must not be shared with the baseline's, or
// the consuming TUs see a target-feature mismatch. Suffix the cache dir
// with the variant flags when present.
std::string stdPcmKey = std::format("{}-{}-{}", config.target, config.march, GetToolchain().Key());
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
// The flags every wasm unit gets are compiled into the std PCM as well,
// and the importing units have to match them. Two releases that share a
// cache and a compiler, as the Flatpak's branches do, would otherwise
// take turns rebuilding this one std PCM, each time underneath the PCMs
// built against the previous one, and clang crashes on those rather than
// rejecting them.
if (config.target.starts_with("wasm32")) {
stdPcmKey += std::format("-{:08x}", static_cast<std::uint32_t>(std::hash<std::string_view>{}(WasmBaselineFlags)));
}
fix: give the mingw target a sysroot that holds mingw-w64 alone Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree that holds mingw-w64 alone, and adds its include/ to the include path and its lib/ to the library path. For /usr those are the Linux system's own: every header mingw-w64 lacks came from glibc, and libstdc++'s <ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read __libc_single_threaded, which nothing on Windows defines. Every program that used one failed to link, among them every Crafter.Graphics Win32 executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak (#43). The sysroot is now MingwSysroot(): a directory in the cache, one per prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/ and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds mingw-w64, gcc's version and libstdc++ exactly as it does through the mingw gcc on PATH, and nothing else. It is made by the first mingw compile, link or CMake configure of a process, so no other command writes to the cache for it and a failure to make it is that build's error. The Toolchain field is now mingwPrefix, since clang never gets the prefix itself, and it is made absolute when the toolchain is resolved, as it ends up in link targets and directory names. Messages keep naming paths under the prefix rather than under the view. The prefix is also part of the mingw target's VariantId and of its std PCM directory. Units do not track the std PCM they import: when the fixed release built in a directory a broken build had left, the std module was rebuilt and the objects compiled against glibc's headers were only relinked, failing as before. As with a compiler update, a new directory is what gets them rebuilt. It also keeps two releases that share a cache, as the Flatpak's branches do, from rebuilding one std PCM under each other's PCMs, and two prefixes apart. Without a prefix both stay as they were: clang's own discovery has not changed. Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32 loses the directory gcc's version names, so <string> is not found, and clang ignores --gcc-toolchain and --gcc-install-dir for this target. Fixes #43 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
// The same goes for the mingw-w64 prefix the mingw std module is
// compiled against: releases up to the fix for issue #43 gave clang the
// prefix itself as the sysroot and the ones after give it a view of it,
// and two prefixes are two sets of headers.
if (config.target == "x86_64-w64-mingw32" && !GetToolchain().mingwPrefix.empty()) {
stdPcmKey += std::format("-{:08x}", static_cast<std::uint32_t>(std::hash<std::string>{}(GetToolchain().mingwPrefix.string())));
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
for (const std::string& f : config.wasmVariantFlags) {
stdPcmKey += "+";
for (char c : f) stdPcmKey += (c == '/' || c == '\\') ? '_' : c;
}
return GetCacheDir()/stdPcmKey;
}
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
std::string Crafter::WasmCompileFlags(const Configuration& config) {
std::string flags = std::format(" {}", WasmBaselineFlags);
for (const std::string& f : config.wasmVariantFlags) flags += std::format(" {}", f);
return flags;
}
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
fs::path Crafter::ExpectedOutput(const Configuration& config) {
const fs::path dir = config.BinDir();
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
const bool mingwTarget = config.target == "x86_64-w64-mingw32";
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
if (config.type == ConfigurationType::Executable) {
if (config.target.starts_with("wasm32")) return dir / std::format("{}.wasm", config.outputName);
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
return mingwTarget ? dir / std::format("{}.exe", config.outputName) : dir / config.outputName;
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
}
if (config.type == ConfigurationType::LibraryStatic) {
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
// The archive step writes lib<name>.a for every target, mingw included:
// llvm-ar does not care about the object format.
return dir / std::format("lib{}.a", config.outputName);
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
}
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
// LibraryDynamic — the .dll on the mingw target so an mtime check sees the
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
// freshly linked library; on Unix the .so is that file.
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
return mingwTarget ? dir / std::format("{}.dll", config.outputName) : dir / std::format("lib{}.so", config.outputName);
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
CompileCommand Crafter::GetCompileCommand(const Configuration& config) {
CompileCommand out;
out.stdPcmDir = StdPcmDir(config);
out.pcmDir = config.PcmDir();
std::string editedTarget = config.target;
std::replace(editedTarget.begin(), editedTarget.end(), '-', '_');
// wasm32 targets reject -march and silently ignore -mtune (clang errors on
// the former). Skip both for any wasm32-* triple.
feat(lint): const-local and constexpr-constant rules Two rules the AST makes possible, plus the mutation analysis behind them. const-local reports a local that is never written. It is restricted to SCALARS — integers, bools, enums, floating types — and that restriction is what makes the answer exact rather than a guess: a scalar has no member functions, so the only ways to write one are assignment, ++/--, having its address taken, or binding to a non-const reference. All four are now tracked in the walk: - assignment and compound assignment visit their LEFT operand in a write context, the right one normally; - ++/-- and & write their operand; - a call argument is checked against the callee's parameter type, so passing to `const int&` or by value is a read while `int&` is a write; - initialising a non-const reference writes what it binds to. For a class type a non-const method call could mutate it, and deciding that is the whole-program analysis clang-tidy does, so those are simply out of scope rather than guessed at. constexpr-constant promotes a const constant whose initialiser is made only of literals and operators, so `const int A = 1 << 4;` qualifies and `const int B = Compute();` does not. On this repository const-local found 103 candidates, which was too many to be useful, and the reason was informative: most were range-for bindings and pointer locals. `for (T* const x : …)` and `T* const p` are not spellings anybody writes, and the useful constness for a pointer is on the pointee, which this rule cannot advise on. Excluding both leaves 36, all plain bool or enum locals worth fixing — isWasm, isPe, exists, writes, isC and so on. Those 36 are fixed in this commit; the compiler verified every one. Both rules are report-only. The analysis is exact, but adding const is a judgement about intent as much as mechanics, and a wrong suggestion should cost a glance rather than a build. const-local also deliberately does not become a transform: inserting `const` before a shared type would apply it to every declarator in a multi-declarator statement, including any that IS written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 00:50:48 +02:00
const bool isWasm = config.target.starts_with("wasm32");
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
std::string archFlags = isWasm
? std::string()
: std::format(" -march={} -mtune={}", config.march, config.mtune);
out.command = std::format("{} --target={}{} -std=c++26 -D CRAFTER_BUILD_CONFIGURATION_TARGET=\\\"{}\\\" -D CRAFTER_BUILD_CONFIGURATION_TARGET_{} -fprebuilt-module-path={} -fprebuilt-module-path={}", GetBaseCommand(config), config.target, archFlags, editedTarget, editedTarget, out.stdPcmDir.string(), out.pcmDir.string());
if (!config.sysroot.empty()) {
out.command += std::format(" --sysroot={}", config.sysroot);
// Same reason as in BuildStdPcm: the driver ranks the host
// toolchain's own libc++ headers (<install>/../include/c++/v1) above
// --sysroot. A unit that only does `import std;` never notices, but
// one that includes a standard header textually (vendored C++ such as
// Crafter.Asset's gdeflate) would get host libc++ configured for the
// host libc, which fails on a musl sysroot with "unknown rune table
// for this platform". Force the sysroot's copy. -nostdinc++ leaves
// -stdlib=libc++ unused on every compile ("argument unused during
// compilation"), hence the silencer; the price is that it also hides
// that diagnostic for a user's own misplaced flag (a -l or -Wl,
// option in compileFlags) on sysroot builds. Not for mingw, which
// uses libstdc++ and has no libc++ in its sysroot.
if (!isWasm && config.target != "x86_64-w64-mingw32") {
out.command += std::format(" -nostdinc++ -isystem {}/usr/include/c++/v1 -Wno-unused-command-line-argument", config.sysroot);
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
}
if (isWasm) {
// -mllvm is consumed by codegen but not the link driver, which is the
// same command line; quiet the unused-flag warning rather than split
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
// compile and link commands. An active variant pass (see the variant
// driver at the end of Build) adds its codegen flags (e.g.
// -mrelaxed-simd) to every TU; they are part of VariantId, so those
// objects/PCMs land in their own dir.
out.command += std::format("{} -Wno-unused-command-line-argument", WasmCompileFlags(config));
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
}
if (config.target == "x86_64-w64-mingw32") {
// mingw libstdc++ defines TLS via __emutls_v.* (emulated TLS); without
// -femulated-tls clang generates native-TLS references that don't
// match. Symptom: undefined std::__once_callable / __once_call at
// link time. Also -Wno-unused… because -femulated-tls is a codegen
// flag the link driver doesn't consume.
out.command += " -femulated-tls -Wno-unused-command-line-argument";
}
if(config.type == ConfigurationType::LibraryDynamic) {
out.command += " -fPIC -D CRAFTER_BUILD_CONFIGURATION_TYPE_SHARED_LIBRARY";
} else if(config.type == ConfigurationType::Executable) {
out.command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_EXECUTABLE";
} else {
out.command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_LIBRARY";
}
// -I propagation that's valid for both C and C++ compiles. Module-only
// bits (-fprebuilt-module-path) stay on `command` only.
{
std::unordered_set<Configuration*> seen;
std::function<void(Configuration*)> addFlags = [&](Configuration* dep) {
if (!seen.insert(dep).second) return;
for (const auto& entry : fs::recursive_directory_iterator(dep->path)) {
if (entry.is_directory() && entry.path().filename() == "include") {
out.includeFlags += std::format(" -I{}", entry.path().string());
}
}
out.includeFlags += std::format(" -I{}", dep->path.string());
out.command += std::format(" -fprebuilt-module-path={}", dep->PcmDir().string());
for (Configuration* sub : dep->dependencies) {
addFlags(sub);
}
};
for (Configuration* dep : config.dependencies) {
addFlags(dep);
}
}
out.command += out.includeFlags;
feat(lint): no-char-pointer reads the AST, retiring the interop denylist The rule was `\bchar\s*\*` over the text minus a substring denylist — argv, getenv, setenv, dlerror, c_str, .data(, reinterpret_cast, extern ". Every entry was a patch for one interop site, the list could only grow as libraries arrived, and each entry disabled the rule for the whole LINE it appeared on. It now walks declarations and asks the question the denylist was approximating: whose header dictates this spelling? A declaration with C language linkage, or one whose initialiser binds to an entity declared outside the project root, is somebody else's API and keeps its spelling. getenv, c_str and friends are exempt because of where they are declared, not because they are named here, so a new external library needs no new entry. Two bugs found while testing this, both of which had made the rule silently pass over the entire repository: clang_getCursorLanguage cannot be used to detect extern "C". Its default answer is CXLanguage_C for a plain function, variable or parameter even in a C++ translation unit, so isExternC was true almost everywhere and exempted everything. Replaced by tracking CXCursor_LinkageSpec depth during the walk, reading the extent text to tell extern "C" from extern "C++". Attributing any foreign reference in a subtree to the enclosing declaration was too broad: a function that merely touched libc++ somewhere in its body would exempt its own signature. Narrowed to initialiser contexts — a variable, field or parameter — which is where a binding to a foreign API actually occurs. Also: functions now carry their RESULT type rather than the whole function type, since the parameters arrive as their own declarations and would otherwise be reported twice. main's parameters are exempt structurally, its signature being fixed by the language rather than chosen here. Two sites keep an explicit lint-disable, both Crafter::Run taking main's argv verbatim. That is two visible, reasoned suppressions in place of a denylist that silently disabled the rule for every line mentioning one of eight tokens. ExternalCloneDir and ExternalIncludeFlags are now exposed from :External, so a source that includes an external dependency's headers can be parsed without running a build to discover where they are. BuildExternal derives its own working directory through the same function, so the two cannot drift. Crafter.Build-Shader.cpp needed this to parse at all. A file with no compile command — project.cpp, which LoadProject builds with its own flags — is not a translation unit of the build graph, so AST rules skip it the way a rule self-filters by extension. That is distinct from a file that should have parsed and did not, which stays an error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:46:32 +02:00
// External dependency includes, gathered transitively. Kept off `command`
// so it stays exactly what it was before this was extracted.
{
std::unordered_set<const Configuration*> seen;
std::function<void(const Configuration*)> addExternal = [&](const Configuration* cfg) {
if (!seen.insert(cfg).second) return;
for (const ExternalDependency& dep : cfg->externalDependencies) {
for (const std::string& flag : ExternalIncludeFlags(dep, cfg->target)) {
out.externalIncludeFlags += std::format(" {}", flag);
}
}
for (const Configuration* sub : cfg->dependencies) addExternal(sub);
};
addExternal(&config);
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
// Defines belong on both C and C++ compiles so vendored C dependencies
// can see configuration-level macros consistently with module sources.
for(const Define& define : config.defines) {
if(define.value.empty()) {
out.defineFlags += std::format(" -D {}", define.name);
} else {
out.defineFlags += std::format(" -D {}={}", define.name, define.value);
}
}
out.command += out.defineFlags;
// Track caller-provided compileFlags separately so the .c compile can
// pick them up too (vendored C deps usually need -I from this set).
for(const std::string& flag : config.compileFlags) {
out.userFlags += std::format(" {}", flag);
}
out.command += out.userFlags;
// Behaviour-neutral release performance for the binaries we emit: ThinLTO
// (cross-TU inlining) plus dead-section GC and safe identical-code folding.
// Default-on in Release, never Debug (keeps builds fast and debuggable).
// Excluded for wasm32 — its -mllvm/sjlj codegen flags don't compose with
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
// LTO here, and wasi-sdk lists exceptions under LTO as broken
// (WebAssembly/wasi-sdk#629) — and for nvcc .cu objects below (they can't
// emit LLVM bitcode, so they link in as ordinary objects alongside the
// bitcode ones). ThinLTO rather than monolithic -flto so link time and
// memory scale with arbitrary user project sizes. Compile-side flags
// (-flto, -ffunction/-fdata-sections) ride on `command`, which is reused
// as the link base; the link-only -Wl flags go on linkExtras so they
// don't warn on each -c compile.
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
out.useLto = !config.debug && !isWasm;
out.ltoCompileFlags = out.useLto ? " -flto=thin -ffunction-sections -fdata-sections" : "";
out.ltoLinkFlags = out.useLto ? " -flto=thin -Wl,--gc-sections -Wl,--icf=safe" : "";
if(config.debug) {
out.command += " -g -D CRAFTER_BUILD_CONFIGURATION_DEBUG";
} else {
out.command += " -O3";
}
out.command += out.ltoCompileFlags;
return out;
}
fix(external): fetch and build each clone once per build pass `crafter-build test` builds a project's tests side by side, and a project whose tests each copy its externalDependencies hands every one of them the same unpinned dependency. Each test's Build() pulled the shared clone. Before the clone had a lock, those pulls ran at the same time and broke its .git/FETCH_HEAD, which failed every test of Crafter.Graphics (issue #34). Since 653dd0b they wait for each other, but every one is still a network round trip: 26 tests cost 26 pulls of each dependency, one after another, and for a CMake dependency as many configure checks and builds. depResults already records one build pass, keyed by the PcmDir of every dependency configuration built in it. An external dependency now goes in the same map, keyed by the directory its build leaves (ExternalBuildDir: the clone, or for CMake its build directory). The first configuration that needs it fetches and builds it; the others wait for that and take its outcome. Keying on the build directory rather than the clone keeps a CMake declaration from taking the outcome of a header-only declaration of the same clone, which only fetched it. A fresh depResults is a fresh pass, so the next build still pulls, and every caller of Build() gets this: RunTests, a plain build, and the pragma dependencies LoadProject builds. To make that possible BuildExternal is split. BuildExternal is now the shared work, lock, fetch, configure and build, and returns the error. ExternalOutputs computes what one declaration compiles and links with, because two declarations of one clone can differ in includeDirs, libDirs and libs; ExternalBuildResult loses its error field. The work no longer stops between steps when the configuration that started it is cancelled: others may be waiting on it, and a run cut short would leave them a clone nobody fetched. A dependency configuration's Build() already runs to completion for the same reason. The promise and future the dependency threads used to build or wait for a configuration are now one helper, OncePerPass, which both kinds use. An exception thrown in BuildExternal used to escape its thread into std::terminate; it is now a build error, as for a dependency. With eight tests that share a local dependency, a test run fetched it eight times before this and fetches it once now (SharedExternalFetch, next commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 02:09:24 +01:00
namespace {
// The outcome of `work` for `key` in this build pass. The first caller
// runs it; every other caller, whether at the same time or later, waits
// for that run and gets its result or its exception. The keys are the
// directories the work writes, a configuration's PcmDir and an external
// dependency's ExternalBuildDir, so the two kinds never collide.
std::shared_future<BuildResult> OncePerPass(std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex, const fs::path& key, const std::function<BuildResult()>& work) {
std::promise<BuildResult> promise;
std::shared_future<BuildResult> outcome = promise.get_future().share();
{
std::lock_guard lock(depMutex);
if (auto [it, inserted] = depResults.try_emplace(key, outcome); !inserted) return it->second;
}
try {
promise.set_value(work());
} catch (...) {
promise.set_exception(std::current_exception());
}
return outcome;
}
// Two configurations among `config` and its transitive dependencies that
// export a module of the same name: the same project loaded along two
// paths with different args, say, or from a checkout on one path and a
// clone on the other. A program can hold one module of each name. Built
// anyway, both PCM dirs went onto the consumer's -fprebuilt-module-path,
// clang resolved the module by name to the first for every import, the
// ones inside the other build's consumers included, and nothing checked
// that it was the build they had been compiled against. The program then
// linked code compiled against two layouts of the same types, and ran
// (issue #41). Empty when the dependencies agree, otherwise the error,
// naming the path to each of the two builds.
std::string DuplicateModuleError(const Configuration& config) {
struct Owner {
const Configuration* cfg;
std::string chain;
};
const auto describe = [](const Owner& owner) {
return std::format("{} ({}, {}, project args {})", owner.chain, owner.cfg->path.string(), owner.cfg->VariantId(), owner.cfg->projectArgs);
};
std::unordered_map<std::string, Owner> owners;
std::unordered_set<const Configuration*> visited;
std::string error;
std::function<void(const Configuration&, const std::string&)> walk = [&](const Configuration& c, const std::string& chain) {
if (!error.empty() || !visited.insert(&c).second) return;
for (const std::unique_ptr<Module>& iface : c.interfaces) {
const auto [it, inserted] = owners.emplace(iface->name, Owner{ &c, chain });
if (inserted || it->second.cfg == &c) continue;
error = std::format("module {} is built twice for {}:\n {}\n {}\nA program can hold one build of each module: load {} with the same args, and from the same place, along every path.", iface->name, config.name, describe(it->second), describe(Owner{ &c, chain }), c.name);
return;
}
for (const Configuration* dep : c.dependencies) walk(*dep, std::format("{} -> {}", chain, dep->name));
};
walk(config, config.name);
return error;
}
fix(wasm): what the review found -flto in a wasm configuration's compileFlags or linkFlags reached the link, where LTO code generation runs without the backend options that keep exception handling: every landing pad was dropped, and the build succeeded with throws that escaped their catch blocks. -fno-exceptions there made every `import std;` fail on the std module built with exceptions, and a wasm shared library failed to link against the eh/ libc++, which is static archives built without -fPIC. Build() refuses all three now, each with a line that says why, before it makes a directory or starts a thread. The sysroot check sat in BuildStdPcm, which the linter calls as well. There the error only dropped the compile command, so lint ran no AST rule on a wasm configuration and reported it clean. It also refused static libraries, which never link, fired after the shader and external threads had started, and called a missing sysroot, or one without the target, "no C++ exception support". It is part of the same early check now, which first says when the sysroot or the target's directory is missing, and asks for libunwind only where something links. C sources got only the signal.h define. setjmp.h refused to compile in C, and a C object given -wasm-enable-sjlj through compileFlags used the legacy exception encoding next to C++ objects in the standard one, a mix engines refuse to load. They get WasmCompileFlags now, like the C++ units, a variant pass's flags included. The baseline probes include simd: V8 and SpiderMonkey switch SIMD off on x86 CPUs without SSE4.1, which virtual machines often present, and such an engine passed the exception-handling probe only to fail compiling the module; the manifest tests expect it on every entry. The Flatpak pins WASI_SDK_EXCEPTIONS=DUAL, which wasi-sdk 34 defaults to only with clang 23 or newer, and checks that the eh/ libunwind got installed: the wasm tests skip without it, so CI would otherwise pass a Flatpak whose every wasm build fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:10:41 +01:00
// What a wasm32 configuration cannot be built with now that every unit
// has C++ exceptions, and what its sysroot has to have for that; empty
// for any other target and when nothing is wrong. Each would otherwise
// fail at the first compile or link with a message that does not say
// why, or not fail at all.
std::string WasmBuildError(const Configuration& config) {
if (!config.target.starts_with("wasm32")) return "";
for (const std::vector<std::string>* flags : { &config.compileFlags, &config.linkFlags }) {
for (const std::string& flag : *flags) {
// The compile has the backend options that keep exception
// handling; LTO code generation at the link does not, so it
// drops every landing pad and the build succeeds with throws
// that escape their catch blocks (WebAssembly/wasi-sdk#629).
if (flag.starts_with("-flto")) {
return std::format("{}: {} is not supported on wasm32: LTO code generation drops C++ exception handling (WebAssembly/wasi-sdk#629), so every throw would escape its catch", config.name, flag);
}
// The std module is built with exceptions, and a unit built
// without them cannot import it.
if (flag == "-fno-exceptions" || flag == "-fno-cxx-exceptions") {
return std::format("{}: {} is not supported on wasm32: wasm builds have C++ exceptions, the std module included, and a unit compiled without them cannot import std", config.name, flag);
}
}
}
// wasi-sdk's exception support does not cover shared libraries: the
// libc++ under lib/<triple>/eh is static archives built without -fPIC.
if (config.type == ConfigurationType::LibraryDynamic) {
return std::format("{}: wasm32 shared libraries are not supported: the exception-enabled libc++ they would link has static archives only", config.name);
}
const fs::path sysroot = config.sysroot;
if (!fs::is_directory(sysroot)) {
return std::format("no WASI sysroot at {}: the Flatpak bundles one, and CRAFTER_BUILD_WASI_SYSROOT selects another", sysroot.string());
}
const fs::path libDir = sysroot / "lib" / NormalizeTarget(config.target);
if (!fs::is_directory(libDir)) {
return std::format("the WASI sysroot at {} has nothing for {} (no {})", sysroot.string(), config.target, libDir.string());
}
// Only a link takes the exception-enabled libraries: a static
// library's objects compile against the headers alone. The driver
// looks in eh/ first and then in the triple's own directory, where a
// sysroot built with exceptions only (WASI_SDK_EXCEPTIONS=ON) has them.
// Without them the first link fails with "unable to find library
// -lunwind", which says nothing about why it was asked for.
if (config.type == ConfigurationType::Executable && !fs::exists(libDir / "eh" / "libunwind.a") && !fs::exists(libDir / "libunwind.a")) {
return std::format("the WASI sysroot at {} has no C++ exception support (no {}): wasm builds use WebAssembly exception handling and need the exception-enabled libc++ and libunwind that wasi-sdk 33 and later ship. The Flatpak bundles such a sysroot; CRAFTER_BUILD_WASI_SYSROOT selects another", sysroot.string(), (libDir / "eh" / "libunwind.a").string());
}
return "";
}
fix(external): fetch and build each clone once per build pass `crafter-build test` builds a project's tests side by side, and a project whose tests each copy its externalDependencies hands every one of them the same unpinned dependency. Each test's Build() pulled the shared clone. Before the clone had a lock, those pulls ran at the same time and broke its .git/FETCH_HEAD, which failed every test of Crafter.Graphics (issue #34). Since 653dd0b they wait for each other, but every one is still a network round trip: 26 tests cost 26 pulls of each dependency, one after another, and for a CMake dependency as many configure checks and builds. depResults already records one build pass, keyed by the PcmDir of every dependency configuration built in it. An external dependency now goes in the same map, keyed by the directory its build leaves (ExternalBuildDir: the clone, or for CMake its build directory). The first configuration that needs it fetches and builds it; the others wait for that and take its outcome. Keying on the build directory rather than the clone keeps a CMake declaration from taking the outcome of a header-only declaration of the same clone, which only fetched it. A fresh depResults is a fresh pass, so the next build still pulls, and every caller of Build() gets this: RunTests, a plain build, and the pragma dependencies LoadProject builds. To make that possible BuildExternal is split. BuildExternal is now the shared work, lock, fetch, configure and build, and returns the error. ExternalOutputs computes what one declaration compiles and links with, because two declarations of one clone can differ in includeDirs, libDirs and libs; ExternalBuildResult loses its error field. The work no longer stops between steps when the configuration that started it is cancelled: others may be waiting on it, and a run cut short would leave them a clone nobody fetched. A dependency configuration's Build() already runs to completion for the same reason. The promise and future the dependency threads used to build or wait for a configuration are now one helper, OncePerPass, which both kinds use. An exception thrown in BuildExternal used to escape its thread into std::terminate; it is now a build error, as for a dependency. With eight tests that share a local dependency, a test run fetched it eight times before this and fetches it once now (SharedExternalFetch, next commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 02:09:24 +01:00
}
2026-04-27 07:04:42 +02:00
BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex) {
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
// Before anything is spawned: one message naming the missing compiler
// beats a clang++-not-found per translation unit.
if (const Toolchain& tc = GetToolchain(); !tc.error.empty()) {
return {tc.error, false, {}};
}
// Before any directory is made or dependency built, so a refused graph
// leaves nothing behind.
if (std::string duplicate = DuplicateModuleError(config); !duplicate.empty()) {
return {std::move(duplicate), false, {}};
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
// Reset per-build cached state on every Module/ModulePartition so that
// successive Build() calls on the same Configuration re-evaluate mtimes
fix: scope per-build module-state reset to the config being built Build() resets each Module/ModulePartition's per-build `compiled`/`checked` flags so a reused Configuration re-evaluates mtimes. That reset recursed into cfg.dependencies — but dependency Configurations are shared across the build DAG and each is compiled concurrently by its own Build() call. A parent/sibling's recursive reset could therefore clear a shared dependency's module `compiled` atomic *after* that dependency's module-compile thread had set it true and exited, but before an intra-config waiter (its impl, or a dependent partition) ran compiled.wait(false). The waiter then blocked forever on a flag nothing would re-signal: the build froze mid-compile, idle, with no compiler process alive — exactly the hang in issue #16. Reset only the current configuration's own modules. Every config in the tree already gets its own Build() call (the per-PcmDir builder registered in depResults), which resets its own state at the top of that call, sequenced before its compile threads spawn. Cross-config module state is consulted only via PCM file mtimes and the depResults futures, never via these flags, so the narrower reset is correct and removes the data race entirely. Adds ConcurrentDependencyReset: builds a static-lib dependency fully, then builds a consumer that depends on it while the dependency is already cached in depResults (so it is never rebuilt), and asserts the consumer build leaves the dependency's module `compiled` flag intact. Fails deterministically on the old recursive reset; passes with the fix. Resolves #16 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 11:32:46 +00:00
// (incremental-rebuild test scenarios).
//
// Reset ONLY this configuration's own modules — never recurse into
// dependencies. Configuration objects are shared across the dependency DAG
// (diamond deps point at the same Configuration*), and every config in the
// tree gets its own Build() call: the per-PcmDir builder registered in
// depResults resets its own state here, at the top of its own Build(),
// sequenced before that config's compile threads spawn. Recursing into
// dependencies from here would re-clear a shared dependency's `compiled`
// atomic from a parent/sibling thread *while that dependency's own Build()
// is concurrently compiling it* — after its module-compile thread set the
// flag true and exited but before its impl/partition waiter ran
// compiled.wait(false). The waiter would then block forever on a flag
// nothing re-signals: the build freezes mid-compile, idle, with no compiler
// process alive (issue #16). Cross-config module state is consulted only via
// PCM file mtimes and the depResults futures, never via these flags, so the
// narrower reset is correct.
for (auto& iface : config.interfaces) {
iface->checked = false;
iface->compiled.store(false);
for (auto& part : iface->partitions) {
part->checked = false;
part->compiled.store(false);
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
}
// Sources were scanned when they were declared, which for most callers is
// before `dependencies` exists — AddTest resolves tests/<name>/main.cpp and
// only then hands back a builder whose .Dependencies() supplies the library.
// Any `import <DepModule>;` in such a TU resolved to nothing and so carried
// no staleness edge, which meant a member added to a dependency's interface
// rebuilt the library, relinked the consumer, and silently kept the
// consumer's object compiled against the *old* class layout (issue #27).
// Re-resolving here — the last point before mtimes are compared, with the
// DAG fully wired — closes that window for every caller rather than for the
// ones that remember to declare in the right order.
config.ResolvePendingImports();
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
// Auto-detect the WASI sysroot before any compile step runs so BuildStdPcm
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
// and the main compile command see the same value. Covers all wasm32-*
// triples (wasi, wasip1, wasip2, ...); the sysroot's per-triple subdirs
// handle the differences.
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
if (config.sysroot.empty() && config.target.starts_with("wasm32")) {
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
config.sysroot = GetToolchain().wasiSysroot.string();
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
}
fix(wasm): what the review found -flto in a wasm configuration's compileFlags or linkFlags reached the link, where LTO code generation runs without the backend options that keep exception handling: every landing pad was dropped, and the build succeeded with throws that escaped their catch blocks. -fno-exceptions there made every `import std;` fail on the std module built with exceptions, and a wasm shared library failed to link against the eh/ libc++, which is static archives built without -fPIC. Build() refuses all three now, each with a line that says why, before it makes a directory or starts a thread. The sysroot check sat in BuildStdPcm, which the linter calls as well. There the error only dropped the compile command, so lint ran no AST rule on a wasm configuration and reported it clean. It also refused static libraries, which never link, fired after the shader and external threads had started, and called a missing sysroot, or one without the target, "no C++ exception support". It is part of the same early check now, which first says when the sysroot or the target's directory is missing, and asks for libunwind only where something links. C sources got only the signal.h define. setjmp.h refused to compile in C, and a C object given -wasm-enable-sjlj through compileFlags used the legacy exception encoding next to C++ objects in the standard one, a mix engines refuse to load. They get WasmCompileFlags now, like the C++ units, a variant pass's flags included. The baseline probes include simd: V8 and SpiderMonkey switch SIMD off on x86 CPUs without SSE4.1, which virtual machines often present, and such an engine passed the exception-handling probe only to fail compiling the module; the manifest tests expect it on every entry. The Flatpak pins WASI_SDK_EXCEPTIONS=DUAL, which wasi-sdk 34 defaults to only with clang 23 or newer, and checks that the eh/ libunwind got installed: the wasm tests skip without it, so CI would otherwise pass a Flatpak whose every wasm build fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:10:41 +01:00
// As early as the duplicate-module check, so a refused build leaves no
// directory behind and starts no thread, but after the sysroot is known.
if (std::string refused = WasmBuildError(config); !refused.empty()) {
return {std::move(refused), false, {}};
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
2026-04-30 02:20:19 +02:00
fs::path buildDir = config.BuildDir();
fs::path outputDir = config.BinDir();
2026-04-23 01:57:25 +02:00
if (!fs::exists(buildDir)) {
fs::create_directories(buildDir);
}
if (!fs::exists(outputDir)) {
fs::create_directories(outputDir);
}
2026-05-19 16:53:24 +02:00
BuildResult buildResult{};
2026-04-27 07:04:42 +02:00
2026-05-02 21:08:51 +02:00
// glslang #include search paths for every shader compiled in this
// configuration: each transitive (incl. self) buildFiles entry's parent
// dir, or the entry itself if it points at a directory. Collected once
// up front so the per-shader threads can capture the resulting span by
// reference. No file copy involved — glslang reads the includes in
// place from the dep's source tree.
std::vector<fs::path> shaderIncludeDirs;
{
std::unordered_set<std::string> seenDirs;
std::unordered_set<const Configuration*> seenCfg;
std::function<void(const Configuration*)> collect = [&](const Configuration* c) {
if (!seenCfg.insert(c).second) return;
for (const fs::path& bf : c->buildFiles) {
fs::path dir = fs::is_directory(bf) ? bf : bf.parent_path();
if (seenDirs.insert(dir.string()).second) {
shaderIncludeDirs.push_back(std::move(dir));
}
}
for (const Configuration* sub : c->dependencies) collect(sub);
};
collect(&config);
}
2026-04-23 01:57:25 +02:00
std::vector<std::thread> threads;
threads.reserve(config.shaders.size() + 1 + config.interfaces.size() + config.implementations.size());
std::string buildError;
std::atomic<bool> buildCancelled{false};
feat(shader): per-entry defines and a named output variant Crafter.Graphics has to ship each of its compute shaders in two builds of one source: one that indexes VK_EXT_descriptor_heap and one that reads the same slots from a descriptor set, for drivers without the extension. A Shader was a path, an entry point and a stage, and its output name was derived from the path alone, so one source could only be compiled once. Shader gains `defines`, spelled like the compiler's -D ("NAME" is defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a preamble, and `variant`, which names the output: "ui.comp.glsl" with variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one place that name is formed; Check, Compile and the forwarding of a dependency's shaders into the consumer's bin dir all use it. Two fields make one mistake easy: defines set, variant forgotten. Both entries then compiled in parallel into the same file, the build reported success with whichever finished last, and Check() called both up to date from then on. The same has always been true of a.vert next to a.frag. Build() now refuses a configuration in which two entries share an output name and says which. A define without a name, or one spanning lines, and a variant with a path separator are rejected by name. Left to glslang the first two surface as "-1:<line>" with no file, and the last as a failed open after the whole compile. Not addressed: Check() still compares only the source's mtime, so a change to the defines, or to an included file, does not recompile (#35). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 16:50:41 +02:00
// Two entries with the same output name would compile in parallel into
// one file: whichever finished last won, the build reported success, and
// Check() then called both up to date. The usual way in is one source
// registered twice with different defines and no variant; a.vert next to
// a.frag is another.
{
std::unordered_map<std::string, const Shader*> outputs;
for (const Shader& shader : config.shaders) {
const auto [existing, inserted] = outputs.emplace(shader.OutputName().string(), &shader);
if (inserted) continue;
buildError = std::format("shaders {} and {} both compile to {}: give one of them a distinct Shader::variant", existing->second->path.string(), shader.path.string(), existing->first);
buildCancelled.store(true);
break;
}
}
2026-04-23 01:57:25 +02:00
for (const Shader& shader : config.shaders) {
feat(shader): per-entry defines and a named output variant Crafter.Graphics has to ship each of its compute shaders in two builds of one source: one that indexes VK_EXT_descriptor_heap and one that reads the same slots from a descriptor set, for drivers without the extension. A Shader was a path, an entry point and a stage, and its output name was derived from the path alone, so one source could only be compiled once. Shader gains `defines`, spelled like the compiler's -D ("NAME" is defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a preamble, and `variant`, which names the output: "ui.comp.glsl" with variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one place that name is formed; Check, Compile and the forwarding of a dependency's shaders into the consumer's bin dir all use it. Two fields make one mistake easy: defines set, variant forgotten. Both entries then compiled in parallel into the same file, the build reported success with whichever finished last, and Check() called both up to date from then on. The same has always been true of a.vert next to a.frag. Build() now refuses a configuration in which two entries share an output name and says which. A define without a name, or one spanning lines, and a variant with a path separator are rejected by name. Left to glslang the first two surface as "-1:<line>" with no file, and the last as a failed open after the whole compile. Not addressed: Check() still compares only the source's mtime, so a change to the defines, or to an included file, does not recompile (#35). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 16:50:41 +02:00
if (buildCancelled.load(std::memory_order_relaxed)) break;
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
if (shader.Check(outputDir, buildDir, shaderIncludeDirs)) continue;
threads.emplace_back([&shader, &outputDir, &buildDir, &shaderIncludeDirs, &buildError, &buildCancelled]() {
feat(shader): per-entry defines and a named output variant Crafter.Graphics has to ship each of its compute shaders in two builds of one source: one that indexes VK_EXT_descriptor_heap and one that reads the same slots from a descriptor set, for drivers without the extension. A Shader was a path, an entry point and a stage, and its output name was derived from the path alone, so one source could only be compiled once. Shader gains `defines`, spelled like the compiler's -D ("NAME" is defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a preamble, and `variant`, which names the output: "ui.comp.glsl" with variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one place that name is formed; Check, Compile and the forwarding of a dependency's shaders into the consumer's bin dir all use it. Two fields make one mistake easy: defines set, variant forgotten. Both entries then compiled in parallel into the same file, the build reported success with whichever finished last, and Check() called both up to date from then on. The same has always been true of a.vert next to a.frag. Build() now refuses a configuration in which two entries share an output name and says which. A define without a name, or one spanning lines, and a variant with a path separator are rejected by name. Left to glslang the first two surface as "-1:<line>" with no file, and the last as a failed open after the whole compile. Not addressed: Check() still compares only the source's mtime, so a change to the defines, or to an included file, does not recompile (#35). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 16:50:41 +02:00
// A source registered more than once shows up once per variant;
// name the variant so the lines can be told apart.
Progress::Task task(shader.variant.empty() ? std::format("Compiling shader {}", shader.path.filename().string()) : std::format("Compiling shader {} ({})", shader.path.filename().string(), shader.variant));
2026-04-23 01:57:25 +02:00
if (buildCancelled.load(std::memory_order_relaxed)) return;
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
std::string result = shader.Compile(outputDir, buildDir, shaderIncludeDirs);
2026-04-23 01:57:25 +02:00
if (result.empty()) return;
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result);
}
});
}
2026-05-12 03:44:14 +02:00
// Asset compilation: each cfg.assets entry is either a single .png/.obj
// file (flat output: outputDir/<filename>.ctex/cmesh — preserves the
// original behavior) or a directory (recursed, with the relative tree
// mirrored under outputDir/<dirname>/; .png/.obj are compressed, every
// other file is copied through unchanged). Directory mode lets mod/map
// trees keep their nested layout so mod.json paths like
// "cannon/base.cmesh" resolve correctly at runtime.
// Skipped per-file if the output is newer than the source. Each
// compress runs in its own thread; passthrough copies are bundled into
// a single thread to match the cfg.files pattern.
auto compressedName = [](const fs::path& src) -> std::optional<fs::path> {
std::string ext = src.extension().string();
2026-07-23 01:24:42 +02:00
for (char& c : ext) c = static_cast<char>(std::tolower(static_cast<std::uint8_t>(c)));
2026-05-19 00:50:06 +02:00
// stb_image (used by CompressAsset → TextureAsset::LoadPNG) handles
// png/tga/jpg/bmp; all map to .ctex.
if (ext == ".png" || ext == ".tga" || ext == ".jpg" || ext == ".jpeg" || ext == ".bmp") {
return fs::path(src.filename()).replace_extension(".ctex");
}
2026-05-12 03:44:14 +02:00
if (ext == ".obj") return fs::path(src.filename()).replace_extension(".cmesh");
2026-05-12 01:16:40 +02:00
return std::nullopt;
};
2026-05-12 03:44:14 +02:00
auto submitCompress = [&](fs::path sourcePath, fs::path outRelative) {
fs::path out = outputDir / outRelative;
if (fs::exists(out) && fs::exists(sourcePath) && fs::last_write_time(sourcePath) <= fs::last_write_time(out)) return;
threads.emplace_back([sourcePath = std::move(sourcePath), out = std::move(out), &buildError, &buildCancelled]() {
Progress::Task task(std::format("Compressing asset {}", sourcePath.filename().string()));
2026-05-12 01:16:40 +02:00
if (buildCancelled.load(std::memory_order_relaxed)) return;
2026-05-12 03:44:14 +02:00
std::error_code ec;
fs::create_directories(out.parent_path(), ec);
std::string result = CompressAsset(sourcePath, out);
2026-05-12 01:16:40 +02:00
if (result.empty()) return;
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result);
}
});
2026-05-12 03:44:14 +02:00
};
std::vector<std::pair<fs::path, fs::path>> assetPassthroughs;
for (const fs::path& asset : config.assets) {
if (fs::is_directory(asset)) {
fs::path topName = asset.filename();
for (const auto& entry : fs::recursive_directory_iterator(asset)) {
if (!entry.is_regular_file()) continue;
fs::path rel = fs::relative(entry.path(), asset);
if (std::optional<fs::path> compName = compressedName(entry.path())) {
submitCompress(entry.path(), topName / rel.parent_path() / *compName);
} else {
assetPassthroughs.emplace_back(entry.path(), topName / rel);
}
}
} else {
std::optional<fs::path> outName = compressedName(asset);
if (!outName) {
buildCancelled.store(true);
2026-05-19 00:50:06 +02:00
buildError = std::format("{}: unsupported asset extension (expected .png/.tga/.jpg/.bmp/.obj, or a directory)", asset.string());
2026-05-12 03:44:14 +02:00
break;
}
submitCompress(asset, *outName);
}
}
if (!assetPassthroughs.empty()) {
threads.emplace_back([passthroughs = std::move(assetPassthroughs), &outputDir, &buildCancelled, &buildError]() {
Progress::Task task("Copying asset passthrough files");
if (buildCancelled.load(std::memory_order_relaxed)) return;
try {
for (const auto& [src, rel] : passthroughs) {
fs::path dst = outputDir / rel;
fs::create_directories(dst.parent_path());
if (!fs::exists(dst)) {
fs::copy_file(src, dst);
} else if (fs::last_write_time(src) > fs::last_write_time(dst)) {
fs::copy_file(src, dst, fs::copy_options::overwrite_existing);
}
}
} catch (const std::exception& e) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = e.what();
}
}
});
2026-05-12 01:16:40 +02:00
}
2026-04-23 01:57:25 +02:00
threads.emplace_back([&config, &outputDir, &buildCancelled, &buildError]() {
2026-04-29 03:27:11 +02:00
Progress::Task task(std::format("Copying files for {}", config.name));
2026-04-23 01:57:25 +02:00
if (buildCancelled.load(std::memory_order_relaxed)) return;
try {
2026-05-02 21:08:51 +02:00
for (const fs::path& additionalFile : config.files) {
2026-04-23 01:57:25 +02:00
fs::path destination = outputDir / additionalFile.filename();
if (fs::is_directory(additionalFile)) {
for (const auto& entry : fs::recursive_directory_iterator(additionalFile)) {
const fs::path& sourcePath = entry.path();
fs::path relativePath = fs::relative(sourcePath, additionalFile);
fs::path destPath = destination / relativePath;
if (entry.is_directory()) {
2026-05-02 21:08:51 +02:00
if (!fs::exists(destPath)) fs::create_directories(destPath);
2026-04-23 01:57:25 +02:00
} else if (entry.is_regular_file()) {
fs::create_directories(destPath.parent_path());
if (!fs::exists(destPath)) {
fs::copy_file(sourcePath, destPath);
2026-05-02 21:08:51 +02:00
} else if (fs::last_write_time(sourcePath) > fs::last_write_time(destPath)) {
2026-04-23 01:57:25 +02:00
fs::copy_file(sourcePath, destPath, fs::copy_options::overwrite_existing);
}
}
}
} else {
if (!fs::exists(destination)) {
fs::copy_file(additionalFile, destination);
2026-05-02 21:08:51 +02:00
} else if (fs::last_write_time(additionalFile) > fs::last_write_time(destination)) {
2026-04-23 01:57:25 +02:00
fs::copy_file(additionalFile, destination, fs::copy_options::overwrite_existing);
}
}
}
} catch(std::exception& e) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = e.what();
}
}
});
2026-04-27 07:04:42 +02:00
std::vector<ExternalBuildResult> externalResults(config.externalDependencies.size());
std::vector<std::thread> externalThreads;
externalThreads.reserve(config.externalDependencies.size());
for (std::size_t i = 0; i < config.externalDependencies.size(); ++i) {
externalThreads.emplace_back([&, i]() {
fix(external): fetch and build each clone once per build pass `crafter-build test` builds a project's tests side by side, and a project whose tests each copy its externalDependencies hands every one of them the same unpinned dependency. Each test's Build() pulled the shared clone. Before the clone had a lock, those pulls ran at the same time and broke its .git/FETCH_HEAD, which failed every test of Crafter.Graphics (issue #34). Since 653dd0b they wait for each other, but every one is still a network round trip: 26 tests cost 26 pulls of each dependency, one after another, and for a CMake dependency as many configure checks and builds. depResults already records one build pass, keyed by the PcmDir of every dependency configuration built in it. An external dependency now goes in the same map, keyed by the directory its build leaves (ExternalBuildDir: the clone, or for CMake its build directory). The first configuration that needs it fetches and builds it; the others wait for that and take its outcome. Keying on the build directory rather than the clone keeps a CMake declaration from taking the outcome of a header-only declaration of the same clone, which only fetched it. A fresh depResults is a fresh pass, so the next build still pulls, and every caller of Build() gets this: RunTests, a plain build, and the pragma dependencies LoadProject builds. To make that possible BuildExternal is split. BuildExternal is now the shared work, lock, fetch, configure and build, and returns the error. ExternalOutputs computes what one declaration compiles and links with, because two declarations of one clone can differ in includeDirs, libDirs and libs; ExternalBuildResult loses its error field. The work no longer stops between steps when the configuration that started it is cancelled: others may be waiting on it, and a run cut short would leave them a clone nobody fetched. A dependency configuration's Build() already runs to completion for the same reason. The promise and future the dependency threads used to build or wait for a configuration are now one helper, OncePerPass, which both kinds use. An exception thrown in BuildExternal used to escape its thread into std::terminate; it is now a build error, as for a dependency. With eight tests that share a local dependency, a test run fetched it eight times before this and fetches it once now (SharedExternalFetch, next commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 02:09:24 +01:00
const ExternalDependency& dep = config.externalDependencies[i];
Progress::Task task(std::format("Building external dep {}", dep.name));
2026-04-27 07:04:42 +02:00
if (buildCancelled.load(std::memory_order_relaxed)) return;
fix(external): fetch and build each clone once per build pass `crafter-build test` builds a project's tests side by side, and a project whose tests each copy its externalDependencies hands every one of them the same unpinned dependency. Each test's Build() pulled the shared clone. Before the clone had a lock, those pulls ran at the same time and broke its .git/FETCH_HEAD, which failed every test of Crafter.Graphics (issue #34). Since 653dd0b they wait for each other, but every one is still a network round trip: 26 tests cost 26 pulls of each dependency, one after another, and for a CMake dependency as many configure checks and builds. depResults already records one build pass, keyed by the PcmDir of every dependency configuration built in it. An external dependency now goes in the same map, keyed by the directory its build leaves (ExternalBuildDir: the clone, or for CMake its build directory). The first configuration that needs it fetches and builds it; the others wait for that and take its outcome. Keying on the build directory rather than the clone keeps a CMake declaration from taking the outcome of a header-only declaration of the same clone, which only fetched it. A fresh depResults is a fresh pass, so the next build still pulls, and every caller of Build() gets this: RunTests, a plain build, and the pragma dependencies LoadProject builds. To make that possible BuildExternal is split. BuildExternal is now the shared work, lock, fetch, configure and build, and returns the error. ExternalOutputs computes what one declaration compiles and links with, because two declarations of one clone can differ in includeDirs, libDirs and libs; ExternalBuildResult loses its error field. The work no longer stops between steps when the configuration that started it is cancelled: others may be waiting on it, and a run cut short would leave them a clone nobody fetched. A dependency configuration's Build() already runs to completion for the same reason. The promise and future the dependency threads used to build or wait for a configuration are now one helper, OncePerPass, which both kinds use. An exception thrown in BuildExternal used to escape its thread into std::terminate; it is now a build error, as for a dependency. With eight tests that share a local dependency, a test run fetched it eight times before this and fetches it once now (SharedExternalFetch, next commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 02:09:24 +01:00
std::string error;
try {
// Every configuration of the pass that declares this
// dependency shares one fetch and build of it. A project's
// tests often each copy its externalDependencies, and RunTests
// builds them side by side: each of them pulling the clone
// cost a network round trip per test, and before the clone had
// a lock, the simultaneous pulls broke its FETCH_HEAD and
// failed every test (issue #34).
const std::shared_future<BuildResult> built = OncePerPass(depResults, depMutex, ExternalBuildDir(dep, config.target), [&] {
return BuildResult{BuildExternal(dep, config.target), false, {}};
});
error = built.get().result;
if (error.empty()) externalResults[i] = ExternalOutputs(dep, config.target);
} catch (const std::exception& e) {
error = std::format("external dep {} threw: {}", dep.name, e.what());
}
if (error.empty()) return;
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(error);
2026-04-27 07:04:42 +02:00
}
});
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
fs::path stdPcmDir = StdPcmDir(config);
2026-04-23 01:57:25 +02:00
if (!fs::exists(stdPcmDir)) {
fs::create_directories(stdPcmDir);
}
2026-04-29 03:27:11 +02:00
std::string stdPcmResult;
{
Progress::Task task(std::format("Building std PCM ({}-{})", config.target, config.march));
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
// The shader and external threads are running: an exception leaving
// here would destroy them joinable, which terminates the process.
try {
stdPcmResult = BuildStdPcm(config, stdPcmDir/"std.pcm");
} catch (const std::exception& e) {
stdPcmResult = std::format("building the std module for {}: {}", config.target, e.what());
}
2026-04-29 03:27:11 +02:00
}
2026-04-23 01:57:25 +02:00
if(!stdPcmResult.empty()) {
2026-04-27 07:04:42 +02:00
buildCancelled.store(true);
for(std::thread& thread : threads) thread.join();
for(std::thread& thread : externalThreads) thread.join();
return {stdPcmResult, false, {}};
2026-04-23 01:57:25 +02:00
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
fs::path pcmDir = config.PcmDir();
2026-04-23 01:57:25 +02:00
2026-04-27 07:04:42 +02:00
fs::copy_file(stdPcmDir/"std.pcm", pcmDir/"std.pcm", fs::copy_options::update_existing);
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
// Everything about the compile command that is a pure function of the
// Configuration is assembled by GetCompileCommand, so that anything which
// needs to PARSE these sources with the same flags — the linter's AST
// layer — cannot drift from what actually built the PCMs. Build appends
// only what depends on work having happened: dependency public flags and
// external dependency flags, further down.
CompileCommand compile = GetCompileCommand(config);
std::string command = compile.command;
const std::string& includeFlags = compile.includeFlags;
const std::string& defineFlags = compile.defineFlags;
const std::string& userFlags = compile.userFlags;
const std::string& ltoCompileFlags = compile.ltoCompileFlags;
const std::string& ltoLinkFlags = compile.ltoLinkFlags;
const bool isWasm = config.target.starts_with("wasm32");
2026-04-23 01:57:25 +02:00
std::string files;
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
std::set<std::string> libSet;
std::set<std::string> publicFlagSet;
2026-04-23 01:57:25 +02:00
std::mutex fileMutex;
std::vector<std::thread> depThreads;
2026-04-27 07:04:42 +02:00
depThreads.reserve(config.dependencies.size());
2026-04-23 01:57:25 +02:00
std::atomic<bool> repack(false);
for(Configuration* dep : config.dependencies) {
2026-04-27 07:04:42 +02:00
depThreads.emplace_back([&, dep](){
try {
if (buildCancelled.load(std::memory_order_relaxed)) return;
fix(external): fetch and build each clone once per build pass `crafter-build test` builds a project's tests side by side, and a project whose tests each copy its externalDependencies hands every one of them the same unpinned dependency. Each test's Build() pulled the shared clone. Before the clone had a lock, those pulls ran at the same time and broke its .git/FETCH_HEAD, which failed every test of Crafter.Graphics (issue #34). Since 653dd0b they wait for each other, but every one is still a network round trip: 26 tests cost 26 pulls of each dependency, one after another, and for a CMake dependency as many configure checks and builds. depResults already records one build pass, keyed by the PcmDir of every dependency configuration built in it. An external dependency now goes in the same map, keyed by the directory its build leaves (ExternalBuildDir: the clone, or for CMake its build directory). The first configuration that needs it fetches and builds it; the others wait for that and take its outcome. Keying on the build directory rather than the clone keeps a CMake declaration from taking the outcome of a header-only declaration of the same clone, which only fetched it. A fresh depResults is a fresh pass, so the next build still pulls, and every caller of Build() gets this: RunTests, a plain build, and the pragma dependencies LoadProject builds. To make that possible BuildExternal is split. BuildExternal is now the shared work, lock, fetch, configure and build, and returns the error. ExternalOutputs computes what one declaration compiles and links with, because two declarations of one clone can differ in includeDirs, libDirs and libs; ExternalBuildResult loses its error field. The work no longer stops between steps when the configuration that started it is cancelled: others may be waiting on it, and a run cut short would leave them a clone nobody fetched. A dependency configuration's Build() already runs to completion for the same reason. The promise and future the dependency threads used to build or wait for a configuration are now one helper, OncePerPass, which both kinds use. An exception thrown in BuildExternal used to escape its thread into std::terminate; it is now a build error, as for a dependency. With eight tests that share a local dependency, a test run fetched it eight times before this and fetches it once now (SharedExternalFetch, next commit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 02:09:24 +01:00
const std::shared_future<BuildResult> built = OncePerPass(depResults, depMutex, dep->PcmDir(), [&] { return Build(*dep, depResults, depMutex); });
const BuildResult& result = built.get();
2026-04-27 07:04:42 +02:00
fileMutex.lock();
for (const std::string& lib : result.libs) libSet.insert(lib);
2026-05-01 19:02:14 +02:00
for (const std::string& f : result.publicCompileFlags) publicFlagSet.insert(f);
2026-04-27 07:04:42 +02:00
fileMutex.unlock();
if (!result.result.empty()) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = result.result;
2026-04-23 01:57:25 +02:00
}
2026-04-27 07:04:42 +02:00
}
if (result.repack) {
repack = true;
}
} catch (const std::exception& e) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::format("dep build for {} threw: {}", dep->path.string(), e.what());
2026-04-23 01:57:25 +02:00
}
}
2026-04-27 07:04:42 +02:00
});
2026-04-23 01:57:25 +02:00
}
refactor: extract GetCompileCommand and StdPcmDir out of Build The clang invocation was assembled inline across four regions of Build, interleaved with the dependency-graph walk, so nothing else could ask "what flags does this Configuration compile with". The linter's AST layer needs exactly that, and it cannot approximate it: a precompiled module is rejected outright by a translation unit whose target features differ from the one that wrote it. Dropping just -march=native produces hundreds of "compiled with the target feature '+avx512bw' but the current translation unit is not" errors and no usable parse, so reconstructed flags fail hard rather than degrade. GetCompileCommand is the config-pure part: target, arch, standard, configuration defines, module search paths, includes, user compileFlags, optimisation and LTO. Build appends only what depends on work having happened — dependency public flags and external dependency flags. The sub-strings it also needs on their own (includes, defines, user flags, LTO) come back as struct members, so the .c compile path is unchanged. Verified by probing `command` at the equivalent point before and after and diffing: byte-identical across all 23 configurations exercised by a full build plus the test suite. Two incidental simplifications fell out. pcmDir was recomputing what Configuration::PcmDir() already returns, and cmakeBuildType is now a one-liner. GetCompileCommand is also most of what a compile_commands.json would need, which this repo lacks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:15:20 +02:00
// Only the CMake build type is still derived here; the compile flags it
// used to sit alongside now come from GetCompileCommand.
const std::string cmakeBuildType = config.debug ? "Debug" : "Release";
2026-04-23 01:57:25 +02:00
2026-05-18 05:23:11 +02:00
// Same target-aware setup as the C++ compile path (line 459-): wasm32
// rejects -march, silently ignores -mtune, and needs --sysroot to find
// wasi-libc headers. Build the prefix once so all C compiles share it.
fix(wasm): what the review found -flto in a wasm configuration's compileFlags or linkFlags reached the link, where LTO code generation runs without the backend options that keep exception handling: every landing pad was dropped, and the build succeeded with throws that escaped their catch blocks. -fno-exceptions there made every `import std;` fail on the std module built with exceptions, and a wasm shared library failed to link against the eh/ libc++, which is static archives built without -fPIC. Build() refuses all three now, each with a line that says why, before it makes a directory or starts a thread. The sysroot check sat in BuildStdPcm, which the linter calls as well. There the error only dropped the compile command, so lint ran no AST rule on a wasm configuration and reported it clean. It also refused static libraries, which never link, fired after the shader and external threads had started, and called a missing sysroot, or one without the target, "no C++ exception support". It is part of the same early check now, which first says when the sysroot or the target's directory is missing, and asks for libunwind only where something links. C sources got only the signal.h define. setjmp.h refused to compile in C, and a C object given -wasm-enable-sjlj through compileFlags used the legacy exception encoding next to C++ objects in the standard one, a mix engines refuse to load. They get WasmCompileFlags now, like the C++ units, a variant pass's flags included. The baseline probes include simd: V8 and SpiderMonkey switch SIMD off on x86 CPUs without SSE4.1, which virtual machines often present, and such an engine passed the exception-handling probe only to fail compiling the module; the manifest tests expect it on every entry. The Flatpak pins WASI_SDK_EXCEPTIONS=DUAL, which wasi-sdk 34 defaults to only with clang 23 or newer, and checks that the eh/ libunwind got installed: the wasm tests skip without it, so CI would otherwise pass a Flatpak whose every wasm build fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:10:41 +01:00
std::string cArchFlags = isWasm
2026-05-18 05:23:11 +02:00
? std::string()
: std::format(" -march={} -mtune={}", config.march, config.mtune);
if (!config.sysroot.empty()) {
cArchFlags += std::format(" --sysroot={}", config.sysroot);
fix(mingw): view only a prefix that holds a Linux system The first fix showed clang every mingw prefix through a view of its mingw-w64 parts. That broke a prefix that holds mingw-w64 alone, where Windows libraries installed with --prefix land in include/ and lib/: with CRAFTER_MINGW_DIR=/opt/mingw, <SDL2/SDL.h> and -lSDL2 were no longer found. Such a prefix is clang's sysroot whole again. Only a prefix that also holds a Linux system gets the view (Toolchain::mingwPrefixIsSystem): the host's /usr inside the Flatpak, whatever its include/ holds, and a prefix whose include/ has a C library's features.h, which every glibc and musl header includes and mingw-w64 has no use for, as CRAFTER_MINGW_DIR=/usr has. MakeMingwSysroot makes the view in the cache, or in $XDG_RUNTIME_DIR/crafter.build when the cache's filesystem takes no symbolic links (vfat, exFAT, CIFS without unix extensions): the XDG spec promises them there, and nothing else in the cache needs them. Its path is absolute, because CMake runs the compiler from directories of its own and a relative cache did not resolve from there. The prefix is resolved once, absolute and lexically normal, so /opt/mingw/ and /opt/mingw are one prefix and not two builds. clang names a header by the path it found it at, so every diagnostic from a viewed mingw-w64 header named a hashed cache directory. RunBuildCommand now writes paths into the view as paths into the prefix, as BuildStdPcm did for its manifest messages already. Also from the review of PR #44: SysrootCompileFlags builds a mingw command, which can make a view, so it redirects the cache like the other tests; the manifest filter's comment names what it still keeps out; the C path asks MingwSysroot() once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:58:30 +01:00
} else if (config.target == "x86_64-w64-mingw32") {
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
// The C++ side gets it through GetBaseCommand.
fix(mingw): view only a prefix that holds a Linux system The first fix showed clang every mingw prefix through a view of its mingw-w64 parts. That broke a prefix that holds mingw-w64 alone, where Windows libraries installed with --prefix land in include/ and lib/: with CRAFTER_MINGW_DIR=/opt/mingw, <SDL2/SDL.h> and -lSDL2 were no longer found. Such a prefix is clang's sysroot whole again. Only a prefix that also holds a Linux system gets the view (Toolchain::mingwPrefixIsSystem): the host's /usr inside the Flatpak, whatever its include/ holds, and a prefix whose include/ has a C library's features.h, which every glibc and musl header includes and mingw-w64 has no use for, as CRAFTER_MINGW_DIR=/usr has. MakeMingwSysroot makes the view in the cache, or in $XDG_RUNTIME_DIR/crafter.build when the cache's filesystem takes no symbolic links (vfat, exFAT, CIFS without unix extensions): the XDG spec promises them there, and nothing else in the cache needs them. Its path is absolute, because CMake runs the compiler from directories of its own and a relative cache did not resolve from there. The prefix is resolved once, absolute and lexically normal, so /opt/mingw/ and /opt/mingw are one prefix and not two builds. clang names a header by the path it found it at, so every diagnostic from a viewed mingw-w64 header named a hashed cache directory. RunBuildCommand now writes paths into the view as paths into the prefix, as BuildStdPcm did for its manifest messages already. Also from the review of PR #44: SysrootCompileFlags builds a mingw command, which can make a view, so it redirects the cache like the other tests; the manifest filter's comment names what it still keeps out; the C path asks MingwSysroot() once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:58:30 +01:00
if (const fs::path& sysroot = MingwSysroot(); !sysroot.empty()) cArchFlags += std::format(" --sysroot={}", sysroot.string());
2026-05-18 05:23:11 +02:00
}
fix(wasm): what the review found -flto in a wasm configuration's compileFlags or linkFlags reached the link, where LTO code generation runs without the backend options that keep exception handling: every landing pad was dropped, and the build succeeded with throws that escaped their catch blocks. -fno-exceptions there made every `import std;` fail on the std module built with exceptions, and a wasm shared library failed to link against the eh/ libc++, which is static archives built without -fPIC. Build() refuses all three now, each with a line that says why, before it makes a directory or starts a thread. The sysroot check sat in BuildStdPcm, which the linter calls as well. There the error only dropped the compile command, so lint ran no AST rule on a wasm configuration and reported it clean. It also refused static libraries, which never link, fired after the shader and external threads had started, and called a missing sysroot, or one without the target, "no C++ exception support". It is part of the same early check now, which first says when the sysroot or the target's directory is missing, and asks for libunwind only where something links. C sources got only the signal.h define. setjmp.h refused to compile in C, and a C object given -wasm-enable-sjlj through compileFlags used the legacy exception encoding next to C++ objects in the standard one, a mix engines refuse to load. They get WasmCompileFlags now, like the C++ units, a variant pass's flags included. The baseline probes include simd: V8 and SpiderMonkey switch SIMD off on x86 CPUs without SSE4.1, which virtual machines often present, and such an engine passed the exception-handling probe only to fail compiling the module; the manifest tests expect it on every entry. The Flatpak pins WASI_SDK_EXCEPTIONS=DUAL, which wasi-sdk 34 defaults to only with clang 23 or newer, and checks that the eh/ libunwind got installed: the wasm tests skip without it, so CI would otherwise pass a Flatpak whose every wasm build fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:10:41 +01:00
if (isWasm) {
// The C++ path's wasm flags, all of them, a variant pass's included.
// With only the signal.h define, wasi-libc's setjmp.h refused to
// compile in C, and a C object given -wasm-enable-sjlj through
// compileFlags used LLVM's legacy exception encoding next to C++
// objects in the standard one, a mix engines refuse to load.
cArchFlags += WasmCompileFlags(config);
2026-05-18 05:23:11 +02:00
}
2026-04-27 07:04:42 +02:00
for (const fs::path& cFile : config.cFiles) {
2026-04-23 01:57:25 +02:00
files += std::format(" {}_source.o ", (buildDir / cFile.filename()).string());
2026-07-23 01:24:42 +02:00
const std::string objPath = std::format("{}_source.o", (buildDir / cFile.filename()).string());
const std::string srcPath = std::format("{}.c", cFile.string());
// C sources carry their dependencies entirely in #includes, so the
// depfile the compile below writes is the whole record of what this
// object was built from beyond the .c itself.
if (!fs::exists(objPath) || (fs::exists(srcPath) && fs::last_write_time(srcPath) > fs::last_write_time(objPath)) || NewestPrerequisite(std::format("{}.d", objPath)) > fs::last_write_time(objPath)) {
2026-06-08 19:28:20 +02:00
threads.emplace_back([&cFile, &buildDir, &buildError, &buildCancelled, &config, &includeFlags, &defineFlags, &userFlags, &cArchFlags, &ltoCompileFlags]() {
2026-04-29 03:27:11 +02:00
Progress::Task task(std::format("Compiling {}.c", cFile.filename().string()));
2026-04-23 01:57:25 +02:00
if (buildCancelled.load(std::memory_order_relaxed)) return;
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
std::string result = RunBuildCommand(std::format("{8} {0}.c --target={1}{2} -O3{3} -c{4}{5}{6} -MD -MF {7}_source.o.d -o {7}_source.o", cFile.string(), config.target, cArchFlags, ltoCompileFlags, includeFlags, defineFlags, userFlags, (buildDir / cFile.filename()).string(), GetToolchain().clang.string()));
2026-04-23 01:57:25 +02:00
if (result.empty()) return;
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result);
}
});
}
}
for (const fs::path& cFile : config.cuda) {
files += std::format(" {}_source.o ", (buildDir / cFile.filename()).string());
2026-07-23 01:24:42 +02:00
const std::string objPath = std::format("{}_source.o", (buildDir / cFile.filename()).string());
const std::string srcPath = std::format("{}.cu", cFile.string());
// Still .cu-mtime only: the header tracking the C and C++ paths get
// comes from asking the compiler for a depfile, and nvcc's -MD/-MF are
// untested here (no CUDA toolkit in CI), so a .cu that only sees a
// changed header does not rebuild yet.
2026-04-30 02:20:19 +02:00
if (!fs::exists(objPath) || (fs::exists(srcPath) && fs::last_write_time(srcPath) > fs::last_write_time(objPath))) {
2026-04-23 01:57:25 +02:00
threads.emplace_back([&cFile, &buildDir, &buildError, &buildCancelled]() {
2026-04-29 03:27:11 +02:00
Progress::Task task(std::format("Compiling {}.cu", cFile.filename().string()));
2026-04-23 01:57:25 +02:00
if (buildCancelled.load(std::memory_order_relaxed)) return;
std::string result = RunBuildCommand(std::format("nvcc {}.cu -c -o {}_source.o -O3 -arch=sm_89", cFile.string(), (buildDir / cFile.filename()).string()));
2026-04-23 01:57:25 +02:00
if (result.empty()) return;
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result);
}
});
}
}
for(std::thread& thread : depThreads) {
thread.join();
}
2026-04-27 07:04:42 +02:00
for(std::thread& thread : externalThreads) {
thread.join();
}
2026-04-23 01:57:25 +02:00
if(buildCancelled.load()) {
2026-04-27 07:04:42 +02:00
for(std::thread& thread : threads) thread.join();
2026-04-23 01:57:25 +02:00
return {buildError, false, {}};
}
2026-05-02 21:08:51 +02:00
// Ship runtime artifacts from transitive deps' bin dirs alongside the
// executable: compiled .spv files (cfg.shaders) and asset files/dirs
// (cfg.files). The lib already mirrored these into its own bin dir
// during its build, but a consumer exe loads them from its own dir at
// runtime. Only an exe is a deployment unit; intermediate libs don't
// need to forward since the exe walks all transitive deps. Runs outside
// the repack gate because the relink mtime check above only watches
// .so/.dll/.a, so a shader/file-only change in a dep wouldn't trigger
// repack but still needs the new artifact copied across.
// (cfg.buildFiles use a different mechanism — they're exposed to shader
// compiles as #include search paths in place, no copy.)
2026-05-01 19:16:13 +02:00
if (config.type == ConfigurationType::Executable) {
try {
2026-05-02 21:08:51 +02:00
auto copyTree = [](const fs::path& src, const fs::path& dest) {
if (fs::is_directory(src)) {
for (const auto& entry : fs::recursive_directory_iterator(src)) {
fs::path rel = fs::relative(entry.path(), src);
fs::path destPath = dest / rel;
if (entry.is_directory()) {
if (!fs::exists(destPath)) fs::create_directories(destPath);
} else if (entry.is_regular_file()) {
fs::create_directories(destPath.parent_path());
if (!fs::exists(destPath)) {
fs::copy_file(entry.path(), destPath);
} else if (fs::last_write_time(entry.path()) > fs::last_write_time(destPath)) {
fs::copy_file(entry.path(), destPath, fs::copy_options::overwrite_existing);
}
}
}
} else {
if (!fs::exists(dest)) {
fs::copy_file(src, dest);
} else if (fs::last_write_time(src) > fs::last_write_time(dest)) {
2026-05-01 19:16:13 +02:00
fs::copy_file(src, dest, fs::copy_options::overwrite_existing);
}
}
};
2026-05-02 21:08:51 +02:00
std::unordered_set<Configuration*> seen;
std::function<void(Configuration*)> forwardDepArtifacts = [&](Configuration* dep) {
if (!seen.insert(dep).second) return;
fs::path depBinDir = dep->BinDir();
for (const Shader& shader : dep->shaders) {
feat(shader): per-entry defines and a named output variant Crafter.Graphics has to ship each of its compute shaders in two builds of one source: one that indexes VK_EXT_descriptor_heap and one that reads the same slots from a descriptor set, for drivers without the extension. A Shader was a path, an entry point and a stage, and its output name was derived from the path alone, so one source could only be compiled once. Shader gains `defines`, spelled like the compiler's -D ("NAME" is defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a preamble, and `variant`, which names the output: "ui.comp.glsl" with variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one place that name is formed; Check, Compile and the forwarding of a dependency's shaders into the consumer's bin dir all use it. Two fields make one mistake easy: defines set, variant forgotten. Both entries then compiled in parallel into the same file, the build reported success with whichever finished last, and Check() called both up to date from then on. The same has always been true of a.vert next to a.frag. Build() now refuses a configuration in which two entries share an output name and says which. A define without a name, or one spanning lines, and a variant with a path separator are rejected by name. Left to glslang the first two surface as "-1:<line>" with no file, and the last as a failed open after the whole compile. Not addressed: Check() still compares only the source's mtime, so a change to the defines, or to an included file, does not recompile (#35). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 16:50:41 +02:00
fs::path src = depBinDir / shader.OutputName();
2026-05-02 21:08:51 +02:00
if (!fs::exists(src)) continue;
copyTree(src, outputDir / src.filename());
}
for (const fs::path& additionalFile : dep->files) {
fs::path src = depBinDir / additionalFile.filename();
if (!fs::exists(src)) continue;
copyTree(src, outputDir / additionalFile.filename());
}
2026-05-12 01:16:40 +02:00
for (const fs::path& asset : dep->assets) {
2026-05-12 03:44:14 +02:00
// Directory entry: the dep already mirrored the
// (compressed + passthrough) tree under
// depBinDir/<asset.filename()>/. Forward it wholesale
// so our bin dir gets the same layout.
if (fs::is_directory(asset)) {
fs::path src = depBinDir / asset.filename();
if (!fs::exists(src)) continue;
copyTree(src, outputDir / asset.filename());
continue;
}
2026-05-12 01:16:40 +02:00
std::string ext = asset.extension().string();
2026-07-23 01:24:42 +02:00
for (char& c : ext) c = static_cast<char>(std::tolower(static_cast<std::uint8_t>(c)));
2026-05-12 01:16:40 +02:00
fs::path srcName = asset.filename();
2026-05-19 00:50:06 +02:00
if (ext == ".png" || ext == ".tga" || ext == ".jpg" || ext == ".jpeg" || ext == ".bmp") {
srcName.replace_extension(".ctex");
} else if (ext == ".obj") {
srcName.replace_extension(".cmesh");
} else {
continue;
}
2026-05-12 01:16:40 +02:00
fs::path src = depBinDir / srcName;
if (!fs::exists(src)) continue;
copyTree(src, outputDir / srcName);
}
2026-05-02 21:08:51 +02:00
for (Configuration* sub : dep->dependencies) forwardDepArtifacts(sub);
};
for (Configuration* dep : config.dependencies) forwardDepArtifacts(dep);
2026-05-01 19:16:13 +02:00
} catch (const fs::filesystem_error& e) {
for(std::thread& thread : threads) thread.join();
return {e.what(), false, {}};
}
}
2026-04-27 07:04:42 +02:00
if(repack.load()) {
buildResult.repack = true;
2026-04-23 01:57:25 +02:00
}
2026-04-27 07:04:42 +02:00
buildResult.libs = std::move(libSet);
for(const std::string& flag : config.linkFlags) {
buildResult.libs.insert(flag);
}
2026-05-01 19:02:14 +02:00
// Public compile flags propagated from sub-deps. Add them to this build's
// command so config sees the headers its deps expose, and re-publish them
// so config's own consumers see them transitively.
2026-07-23 01:24:42 +02:00
for (const std::string& flag : publicFlagSet) command += std::format(" {}", flag);
2026-05-01 19:02:14 +02:00
buildResult.publicCompileFlags = std::move(publicFlagSet);
2026-04-27 07:04:42 +02:00
fs::file_time_type externalFloor = fs::file_time_type::min();
for(const ExternalBuildResult& ext : externalResults) {
for(const std::string& flag : ext.compileFlags) {
2026-07-23 01:24:42 +02:00
command += std::format(" {}", flag);
2026-05-01 19:02:14 +02:00
// Headers a dep links via ExternalDependency are part of its
// public surface (its modules can include them in declarations
// visible to consumers), so propagate the -I to consumers.
buildResult.publicCompileFlags.insert(flag);
2026-04-23 01:57:25 +02:00
}
2026-04-27 07:04:42 +02:00
for(const std::string& flag : ext.linkFlags) {
buildResult.libs.insert(flag);
2026-04-23 01:57:25 +02:00
}
2026-04-27 07:04:42 +02:00
if (ext.latestArtifact > externalFloor) externalFloor = ext.latestArtifact;
2026-04-23 01:57:25 +02:00
}
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
// Check every interface before compiling any of them. Module::Compile waits
// on the `compiled` flag of each sibling module it imports, and that flag is
// set either by a Compile that runs or by the Check that decides none is
// needed — so a Check still pending while another module's thread is already
// waiting would block on a flag nothing goes on to raise.
std::vector<Module*> staleInterfaces;
2026-04-27 07:04:42 +02:00
for(std::unique_ptr<Module>& interface : config.interfaces) {
if(interface->Check(pcmDir, externalFloor)) {
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
staleInterfaces.push_back(interface.get());
2026-04-23 01:57:25 +02:00
buildResult.repack = true;
}
2026-04-27 07:04:42 +02:00
files += std::format(" {}/{}.o", buildDir.string(), interface->path.filename().string());
for(std::unique_ptr<ModulePartition>& part : interface->partitions) {
2026-04-23 01:57:25 +02:00
files += std::format(" {}/{}.o", buildDir.string(), part->path.filename().string());
}
}
fix: track what a primary module interface imports A primary module interface unit — `export module Widget;`, no partitions — recorded nothing about what it imported. GetInterfacesAndImplementations registered the Module and then erased the file from the scan list, so the import pass only ever saw partitions, and Module had no vectors to hold an edge anyway. Two consequences, both reported as issue #26: Module::Check consulted only its own .cppm and its partitions. A data member added to an imported module left Widget.pcm, Widget.o and every consumer object untouched while the imported library rebuilt and both binaries relinked — one executable holding two class layouts, no diagnostic, and a crash somewhere unrelated. Wiping build/ was the only cure, so `crafter-build test` could not be trusted straight after an interface edit. Module::Compile waited on nothing. Two modules in one Configuration compile on concurrent threads, so a primary interface importing a sibling was a coin flip between working and "module 'Base' not found". Partitions never had either problem — they carry the same three vectors and Check/Compile honour them — which is why the gap only surfaced on a module whose interface is one flat unit. Module now carries moduleDependencies, externalModuleDependencies and pendingImports with the same meanings as on ModulePartition; primary units stay in the scan list so their imports land there; Check sees through them; Compile orders itself behind a local sibling; and ResolvePendingImports sweeps them so an edge survives dependencies being wired up afterwards. Build() now Checks every interface before spawning any compile thread — the `compiled` flag a waiter blocks on is raised either by a Compile that runs or by the Check that decides none is needed, so a Check still pending while another module's thread waits would have hung the build. Resolves #26 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-30 18:19:04 +00:00
for(Module* mod : staleInterfaces) {
threads.emplace_back([mod, &command, &pcmDir, &buildDir, &buildCancelled, &buildError]() {
Progress::Task task(std::format("Compiling interface {}", mod->path.filename().string()));
try {
mod->Compile(command, pcmDir, buildDir, buildCancelled, buildError);
} catch (const std::exception& e) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::format("Module::Compile threw: {}", e.what());
}
}
});
}
2026-04-27 07:04:42 +02:00
for(Implementation& implementation : config.implementations) {
if(implementation.Check(buildDir, pcmDir, externalFloor)) {
2026-04-23 01:57:25 +02:00
buildResult.repack = true;
2026-04-27 07:04:42 +02:00
Implementation* impl = &implementation;
threads.emplace_back([impl, &command, &buildDir, &buildCancelled, &buildError]() {
2026-04-29 03:27:11 +02:00
Progress::Task task(std::format("Compiling {}.cpp", impl->path.filename().string()));
2026-04-27 07:04:42 +02:00
try {
impl->Compile(command, buildDir, buildCancelled, buildError);
} catch (const std::exception& e) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::format("Implementation::Compile threw: {}", e.what());
}
}
});
2026-04-23 01:57:25 +02:00
}
2026-04-27 07:04:42 +02:00
files += std::format(" {}/{}_impl.o", buildDir.string(), implementation.path.filename().string());
2026-04-23 01:57:25 +02:00
}
for(std::thread& thread : threads) {
thread.join();
}
if(buildCancelled.load()) {
return {buildError, false, {}};
}
2026-04-27 07:04:42 +02:00
std::string linkExtras;
for(const std::string& flag : buildResult.libs) {
2026-07-23 01:24:42 +02:00
linkExtras += std::format(" {}", flag);
2026-04-27 07:04:42 +02:00
}
2026-06-08 19:28:20 +02:00
// Link-only LTO/section flags (empty in Debug and for wasm). Every exe/lib
// link below ends with linkExtras, so this reaches them all; the static-lib
// archive path uses ar instead and is handled separately.
linkExtras += ltoLinkFlags;
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
// Host links use the toolchain's own libc++. When the toolchain lives
// outside /usr (a bundled one) the binary needs an rpath to find
// libc++.so.1 at run time. mingw links libstdc++ and every sysroot build
// (wasm included) brings its own libc++.
if (config.target != "x86_64-w64-mingw32" && config.sysroot.empty()) {
linkExtras += GetToolchain().LibcxxLinkFlags();
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
// mingw uses libstdc++; C++26 std::print/format extras live in libstdc++exp.
// libstdc++ on mingw uses winpthreads for std::atomic_wait /
// counting_semaphore / stop_token, so -lpthread is required as soon as
// those primitives appear (they do, transitively, in any non-trivial std
// import). -static-libstdc++ bundles libstdc++ into the exe so we don't
// chase libstdc++-6.dll TLS symbol mismatches across mingw versions and
// the resulting binary stands alone. Auto-link so user projects don't
// carry boilerplate.
if (config.target == "x86_64-w64-mingw32") {
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
// Static-link libstdc++/libgcc/libwinpthread so the resulting .exe
// or .dll doesn't depend on a specific runtime DLL being on the
// consumer's PATH. The mingw runtime ABI varies subtly between
// distributions (Arch UCRT vs msys2 UCRT vs msys2 MSVCRT) and
// STATUS_ENTRYPOINT_NOT_FOUND at LoadLibrary time is the symptom
// of a mismatch. Static linkage trades binary size for portability.
// The -Bstatic/-Bdynamic bracketing forces -lpthread to resolve
// against libwinpthread.a rather than the import lib; everything
// else (KERNEL32, UCRT) stays dynamic. -lstdc++exp adds C++26
// std::print/format extras. -femulated-tls is already on the
// compile so __once_callable et al resolve in static libstdc++.
linkExtras += " -lstdc++exp -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread -Wl,-Bdynamic";
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
}
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
// Code built with -fwasm-exceptions calls into libunwind (the
// __cpp_exception tag, the landing-pad context, _Unwind_RaiseException),
// and -wasm-enable-sjlj turns setjmp and longjmp into calls into
// wasi-libc's libsetjmp (__wasm_setjmp, __wasm_longjmp, the __c_longjmp
// tag). The driver adds neither to the link the way it adds libc++. A
// static archive contributes only what something references, so a
// program without setjmp gets nothing from libsetjmp, and every sysroot
// that passes WasmBuildError ships both.
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
if (isWasm) {
linkExtras += " -lunwind -lsetjmp";
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
// Force a relink if the expected output is missing or older than any dep
// artifact. Missing covers: previous build produced a different outputName,
// or the binary was deleted by hand. Older-than-dep covers: dep's library
// was rebuilt by an earlier run (so dep.repack is false this time around)
// but the consumer was never relinked against the new dep.
{
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
fs::path expected = ExpectedOutput(config);
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
if (!fs::exists(expected)) {
buildResult.repack = true;
} else {
auto consumerMtime = fs::last_write_time(expected);
for (Configuration* dep : config.dependencies) {
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
fs::path depArtifact = ExpectedOutput(*dep);
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
if (fs::exists(depArtifact) && fs::last_write_time(depArtifact) > consumerMtime) {
buildResult.repack = true;
break;
}
}
2026-05-19 16:53:24 +02:00
// Also relink if any .o this archive bundles is newer than the
// archive itself. Covers a build that compiled the .o but never
// reached the link step (interrupt, crash, or a source touched
// after compile but before link): on the next run the .cpp is
// already ≤ .o so Implementation/Module Check returns false and
// nothing else would notice the archive is stale.
if (!buildResult.repack) {
auto objNewer = [&](const fs::path& obj) {
std::error_code ec;
auto t = fs::last_write_time(obj, ec);
return !ec && t > consumerMtime;
};
for (const std::unique_ptr<Module>& iface : config.interfaces) {
2026-07-23 01:24:42 +02:00
if (objNewer(buildDir / std::format("{}.o", iface->path.filename().string()))) {
2026-05-19 16:53:24 +02:00
buildResult.repack = true;
break;
}
bool partHit = false;
for (const std::unique_ptr<ModulePartition>& part : iface->partitions) {
2026-07-23 01:24:42 +02:00
if (objNewer(buildDir / std::format("{}.o", part->path.filename().string()))) {
2026-05-19 16:53:24 +02:00
partHit = true;
break;
}
}
if (partHit) { buildResult.repack = true; break; }
}
if (!buildResult.repack) {
for (const Implementation& impl : config.implementations) {
2026-07-23 01:24:42 +02:00
if (objNewer(buildDir / std::format("{}_impl.o", impl.path.filename().string()))) {
2026-05-19 16:53:24 +02:00
buildResult.repack = true;
break;
}
}
}
// C and CUDA objects are archive members too, and their compile
// loop above is the only thing that knows it rebuilt one — it
// sets no repack flag, so without this a .c edit recompiled the
// object and then linked nothing.
if (!buildResult.repack) {
auto sourceObjNewer = [&](const std::vector<fs::path>& sources) {
for (const fs::path& source : sources) {
if (objNewer(buildDir / std::format("{}_source.o", source.filename().string()))) return true;
}
return false;
};
if (sourceObjNewer(config.cFiles) || sourceObjNewer(config.cuda)) {
buildResult.repack = true;
}
}
2026-05-19 16:53:24 +02:00
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
}
}
2026-04-27 07:04:42 +02:00
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
// The exact command the link (or archive) step runs, kept beside the
// objects once it has succeeded. A different one relinks even with no
// object newer than the output: a link flag changed in project.cpp, or a
// dependency's library moved, as an ExternalDependency's does when its
// options change and it is built into a new cache directory.
std::string linkCommand;
if (config.type == ConfigurationType::Executable) {
if (config.target.starts_with("wasm32")) {
linkCommand = std::format("{}{} -o {}.wasm -fuse-ld=lld{}", command, files, (outputDir/config.outputName).string(), linkExtras);
} else {
linkCommand = std::format("{}{} -o {} -fuse-ld=lld{}", command, files, (outputDir/config.outputName).string(), linkExtras);
}
} else if (config.type == ConfigurationType::LibraryStatic) {
// ThinLTO emits LLVM bitcode objects; plain `ar` writes an archive
// index that omits their symbols, so a consumer's lld link can't
// pull the needed members. llvm-ar writes a bitcode-aware index
// and handles plain ELF and COFF objects just as well, so every
// build uses it.
linkCommand = std::format("{} rcs {}.a {}", GetToolchain().llvmAr.string(), (outputDir/fs::path(std::string("lib")+config.outputName)).string(), files);
} else if (config.target == "x86_64-w64-mingw32") {
// LibraryDynamic. Output names follow each target's convention so
// consumers can link via the standard linker search:
// mingw: <name>.dll + lib<name>.dll.a (lld --out-implib)
// unix: lib<name>.so (rpath $ORIGIN)
fs::path dll = outputDir / std::format("{}.dll", config.outputName);
fs::path implib = outputDir / std::format("lib{}.dll.a", config.outputName);
linkCommand = std::format("{}{} -shared -o {} -Wl,--out-implib,{} -fuse-ld=lld{}", command, files, dll.string(), implib.string(), linkExtras);
} else {
linkCommand = std::format("{}{} -shared -o {}.so -Wl,-rpath,'$ORIGIN' -fuse-ld=lld{}", command, files, (outputDir/(std::string("lib")+config.outputName)).string(), linkExtras);
}
const fs::path linkRecord = buildDir / std::format("{}.link", config.outputName);
if (!buildResult.repack) {
std::ifstream recorded(linkRecord, std::ios::binary);
const std::string previous((std::istreambuf_iterator<char>(recorded)), std::istreambuf_iterator<char>());
if (previous != linkCommand) buildResult.repack = true;
}
2026-04-23 01:57:25 +02:00
if(buildResult.repack) {
2026-04-29 03:27:11 +02:00
Progress::Task task(std::format("Linking {}", config.outputName));
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
// The record goes first, so a link that fails leaves it stale.
std::error_code removed;
fs::remove(linkRecord, removed);
2026-04-27 07:04:42 +02:00
if(config.type == ConfigurationType::Executable) {
if(config.target == "x86_64-w64-mingw32") {
try {
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
// Copy any LibraryDynamic dependency DLLs alongside
// the launcher exe — Windows resolves DLLs from the exe's
// own directory at load time, so this is the simplest
// equivalent of rpath $ORIGIN.
std::unordered_set<Configuration*> dllSeen;
std::function<void(Configuration*)> copyDepDlls = [&](Configuration* dep) {
if (!dllSeen.insert(dep).second) return;
if (dep->type == ConfigurationType::LibraryDynamic && dep->target == "x86_64-w64-mingw32") {
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
// Only the DLL: Windows resolves it from the exe's
// directory at load time. The import lib stays in
// the dependency's own bin dir, which is already
// on the link line through buildResult.libs.
fs::path src = dep->BinDir() / std::format("{}.dll", dep->outputName);
if (fs::exists(src)) {
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
fs::path dest = outputDir / src.filename();
if (!fs::exists(dest) || fs::last_write_time(src) > fs::last_write_time(dest)) {
fs::copy(src, dest, fs::copy_options::overwrite_existing);
}
2026-04-27 07:04:42 +02:00
}
}
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target Linux→mingw cross-compile now produces the same architectural shape as build.cmd (DLL + import lib + launcher exe) instead of a single static binary. The CI Windows artifact becomes a first-class drop-in: a user on Windows can run crafter-build.exe against any project.cpp and have it produce real Windows binaries — for either mingw or MSVC ABI. What changed: project.cpp: when target=mingw or target=msvc, crafter.build-lib is built as LibraryDynamic instead of LibraryStatic so the link emits a DLL + import lib (matching what build.cmd produces natively). Crafter.Build-Clang.cpp Build(): - LibraryDynamic now branches per target — mingw emits <name>.dll + lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll + <name>.lib via /IMPLIB; unix unchanged. - expectedOutputFor returns .dll for Windows-target dynamic libs. - Executable on Windows host now branches per target: mingw target uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps the existing path. Both auto-copy LibraryDynamic dep DLLs + import libs alongside the launcher exe (Windows resolves DLLs from the exe's own directory at load time). - Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so CRAFTER_API resolves to dllimport in their PCMs. - mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread so produced .exe/.dll don't depend on a particular libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole). Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which is now dead weight. - -r flag resolves to an absolute path before std::system, otherwise cmd.exe rejects "./bin/..." with "'.' is not recognized...". Crafter.Build-Platform.cpp: - Split the Windows-host block into shared shell helpers (#if MSVC || MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject / EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm. - Mingw-host LoadProject compiles project.cpp with --target=mingw, --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR), -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept /EXPORT:NAME), and links against libcrafter-build.dll.a from the launcher's directory. - Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target so a mingw-host crafter-build can also build msvc-target outputs (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when the user sets cfg.target = "x86_64-pc-windows-msvc". README adds a Quick start (Windows) section covering both build paths (native MSVC via build.cmd and the cross-compiled mingw artifact), documenting the msys2 UCRT toolchain prerequisite. Verified end-to-end on the winvm: - mingw target: cross-compiled crafter-build.exe builds hello-world's project.cpp, compiles main.cpp, links a hello.exe that runs without any custom PATH (only Windows system DLLs needed). - msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 02:23:42 +02:00
for (Configuration* sub : dep->dependencies) copyDepDlls(sub);
};
for (Configuration* dep : config.dependencies) copyDepDlls(dep);
2026-04-27 07:04:42 +02:00
} catch (const fs::filesystem_error& e) {
return {e.what(), false, {}};
}
}
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
}
buildResult.result = RunBuildCommand(linkCommand);
if (buildResult.result.empty()) {
std::ofstream(linkRecord, std::ios::binary | std::ios::trunc) << linkCommand;
2026-04-23 01:57:25 +02:00
}
}
2026-04-27 07:04:42 +02:00
if (config.type == ConfigurationType::LibraryStatic || config.type == ConfigurationType::LibraryDynamic) {
buildResult.libs.insert(std::format("-L{}", outputDir.string()));
buildResult.libs.insert(std::format("-l{}", config.outputName));
}
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
// Browser-wasm feature-detected variants. The baseline outputName.wasm is
// now built (above). For each declared variant, recompile the entire build
// graph with the variant's extra codegen flags and drop the result next to
// the baseline as outputName.<label>.wasm. Relaxed-SIMD (and friends) is
// per-translation-unit codegen, not a link switch, so a full rebuild — incl.
// dep libs and the std PCM — is the only correct way to produce it.
//
// Guard: only fires on the top-level config that *declared* wasmVariants
// (deps leave it empty) and never on a variant pass itself (those carry
// wasmVariantFlags), so there's no recursion. Skipped on build error.
2026-07-23 01:24:42 +02:00
if (buildResult.result.empty() && config.target.starts_with("wasm32") && config.type == ConfigurationType::Executable && !config.wasmVariants.empty() && config.wasmVariantFlags.empty()) {
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
fs::path baselineBin = config.BinDir();
// The whole transitive graph, so the variant flags reach every TU.
std::vector<Configuration*> graph;
{
std::unordered_set<Configuration*> graphSeen;
std::function<void(Configuration*)> collect = [&](Configuration* c) {
if (!c || !graphSeen.insert(c).second) return;
graph.push_back(c);
for (Configuration* dep : c->dependencies) collect(dep);
};
collect(&config);
}
for (const WasmVariant& variant : config.wasmVariants) {
// A labelless or flagless "variant" is just the baseline under
// another name — nothing extra to compile.
if (variant.label.empty() || variant.flags.empty()) continue;
2026-07-23 01:24:42 +02:00
Progress::Task task(std::format("Building wasm variant {} ({})", variant.label, config.outputName));
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
for (Configuration* c : graph) c->wasmVariantFlags = variant.flags;
fs::path variantBin = config.BinDir();
std::unordered_map<fs::path, std::shared_future<BuildResult>> variantDeps;
std::mutex variantMutex;
BuildResult vr = Build(config, variantDeps, variantMutex);
for (Configuration* c : graph) c->wasmVariantFlags.clear();
if (!vr.result.empty()) {
2026-07-23 01:24:42 +02:00
buildResult.result = std::format("wasm variant '{}' build failed: {}", variant.label, vr.result);
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
break;
}
std::error_code ec;
2026-07-23 01:24:42 +02:00
fs::path src = variantBin / (std::format("{}.wasm", config.outputName));
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
fs::path dst = baselineBin / std::format("{}.{}.wasm", config.outputName, variant.label);
fs::copy_file(src, dst, fs::copy_options::overwrite_existing, ec);
if (ec) {
2026-07-23 01:24:42 +02:00
buildResult.result = std::format("copy wasm variant '{}' {} -> {}: {}", variant.label, src.string(), dst.string(), ec.message());
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
break;
}
}
}
2026-04-23 01:57:25 +02:00
return buildResult;
2026-04-27 07:04:42 +02:00
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
void Crafter::EnableWasiBrowserRuntime(Configuration& cfg) {
fs::path runtimeDir = GetCrafterBuildHome() / "wasi-runtime";
fs::path runtimeJs = runtimeDir / "runtime.js";
fs::path htmlTemplate = runtimeDir / "index.html.in";
if (!fs::exists(runtimeJs) || !fs::exists(htmlTemplate)) {
2026-07-23 01:24:42 +02:00
throw std::runtime_error(std::format("wasi-runtime assets missing under {} (set CRAFTER_BUILD_HOME or reinstall)", runtimeDir.string()));
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
}
fs::path htmlOutDir = cfg.path / "build" / "wasi-runtime" / cfg.name;
fs::create_directories(htmlOutDir);
fs::path htmlPath = htmlOutDir / "index.html";
2026-05-18 05:23:11 +02:00
// Walk the dep graph for env-style JS bridges that need to load BEFORE
// runtime.js so they can populate `window.crafter_webbuild_env`. Any
// `*.js` entry in a (transitive) dep's `cfg.files` qualifies — the
// file is already going to be copied into the consumer's bin dir, we
// just need its basename for a `<script src=...>` tag.
std::vector<std::string> envScripts;
std::unordered_set<Configuration*> seen;
std::function<void(Configuration*)> walk = [&](Configuration* c) {
if (!c || !seen.insert(c).second) return;
for (const fs::path& f : c->files) {
if (f.extension() == ".js" && f.filename() != "runtime.js") {
std::string name = f.filename().string();
if (std::find(envScripts.begin(), envScripts.end(), name) == envScripts.end()) {
envScripts.push_back(std::move(name));
}
}
}
for (Configuration* dep : c->dependencies) walk(dep);
};
walk(&cfg);
2026-05-26 22:50:08 +02:00
// Per-build cache-busting token. Stamped onto every script src + the
// wasm URL so a regular browser reload sees fresh files even though
// the dev server (python -m http.server) sends no Cache-Control
// headers. Using ms-since-epoch is enough to be unique per build
// without invoking any version-control machinery.
2026-07-23 01:24:42 +02:00
const std::string buildId = std::to_string(std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count());
2026-05-26 22:50:08 +02:00
2026-05-18 05:23:11 +02:00
std::string envScriptTags;
for (const std::string& name : envScripts) {
2026-07-23 01:24:42 +02:00
envScriptTags += std::format(" <script src=\"{}?v={}\" type=\"module\"></script>\n", name, buildId);
2026-05-18 05:23:11 +02:00
}
// Walk the dep graph again for non-JS assets — these get pre-loaded by
// runtime.js into an in-memory VFS so the wasm's std::ifstream et al.
// can actually read them (the wasi-runtime in this repo otherwise
2026-05-19 03:28:27 +02:00
// stubs every fd syscall to zero).
//
// The manifest lists *relative paths* (e.g. "assets/Inter.ttf") so
// runtime.js's fetch() resolves against the bin-dir layout the asset
// copy step actually emits. The VFS is keyed by basename — path_open
// strips to basename on lookup, so subdir layouts collapse on the
// wasm side. Basename collisions across subdirs aren't supported on
// the wasi runtime today; if two assets share a basename, the last
// one preloaded wins. Avoid collisions in the source tree.
auto compressedExt = [](const fs::path& src) -> std::optional<std::string> {
2026-05-19 00:50:06 +02:00
std::string ext = src.extension().string();
2026-07-23 01:24:42 +02:00
for (char& c : ext) c = static_cast<char>(std::tolower(static_cast<std::uint8_t>(c)));
2026-05-19 00:50:06 +02:00
if (ext == ".png" || ext == ".tga" || ext == ".jpg" || ext == ".jpeg" || ext == ".bmp") {
2026-05-19 03:28:27 +02:00
return std::string(".ctex");
2026-05-19 00:50:06 +02:00
}
2026-05-19 03:28:27 +02:00
if (ext == ".obj") return std::string(".cmesh");
return std::nullopt;
};
auto compressedRel = [&](const fs::path& rel) -> fs::path {
if (auto ext = compressedExt(rel)) {
fs::path out = rel;
out.replace_extension(*ext);
return out;
}
return rel;
2026-05-19 00:50:06 +02:00
};
2026-05-18 05:23:11 +02:00
std::vector<std::string> assetFiles;
2026-05-19 03:28:27 +02:00
auto pushUnique = [&](std::string name) {
if (name.empty()) return;
if (std::find(assetFiles.begin(), assetFiles.end(), name) == assetFiles.end()) {
assetFiles.push_back(std::move(name));
}
};
2026-05-18 05:23:11 +02:00
seen.clear();
std::function<void(Configuration*)> walkAssets = [&](Configuration* c) {
if (!c || !seen.insert(c).second) return;
for (const fs::path& f : c->files) {
std::string ext = f.extension().string();
if (ext == ".js" || ext == ".html") continue;
if (f.filename() == "runtime.js") continue;
2026-05-19 03:28:27 +02:00
// cfg.files lands flat next to the .wasm by `name = filename()`.
pushUnique(f.filename().string());
2026-05-18 05:23:11 +02:00
}
2026-05-19 03:28:27 +02:00
// cfg.assets — mirror the bin-dir layout the build emits: a
// directory entry becomes <topName>/<rel inside dir>, single
// files land flat at the bin root. .png/.obj are compressed in
// place; everything else passes through under its original name.
2026-05-19 00:50:06 +02:00
for (const fs::path& a : c->assets) {
if (fs::is_directory(a)) {
2026-05-19 03:28:27 +02:00
const fs::path topName = a.filename();
2026-05-19 00:50:06 +02:00
std::error_code ec;
for (const auto& entry : fs::recursive_directory_iterator(a, ec)) {
if (ec) break;
if (!entry.is_regular_file()) continue;
2026-05-19 03:28:27 +02:00
fs::path rel = fs::relative(entry.path(), a);
pushUnique((topName / compressedRel(rel)).generic_string());
2026-05-19 00:50:06 +02:00
}
} else {
2026-05-19 03:28:27 +02:00
pushUnique(compressedRel(a.filename()).generic_string());
2026-05-19 00:50:06 +02:00
}
}
2026-05-18 05:23:11 +02:00
for (Configuration* dep : c->dependencies) walkAssets(dep);
};
walkAssets(&cfg);
fs::path manifestPath = htmlOutDir / "files.json";
{
std::ofstream m(manifestPath);
m << "[";
for (std::size_t i = 0; i < assetFiles.size(); ++i) {
if (i) m << ",";
m << "\"" << assetFiles[i] << "\"";
}
m << "]";
}
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
// Variant manifest. runtime.js fetches this, runs each entry's probes in
// order, and instantiates the first whose probes all pass — so the
// preferred (most feature-rich) variants come first and the baseline last
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
// as the fallback. Every entry carries WasmBaselineProbes besides its own:
// every file is built with the baseline flags, so an engine without
// exception handling can run none of them, and runtime.js tells it so
// instead of failing to compile the module. A variant with a non-empty
// label maps to outputName.<label>.wasm (produced by Build's variant
// driver); the baseline maps to outputName.wasm. Even with no declared
// variants we still emit a one-entry manifest so the runtime has a
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
// single, uniform code path.
auto jsonEscape = [](std::string_view s) {
std::string out;
for (char c : s) {
if (c == '"' || c == '\\') out += '\\';
out += c;
}
return out;
};
fs::path variantsPath = htmlOutDir / "variants.json";
{
std::ofstream v(variantsPath);
v << "[";
bool first = true;
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
auto emit = [&](const std::string& label, const std::vector<std::string>& ownProbes) {
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
if (!first) v << ",";
first = false;
std::string url = label.empty()
2026-07-23 01:24:42 +02:00
? std::format("{}.wasm", cfg.outputName)
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
: std::format("{}.{}.wasm", cfg.outputName, label);
feat(wasm): compile C++ with WebAssembly exception handling wasm builds were compiled with -fno-exceptions. A throw did not compile, libc++'s own throws (vector::at, std::format, filesystem) aborted the program, and code that relies on exceptions could not be built for the browser at all. Every wasm unit and the std module now get -fwasm-exceptions, and the link adds -lunwind, which code built that way calls into and the driver does not add. The flag also selects the libc++ under the sysroot's lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh one. Those libraries use the standardized encoding (try_table, exnref) while LLVM 23 still emits the legacy one by default, which wasmtime refuses to load and which would put both in one binary, so the flags ask for the standard form with -wasm-use-legacy-eh=false. That makes the setjmp lowering loadable under wasmtime too. The flags live in one constant, WasmBaselineFlags, which the compile command and the std module both take through WasmCompileFlags: a BMI is refused by a unit whose exception setting differs from its own. VariantId and StdPcmDir hash it, so the first build after the update starts fresh instead of importing PCMs built without exceptions, and two releases sharing the Flatpak's cache don't take turns rebuilding one std PCM. A sysroot without the exception-enabled libraries stops the build with a line saying so, rather than at the link with "unable to find library -lunwind". The result needs an engine with exnref: Chrome 137, Firefox 131, Safari 18.4, the bundled wasmtime. The runtime's exception-handling probe tested the legacy encoding; it now tests try_table, every variants.json entry carries it, and a browser without it is told which feature it lacks instead of failing with a CompileError. There is no switch back to -fno-exceptions: engines without exnref are old enough not to keep a second build for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
std::vector<std::string> probes = ownProbes;
probes.insert(probes.end(), WasmBaselineProbes.begin(), WasmBaselineProbes.end());
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
v << "{\"label\":\"" << jsonEscape(label) << "\""
<< ",\"url\":\"" << jsonEscape(url) << "\""
<< ",\"probes\":[";
for (std::size_t i = 0; i < probes.size(); ++i) {
if (i) v << ",";
v << "\"" << jsonEscape(probes[i]) << "\"";
}
v << "]}";
};
for (const WasmVariant& variant : cfg.wasmVariants) {
if (variant.label.empty()) continue; // baseline emitted below
emit(variant.label, variant.probes);
}
emit("", {}); // baseline fallback, always last
v << "]";
}
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
std::ifstream in(htmlTemplate);
std::stringstream buf;
buf << in.rdbuf();
2026-05-18 05:23:11 +02:00
std::string html = buf.str();
2026-07-23 01:24:42 +02:00
html = std::regex_replace(html, std::regex(R"(\{\{WASM\}\})"), std::format("{}.wasm", cfg.outputName));
2026-05-18 05:23:11 +02:00
html = std::regex_replace(html, std::regex(R"(\{\{ENV_SCRIPTS\}\})"), envScriptTags);
2026-05-26 22:50:08 +02:00
html = std::regex_replace(html, std::regex(R"(\{\{BUILDID\}\})"), buildId);
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
std::ofstream out(htmlPath);
out << html;
out.close();
cfg.files.push_back(runtimeJs);
cfg.files.push_back(htmlPath);
2026-05-18 05:23:11 +02:00
cfg.files.push_back(manifestPath);
feat: feature-detected browser-wasm variants (relaxed-SIMD) + variant-aware runtime The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and baked a single wasm URL into index.html, so newer codegen features that aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD revisions later) couldn't be adopted without dropping the browsers that lack them. Add a general, feature-parameterized mechanism owned entirely by Crafter.Build: - Configuration::wasmVariants declares N codegen variants (label, extra -m flags, runtime probes). Build() compiles the baseline plus one outputName.<label>.wasm per variant, recompiling the whole graph (incl. dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU codegen, not a link switch. wasmVariantFlags folds into VariantId so each variant's objects/PCMs land in their own build+bin dir. - EnableWasiBrowserRuntime emits a variants.json manifest (label -> url + probes), preferred-first with the baseline as the universal fallback. - The shipped runtime.js runs inlined wasm-feature-detect probes (relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads), picks the first variant whose probes all pass, and falls back to the single baked CRAFTER_WASM_URL when no manifest is present (backward compatible). - EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari still flag-gates it as of mid-2026). Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the relaxed-simd variant and runs it. Resolves #24 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:14:18 +00:00
cfg.files.push_back(variantsPath);
}
void Crafter::EnableWasiRelaxedSimdVariant(Configuration& cfg) {
for (const WasmVariant& v : cfg.wasmVariants) {
if (v.label == "relaxed-simd") return; // idempotent
}
cfg.wasmVariants.push_back(WasmVariant{
.label = "relaxed-simd",
.flags = {"-mrelaxed-simd"},
.probes = {"relaxed-simd"},
});
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
}
2026-04-29 18:59:01 +02:00
std::string Crafter::HostTarget() {
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
return GetToolchain().hostTriple;
}
std::string Crafter::NormalizeTarget(std::string_view triple) {
// <arch>-<vendor>-<os>-<env>: drop the vendor. A three-part triple only
// carries one when its middle part is a vendor name rather than an OS
// (x86_64-unknown-wasip1 yes, x86_64-linux-gnu no).
std::vector<std::string_view> parts;
for (std::size_t begin = 0;;) {
std::size_t dash = triple.find('-', begin);
parts.push_back(dash == std::string_view::npos ? triple.substr(begin) : triple.substr(begin, dash - begin));
if (dash == std::string_view::npos) break;
begin = dash + 1;
}
const bool hasVendor = parts.size() == 4 || (parts.size() == 3 && (parts[1] == "unknown" || parts[1] == "pc"));
if (hasVendor) parts.erase(parts.begin() + 1);
std::string out;
for (std::string_view part : parts) {
if (!out.empty()) out += '-';
out += part;
}
return out;
}
bool Crafter::SameTarget(std::string_view a, std::string_view b) {
return a == b || NormalizeTarget(a) == NormalizeTarget(b);
2026-04-29 18:59:01 +02:00
}
bool Crafter::ArgQuery::Has(std::string_view flag) const {
for (std::string_view a : args) if (a == flag) return true;
return false;
}
std::optional<std::string> Crafter::ArgQuery::Get(std::string_view prefix) const {
for (std::string_view a : args) {
if (a.starts_with(prefix)) return std::string(a.substr(prefix.size()));
}
return std::nullopt;
}
2026-04-30 02:20:19 +02:00
ArgQuery Crafter::ApplyStandardArgs(Configuration& cfg, std::span<const std::string_view> args) {
2026-04-29 18:59:01 +02:00
if (const char* envMarch = std::getenv("CRAFTER_BUILD_MARCH"); envMarch && *envMarch) {
cfg.march = envMarch;
}
if (const char* envMtune = std::getenv("CRAFTER_BUILD_MTUNE"); envMtune && *envMtune) {
cfg.mtune = envMtune;
}
2026-07-23 01:24:42 +02:00
bool sawLib = false;
bool sawShared = false;
2026-04-29 18:59:01 +02:00
for (std::string_view a : args) {
if (a == "--debug") cfg.debug = true;
2026-04-30 02:20:19 +02:00
else if (a == "--lib") sawLib = true;
else if (a == "--shared") sawShared = true;
2026-04-29 18:59:01 +02:00
else if (a.starts_with("--target=")) cfg.target = std::string(a.substr(std::string_view("--target=").size()));
else if (a.starts_with("--march=")) cfg.march = std::string(a.substr(std::string_view("--march=").size()));
else if (a.starts_with("--mtune=")) cfg.mtune = std::string(a.substr(std::string_view("--mtune=").size()));
2026-07-22 18:25:39 +02:00
else if (a.starts_with("--sysroot=")) cfg.sysroot = std::string(a.substr(std::string_view("--sysroot=").size()));
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
// Anything else is the project's own flag. Its effect on the output is
// opaque to the framework, so it has to key into VariantId or two flag
// settings share a bin dir and leave a bundle matching neither.
else cfg.projectArgs.emplace_back(a);
2026-04-29 18:59:01 +02:00
}
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
// Sorted and deduplicated so the identity depends on the set of flags, not
// on how they were ordered or repeated.
std::ranges::sort(cfg.projectArgs);
cfg.projectArgs.erase(std::ranges::unique(cfg.projectArgs).begin(), cfg.projectArgs.end());
2026-04-30 02:20:19 +02:00
if (sawLib && cfg.type == ConfigurationType::Executable) cfg.type = ConfigurationType::LibraryStatic;
if (sawShared && cfg.type == ConfigurationType::LibraryStatic) cfg.type = ConfigurationType::LibraryDynamic;
2026-05-18 05:23:11 +02:00
// WASI sysroot autodetect, applied at config-load time so the VariantId
// includes it. (Build() also runs this once more for callers that bypassed
// ApplyStandardArgs, but doing it here makes dep PcmDirs consistent
// between the consumer's command-construction and the dep's own build.)
if (cfg.sysroot.empty() && cfg.target.starts_with("wasm32")) {
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
cfg.sysroot = GetToolchain().wasiSysroot.string();
2026-05-18 05:23:11 +02:00
}
2026-04-30 02:20:19 +02:00
return ArgQuery{args};
2026-04-29 18:59:01 +02:00
}
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
std::vector<fs::path> Crafter::CleanProject(const fs::path& projectFile) {
fs::path projectDir = fs::absolute(projectFile).lexically_normal().parent_path();
std::vector<fs::path> removed;
for (std::string_view name : { "bin", "build" }) {
fs::path dir = projectDir / name;
std::error_code ec;
if (!fs::is_directory(dir, ec)) continue;
if (fs::remove_all(dir, ec) == static_cast<std::uintmax_t>(-1) || ec) {
throw std::runtime_error(std::format("could not remove {}: {}", dir.string(), ec.message()));
}
removed.push_back(std::move(dir));
}
return removed;
}
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
static void PrintVersion() {
std::println("crafter-build {}", CrafterBuildVersion());
const Toolchain& tc = GetToolchain();
if (!tc.error.empty()) {
std::println("compiler: {}", tc.error);
return;
}
std::println("compiler {} ({})", tc.clangVersion, tc.clangxx.string());
fix(sysroot): take the std module source from the sysroot's libc++ release A cross build against a sysroot whose libc++ differs from the toolchain's failed while precompiling the std module, with a page of "no member named 'runtime_format' in namespace 'std'". The module source is a list of using-declarations for what one libc++ release declares, so it only compiles against that release's headers. BuildStdPcm wants the sysroot's own <sysroot>/usr/share/libc++/v1/std.cppm, but Alpine packages the headers without it, and the fallback was the toolchain's copy whatever its release. That held while the compiler came from the same distro as the sysroot; with the Flatpak's libc++ 22.1.8 and Alpine edge on 23.1.2 it no longer does. The release is now read from _LIBCPP_VERSION in the sysroot's and the toolchain's <__config>, and SysrootStdModule picks in this order: the sysroot's own source, the toolchain's when both are the same major release (a release branch only takes fixes), the copy crafter-build ships for that release under share/crafter-build/libc++-modules/<major>. When none fits the build stops with one line naming both releases and the directory the matching source belongs in, before clang is run. If either release cannot be read the toolchain's source is tried as before, since there is nothing to compare. Matching sources live in crafter-build's own share directory rather than the LLVM prefix: they are data this tool ships for sysroots, not part of any one compiler install, and "sources ship" already describes that directory. `--version` prints the toolchain's libc++ release and the releases sources are shipped for, the first thing to look at when such a build fails. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-03 16:37:00 +02:00
if (tc.libcxxVersion != 0) {
// A sysroot on another libc++ release builds only when its std
// module source is in the sysroot or among these; printed here
// because that is the first question when such a build fails.
std::vector<std::string> shipped;
std::error_code ec;
for (const fs::directory_entry& entry : fs::directory_iterator(tc.stdModulesDir, ec)) {
if (fs::exists(entry.path() / "std.cppm")) shipped.push_back(entry.path().filename().string());
}
std::ranges::sort(shipped);
std::string also;
for (const std::string& release : shipped) also += std::format(", {}", release);
std::println("libc++ {}{}", LibcxxVersionString(tc.libcxxVersion), also.empty() ? std::string() : std::format(" (std module sources for sysroots also for{})", also.substr(1)));
}
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
std::println("host {}", tc.hostTriple);
}
2026-04-29 04:00:07 +02:00
static void PrintHelp(std::string_view argv0) {
std::println(
R"(Usage:
{0} [options] [-- project-args...] Build the project in the current directory
{0} test [test-options] [globs...] Build and run the project's tests
2026-07-23 01:24:42 +02:00
{0} lint [lint-options] [globs...] Run the project's lint rules over its sources
{0} format [format-options] [globs...] Apply the project's transform rules (rewrites files)
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
{0} run [script [args...]] Run a script the project defines, or list them
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
{0} clean Delete the project's bin/ and build/ trees
2026-04-29 04:00:07 +02:00
{0} help | -h | --help Show this help
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
{0} --version Print the version and the compiler in use
2026-04-29 04:00:07 +02:00
Loads ./project.cpp (override with --project=<path>), compiles it to a shared
object, and invokes its CrafterBuildProject() to obtain a Configuration that
drives the build. Outputs land at bin/<name>-<target>-<march>/, intermediates
at build/<name>-<target>-<march>/.
Build options:
-r Run the produced executable after a successful build
(host targets only; libraries cannot be run).
-v, --verbose Verbose progress output.
-q, --quiet Suppress progress output.
--project=<path> Path to the project file (default: ./project.cpp).
-- Ends this tool's options. Every later word is a
project arg, even one that looks like an option or
a verb; after test, lint or format, a glob; after
run, the script's name and then its args.
2026-04-29 04:00:07 +02:00
Test options (after the `test` subcommand):
--list Enumerate matching tests without running them.
--jobs=<N> Parallel job count (default: hardware_concurrency).
--timeout=<seconds> Per-test timeout override.
--runner=<spec> Override the test runner for this run. Specs:
local
cmd:<command> (e.g. cmd:wine)
2026-05-27 19:45:05 +02:00
--target=<triple> Filter to tests whose cfg.target matches. Default:
sweep across every distinct target declared by the
project's tests plus the host triple.
2026-04-29 04:00:07 +02:00
<glob> One or more name globs to filter tests (e.g. 'Unit*').
2026-07-23 01:24:42 +02:00
Lint options (after the `lint` subcommand):
--list Enumerate matching lint rules without running them.
feat(lint): AST layer over libclang cursors Adds LintContext::Decls() — clang's view of the declarations written in the file — plus AddAstLintRule to register a rule that reads it. No rule uses it yet; the three that will are migrated separately. The declarations come back as a flat vector with parent indices rather than an opaque cursor handle: no lifetimes cross the project-DLL boundary, no callback hops back into project.so per node, and "is this at namespace scope or inside a function?" becomes an index lookup instead of a hand-rolled brace stack. Three things had to be solved for this to work at all on this codebase. libclang cannot see through `export`. A C++20 export declaration has no CXCursorKind, so `export namespace Crafter { … }` arrives as a childless CXCursor_UnexposedDecl and clang_visitChildren does not descend. Five of the eleven interfaces here are written that way — 677 lines, including Configuration and LintContext, yielding zero usable cursors. A plain `namespace` IS descended into, so the fix is to blank the keyword before parsing, byte-length preserving so every line and column still lands on the original file. `export module` is left alone or the unit stops being a module interface. Verified end-to-end against a fixture whose asserted line numbers match the unblanked file. PCMs are flag-locked, so each file has to parse with the flags that built it. CollectConfigSources now records which Configuration owns each source instead of flattening to a set, because three regimes coexist: the library, each test (carrying its own target, defines and -march), and project.cpp, which Build never touches and which therefore gets no command at all. libclang resolves its builtin headers relative to its own install path, which need not match the clang++ that wrote the PCMs. When it doesn't, every parse dies on "'stddef.h' file not found", so -resource-dir is passed explicitly from `clang++ -print-resource-dir`. Two flags on each declaration replace what would otherwise become more substring denylists: isExternC, and isForeignApi for a declaration that binds to an entity declared outside the project root — resolved through clang_getCursorReferenced and the same inside-the-root test the dependency walk already uses. Parameters and fields inherit it, so an exemption covers a whole signature rather than the one node that named the foreign entity. Failure is never silent. A fatal diagnostic leaves a fragment that is indistinguishable from a file declaring nothing, so it is reported as an error instead: the rule is skipped, a finding explains why, and summary.errors makes the run fail. --no-ast opts out deliberately and exits normally. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:30:41 +02:00
--no-ast Skip rules that need an AST instead of building the
module PCMs they require.
2026-07-23 01:24:42 +02:00
<glob> One or more name globs to filter rules (e.g. 'spdx*').
Lint rules are defined in project.cpp via cfg.AddLintRule(name, callback) —
C++ callbacks run once per source file. No rules ship by default.
feat(lint): AST layer over libclang cursors Adds LintContext::Decls() — clang's view of the declarations written in the file — plus AddAstLintRule to register a rule that reads it. No rule uses it yet; the three that will are migrated separately. The declarations come back as a flat vector with parent indices rather than an opaque cursor handle: no lifetimes cross the project-DLL boundary, no callback hops back into project.so per node, and "is this at namespace scope or inside a function?" becomes an index lookup instead of a hand-rolled brace stack. Three things had to be solved for this to work at all on this codebase. libclang cannot see through `export`. A C++20 export declaration has no CXCursorKind, so `export namespace Crafter { … }` arrives as a childless CXCursor_UnexposedDecl and clang_visitChildren does not descend. Five of the eleven interfaces here are written that way — 677 lines, including Configuration and LintContext, yielding zero usable cursors. A plain `namespace` IS descended into, so the fix is to blank the keyword before parsing, byte-length preserving so every line and column still lands on the original file. `export module` is left alone or the unit stops being a module interface. Verified end-to-end against a fixture whose asserted line numbers match the unblanked file. PCMs are flag-locked, so each file has to parse with the flags that built it. CollectConfigSources now records which Configuration owns each source instead of flattening to a set, because three regimes coexist: the library, each test (carrying its own target, defines and -march), and project.cpp, which Build never touches and which therefore gets no command at all. libclang resolves its builtin headers relative to its own install path, which need not match the clang++ that wrote the PCMs. When it doesn't, every parse dies on "'stddef.h' file not found", so -resource-dir is passed explicitly from `clang++ -print-resource-dir`. Two flags on each declaration replace what would otherwise become more substring denylists: isExternC, and isForeignApi for a declaration that binds to an entity declared outside the project root — resolved through clang_getCursorReferenced and the same inside-the-root test the dependency walk already uses. Parameters and fields inherit it, so an exemption covers a whole signature rather than the one node that named the foreign entity. Failure is never silent. A fatal diagnostic leaves a fragment that is indistinguishable from a file declaring nothing, so it is reported as an error instead: the rule is skipped, a finding explains why, and summary.errors makes the run fail. --no-ast opts out deliberately and exits normally. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:30:41 +02:00
cfg.AddAstLintRule registers a rule that reads ctx.Decls(), clang's view of
the declarations in the file. Those need the module PCMs, which the run
builds if they are missing; a file whose AST cannot be produced is an error,
never a silent pass. --no-ast skips them instead.
2026-07-23 01:24:42 +02:00
Format options (after the `format` subcommand):
--check Dry run: list files that would change, exit 1 if any.
--list Enumerate matching rules without running them.
feat(lint): AST layer over libclang cursors Adds LintContext::Decls() — clang's view of the declarations written in the file — plus AddAstLintRule to register a rule that reads it. No rule uses it yet; the three that will are migrated separately. The declarations come back as a flat vector with parent indices rather than an opaque cursor handle: no lifetimes cross the project-DLL boundary, no callback hops back into project.so per node, and "is this at namespace scope or inside a function?" becomes an index lookup instead of a hand-rolled brace stack. Three things had to be solved for this to work at all on this codebase. libclang cannot see through `export`. A C++20 export declaration has no CXCursorKind, so `export namespace Crafter { … }` arrives as a childless CXCursor_UnexposedDecl and clang_visitChildren does not descend. Five of the eleven interfaces here are written that way — 677 lines, including Configuration and LintContext, yielding zero usable cursors. A plain `namespace` IS descended into, so the fix is to blank the keyword before parsing, byte-length preserving so every line and column still lands on the original file. `export module` is left alone or the unit stops being a module interface. Verified end-to-end against a fixture whose asserted line numbers match the unblanked file. PCMs are flag-locked, so each file has to parse with the flags that built it. CollectConfigSources now records which Configuration owns each source instead of flattening to a set, because three regimes coexist: the library, each test (carrying its own target, defines and -march), and project.cpp, which Build never touches and which therefore gets no command at all. libclang resolves its builtin headers relative to its own install path, which need not match the clang++ that wrote the PCMs. When it doesn't, every parse dies on "'stddef.h' file not found", so -resource-dir is passed explicitly from `clang++ -print-resource-dir`. Two flags on each declaration replace what would otherwise become more substring denylists: isExternC, and isForeignApi for a declaration that binds to an entity declared outside the project root — resolved through clang_getCursorReferenced and the same inside-the-root test the dependency walk already uses. Parameters and fields inherit it, so an exemption covers a whole signature rather than the one node that named the foreign entity. Failure is never silent. A fatal diagnostic leaves a fragment that is indistinguishable from a file declaring nothing, so it is reported as an error instead: the rule is skipped, a finding explains why, and summary.errors makes the run fail. --no-ast opts out deliberately and exits normally. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:30:41 +02:00
--no-ast Skip rules that need an AST (see `lint --no-ast`).
2026-07-23 01:24:42 +02:00
<glob> One or more name globs to filter rules.
Rules are shared with lint — a rule that calls ctx.SetContent is a
transform. `format` rewrites changed files in place; `lint` reports the
same transforms as would-reformat findings without writing.
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
Run (after the `run` subcommand):
<script> [args...] Run the named script. Everything after the name is
handed to the script untouched, including what
looks like an option of crafter-build itself, so
-v, -q, --project= and project args go before it.
--list List the project's scripts. A bare `run` does too.
Scripts are defined in project.cpp, as entries of cfg.scripts:
cfg.scripts.push_back({{"docs", [](Crafter::ScriptContext& ctx) {{
return ctx.Run("python3 tools/gen-docs.py");
}}}});
The callback returns the exit status. ctx.Run runs a shell command with the
terminal attached and returns its status; ctx.args holds the arguments and
ctx.projectDir the project's directory. Commands run in the directory
crafter-build was started from. Only the scripts of the project being run
are available, never a dependency's.
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
Clean (after the `clean` subcommand):
Removes bin/ and build/ next to the project file. Does not load project.cpp,
so it works when the project itself no longer compiles. Every target, variant
and dependency artifact under those trees goes with it.
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md An agent working on a crafter-build project reads its AGENTS.md, and learns nothing there about the libraries the project builds against unless someone copied that in by hand. This repository shows the cost: the code style section of its AGENTS.md restates what Crafter.LintRules enforces, and goes stale whenever the rules change. A library now ships those notes itself, as AGENTS.consumer.md beside its project.cpp; nothing in its project.cpp changes. When a consumer has an AGENTS.md beside its own project.cpp, `crafter-build` and `crafter-build test` keep a section in it, between two marker comments, holding its dependencies' notes. The section is appended the first time and rewritten in place after that. The file is written only when the section changes, and an AGENTS.md is never created. lint, format, run and clean leave it alone. The update runs after loading, which has just fetched every dependency, and before compiling, so a build that fails still has the notes. cfg.dependencyNotes picks the dependencies. Transitive, the default, follows `dependencies` all the way down from the project's own configurations, its tests and its `#pragma crafter dependency` tooling; Direct stops at the first loaded project on each path; None removes the section. A dependency's own pragma tooling never counts: it is about how that dependency is built, not how it is used, and being no `dependencies` edge it is never reached. Nor are a dependency's tests: nothing builds them for the consumer, and their configurations can point at a local their project file has since moved out of. The walk has to know which Configurations belong to another project, and where that project lives, and cfg.path cannot say: only the Configuration GitProject and LocalProject return is rebased, so a dependency's internal configurations keep "./", which resolves to the consumer. The loader records it instead, LoadedProjectOf beside the project cache, and keeps what the root's pragmas resolved to past the frame LoadProject pops, PragmaDependencies. Neither is exported. The markers count only as whole lines outside fenced code blocks, so documentation can quote them. Unbalanced markers and a document ending inside a code block leave the file alone with a warning: guessing which part of someone's file is generated would overwrite their text, and missing the section would add another one on every build. A note that would break the section when it is read back, through a marker line of its own or a code block it never closes, is left out with a warning. The file is replaced through a temporary renamed onto the symlink's target, so a linked AGENTS.md stays a link and a write cut short leaves the old file rather than part of the new one. The code is a new partition, Crafter.Build:Agents, listed in build.sh, CrafterBuildModules, project.cpp and the umbrella. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
AGENTS.md:
feat(agents): write AGENTS.md from a tracked AGENTS.template.md The dependency notes were kept in a section of the AGENTS.md beside project.cpp, the file a project commits, so a build could change a tracked file at any time: a dependency editing its AGENTS.consumer.md, an unpinned one moving on, a flag pulling a dependency in or out, each showed up in the consumer's diff as a change nobody made. Users reported it as the feature clobbering their diffs. A project now commits its instructions as AGENTS.template.md, and a build and a test run write AGENTS.md from it: a first line saying the file is generated and where to edit, the template's text as written, then the notes section, after it or where the template's marker lines put it. AGENTS.md is build output and goes into .gitignore. It is created when missing, since a fresh clone has none, and otherwise written as before: only when its text changes, in place and under a lock, so links, ownership and a read-only file are respected. The first line is also how a build tells its own AGENTS.md from one someone wrote. Beside a template the second is easily there, as a committed AGENTS.md nobody has removed yet; it is never replaced, and each build warns until it is deleted. An empty file counts as a build's, since that is what a write cut short leaves of a new one. An AGENTS.md linked to the template is refused: writing it would write the committed file again. A template the notes cannot be placed in leaves no empty AGENTS.md behind. The in-place mode is removed rather than kept as the fallback for a project without a template. It is the default that put generated text into tracked files unasked, the moment any dependency started shipping notes. Without a template a build no longer touches AGENTS.md. One that still holds a section from 3.0.0 would go stale silently, under a preamble promising that every build rewrites it, so it gets a warning naming the move instead. MergeDependencyNotes keeps its rules and now runs on the template. The test changes in this commit rather than its own, because it pinned the in-place behaviour this removes: it now covers the generated file's shape, edits and markers in the template, the ownership check, a CRLF template, a link to the template and the cases without one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 16:53:42 +01:00
When an AGENTS.template.md sits beside the project file, a build and a test
run write AGENTS.md from it, with a section holding the AGENTS.consumer.md
each dependency ships beside its project.cpp, and only when the text changes.
Commit the template and keep AGENTS.md out of version control. An AGENTS.md
no build wrote is never replaced, and without a template AGENTS.md is left
alone. cfg.dependencyNotes picks the dependencies: Transitive (the default),
Direct, or None.
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md An agent working on a crafter-build project reads its AGENTS.md, and learns nothing there about the libraries the project builds against unless someone copied that in by hand. This repository shows the cost: the code style section of its AGENTS.md restates what Crafter.LintRules enforces, and goes stale whenever the rules change. A library now ships those notes itself, as AGENTS.consumer.md beside its project.cpp; nothing in its project.cpp changes. When a consumer has an AGENTS.md beside its own project.cpp, `crafter-build` and `crafter-build test` keep a section in it, between two marker comments, holding its dependencies' notes. The section is appended the first time and rewritten in place after that. The file is written only when the section changes, and an AGENTS.md is never created. lint, format, run and clean leave it alone. The update runs after loading, which has just fetched every dependency, and before compiling, so a build that fails still has the notes. cfg.dependencyNotes picks the dependencies. Transitive, the default, follows `dependencies` all the way down from the project's own configurations, its tests and its `#pragma crafter dependency` tooling; Direct stops at the first loaded project on each path; None removes the section. A dependency's own pragma tooling never counts: it is about how that dependency is built, not how it is used, and being no `dependencies` edge it is never reached. Nor are a dependency's tests: nothing builds them for the consumer, and their configurations can point at a local their project file has since moved out of. The walk has to know which Configurations belong to another project, and where that project lives, and cfg.path cannot say: only the Configuration GitProject and LocalProject return is rebased, so a dependency's internal configurations keep "./", which resolves to the consumer. The loader records it instead, LoadedProjectOf beside the project cache, and keeps what the root's pragmas resolved to past the frame LoadProject pops, PragmaDependencies. Neither is exported. The markers count only as whole lines outside fenced code blocks, so documentation can quote them. Unbalanced markers and a document ending inside a code block leave the file alone with a warning: guessing which part of someone's file is generated would overwrite their text, and missing the section would add another one on every build. A note that would break the section when it is read back, through a marker line of its own or a code block it never closes, is left out with a warning. The file is replaced through a temporary renamed onto the symlink's target, so a linked AGENTS.md stays a link and a write cut short leaves the old file rather than part of the new one. The code is a new partition, Crafter.Build:Agents, listed in build.sh, CrafterBuildModules, project.cpp and the umbrella. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
2026-04-29 04:00:07 +02:00
Project args:
Any flag not consumed above is forwarded verbatim to CrafterBuildProject as
part of its `args` span. Project-specific flags (e.g. --target=, custom
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
feature toggles) live there. Flags ApplyStandardArgs does not itself
interpret are folded into the variant hash, so switching one lands in its own
bin/ and build/ directory instead of overwriting the other setting's.
2026-04-29 04:00:07 +02:00
Environment:
CRAFTER_BUILD_MARCH Override -march (default: native).
CRAFTER_BUILD_MTUNE Override -mtune (default: native).
CRAFTER_BUILD_RUNNER_<TARGET> Default test runner for a target triple.
Replace '-' and '.' with '_' in the
triple. CLI --runner= overrides this.
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
CRAFTER_BUILD_LLVM_DIR LLVM prefix to use instead of a toolchain
installed beside crafter-build or PATH.
CRAFTER_BUILD_LIBCXX_DIR libc++ install to compile and link against
instead of the compiler's own.
CRAFTER_BUILD_LIBCLANG Exact libclang to load for lint.
CRAFTER_BUILD_WASI_SYSROOT WASI sysroot for wasm32 targets.
fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is - Shaders record the files glslang opened (a depfile) and their settings (source path, stage, entry point, defines, include directories) in the build directory, and rebuild when an included file or a setting changes, not only the shader's own source. - An executable relinks when its link command changed, not only when an object is newer: a link flag in project.cpp, or a dependency's library that moved (an ExternalDependency built into a new cache directory). The command is recorded beside the objects; its flags come from ordered sets, so the order the dependency threads finish in can't change it. - An external's newest artifact counts its libDirs and versioned .so names. - Submodules are checked out at the revisions the fetched commit records, from the URLs its .gitmodules gives, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules, and one moved by hand is put back; with no .gitmodules, git isn't asked. - Each external clone's fetch and build hold a file lock: parallel builds pulled into one clone at once ("cannot lock ref"). - mingw: - -stdlib=libstdc++ always; - the std module's source is found through libstdc++.modules.json in the mingw toolchain's own library directories (the generic lib/ holds the host's Linux one), asking the compiler once a run; - a toolchain that isn't there is a build error saying where it looked, not std::terminate; - CRAFTER_MINGW_DIR names the prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/; - inside the Flatpak the host's /usr mingw is used. - RunScript's exit-status check reads as the one range it is. - Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules, LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and ToolchainResolve extended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:08:13 +02:00
CRAFTER_MINGW_DIR mingw-w64 prefix (holding x86_64-w64-mingw32/
fix: give the mingw target a sysroot that holds mingw-w64 alone Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree that holds mingw-w64 alone, and adds its include/ to the include path and its lib/ to the library path. For /usr those are the Linux system's own: every header mingw-w64 lacks came from glibc, and libstdc++'s <ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read __libc_single_threaded, which nothing on Windows defines. Every program that used one failed to link, among them every Crafter.Graphics Win32 executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr did the same outside the Flatpak (#43). The sysroot is now MingwSysroot(): a directory in the cache, one per prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/ and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds mingw-w64, gcc's version and libstdc++ exactly as it does through the mingw gcc on PATH, and nothing else. It is made by the first mingw compile, link or CMake configure of a process, so no other command writes to the cache for it and a failure to make it is that build's error. The Toolchain field is now mingwPrefix, since clang never gets the prefix itself, and it is made absolute when the toolchain is resolved, as it ends up in link targets and directory names. Messages keep naming paths under the prefix rather than under the view. The prefix is also part of the mingw target's VariantId and of its std PCM directory. Units do not track the std PCM they import: when the fixed release built in a directory a broken build had left, the std module was rebuilt and the objects compiled against glibc's headers were only relinked, failing as before. As with a compiler update, a new directory is what gets them rebuilt. It also keeps two releases that share a cache, as the Flatpak's branches do, from rebuilding one std PCM under each other's PCMs, and two prefixes apart. Without a prefix both stay as they were: clang's own discovery has not changed. Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32 loses the directory gcc's version names, so <string> is not found, and clang ignores --gcc-toolchain and --gcc-install-dir for this target. Fixes #43 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
and lib/gcc/). Default: the mingw gcc on
PATH; in the Flatpak, the host's /usr when
fix(mingw): view only a prefix that holds a Linux system The first fix showed clang every mingw prefix through a view of its mingw-w64 parts. That broke a prefix that holds mingw-w64 alone, where Windows libraries installed with --prefix land in include/ and lib/: with CRAFTER_MINGW_DIR=/opt/mingw, <SDL2/SDL.h> and -lSDL2 were no longer found. Such a prefix is clang's sysroot whole again. Only a prefix that also holds a Linux system gets the view (Toolchain::mingwPrefixIsSystem): the host's /usr inside the Flatpak, whatever its include/ holds, and a prefix whose include/ has a C library's features.h, which every glibc and musl header includes and mingw-w64 has no use for, as CRAFTER_MINGW_DIR=/usr has. MakeMingwSysroot makes the view in the cache, or in $XDG_RUNTIME_DIR/crafter.build when the cache's filesystem takes no symbolic links (vfat, exFAT, CIFS without unix extensions): the XDG spec promises them there, and nothing else in the cache needs them. Its path is absolute, because CMake runs the compiler from directories of its own and a relative cache did not resolve from there. The prefix is resolved once, absolute and lexically normal, so /opt/mingw/ and /opt/mingw are one prefix and not two builds. clang names a header by the path it found it at, so every diagnostic from a viewed mingw-w64 header named a hashed cache directory. RunBuildCommand now writes paths into the view as paths into the prefix, as BuildStdPcm did for its manifest messages already. Also from the review of PR #44: SysrootCompileFlags builds a mingw command, which can make a view, so it redirects the cache like the other tests; the manifest filter's comment names what it still keeps out; the C path asks MingwSysroot() once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:58:30 +01:00
it has mingw-w64. Of a prefix that holds a
Linux system too, such as /usr, clang sees
only those directories.
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
CRAFTER_BUILD_HOME Directory holding the module sources.
2026-04-29 04:00:07 +02:00
Exit status:
2026-07-23 01:24:42 +02:00
0 success / all non-skipped tests passed / lint clean / format applied
1 build failure, one or more tests failed, lint findings, no rules defined,
format write errors, or `format --check` found files that would change
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
`run` exits with the status its script returned, and with 1 when there is no
such script, when the script throws, or when it returns a value outside 0..255
2026-04-29 04:00:07 +02:00
)", argv0);
}
feat(lint): no-char-pointer reads the AST, retiring the interop denylist The rule was `\bchar\s*\*` over the text minus a substring denylist — argv, getenv, setenv, dlerror, c_str, .data(, reinterpret_cast, extern ". Every entry was a patch for one interop site, the list could only grow as libraries arrived, and each entry disabled the rule for the whole LINE it appeared on. It now walks declarations and asks the question the denylist was approximating: whose header dictates this spelling? A declaration with C language linkage, or one whose initialiser binds to an entity declared outside the project root, is somebody else's API and keeps its spelling. getenv, c_str and friends are exempt because of where they are declared, not because they are named here, so a new external library needs no new entry. Two bugs found while testing this, both of which had made the rule silently pass over the entire repository: clang_getCursorLanguage cannot be used to detect extern "C". Its default answer is CXLanguage_C for a plain function, variable or parameter even in a C++ translation unit, so isExternC was true almost everywhere and exempted everything. Replaced by tracking CXCursor_LinkageSpec depth during the walk, reading the extent text to tell extern "C" from extern "C++". Attributing any foreign reference in a subtree to the enclosing declaration was too broad: a function that merely touched libc++ somewhere in its body would exempt its own signature. Narrowed to initialiser contexts — a variable, field or parameter — which is where a binding to a foreign API actually occurs. Also: functions now carry their RESULT type rather than the whole function type, since the parameters arrive as their own declarations and would otherwise be reported twice. main's parameters are exempt structurally, its signature being fixed by the language rather than chosen here. Two sites keep an explicit lint-disable, both Crafter::Run taking main's argv verbatim. That is two visible, reasoned suppressions in place of a denylist that silently disabled the rule for every line mentioning one of eight tokens. ExternalCloneDir and ExternalIncludeFlags are now exposed from :External, so a source that includes an external dependency's headers can be parsed without running a build to discover where they are. BuildExternal derives its own working directory through the same function, so the two cannot drift. Crafter.Build-Shader.cpp needed this to parse at all. A file with no compile command — project.cpp, which LoadProject builds with its own flags — is not a translation unit of the build graph, so AST rules skip it the way a rule self-filters by extension. That is distinct from a file that should have parsed and did not, which stays an error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:46:32 +02:00
// lint-disable-next-line no-char-pointer
feat(lint): fixed-width-types keeps widths that a foreign API chose The rewrite itself stays token-shaped — it edits type SPELLINGS, which an AST discards — but what it must not touch now comes from the AST. The old exemption was per LINE: `int main`, `argc`, `argv`, `extern "`. Being a transform, a missed exemption here does not over-report, it emits code that no longer matches the API being called, so this is the rule where guessing from substrings mattered most. And being per-line, it also disabled the rule for anything sharing a line with one of those words. Now a declaration with C language linkage, or one whose initialiser binds to an entity declared outside the project, contributes a protected byte range and keeps its spelling. That answers the case directly: a function declared in somebody else's header taking `unsigned int` keeps `unsigned int`, and a local initialised from strtoul keeps `unsigned long`, because of where those are declared rather than because of what the line says. main is protected from the start of its declaration to the opening brace of its body, not for its whole extent. Its signature is fixed by the language; its body is ordinary code. Three findings on this repository came out of that distinction, all correct: for (int i = 1; i < argc; ++i) -> for (std::int32_t i = 1; ...) skipped before only because `argc` appeared on the line, plus Crafter::Run's own `int argc` and return type, which are ours rather than the language's. All three AST rules now share one interop test instead of carrying a denylist each, and it is the same test: whose header dictates this spelling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:59:46 +02:00
std::int32_t Crafter::Run(std::int32_t argc, char** argv) {
2026-04-27 07:04:42 +02:00
try {
2026-04-29 04:00:07 +02:00
std::string_view argv0 = argc > 0 ? argv[0] : "crafter-build";
2026-04-27 07:04:42 +02:00
fs::path projectFile = "./project.cpp";
std::vector<std::string_view> projectArgs;
projectArgs.reserve(argc);
bool runTests = false;
2026-07-23 01:24:42 +02:00
bool runLint = false;
bool runFormat = false;
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
bool runClean = false;
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
bool runScript = false;
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
bool runAfterBuild = false;
RunTestsOptions testOpts;
2026-07-23 01:24:42 +02:00
RunLintOptions lintOpts;
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
RunScriptOptions scriptOpts;
2026-04-29 03:27:11 +02:00
Progress::Verbosity verbosity = Progress::Verbosity::Default;
// Set by `--`: every later word is an operand of the verb, never an
// option or a verb of this tool.
bool operandsOnly = false;
feat(lint): fixed-width-types keeps widths that a foreign API chose The rewrite itself stays token-shaped — it edits type SPELLINGS, which an AST discards — but what it must not touch now comes from the AST. The old exemption was per LINE: `int main`, `argc`, `argv`, `extern "`. Being a transform, a missed exemption here does not over-report, it emits code that no longer matches the API being called, so this is the rule where guessing from substrings mattered most. And being per-line, it also disabled the rule for anything sharing a line with one of those words. Now a declaration with C language linkage, or one whose initialiser binds to an entity declared outside the project, contributes a protected byte range and keeps its spelling. That answers the case directly: a function declared in somebody else's header taking `unsigned int` keeps `unsigned int`, and a local initialised from strtoul keeps `unsigned long`, because of where those are declared rather than because of what the line says. main is protected from the start of its declaration to the opening brace of its body, not for its whole extent. Its signature is fixed by the language; its body is ordinary code. Three findings on this repository came out of that distinction, all correct: for (int i = 1; i < argc; ++i) -> for (std::int32_t i = 1; ...) skipped before only because `argc` appeared on the line, plus Crafter::Run's own `int argc` and return type, which are ours rather than the language's. All three AST rules now share one interop test instead of carrying a denylist each, and it is the same test: whose header dictates this spelling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:59:46 +02:00
for (std::int32_t i = 1; i < argc; ++i) {
2026-04-27 07:04:42 +02:00
std::string_view arg = argv[i];
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
// The two `run` branches come before everything else on purpose.
// What follows a script's name belongs to the script, whatever
// it looks like: `run package --help` must not print this
// tool's help, `run package clean` must not delete build/, and
// none of it may reach projectArgs, where it would change the
// variant hash and so the directory the project builds into.
if (scriptOpts.named) {
scriptOpts.args.emplace_back(arg);
} else if (runScript && (operandsOnly || !arg.starts_with("-"))) {
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
// The first word after `run`, even one that is a verb
// elsewhere. Options before it keep their usual meaning.
scriptOpts.name = arg;
scriptOpts.named = true;
} else if (operandsOnly) {
if (runTests) {
testOpts.globs.emplace_back(arg);
} else if (runLint || runFormat) {
lintOpts.globs.emplace_back(arg);
} else {
projectArgs.push_back(arg);
}
} else if (arg == "--") {
// Ends this tool's options and is never forwarded itself. It
// used to land in projectArgs as a word of its own, giving the
// project a variant hash of its own, and every sibling project
// that inherits the args one as well.
operandsOnly = true;
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
} else if (arg == "--version") {
feat(toolchain): resolve the compiler in one place Every clang++, clang and llvm-ar the tool spawned was a bare name on PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI sysroot /usr/share/wasi-sysroot and the mingw headers /usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A toolchain installed anywhere else, which is what the Flatpak ships under /app, had no way in. Crafter.Build:Toolchain resolves all of it once per process: CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build (<prefix>/bin, found through the same walk GetCrafterBuildHome does), else PATH. What sits beside the compiler is used when present: its libclang.so, its clang-c headers, its std module source, its WASI sysroot. An explicit prefix without a clang++ is an error rather than a fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other than the compiler's own, and CRAFTER_MINGW_DIR now does what its help line always claimed: it is passed as --sysroot for the mingw target. Binaries linked by a toolchain outside /usr get an rpath to its lib dir, since its libc++.so.1 is not on the loader's default path. HostTarget() is the resolved clang's -print-target-triple. A clang built without a vendor reports x86_64-unknown-linux-gnu where Arch's reports x86_64-pc-linux-gnu, so target equality that decides whether a test runs locally and which targets a bare `test` sweeps goes through SameTarget, which ignores the vendor field. The strings themselves are never rewritten: clang rejects a PCM whose triple differs from the importing unit's. The examples and README stop hardcoding the host triple; the default is already the host. `crafter-build --version` prints the version baked in at build time (CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the compiler and the host triple, and is answered before any argument is forwarded to project.cpp. The version is a define on the library, so each version is its own library variant and relinks the executable and tests once per commit. build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already there, and writes a fresh archive instead of appending to one that may hold members from an older layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:26:31 +02:00
// Answered here, before the project file is looked for, so
// it works in any directory and is never forwarded to
// project.cpp as an unknown argument.
PrintVersion();
return 0;
} else if (arg == "-h" || arg == "--help" || (!runTests && !runLint && !runFormat && arg == "help")) {
2026-04-29 04:00:07 +02:00
PrintHelp(argv0);
return 0;
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
} else if (!runLint && !runFormat && !runClean && arg == "test") {
runTests = true;
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
} else if (!runTests && !runFormat && !runClean && arg == "lint") {
2026-07-23 01:24:42 +02:00
runLint = true;
} else if (!runTests && !runLint && !runFormat && arg == "clean") {
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
runClean = true;
} else if (!runTests && !runLint && !runClean && arg == "format") {
2026-07-23 01:24:42 +02:00
runFormat = true;
lintOpts.mode = LintMode::Apply;
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
} else if (!runTests && !runLint && !runFormat && !runClean && arg == "run") {
runScript = true;
} else if (runScript && arg == "--list") {
scriptOpts.listOnly = true;
2026-07-23 01:24:42 +02:00
} else if (runFormat && arg == "--check") {
lintOpts.mode = LintMode::Check;
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
} else if (arg == "-r") {
runAfterBuild = true;
2026-04-29 03:27:11 +02:00
} else if (arg == "-v" || arg == "--verbose") {
verbosity = Progress::Verbosity::Verbose;
} else if (arg == "-q" || arg == "--quiet") {
verbosity = Progress::Verbosity::Quiet;
} else if (arg.starts_with("--project=")) {
2026-04-27 07:04:42 +02:00
projectFile = arg.substr(std::string_view("--project=").size());
} else if (runTests && arg.starts_with("--jobs=")) {
testOpts.jobs = std::stoi(std::string(arg.substr(std::string_view("--jobs=").size())));
} else if (runTests && arg.starts_with("--timeout=")) {
testOpts.timeoutOverride = std::chrono::seconds(std::stoi(std::string(arg.substr(std::string_view("--timeout=").size()))));
} else if (runTests && arg == "--list") {
testOpts.listOnly = true;
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
} else if (runTests && arg.starts_with("--runner=")) {
testOpts.runnerOverride = std::string(arg.substr(std::string_view("--runner=").size()));
} else if (runTests && !arg.starts_with("-")) {
testOpts.globs.emplace_back(arg);
2026-07-23 01:24:42 +02:00
} else if ((runLint || runFormat) && arg == "--list") {
lintOpts.listOnly = true;
feat(lint): AST layer over libclang cursors Adds LintContext::Decls() — clang's view of the declarations written in the file — plus AddAstLintRule to register a rule that reads it. No rule uses it yet; the three that will are migrated separately. The declarations come back as a flat vector with parent indices rather than an opaque cursor handle: no lifetimes cross the project-DLL boundary, no callback hops back into project.so per node, and "is this at namespace scope or inside a function?" becomes an index lookup instead of a hand-rolled brace stack. Three things had to be solved for this to work at all on this codebase. libclang cannot see through `export`. A C++20 export declaration has no CXCursorKind, so `export namespace Crafter { … }` arrives as a childless CXCursor_UnexposedDecl and clang_visitChildren does not descend. Five of the eleven interfaces here are written that way — 677 lines, including Configuration and LintContext, yielding zero usable cursors. A plain `namespace` IS descended into, so the fix is to blank the keyword before parsing, byte-length preserving so every line and column still lands on the original file. `export module` is left alone or the unit stops being a module interface. Verified end-to-end against a fixture whose asserted line numbers match the unblanked file. PCMs are flag-locked, so each file has to parse with the flags that built it. CollectConfigSources now records which Configuration owns each source instead of flattening to a set, because three regimes coexist: the library, each test (carrying its own target, defines and -march), and project.cpp, which Build never touches and which therefore gets no command at all. libclang resolves its builtin headers relative to its own install path, which need not match the clang++ that wrote the PCMs. When it doesn't, every parse dies on "'stddef.h' file not found", so -resource-dir is passed explicitly from `clang++ -print-resource-dir`. Two flags on each declaration replace what would otherwise become more substring denylists: isExternC, and isForeignApi for a declaration that binds to an entity declared outside the project root — resolved through clang_getCursorReferenced and the same inside-the-root test the dependency walk already uses. Parameters and fields inherit it, so an exemption covers a whole signature rather than the one node that named the foreign entity. Failure is never silent. A fatal diagnostic leaves a fragment that is indistinguishable from a file declaring nothing, so it is reported as an error instead: the rule is skipped, a finding explains why, and summary.errors makes the run fail. --no-ast opts out deliberately and exits normally. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:30:41 +02:00
} else if ((runLint || runFormat) && arg == "--no-ast") {
lintOpts.noAst = true;
2026-07-23 01:24:42 +02:00
} else if ((runLint || runFormat) && !arg.starts_with("-")) {
lintOpts.globs.emplace_back(arg);
2026-04-27 07:04:42 +02:00
} else {
projectArgs.push_back(arg);
}
}
2026-04-29 03:27:11 +02:00
Progress::SetVerbosity(verbosity);
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
// The test run is target-scoped: only tests whose cfg.target equals
// testOpts.targetFilter are included. Default = host triple, so a
// bare `crafter-build test` runs everything that targets host.
for (auto& a : projectArgs) {
if (a.starts_with("--target=")) {
testOpts.targetFilter = std::string(a.substr(std::string_view("--target=").size()));
}
}
2026-04-27 07:04:42 +02:00
if (!fs::exists(projectFile)) {
std::println(std::cerr, "No project file at {}", projectFile.string());
return 1;
}
fix: key the host PCM cache on source content, add clean, hash project args Three follow-ons to the stale-build report, all cases of an identity not capturing something that changes the output. The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by every crafter-build on the machine, and freshness was a per-file mtime comparison. That cannot tell "this PCM is newer than my source" from "this PCM was built from different sources that happen to be newer", so a package install and a working checkout — or two checkouts of different versions — silently compiled their project.cpp against each other's declarations. Invalidation now keys on a stamp over the bytes of every module source, which also covers the case one file's mtime never could: the cached PCMs import each other, so a change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm untouched. Project args ApplyStandardArgs does not itself interpret are now folded into VariantId. Such a flag typically decides what gets compiled or bundled — the report's example is --no-webgpu dropping entries from cfg.files — and without it both settings shared one bin dir and interleaved their outputs there, leaving a bundle matching neither. Sorted and deduplicated so flag order doesn't split the cache, and inherited by test Configurations. `crafter-build clean` removes the project's bin/ and build/ trees. It deliberately does not load project.cpp: cleaning is most often reached when something is already wrong, and a clean that first needs the project to compile is useless exactly then.
2026-07-30 17:43:09 +00:00
// Ahead of LoadProject on purpose — see CleanProject.
if (runClean) {
std::vector<fs::path> removed = CleanProject(projectFile);
if (removed.empty()) {
std::println("Nothing to clean");
} else {
for (const fs::path& dir : removed) std::println("Removed {}", dir.string());
}
return 0;
}
2026-04-27 07:04:42 +02:00
Configuration config = LoadProject(projectFile, projectArgs);
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
SetParentProject(&config);
2026-04-27 07:04:42 +02:00
feat(script): scripts in project.cpp, run with `crafter-build run` A project.cpp could describe how a project is built, tested and linted, but not the chores around it: packaging, installing, publishing. Those stayed shell lines in a README or in someone's history. Configuration gains `scripts`, a plain list like `defines`. A Script is a name and a C++ callback, as a lint rule is; there is no script language and no config file. The callback gets a ScriptContext (the project, the project file's directory, the arguments) and returns the exit status. ctx.Run runs a shell command with the terminal attached and returns its status. It is only stored when project.cpp loads, which happens for every verb and for every project that depends on this one. `crafter-build run <name> [args...]` runs one, a bare `run` lists them. Only the scripts of the project crafter-build was pointed at are runnable, never a dependency's. Everything after the script's name is handed to the script untouched. The two branches that do this sit ahead of every other branch of the parse loop, --version and --help included. Below the verbs, `run package clean` would delete build/, and any other word would land in the project's args and so in the variant hash, moving the build to another directory. Options before the name keep their meaning, so project args go there. The driver is its own partition, :Script. It refuses a project with no scripts, a name that is empty or starts with '-' (it could never be typed), a name declared twice (running either would be a guess) and an unknown name, each with exit 1 and without running anything. A script that throws exits 1. A return value outside 0..255 becomes 1: an exit status is eight bits, and a raw wait status such as 256 would otherwise wrap to success. Two helpers in :Platform: - RunCommandAttached runs a command with inherited stdio. Every helper so far captured through popen, which is right for a compile step and useless for a command that runs for half an hour or asks a question. It reads the wait status through DecodeWaitStatus, which RunCommandChecked and RunCommandWithTimeout now share. - ShellQuote is exported. It existed twice, as ShellQuote in :External and ShellQuoteSh in :Test, with the same behaviour; both are gone. A script that puts a path into a command needs it too. Not included: a string shorthand for one-command scripts (a one-line lambda covers it) and a ctx.Build() helper (Build() is already exported). A new partition and new exported symbols: run ./build.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 18:16:00 +02:00
if (runScript) {
scriptOpts.projectFile = fs::absolute(projectFile).lexically_normal();
return RunScript(config, scriptOpts);
}
2026-07-23 01:24:42 +02:00
if (runLint || runFormat) {
// lexically_normal: fs::absolute keeps a "./" component, which
// would leak into displayed paths and the project-root derivation.
lintOpts.projectFile = fs::absolute(projectFile).lexically_normal();
LintSummary lintSummary = RunLint(config, lintOpts);
if (!runFormat) return lintSummary.Clean() ? 0 : 1;
if (lintSummary.noRulesDefined || lintSummary.errors > 0) return 1;
// Apply: formatting files is success (gofmt convention).
// Check: a would-change file is the CI failure signal.
return lintOpts.mode == LintMode::Check && !lintSummary.changedFiles.empty() ? 1 : 0;
}
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md An agent working on a crafter-build project reads its AGENTS.md, and learns nothing there about the libraries the project builds against unless someone copied that in by hand. This repository shows the cost: the code style section of its AGENTS.md restates what Crafter.LintRules enforces, and goes stale whenever the rules change. A library now ships those notes itself, as AGENTS.consumer.md beside its project.cpp; nothing in its project.cpp changes. When a consumer has an AGENTS.md beside its own project.cpp, `crafter-build` and `crafter-build test` keep a section in it, between two marker comments, holding its dependencies' notes. The section is appended the first time and rewritten in place after that. The file is written only when the section changes, and an AGENTS.md is never created. lint, format, run and clean leave it alone. The update runs after loading, which has just fetched every dependency, and before compiling, so a build that fails still has the notes. cfg.dependencyNotes picks the dependencies. Transitive, the default, follows `dependencies` all the way down from the project's own configurations, its tests and its `#pragma crafter dependency` tooling; Direct stops at the first loaded project on each path; None removes the section. A dependency's own pragma tooling never counts: it is about how that dependency is built, not how it is used, and being no `dependencies` edge it is never reached. Nor are a dependency's tests: nothing builds them for the consumer, and their configurations can point at a local their project file has since moved out of. The walk has to know which Configurations belong to another project, and where that project lives, and cfg.path cannot say: only the Configuration GitProject and LocalProject return is rebased, so a dependency's internal configurations keep "./", which resolves to the consumer. The loader records it instead, LoadedProjectOf beside the project cache, and keeps what the root's pragmas resolved to past the frame LoadProject pops, PragmaDependencies. Neither is exported. The markers count only as whole lines outside fenced code blocks, so documentation can quote them. Unbalanced markers and a document ending inside a code block leave the file alone with a warning: guessing which part of someone's file is generated would overwrite their text, and missing the section would add another one on every build. A note that would break the section when it is read back, through a marker line of its own or a code block it never closes, is left out with a warning. The file is replaced through a temporary renamed onto the symlink's target, so a linked AGENTS.md stays a link and a write cut short leaves the old file rather than part of the new one. The code is a new partition, Crafter.Build:Agents, listed in build.sh, CrafterBuildModules, project.cpp and the umbrella. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
// Builds and test runs only: lint and `format --check` promise to
// write nothing, `format` rewrites the project's sources and nothing
fix(agents): what the review of dependency notes found A review of the feature turned up the following, fixed here. Writing. AGENTS.md was replaced through a temporary renamed over it, which swaps in another file: a read-only AGENTS.md was rewritten anyway, a hard link such as a CLAUDE.md beside it kept the old text, ACLs and ownership were dropped, and without an fsync a power cut could still leave it empty. The temporary was named after the PID, which every Flatpak instance shares as PID 2. The file is now written in place, from the first byte that differs, then truncated and synced, under an flock on the file itself. A write that fails partway is undone where the disk allows; with the section last, where it is appended, only the section can be damaged. A read-only file is opened for writing only when something would change, so it is refused then and silent otherwise. Reading. libc++'s filebuf reports a failed read as the end of the file, so an I/O error came back as a short file and the section was written over what was left. Reads go through read(2). Code fences. ```make``` was taken for a fence, and a fence in a list item was missed while its indented closer opened one, so markers a user quoted in a code block could be taken for the section, or the real one hidden and appended again. Backtick info strings, block quotes and list items are now followed, and a block ends with its container. Warnings with nothing to write. With no notes, nothing short of a well-formed section is reported, so an AGENTS.md that ends in an open code block, or a project set to None, no longer warns on every build. An unclosed fence after a section no longer stops it being rewritten in place. Labels. The `from` line carried the git URL, credentials and all, or a path relative to the consumer that changed with the machine, the cache location and --local. It is now the repository's or the directory's name. Notes are deduplicated by repository rather than by clone, so one library pinned two ways is described once, as the nearer spec has it. The walk. It no longer looks at the project's tests: a plain build never did, and a test's dependency can point at the Configuration the project file returned by value. It goes ring by ring over projects rather than configurations, so what the project names itself comes first even with the library/executable split. The project's own AGENTS.consumer.md is excluded by path, also when a dependency's project file sits beside the project's. Removing a section dropped a blank line even from between two paragraphs, joining them; only a section that ends the file takes it now. A CRLF file gets a CRLF section. A note was checked on its raw bytes and again, under other whitespace rules, on the trimmed ones, so it could pass the first check and fail the merge, which then dropped every note. It is normalised once and checked on exactly what is merged. `test --list` only lists, and no longer writes AGENTS.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:02:28 +01:00
// else, `run` is for chores and `test --list` only lists. Here,
// before compiling, because loading has just fetched every
// dependency, so the notes are current, and an agent fixing a build
// that fails needs them.
if (!testOpts.listOnly) UpdateAgentsMd(config, projectFile);
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md An agent working on a crafter-build project reads its AGENTS.md, and learns nothing there about the libraries the project builds against unless someone copied that in by hand. This repository shows the cost: the code style section of its AGENTS.md restates what Crafter.LintRules enforces, and goes stale whenever the rules change. A library now ships those notes itself, as AGENTS.consumer.md beside its project.cpp; nothing in its project.cpp changes. When a consumer has an AGENTS.md beside its own project.cpp, `crafter-build` and `crafter-build test` keep a section in it, between two marker comments, holding its dependencies' notes. The section is appended the first time and rewritten in place after that. The file is written only when the section changes, and an AGENTS.md is never created. lint, format, run and clean leave it alone. The update runs after loading, which has just fetched every dependency, and before compiling, so a build that fails still has the notes. cfg.dependencyNotes picks the dependencies. Transitive, the default, follows `dependencies` all the way down from the project's own configurations, its tests and its `#pragma crafter dependency` tooling; Direct stops at the first loaded project on each path; None removes the section. A dependency's own pragma tooling never counts: it is about how that dependency is built, not how it is used, and being no `dependencies` edge it is never reached. Nor are a dependency's tests: nothing builds them for the consumer, and their configurations can point at a local their project file has since moved out of. The walk has to know which Configurations belong to another project, and where that project lives, and cfg.path cannot say: only the Configuration GitProject and LocalProject return is rebased, so a dependency's internal configurations keep "./", which resolves to the consumer. The loader records it instead, LoadedProjectOf beside the project cache, and keeps what the root's pragmas resolved to past the frame LoadProject pops, PragmaDependencies. Neither is exported. The markers count only as whole lines outside fenced code blocks, so documentation can quote them. Unbalanced markers and a document ending inside a code block leave the file alone with a warning: guessing which part of someone's file is generated would overwrite their text, and missing the section would add another one on every build. A note that would break the section when it is read back, through a marker line of its own or a code block it never closes, is left out with a warning. The file is replaced through a temporary renamed onto the symlink's target, so a linked AGENTS.md stays a link and a write cut short leaves the old file rather than part of the new one. The code is a new partition, Crafter.Build:Agents, listed in build.sh, CrafterBuildModules, project.cpp and the umbrella. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
if (runTests) {
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
TestSummary summary = RunTests(config, testOpts, projectArgs);
return summary.AllPassed() ? 0 : 1;
}
2026-04-27 07:04:42 +02:00
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
BuildResult result = Build(config, depResults, depMutex);
if (!result.result.empty()) {
2026-04-29 03:27:11 +02:00
Progress::Clear();
2026-04-27 07:04:42 +02:00
std::println(std::cerr, "{}", result.result);
return 1;
}
2026-04-29 03:27:11 +02:00
Progress::Finalize();
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
if (runAfterBuild) {
if (config.type != ConfigurationType::Executable) {
std::println(std::cerr, "-r: cannot run a library");
return 1;
}
2026-04-30 02:20:19 +02:00
fs::path dir = config.BinDir();
feat: project.cpp declares its own dependencies with #pragma crafter dependency A project.cpp could only import std and Crafter.Build, so anything shared between build descriptions had to be a header every consumer copies. Now #pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"] names a sibling crafter-build project that project.cpp itself imports a module from. LoadProject scans for the directive, resolves it through GitProject (a URL) or LocalProject (a path relative to the project file), forces the result into host-tooling mode (the host triple LoadProject's own cache is keyed on, native march, no debug, -fPIC so the archive links into a shared object), builds it, and puts its BMIs and archive on the project.so command line. The dependency's own project.cpp goes through the same loader, so its directives resolve recursively; a cycle between project files is an error instead of unbounded recursion. The dependency is built on every load, fresh or not: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the module depfile — which Module::Check already reads — is what notices the host PCM cache was regenerated. Its artifact mtime is an input to project.so's staleness check, so editing the dependency recompiles project.cpp without touching it. The Linux link gains -fuse-ld=lld because release archives hold ThinLTO bitcode. ProjectDependency(name) hands the same Configuration* back from inside CrafterBuildProject, by configuration name or exported module name, so a test that links the module needn't spell the source twice. It reads a thread_local frame stack that an RAII guard keeps for the whole of LoadProject, which also provides the cycle check. ExpectedOutput() is the link gate's artifact-path lambda hoisted into the :Clang partition, so the staleness check and the -r runner share the one rule. Windows branches are wired identically but unverified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 20:03:22 +02:00
fs::path artifact = fs::absolute(ExpectedOutput(config));
2026-05-18 05:23:11 +02:00
fs::path absDir = fs::absolute(dir);
// wasm targets need either a wasm runtime (wasi-cli) or an HTTP
// server (browser build with index.html). std::system on the
// .wasm path goes nowhere useful — replace with detection.
if (config.target.starts_with("wasm32")) {
feat(lint): const-local and constexpr-constant rules Two rules the AST makes possible, plus the mutation analysis behind them. const-local reports a local that is never written. It is restricted to SCALARS — integers, bools, enums, floating types — and that restriction is what makes the answer exact rather than a guess: a scalar has no member functions, so the only ways to write one are assignment, ++/--, having its address taken, or binding to a non-const reference. All four are now tracked in the walk: - assignment and compound assignment visit their LEFT operand in a write context, the right one normally; - ++/-- and & write their operand; - a call argument is checked against the callee's parameter type, so passing to `const int&` or by value is a read while `int&` is a write; - initialising a non-const reference writes what it binds to. For a class type a non-const method call could mutate it, and deciding that is the whole-program analysis clang-tidy does, so those are simply out of scope rather than guessed at. constexpr-constant promotes a const constant whose initialiser is made only of literals and operators, so `const int A = 1 << 4;` qualifies and `const int B = Compute();` does not. On this repository const-local found 103 candidates, which was too many to be useful, and the reason was informative: most were range-for bindings and pointer locals. `for (T* const x : …)` and `T* const p` are not spellings anybody writes, and the useful constness for a pointer is on the pointee, which this rule cannot advise on. Excluding both leaves 36, all plain bool or enum locals worth fixing — isWasm, isPe, exists, writes, isC and so on. Those 36 are fixed in this commit; the compiler verified every one. Both rules are report-only. The analysis is exact, but adding const is a judgement about intent as much as mechanics, and a wrong suggestion should cost a glance rather than a build. const-local also deliberately does not become a transform: inserting `const` before a shared type would apply it to every declarator in a multi-declarator statement, including any that IS written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 00:50:48 +02:00
const bool browserBuild = fs::exists(absDir / "index.html");
2026-05-18 05:23:11 +02:00
auto have = [](std::string_view exe) {
std::string probe = std::format("command -v {} > /dev/null 2>&1", exe);
return std::system(probe.c_str()) == 0;
};
if (browserBuild) {
2026-05-19 00:50:06 +02:00
// Probe-bind to find a free port starting at 8080 — if the
// user has another dev server running we shift up rather
// than letting caddy/python exit with EADDRINUSE.
2026-07-23 01:24:42 +02:00
auto findFreePort = [](std::int32_t basePort, std::int32_t span) -> std::int32_t {
using Sock = std::int32_t;
const Sock invalid = -1;
auto closesock = [](Sock s){ ::close(s); };
for (std::int32_t p = basePort; p < basePort + span; ++p) {
Sock s = ::socket(AF_INET, SOCK_STREAM, 0);
2026-05-19 00:50:06 +02:00
if (s == invalid) return basePort;
sockaddr_in addr{};
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = htonl(INADDR_ANY);
addr.sin_port = htons(static_cast<std::uint16_t>(p));
2026-05-19 00:50:06 +02:00
bool ok = ::bind(s, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) == 0;
closesock(s);
if (ok) return p;
}
return basePort;
};
2026-07-23 01:24:42 +02:00
const std::int32_t port = findFreePort(8080, 16);
2026-05-26 22:50:08 +02:00
// Cross-origin isolation: the browser coarsens
// performance.now() (and chrono::steady_clock under wasi)
// to ~0.1ms unless the response carries COOP/COEP, which
// floors the --timing overlay's sub-ms phase counters to
// zero. Emitting same-origin / require-corp drops the
// resolution to ~5µs and also unlocks SharedArrayBuffer
// for any future threading work. CORP keeps the local
// asset fetches passing under require-corp.
auto writeFile = [](const fs::path& p, std::string_view contents) {
std::ofstream f(p, std::ios::binary | std::ios::trunc);
f.write(contents.data(), static_cast<std::streamsize>(contents.size()));
};
2026-05-18 05:23:11 +02:00
std::string cmd;
std::string_view picked;
2026-05-26 22:50:08 +02:00
bool isolated = false;
2026-05-18 05:23:11 +02:00
if (have("caddy")) {
picked = "caddy";
2026-05-26 22:50:08 +02:00
isolated = true;
// caddy file-server has no --header flag; write a
// Caddyfile next to the build output. Adapter is
// inferred from the .caddyfile extension when run
// via `caddy run`.
fs::path cf = absDir / "Caddyfile.coi";
writeFile(cf, std::format(
":{} {{\n"
" root * {}\n"
" header Cross-Origin-Opener-Policy \"same-origin\"\n"
" header Cross-Origin-Embedder-Policy \"require-corp\"\n"
" header Cross-Origin-Resource-Policy \"same-origin\"\n"
" header Cache-Control \"no-store\"\n"
2026-07-30 23:01:10 +02:00
// Every EnableWasiBrowserRuntime consumer is a
// single-page wasm app by construction: the
// generated index.html ships an empty <body> and
// the module builds the DOM at runtime. So an app
// that routes on window.location has no file on
// disk for any path but "/", and a bare
// file_server 404s every deep link, refresh and
// shared URL during development.
//
// try_files falls through to index.html only for
// paths that are not real files, so static assets
// still serve normally and a genuinely missing
// asset becomes a visible wrong-content-type
// rather than a silent 404 the app can't see.
" try_files {{path}} /index.html\n"
2026-05-26 22:50:08 +02:00
" file_server\n"
"}}\n",
port, absDir.string()));
2026-07-23 01:24:42 +02:00
cmd = std::format("caddy run --config {} --adapter caddyfile", cf.string());
2026-05-26 22:50:08 +02:00
} else if (have("python3") || have("python")) {
std::string_view py = have("python3") ? "python3" : "python";
picked = py;
isolated = true;
// Inline a tiny SimpleHTTPRequestHandler subclass
// that appends the COI headers on every response.
// Lives in absDir so the user can re-run it manually
// (`python3 absDir/.serve-coi.py 8080`).
fs::path sp = absDir / ".serve-coi.py";
writeFile(sp,
"import http.server, socketserver, sys, os\n"
"class H(http.server.SimpleHTTPRequestHandler):\n"
" def end_headers(self):\n"
" self.send_header('Cross-Origin-Opener-Policy', 'same-origin')\n"
" self.send_header('Cross-Origin-Embedder-Policy', 'require-corp')\n"
" self.send_header('Cross-Origin-Resource-Policy', 'same-origin')\n"
" self.send_header('Cache-Control', 'no-store')\n"
" super().end_headers()\n"
"socketserver.TCPServer.allow_reuse_address = True\n"
"port = int(sys.argv[1]) if len(sys.argv) > 1 else 8080\n"
"os.chdir(os.path.dirname(os.path.abspath(__file__)))\n"
"with socketserver.TCPServer(('', port), H) as s:\n"
" s.serve_forever()\n");
cmd = std::format("{} {} {}", py, sp.string(), port);
2026-05-18 05:23:11 +02:00
} else if (have("php")) {
picked = "php";
2026-05-26 22:50:08 +02:00
// php -S supports a router script — emit one that
// sets COI headers then delegates to the built-in
// static-file handler by returning false.
fs::path rp = absDir / ".serve-coi.php";
writeFile(rp,
"<?php\n"
"header('Cross-Origin-Opener-Policy: same-origin');\n"
"header('Cross-Origin-Embedder-Policy: require-corp');\n"
"header('Cross-Origin-Resource-Policy: same-origin');\n"
"header('Cache-Control: no-store');\n"
"return false;\n");
isolated = true;
2026-07-23 01:24:42 +02:00
cmd = std::format("php -S 0.0.0.0:{} -t {} {}", port, absDir.string(), rp.string());
2026-05-18 05:23:11 +02:00
} else if (have("ruby")) {
picked = "ruby";
cmd = std::format("ruby -run -e httpd {} -p{}", absDir.string(), port);
} else if (have("busybox")) {
picked = "busybox httpd";
cmd = std::format("busybox httpd -f -p {} -h {}", port, absDir.string());
} else if (have("npx")) {
picked = "npx http-server";
cmd = std::format("npx --yes http-server {} -p {} --silent", absDir.string(), port);
} else {
2026-07-23 01:24:42 +02:00
std::println(std::cerr, "-r wasm: no HTTP server found in PATH. Install one of: " "caddy, python3, python, php, ruby, busybox, npx (Node.js).");
2026-05-18 05:23:11 +02:00
return 1;
}
2026-05-31 17:23:34 +02:00
std::println("listening on port :{}", port);
if (!isolated) {
2026-07-23 01:24:42 +02:00
std::println(std::cerr, "warning: {} does not emit COOP/COEP — performance.now() will be coarse " "(~0.1ms). Install caddy, python3, or php for cross-origin isolation.", picked);
2026-05-26 22:50:08 +02:00
}
2026-05-31 17:23:34 +02:00
// Silence the backend's own banner/request logs so the
// experience is identical regardless of which server is
// picked — the line above is the only output.
cmd += " > /dev/null 2>&1";
2026-05-18 05:23:11 +02:00
return std::system(cmd.c_str()) == 0 ? 0 : 1;
}
// wasi-cli wasm — needs a standalone runtime.
if (have("wasmtime")) {
return std::system(std::format("wasmtime {}", artifact.string()).c_str()) == 0 ? 0 : 1;
}
if (have("wasmer")) {
return std::system(std::format("wasmer run {}", artifact.string()).c_str()) == 0 ? 0 : 1;
}
2026-07-23 01:24:42 +02:00
std::println(std::cerr, "-r wasm: no wasm runtime found in PATH. Install wasmtime or wasmer, " "or call EnableWasiBrowserRuntime(cfg) in project.cpp for a browser build.");
2026-05-18 05:23:11 +02:00
return 1;
}
2026-05-01 19:02:14 +02:00
// Run from the artifact's own directory so relative file opens
// (shaders, assets copied alongside the exe via cfg.files) resolve
// against the bin dir rather than the user's cwd. We exit the
// process immediately after, so no cwd restore needed.
fs::current_path(dir);
V2: WASI, -r flag, CI pipeline, examples & tests cleanup WASI / wasm32 target support - Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32") - Skip -march/-mtune for wasm (clang rejects them) - Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync) - .wasm output extension in expectedOutputFor and link command - EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html + runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL set in the templated index.html so a single shim handles any output name -r run flag in the CLI: build then exec the artifact (host targets only; rejects libraries; auto .exe/.wasm extension handling) CI pipeline (.forgejo/workflows/ci.yaml) - Triggers: PR/push to master + manual dispatch - Single arch-latest container job: install deps, bootstrap, self-rebuild, run tests, cross-compile mingw, package both archives, upload artifacts - Rolling 'latest' release published only on push/dispatch to master mingw cross-compile from Linux now works end-to-end: - ExternalDependency cache key includes target so per-target glslang builds don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends 'd' to lib names and breaks linking); cross-compile cmake flags (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...) - project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh link pulls pthread which mingw doesn't link by default) - mingw compile uses -femulated-tls so std::__once_callable etc reference the same emutls symbols libstdc++ provides - mingw link auto-adds -lstdc++exp -lpthread GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux + Windows) now both use it instead of duplicating the resolution. Examples reorg: hello-world, library, with-module, wasi, tests — each with its own README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:24:46 +02:00
return std::system(artifact.string().c_str()) == 0 ? 0 : 1;
}
2026-04-27 07:04:42 +02:00
return 0;
} catch (const std::exception& e) {
2026-04-29 03:27:11 +02:00
Progress::Clear();
2026-04-27 07:04:42 +02:00
std::println(std::cerr, "{}", e.what());
return 1;
}
}