ExternalReconfigure builds a one-file C library from a local repository
as a CMake external, then plants a file in its build directory to stand
for what an old configure left. Built again with the same inputs, the
directory is kept. With its recorded inputs rewritten the way another
crafter-build's configure would have left them, the next build has to
start from an empty directory and build the library again. Before the
fix the planted file survived the reconfigure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StdPcmRebuild builds a project whose main.cpp imports std, then dates
the source before the object and the object before std.pcm, as a std
module rebuilt after the object leaves them, so that nothing else makes
the object stale. The next build has to compile main.cpp again, and the
idle build before it must compile nothing. Without -fmodule-file-deps
the object was only relinked. The test redirects the cache, since
redating a std PCM the real cache shares would rebuild its other users.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MingwSysroot stages a prefix shaped like a Linux system's /usr with a
mingw-w64 cross toolchain in it, without needing a real one: glibc's
<sys/single_threaded.h> in include/, gcc's per-target directory, and a
stand-in mingw-w64 tree whose libstdc++ has an empty std module. A
static library for x86_64-w64-mingw32, so that nothing is linked,
compiles a C++ and a C source that stop with #error when they see the
prefix's include/ or miss mingw-w64's or libstdc++'s headers. Against
the library before the fix, the C++ source stopped on
<sys/single_threaded.h>. The test also checks that the std module is
not cached in the directory older releases built it in against the
prefix itself. It needs only clang, so it runs in CI too, where no
mingw-w64 is installed.
The stand-in std module's source is written through std::format: the
build's import scanner reads string literals as code, and the literal
"export module std;" made the test itself a unit of a module std.
VariantId checks that a prefix gives a mingw configuration a variant of
its own, that two prefixes give two, and that other targets ignore it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WasmBuildRefusals checks that -flto in compileFlags and in linkFlags,
-fno-exceptions and a shared library are each refused with their
cause, and that none of them leaves bin/ or build/ behind. With the
check disabled, each of them failed on a missing std.cppm instead,
after the build directories had been made.
WasiSysrootWithoutExceptions also takes its sysroot through missing
and without the target's directory, each with a message of its own,
checks that a refused build leaves nothing behind, and that a static
library passes without libunwind.
WasmExceptions catches an exception thrown inside libc++.a (stoi).
vector::at throws from a header compiled into the fixture, so the
fixture also passed against the libc++ built without exceptions. Its C
source uses setjmp, which the old C flags did not compile. An
exception that escapes main has to end the run as wasmtime's uncaught
exception, where any nonzero exit used to pass, a trap or a timeout
included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
runtime.js had no test that runs it. WasiBrowserRuntimeExit runs the
real file under node, once per scenario, against a module assembled in
the harness. Between _start and the call that ends it the module has
the frames that caught the old exit: a destructor (catch_all_ref, then
throw_ref), a catch (...) and a noexcept function, each reporting when
it runs. An exit has to run none of them and record its status; a park
has to run none of them, nor the code after it, and leave the program
callable with its state intact; an exit from a later call has to end
the program too, a return from main has to give status 0, and a crash
or an exception nothing caught has to stay an error.
With the JS throw put back into proc_exit, the exit from main and the
exit from a later call both end in the noexcept frame's terminate.
It skips without node, or with a node whose engine lacks the
standardized exception handling. The Flatpak has no node, so CI does
not run it; the native loop does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WasmExceptions builds a fixture for wasm32-wasip1 and runs it under
wasmtime. It throws through four frames whose destructors have to run,
catches the exception, catches one that vector::at throws, and with an
argument lets one escape main, which has to fail the run. With the
previous flags the fixture does not compile ("cannot use 'throw' with
exceptions disabled"), and each part of turning exceptions on fails it
in its own way, from a BMI built without them to the legacy encoding
wasmtime refuses.
WasiSysrootWithoutExceptions takes a scratch sysroot through three
states, no libunwind, libunwind under eh/ and libunwind in the
triple's own directory, and checks that only the first stops the
build, with the message naming the library it looked for. It needs no
working wasm toolchain, so it also runs where the others skip.
WasmHelloRuns and WasmExceptions treat a sysroot without the
exception-enabled libraries as missing and skip. The manifest tests
expect the exception-handling probe on every entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #42, fixes#41. Build() now refuses a configuration whose transitive
dependencies hold two configurations exporting a module of the same
name, such as one project loaded along two paths with different arg
lists, before it builds anything, and names both paths. Until now both
builds ended up in the one program, which compiled cleanly and then
read garbage. Merged cleanly on top of claude/forward-project-args;
the suite, with claude/dependency-notes' fixtures, passes under the
new check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #40. GitProject and LocalProject default their args to the loading
project's own (ProjectArgs()), this tree lints with LintRules' lite
set, and the fixes from the branch's review: a project file gets its
args only for the CrafterBuildProject call, host tooling gets project
cache entries of its own, and `--` ends crafter-build's options.
The conflicts with claude/dependency-notes are all places both sides
added to:
- Platform.cppm and Platform.cpp: PragmaDependencies() beside
ProjectArgs() and LoadingHostTooling(), all three kept.
- LoadProject: the record PragmaDependencies() reads moves up with the
frame's dependencies, which this branch sets before dlopen. Left
where it was, it would have copied preDeps.configs after the vector
was moved into the frame, and recorded nothing.
- README: the pragma section's new last sentence goes before the new
"Dependency notes for agents" section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixture is P -> M and P -> A -> M, where M's own flag --wide
changes a struct it exports. Loaded with --wide --split, P gives M its
args without --wide while A passes them on whole: Build(P) has to fail,
name the module and both paths, and leave no bin/ anywhere. Loaded with
--wide alone, both paths give M the same list, there is one M, and the
build has to go through. Before the fix the first case built cleanly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixture app has `#pragma crafter dependency "../tool"` and loads
../shared for itself, which the tool loads too; the app is loaded with
no args, the case where the two cache keys used to match. With
CRAFTER_BUILD_MARCH=x86-64-v2, the app's copy of shared has to keep
that march and no -fPIC, and the tool's copy has to be host tooling.
Before the fix the app got the tool's copy back: one Configuration,
with march native and -fPIC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in dependency notes: a crafter-build project that ships an
AGENTS.consumer.md beside its project.cpp has it kept in a marked
section of each consumer's AGENTS.md, refreshed by `crafter-build` and
`crafter-build test`, chosen by cfg.dependencyNotes (Transitive,
Direct, None). The new partition is Crafter.Build:Agents; the loader
records where each GitProject and LocalProject came from and what a
project file's pragmas resolved to.
The branch started before master's four external-fetch commits. Both
touch External.cpp, the Clang partition, project.cpp, README.md and
AGENTS.md, but in different places, so the merge needed no hand
reconciliation; the merged tree passes build.sh, the self-host build,
format --check and the test suite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crafter.LintRules now has AddLiteRules: the house rules without the ones
that only decide layout, whitespace and naming. This tree switches to it.
The full set's naming rule now wants camelCase statics and constants,
which this tree spells PascalCase throughout, so `lint` with it reported
49 findings on code that follows the documented convention; with the lite
set `lint` and `format --check` are clean.
AGENTS.md described the 17 rules as the enforced style and told
contributors to let `format` restyle their code. It now says which 8 rules
`lint` enforces, that `format` only rewrites string concatenation and
builtin integer types, and lists layout and naming as conventions kept by
hand.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crafter.Math and Crafter.Asset now get this project's args through the
GitProjectSpec default. Loading this file therefore needs a crafter-build
that exports ProjectArgs: after pulling, run ./build.sh before the
bootstrap binary loads it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ProjectArgs loads a chain app -> lib -> leaf of LocalProjects with --debug
and a project flag. lib, loaded without .args, has to see both, and so
does its own sibling, which takes the default one level further down.
Copies loaded with `.args = {}` and with an explicit list get nothing and
exactly that list. The GitProjectSpec default is checked on the spec
itself, since a GitProject would need a repository to clone. Without the
defaults, five of its checks fail.
ProjectDependency now also loads its consumer with a project flag: the
pragma dependency must still come back with no args. Without the explicit
`.args = {}` in the pragma path it gets the consumer's flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge on its own: the section is appended after one blank line,
merging again changes nothing, a moved section is rewritten where it
stands, and removing it gives the file its own bytes back. Markers
inside a fenced block, one quoting a fenced block of its own, are not
the section's. Unbalanced markers, a document that ends inside a code
block and notes that would break the section are refused, naming the
line.
End to end, through LoadProject on a staged fixture: app names lib
and has tool as host tooling; lib builds against base and has libtool
as its own. Transitive gives tool, lib and base in that order, and
neither app's own notes nor libtool's. A second update writes nothing,
mtime included. Direct drops base, edited notes replace the old ones
in the one section, and None restores the original bytes. A project
without an AGENTS.md does not get one, and a symlinked AGENTS.md is
written through the link, which stays a link. Through Run in process,
lint and run leave the file alone, while `test --list` and a build
that fails at the link both refresh it.
The fixture's AGENTS.md is checked in as AGENTS.md.in: an AGENTS.md in
this tree would read as instructions for the agents working on it.
Moving the CLI hook above the run and lint branches, or letting Direct
walk past the first loaded project, fails four of the checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An agent working on a crafter-build project reads its AGENTS.md, and
learns nothing there about the libraries the project builds against
unless someone copied that in by hand. This repository shows the cost:
the code style section of its AGENTS.md restates what Crafter.LintRules
enforces, and goes stale whenever the rules change.
A library now ships those notes itself, as AGENTS.consumer.md beside
its project.cpp; nothing in its project.cpp changes. When a consumer
has an AGENTS.md beside its own project.cpp, `crafter-build` and
`crafter-build test` keep a section in it, between two marker comments,
holding its dependencies' notes. The section is appended the first time
and rewritten in place after that. The file is written only when the
section changes, and an AGENTS.md is never created. lint, format, run
and clean leave it alone. The update runs after loading, which has just
fetched every dependency, and before compiling, so a build that fails
still has the notes.
cfg.dependencyNotes picks the dependencies. Transitive, the default,
follows `dependencies` all the way down from the project's own
configurations, its tests and its `#pragma crafter dependency` tooling;
Direct stops at the first loaded project on each path; None removes
the section. A dependency's own pragma tooling never counts: it is
about how that dependency is built, not how it is used, and being no
`dependencies` edge it is never reached. Nor are a dependency's tests:
nothing builds them for the consumer, and their configurations can
point at a local their project file has since moved out of.
The walk has to know which Configurations belong to another project,
and where that project lives, and cfg.path cannot say: only the
Configuration GitProject and LocalProject return is rebased, so a
dependency's internal configurations keep "./", which resolves to the
consumer. The loader records it instead, LoadedProjectOf beside the
project cache, and keeps what the root's pragmas resolved to past the
frame LoadProject pops, PragmaDependencies. Neither is exported.
The markers count only as whole lines outside fenced code blocks, so
documentation can quote them. Unbalanced markers and a document ending
inside a code block leave the file alone with a warning: guessing which
part of someone's file is generated would overwrite their text, and
missing the section would add another one on every build. A note that
would break the section when it is read back, through a marker line of
its own or a code block it never closes, is left out with a warning.
The file is replaced through a temporary renamed onto the symlink's
target, so a linked AGENTS.md stays a link and a write cut short leaves
the old file rather than part of the new one.
The code is a new partition, Crafter.Build:Agents, listed in build.sh,
CrafterBuildModules, project.cpp and the umbrella.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight tests that include a header from the same tracked branch of a
local repository go through RunTests at once, and again after the
branch has moved. GIT_TRACE logs every git process, and each fetch of
the repository, clone or pull, runs one `git upload-pack` in it, so the
trace counts the fetches: one per run rather than one per test, and the
second run still pulls and brings the clone to the new commit.
On the commit before the fix both runs fetch eight times. The tests
themselves pass there, because the lock from 653dd0b already keeps the
pulls apart. Going through RunTests, rather than Build() on threads of
the test's own, also holds RunTests to building its tests with one
depResults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Shaders record the files glslang opened (a depfile) and their settings
(source path, stage, entry point, defines, include directories) in the
build directory, and rebuild when an included file or a setting changes,
not only the shader's own source.
- An executable relinks when its link command changed, not only when an
object is newer: a link flag in project.cpp, or a dependency's library
that moved (an ExternalDependency built into a new cache directory). The
command is recorded beside the objects; its flags come from ordered sets,
so the order the dependency threads finish in can't change it.
- An external's newest artifact counts its libDirs and versioned .so names.
- Submodules are checked out at the revisions the fetched commit records,
from the URLs its .gitmodules gives, after every clone, switch, checkout
and pull, so a pinned commit builds against its own submodules, and one
moved by hand is put back; with no .gitmodules, git isn't asked.
- Each external clone's fetch and build hold a file lock: parallel builds
pulled into one clone at once ("cannot lock ref").
- mingw:
- -stdlib=libstdc++ always;
- the std module's source is found through libstdc++.modules.json in the
mingw toolchain's own library directories (the generic lib/ holds the
host's Linux one), asking the compiler once a run;
- a toolchain that isn't there is a build error saying where it looked,
not std::terminate;
- CRAFTER_MINGW_DIR names the prefix, the directory holding
x86_64-w64-mingw32/ and lib/gcc/;
- inside the Flatpak the host's /usr mingw is used.
- RunScript's exit-status check reads as the one range it is.
- Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules,
LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and
ToolchainResolve extended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local Flatpak build was one long flatpak-builder line in AGENTS.md,
with its state wherever the caller happened to stand, and installing
the result was not written down anywhere. It is not a rare chore: the
development box compiles with the distro's clang and the Flatpak with
its bundled LLVM, a major version apart at the moment, so anything
toolchain-sensitive has to be tried in the Flatpak too.
Both are now scripts in project.cpp, the first users of cfg.scripts.
`run flatpak` builds the manifest as it is, from the working tree, and
ends with `crafter-build --version` from inside the result. Its state
is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and
not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in
the sandbox, so it does not depend on the host variant, and a
variant-keyed directory would strand the LLVM cache on every host
compiler update. The manifest already skips build/ when it copies the
sources. flatpak-builder keys its cache on the manifest's path, so the
manifest is never rewritten into a copy the way CI stamps the version
in, and a local build reports 0.0.0-dev. The only argument accepted is
--jobs=<N>: anything that ends the run without a build would leave the
smoke test checking the previous one.
`run install` installs the last export as branch `local` from a user
remote of its own and makes it current, leaving the published branch
installed. It does not build first: flatpak-builder never caches a
`type: dir` module, so every build reruns build.sh and the suite.
remote-add --if-not-exists is followed by remote-modify --url, because
the remote may exist and point at another checkout.
Both refuse to run from the Flatpak build of crafter-build, which has
no flatpak to drive.
The smoke test goes through `flatpak build` and not `<builder> --run`,
which is what CI uses. flatpak-builder 1.4.9 run as the
org.flatpak.Builder app exits 1 when the command it ran succeeded and 0
when it failed (`true` gives 1, `exit 3` gives 0), so the script
reported a good build as a failure.
Measured on the 56-core box with LLVM and the WASI sysroot cached: 260
seconds, the crafter-build module only, 29 tests passing in the sandbox.
A second `run install` after a rebuild moved the installed commit.
`clean` now also drops that state, about 2 GB and a cold LLVM build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
RunScript on hand-built configurations: the named script alone runs,
with the project, the project directory and the arguments exactly as
given; its return value comes back whole from 0 to 255 and as 1 outside
that; a script that throws exits 1. A project without scripts, a
duplicate, an untypeable name, a missing callback and an unknown name
all fail without running anything. ctx.Run returns the command's exit
code and 128+signal for a killed one, and ShellQuote carries quotes,
spaces and expansions through sh intact.
Then the same through Run() with a fixture project, because the parsing
is where this goes wrong. `run record a --help -v clean --debug
--version run` must hand all seven words to the script, leave the one
project arg that stood before `run` alone, and not delete the staged
bin/ and build/. `run clean` and `run help` look for scripts of those
names. Moving the two capture branches below the verbs in the parse
loop fails this test.
The command-line half needs the installed module sources and is skipped
without them, but only when the in-process half reported no failure.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project.cpp could describe how a project is built, tested and linted,
but not the chores around it: packaging, installing, publishing. Those
stayed shell lines in a README or in someone's history.
Configuration gains `scripts`, a plain list like `defines`. A Script is
a name and a C++ callback, as a lint rule is; there is no script
language and no config file. The callback gets a ScriptContext (the
project, the project file's directory, the arguments) and returns the
exit status. ctx.Run runs a shell command with the terminal attached
and returns its status. It is only stored when project.cpp loads, which
happens for every verb and for every project that depends on this one.
`crafter-build run <name> [args...]` runs one, a bare `run` lists them.
Only the scripts of the project crafter-build was pointed at are
runnable, never a dependency's.
Everything after the script's name is handed to the script untouched.
The two branches that do this sit ahead of every other branch of the
parse loop, --version and --help included. Below the verbs,
`run package clean` would delete build/, and any other word would land
in the project's args and so in the variant hash, moving the build to
another directory. Options before the name keep their meaning, so
project args go there.
The driver is its own partition, :Script. It refuses a project with no
scripts, a name that is empty or starts with '-' (it could never be
typed), a name declared twice (running either would be a guess) and an
unknown name, each with exit 1 and without running anything. A script
that throws exits 1. A return value outside 0..255 becomes 1: an exit
status is eight bits, and a raw wait status such as 256 would otherwise
wrap to success.
Two helpers in :Platform:
- RunCommandAttached runs a command with inherited stdio. Every helper
so far captured through popen, which is right for a compile step and
useless for a command that runs for half an hour or asks a question.
It reads the wait status through DecodeWaitStatus, which
RunCommandChecked and RunCommandWithTimeout now share.
- ShellQuote is exported. It existed twice, as ShellQuote in :External
and ShellQuoteSh in :Test, with the same behaviour; both are gone. A
script that puts a path into a command needs it too.
Not included: a string shorthand for one-command scripts (a one-line
lambda covers it) and a ctx.Build() helper (Build() is already
exported).
A new partition and new exported symbols: run ./build.sh.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GetCompileCommand is checked as text, without a compiler: the forced
include directory is present for linux-gnu and linux-musl targets with
a sysroot, and absent without a sysroot, for wasm32 and for mingw.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ShaderCompile registers one source twice and checks that the variant
gets its own output name, takes the other #ifdef branch and receives
the value of a NAME=VALUE define. Staleness is pinned per output: while
only the plain output exists the variant must still be due, or Build()
would never compile a variant added to a built project. Malformed
defines and a variant with a path separator must be refused by name.
ShaderVariantBuild drives the same through Build(), which is where the
entries compile in parallel into one directory: a variant added to an
already built configuration is compiled into its own file, and two
entries with one output name fail the build without touching the
existing output. Its fixture tests the macro with `#if`, which pins
that a bare define is 1 and not empty.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SysrootStdModule runs over staged layouts that need no compiler: a
sysroot with its own module source, one on a point release of the
toolchain's libc++, one on the next major release with a shipped
source, one nothing was shipped for, one without libc++ at all, and the
two cases where a release cannot be read. The last block drives Build()
against a sysroot claiming libc++ 99 and expects the one-line
explanation back instead of compiler output, which is the failure the
fix replaces.
ToolchainResolve additionally checks that the release is read from the
libc++ the module source comes from (the prefix's, an explicit libc++
install's, the system's) and where the shipped sources are looked up.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
packaging/flatpak/ holds the manifest for net.catcrafts.CrafterBuild and
its side files. The runtime is the Freedesktop Sdk, not the Platform: a
build tool needs glibc's headers and crt objects, gcc's libgcc_s, cmake,
ninja, git and python at run time. Everything compiler-shaped is ours
under /app, built from one LLVM 22.1.8 source tree: clang, lld, libclang,
libc++ with its std module, compiler-rt for the host and for wasm32, the
WASI sysroot (wasi-sdk 34 in sysroot-only mode, driven by the clang just
built) and wasmtime for running the result. One tree means the modules
crafter-build builds and the compiler that consumes them cannot drift
apart, and the toolchain resolver finds all of it beside the executable
with no configuration.
The crafter-build module runs the bootstrap, the self-host build and the
test suite inside the sandbox, so a Flatpak that installs has passed the
suite with the toolchain it ships. CRAFTER_BUILD_MARCH is x86-64-v2 for
the shipped binary; user projects keep -march=native.
WasmHelloRuns compiles a fixture for wasm32-wasip1 and runs it under
wasmtime. The existing wasm tests only inspect generated files, so this
is the one that proves the sysroot, the std module for wasm and wasm-ld
work together; it skips where the sysroot or wasmtime is missing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
std.pcm was rebuilt when it was older than std.cppm, and project.so when
it was older than the crafter-build executable. Both inputs are installed
files, and an OSTree deployment (the Flatpak) gives every installed file
mtime 0: std.cppm and the executable always looked older than the cache,
so a clang bump would never rebuild std.pcm and a crafter-build update
would never recompile project.so. clang crashes on a PCM from another
compiler instead of reporting it, so the failure would have been a
backtrace on the user's first build after an update.
Each artifact now has a one-line stamp beside it. std.pcm's covers the
precompile command and the module source's bytes, which also catches a
libc++ update under the same clang. project.so's covers the crafter-build
version, the toolchain key, the module source stamp and the compile
command; project.cpp's and the pre-dependency artifacts' mtimes remain
inputs, since those live next to it with real timestamps. The module
source stamp stands in for the executable: project.so resolves Crafter::
symbols from the running binary, so only declarations matter, and an
implementation-only rebuild of crafter-build no longer recompiles it.
The std.pcm stamp folds into the module cache stamp, so a rebuilt
std.pcm rebuilds the PCMs that import it.
The cache directory carries the toolchain key
(<target>-<march>-<clang version>-<hash>), so a compiler bump starts
cold instead of handing the new clang PCMs it will crash on. LoadProject
and Build() now derive the directory from the same StdPcmDir. Old
<target>-<march> directories are left behind; `clean` never touched the
cache and still does not.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every clang++, clang and llvm-ar the tool spawned was a bare name on
PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI
sysroot /usr/share/wasi-sysroot and the mingw headers
/usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A
toolchain installed anywhere else, which is what the Flatpak ships
under /app, had no way in.
Crafter.Build:Toolchain resolves all of it once per process:
CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build
(<prefix>/bin, found through the same walk GetCrafterBuildHome does),
else PATH. What sits beside the compiler is used when present: its
libclang.so, its clang-c headers, its std module source, its WASI
sysroot. An explicit prefix without a clang++ is an error rather than a
fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already
exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other
than the compiler's own, and CRAFTER_MINGW_DIR now does what its help
line always claimed: it is passed as --sysroot for the mingw target.
Binaries linked by a toolchain outside /usr get an rpath to its lib
dir, since its libc++.so.1 is not on the loader's default path.
HostTarget() is the resolved clang's -print-target-triple. A clang built
without a vendor reports x86_64-unknown-linux-gnu where Arch's reports
x86_64-pc-linux-gnu, so target equality that decides whether a test runs
locally and which targets a bare `test` sweeps goes through SameTarget,
which ignores the vendor field. The strings themselves are never
rewritten: clang rejects a PCM whose triple differs from the importing
unit's. The examples and README stop hardcoding the host triple; the
default is already the host.
`crafter-build --version` prints the version baked in at build time
(CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the
compiler and the host triple, and is answered before any argument is
forwarded to project.cpp. The version is a define on the library, so
each version is its own library variant and relinks the executable and
tests once per commit.
build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR
and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already
there, and writes a fresh archive instead of appending to one that may
hold members from an older layout.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
crafter-build is moving to a single Flatpak distribution, and there is
no Windows host in that picture: no build.cmd, no MSVC or mingw host
build, no DLL plus launcher. Delete every host-Windows path instead of
leaving it to rot unverified: the cmd.exe command runner and quoting,
the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants
with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port
probe, the LoadLibrary libclang loader, the console-width and isatty
shims, and the preprocessor guards around the Linux bodies.
The x86_64-w64-mingw32 cross-compile target stays. It is target logic
(output names, -femulated-tls, the libstdc++ link extras, copying a
dependency DLL next to a mingw exe, wine as the runner) and did not
depend on any of the removed code. The msvc target goes with the host
that was the only thing able to use it.
Three target-path warts fixed on the way: ExpectedOutput named a mingw
static library <name>.lib while the archive step wrote lib<name>.a, so
those libraries relinked on every run; the import lib is no longer
copied next to a mingw exe, since the dependency's own bin dir is
already on the link line; and llvm-ar now writes every archive, not
only the LTO ones, so Debug and Release archives are made the same way.
CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary.
Nothing downstream used them, so they are removed rather than kept as
an empty macro nobody can verify.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Brings in `#pragma crafter dependency`, the switch to Crafter.LintRules
for the house style, and the contributor guide in AGENTS.md.
The branch predates the musl host work on master, and both rewrote the
command that compiles project.so, so the merge reconciles them by hand:
- LoadProject compiles project.so for hostConfig.target (HostTarget(),
from master) with the pre-dependency module path and archives from
the branch. -fuse-ld=lld appears once, with both reasons in the
comment.
- ForceHostTooling added -fPIC only for the literal x86_64-pc-linux-gnu
triple. hostConfig now comes from HostTarget(), so on Alpine the host
is x86_64-alpine-linux-musl and the dependency archive would not have
been position-independent. It now matches any *-linux-gnu or
*-linux-musl host, as project.cpp and the cmake flags do since the
musl change, and its comment no longer argues against HostTarget().
- The ProjectDependency test compared the dependency's target against
the glibc triple, which the musl CI job would fail; it compares
against HostTarget().
- README keeps master's paragraph on the bootstrap triple. AGENTS.md
describes the test block as `if (linuxTarget)`, which is what
project.cpp has now.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
lint-rules.h moves to its own repo as the module Crafter.LintRules, which
project.cpp now imports through `#pragma crafter dependency`. Editing a
rule no longer needs `touch project.cpp`: the rebuilt dependency makes
project.so stale by itself. tests/HouseRules moves with the rules.
Building this checkout needs a crafter-build that understands the
directive (`./build.sh` bootstraps one) and the Crafter.LintRules repo on
Forgejo.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project.cpp could only import std and Crafter.Build, so anything shared
between build descriptions had to be a header every consumer copies. Now
#pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"]
names a sibling crafter-build project that project.cpp itself imports a
module from. LoadProject scans for the directive, resolves it through
GitProject (a URL) or LocalProject (a path relative to the project file),
forces the result into host-tooling mode (the host triple LoadProject's
own cache is keyed on, native march, no debug, -fPIC so the archive links
into a shared object), builds it, and puts its BMIs and archive on the
project.so command line. The dependency's own project.cpp goes through
the same loader, so its directives resolve recursively; a cycle between
project files is an error instead of unbounded recursion.
The dependency is built on every load, fresh or not: clang crashes rather
than erroring when it imports a BMI whose own imported PCM was rebuilt
underneath it, and the module depfile — which Module::Check already reads —
is what notices the host PCM cache was regenerated. Its artifact mtime is
an input to project.so's staleness check, so editing the dependency
recompiles project.cpp without touching it. The Linux link gains
-fuse-ld=lld because release archives hold ThinLTO bitcode.
ProjectDependency(name) hands the same Configuration* back from inside
CrafterBuildProject, by configuration name or exported module name, so a
test that links the module needn't spell the source twice. It reads a
thread_local frame stack that an RAII guard keeps for the whole of
LoadProject, which also provides the cycle check.
ExpectedOutput() is the link gate's artifact-path lambda hoisted into the
:Clang partition, so the staleness check and the -r runner share the one
rule. Windows branches are wired identically but unverified.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Linux host code paths were gated on the x86_64_pc_linux_gnu target
define alone, and LoadProject hardcoded --target=x86_64-pc-linux-gnu when
compiling project.cpp and the module PCMs. Both break on musl hosts: the
glibc release launcher cannot run there at all (no glibc loader; gcompat
lacks libgcc_s and the __isoc23_* symbols), and a musl-built launcher
would still compile project.cpp for the wrong libc.
- The host gates test clang's __linux__ instead of one triple's define:
they mean "Linux host", whatever the libc (or, later, the CPU).
- LoadProject derives the host triple from HostTarget() (clang's
-print-target-triple), which is also more correct on glibc distros
whose triple isn't x86_64-pc-linux-gnu.
- project.cpp / the cmake libc++ flags treat any *-linux-gnu or
*-linux-musl target as Linux.
- build.sh targets the host toolchain's triple (CRAFTER_BUILD_TARGET
overrides) and derives the per-target define from it. On Arch this is
byte-for-byte the previous behaviour.
- CI: a release-musl job bootstraps on alpine:edge, runs the tests, and
attaches crafter-build-linux-x86_64-musl-v2.tar.gz to the rolling
latest release (via the API, so the glibc assets survive). Dynamic
against musl libc++: the launcher dlopens project.so, so it cannot be
fully static.
First consumer: imsd's package CI, which cross-compiles for aarch64 from
an Alpine container.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The fixture puts a #warning in each kind of source the build compiles —
module interface, module implementation, C, consumer — plus an unknown -z
value so ld.lld warns on the link step too, and asserts the build succeeds
cold (every unit compiling, every warning emitted), succeeds with nothing to
do, and succeeds again after each source is touched back into staleness.
The staged copy lives outside the repo because the whole test depends on
units compiling for the first time.
The last pass breaks a source on purpose: "never fails" would satisfy
everything above just as well, so the error path is asserted too — both that
it fails and that it still reports the compiler's diagnostic.
One header per pass, asserting both directions — the objects that include
it recompile, the ones that don't are untouched — then running the binary,
since "was recompiled" only matters if the program agrees with itself.
Covers the module interface (a macro in its global module fragment decides
an exported class's layout), the implementation unit, a C source, an idle
rebuild that must recompile nothing, and an object whose depfile is gone.
The implementation unit's header has a space in its name so the depfile
spells it escaped.
A primary module interface unit — `export module Widget;`, no partitions —
recorded nothing about what it imported. GetInterfacesAndImplementations
registered the Module and then erased the file from the scan list, so the
import pass only ever saw partitions, and Module had no vectors to hold an
edge anyway.
Two consequences, both reported as issue #26:
Module::Check consulted only its own .cppm and its partitions. A data
member added to an imported module left Widget.pcm, Widget.o and every
consumer object untouched while the imported library rebuilt and both
binaries relinked — one executable holding two class layouts, no
diagnostic, and a crash somewhere unrelated. Wiping build/ was the only
cure, so `crafter-build test` could not be trusted straight after an
interface edit.
Module::Compile waited on nothing. Two modules in one Configuration
compile on concurrent threads, so a primary interface importing a
sibling was a coin flip between working and "module 'Base' not found".
Partitions never had either problem — they carry the same three vectors and
Check/Compile honour them — which is why the gap only surfaced on a module
whose interface is one flat unit.
Module now carries moduleDependencies, externalModuleDependencies and
pendingImports with the same meanings as on ModulePartition; primary units
stay in the scan list so their imports land there; Check sees through them;
Compile orders itself behind a local sibling; and ResolvePendingImports
sweeps them so an edge survives dependencies being wired up afterwards.
Build() now Checks every interface before spawning any compile thread — the
`compiled` flag a waiter blocks on is raised either by a Compile that runs
or by the Check that decides none is needed, so a Check still pending while
another module's thread waits would have hung the build.
Resolves#26
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three follow-ons to the stale-build report, all cases of an identity not
capturing something that changes the output.
The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by
every crafter-build on the machine, and freshness was a per-file mtime
comparison. That cannot tell "this PCM is newer than my source" from "this PCM
was built from different sources that happen to be newer", so a package install
and a working checkout — or two checkouts of different versions — silently
compiled their project.cpp against each other's declarations. Invalidation now
keys on a stamp over the bytes of every module source, which also covers the
case one file's mtime never could: the cached PCMs import each other, so a
change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm
untouched.
Project args ApplyStandardArgs does not itself interpret are now folded into
VariantId. Such a flag typically decides what gets compiled or bundled — the
report's example is --no-webgpu dropping entries from cfg.files — and without it
both settings shared one bin dir and interleaved their outputs there, leaving a
bundle matching neither. Sorted and deduplicated so flag order doesn't split the
cache, and inherited by test Configurations.
`crafter-build clean` removes the project's bin/ and build/ trees. It
deliberately does not load project.cpp: cleaning is most often reached when
something is already wrong, and a clean that first needs the project to compile
is useless exactly then.
Adding a data member to a class in a module interface did not rebuild every
object compiled against the old layout. The build succeeded with no error or
warning and the resulting binary mixed both layouts, surfacing later as a
SIGSEGV in a destructor.
GetInterfacesAndImplementations scans a TU's `import X;` statements when the
source is declared. An import that matches neither a module in the
Configuration nor one reachable through `dependencies` was dropped on the
floor, leaving that TU with no staleness edge to the interface it consumes.
`dependencies` is frequently assigned *after* the scan — AddTest does exactly
that, resolving tests/<name>/main.cpp and only then returning a builder whose
.Dependencies() supplies the library — so consumers of a dependency's modules
routinely carried no edge at all. A layout change then rebuilt the library,
relinked the consumer, and kept the consumer's object as it was.
Unresolved names are now remembered on the partition/implementation as
pendingImports, and Configuration::ResolvePendingImports retries them against
the dependency DAG as it stands. Build() calls it immediately before comparing
mtimes, which closes the window for every caller rather than only the ones that
declare in the right order; TestBuilder::Dependencies also calls it so the
Configuration is coherent for anyone inspecting it before the build.
Resolves#27
Canonical LGPL-3.0-only text, GPL-3.0 companion, SPDX headers on all
first-party sources, MIT for examples/. Vendored lib/ untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and
baked a single wasm URL into index.html, so newer codegen features that
aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD
revisions later) couldn't be adopted without dropping the browsers that lack
them.
Add a general, feature-parameterized mechanism owned entirely by
Crafter.Build:
- Configuration::wasmVariants declares N codegen variants (label, extra -m
flags, runtime probes). Build() compiles the baseline plus one
outputName.<label>.wasm per variant, recompiling the whole graph (incl.
dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU
codegen, not a link switch. wasmVariantFlags folds into VariantId so each
variant's objects/PCMs land in their own build+bin dir.
- EnableWasiBrowserRuntime emits a variants.json manifest (label -> url +
probes), preferred-first with the baseline as the universal fallback.
- The shipped runtime.js runs inlined wasm-feature-detect probes
(relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads),
picks the first variant whose probes all pass, and falls back to the single
baked CRAFTER_WASM_URL when no manifest is present (backward compatible).
- EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the
motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari
still flag-gates it as of mid-2026).
Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and
wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the
relaxed-simd variant and runs it.
Resolves#24
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build() resets each Module/ModulePartition's per-build `compiled`/`checked`
flags so a reused Configuration re-evaluates mtimes. That reset recursed into
cfg.dependencies — but dependency Configurations are shared across the build
DAG and each is compiled concurrently by its own Build() call.
A parent/sibling's recursive reset could therefore clear a shared dependency's
module `compiled` atomic *after* that dependency's module-compile thread had
set it true and exited, but before an intra-config waiter (its impl, or a
dependent partition) ran compiled.wait(false). The waiter then blocked forever
on a flag nothing would re-signal: the build froze mid-compile, idle, with no
compiler process alive — exactly the hang in issue #16.
Reset only the current configuration's own modules. Every config in the tree
already gets its own Build() call (the per-PcmDir builder registered in
depResults), which resets its own state at the top of that call, sequenced
before its compile threads spawn. Cross-config module state is consulted only
via PCM file mtimes and the depResults futures, never via these flags, so the
narrower reset is correct and removes the data race entirely.
Adds ConcurrentDependencyReset: builds a static-lib dependency fully, then
builds a consumer that depends on it while the dependency is already cached in
depResults (so it is never rebuilt), and asserts the consumer build leaves the
dependency's module `compiled` flag intact. Fails deterministically on the old
recursive reset; passes with the fix.
Resolves#16
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two crafter-build invocations sharing XDG_CACHE_HOME used to clobber each
other's writes to <cache>/<target>-<march>/std.pcm and the
Crafter.Build-*.pcm modules: each LoadProject path wrote directly to the
final path, so a reader could see a half-written file and die with
"malformed or corrupted precompiled file: 'can't skip to bit X from Y'"
(issue #14). Every BuildStdPcm / EnsureCrafterBuildPcms write now goes via
<final>.tmp.<pid>.<seq> and atomic-renames into place; concurrent writers
always see either the old or the new file, never torn bytes. The mingw-on-
Linux std.cppm copy is per-PID for the same reason. Adds a regression test
(ConcurrentCacheRace) that races four LoadProject() calls against a cold
scratch cache — reproduces the race 5/5 without the fix and passes 5/5
with it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The suite had only HelloWorld, which built and exited an empty exe. Add
in-process tests covering each public surface area users actually touch:
- StaticLib / ModuleInterface / DependencyLink — Build() against
fixtures for libraries, project-local module interfaces, and
cross-config module deps with link verification (runs the built exe).
- ShaderCompile — drives Shader::Compile directly, validates SPIR-V
magic + Check() idempotency.
- StandardArgs — covers --debug, --target=, --march=, --mtune=,
--lib/--shared promotions, and ArgQuery::Has / Get.
- TestRunnerSpec — FromSpec parse rules, ForTarget routing for host,
wasm32-wasip1, aarch64-linux-gnu (+ sysroot QEMU_LD_PREFIX),
i686 → qemu-i386 rewrite, mingw → wine on Linux hosts, FromEnv.
- VariantId — confirms type / debug / sysroot / defines / compileFlags /
target / march all perturb the cache key, plus PcmDir routing.
- WasiBrowserRuntime — calls EnableWasiBrowserRuntime, asserts the
three cfg.files entries get registered and index.html had its
template placeholder substituted.
- RunSingleTestExit — drives RunSingleTest against tiny sh scripts and
pins the documented exit-code mapping (0/77/non-zero) and the
Cmd-prefix runner path.
Closes#12.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Linux→mingw cross-compile now produces the same architectural shape as
build.cmd (DLL + import lib + launcher exe) instead of a single static
binary. The CI Windows artifact becomes a first-class drop-in: a user
on Windows can run crafter-build.exe against any project.cpp and have
it produce real Windows binaries — for either mingw or MSVC ABI.
What changed:
project.cpp: when target=mingw or target=msvc, crafter.build-lib is
built as LibraryDynamic instead of LibraryStatic so the link emits a
DLL + import lib (matching what build.cmd produces natively).
Crafter.Build-Clang.cpp Build():
- LibraryDynamic now branches per target — mingw emits <name>.dll +
lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll +
<name>.lib via /IMPLIB; unix unchanged.
- expectedOutputFor returns .dll for Windows-target dynamic libs.
- Executable on Windows host now branches per target: mingw target
uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps
the existing path. Both auto-copy LibraryDynamic dep DLLs + import
libs alongside the launcher exe (Windows resolves DLLs from the exe's
own directory at load time).
- Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so
CRAFTER_API resolves to dllimport in their PCMs.
- mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic
-lpthread so produced .exe/.dll don't depend on a particular
libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH
(avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole).
Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which
is now dead weight.
- -r flag resolves to an absolute path before std::system, otherwise
cmd.exe rejects "./bin/..." with "'.' is not recognized...".
Crafter.Build-Platform.cpp:
- Split the Windows-host block into shared shell helpers (#if MSVC ||
MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject /
EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm.
- Mingw-host LoadProject compiles project.cpp with --target=mingw,
--sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR),
-femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept
/EXPORT:NAME), and links against libcrafter-build.dll.a from the
launcher's directory.
- Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target
so a mingw-host crafter-build can also build msvc-target outputs
(uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when
the user sets cfg.target = "x86_64-pc-windows-msvc".
README adds a Quick start (Windows) section covering both build paths
(native MSVC via build.cmd and the cross-compiled mingw artifact),
documenting the msys2 UCRT toolchain prerequisite.
Verified end-to-end on the winvm:
- mingw target: cross-compiled crafter-build.exe builds hello-world's
project.cpp, compiles main.cpp, links a hello.exe that runs without
any custom PATH (only Windows system DLLs needed).
- msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe
linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
