runtime.js implemented proc_exit by throwing a JavaScript error through
the wasm stack, and the switch to -fwasm-exceptions made that error
catchable: every cleanup pad, catch (...) and noexcept terminate pad
catches a foreign exception. An exit therefore ran the destructors of
every frame it left, a catch (...) swallowed it and carried on, and
under a noexcept frame it ended in std::terminate. Crafter.Graphics
calls std::_Exit(0) in Window::StartSync to hand the thread to
requestAnimationFrame; once the exit's destructors had torn down
main's locals, every later frame read destroyed state.
proc_exit is now a function of a small helper module, bound directly
as the program's import, that reports to the runtime and then executes
unreachable. A trap is the one thing wasm code cannot catch, and raised
in wasm rather than from JS it does not depend on each engine keeping
a trap uncatchable when it passes back through a JS frame. Nothing of
the program runs on the way out and its shadow stack stays as it was,
which is what exit and _Exit require. window.crafter_wasi.exitCode
holds the status, also after a return from main, so that bridges which
call into the program on their own can stop.
Handing the thread to the browser for good is a separate import,
crafter_runtime.park. It ends the call the same way but leaves the
program alive, and main's locals valid for the calls that follow, so
StartSync keeps its contract that no caller code after it runs. Until
Crafter.Graphics moves to park, its _Exit behaves as before minus the
destructors: the runtime records an exit but cannot stop a bridge that
calls exports itself.
A WASI reactor build, where main returns and state lives in statics,
was the alternative. It breaks StartSync's promise never to return,
which the native targets rely on. JSPI would keep the whole stack
alive, but Safari does not have it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README described the mingw target and said nothing about wasm. It
now says where the WASI sysroot comes from, that C++ exceptions work as
on the other targets and what that takes (-fwasm-exceptions in the
standardized form, the sysroot's exception-enabled libc++, -lunwind
added to the link), which engines run the result, and that an exception
escaping main ends the program without calling std::terminate.
AGENTS.md says where the wasm flags live, that VariantId and the std
PCM directory hash them, and that the wasm tests skip where the
distro's sysroot lacks the eh/ libraries, so they run in the Flatpak.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WasmExceptions builds a fixture for wasm32-wasip1 and runs it under
wasmtime. It throws through four frames whose destructors have to run,
catches the exception, catches one that vector::at throws, and with an
argument lets one escape main, which has to fail the run. With the
previous flags the fixture does not compile ("cannot use 'throw' with
exceptions disabled"), and each part of turning exceptions on fails it
in its own way, from a BMI built without them to the legacy encoding
wasmtime refuses.
WasiSysrootWithoutExceptions takes a scratch sysroot through three
states, no libunwind, libunwind under eh/ and libunwind in the
triple's own directory, and checks that only the first stops the
build, with the message naming the library it looked for. It needs no
working wasm toolchain, so it also runs where the others skip.
WasmHelloRuns and WasmExceptions treat a sysroot without the
exception-enabled libraries as missing and skip. The manifest tests
expect the exception-handling probe on every entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wasm builds were compiled with -fno-exceptions. A throw did not
compile, libc++'s own throws (vector::at, std::format, filesystem)
aborted the program, and code that relies on exceptions could not be
built for the browser at all.
Every wasm unit and the std module now get -fwasm-exceptions, and the
link adds -lunwind, which code built that way calls into and the driver
does not add. The flag also selects the libc++ under the sysroot's
lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh
one. Those libraries use the standardized encoding (try_table, exnref)
while LLVM 23 still emits the legacy one by default, which wasmtime
refuses to load and which would put both in one binary, so the flags
ask for the standard form with -wasm-use-legacy-eh=false. That makes
the setjmp lowering loadable under wasmtime too.
The flags live in one constant, WasmBaselineFlags, which the compile
command and the std module both take through WasmCompileFlags: a BMI
is refused by a unit whose exception setting differs from its own.
VariantId and StdPcmDir hash it, so the first build after the update
starts fresh instead of importing PCMs built without exceptions, and
two releases sharing the Flatpak's cache don't take turns rebuilding
one std PCM. A sysroot without the exception-enabled libraries stops
the build with a line saying so, rather than at the link with "unable
to find library -lunwind".
The result needs an engine with exnref: Chrome 137, Firefox 131,
Safari 18.4, the bundled wasmtime. The runtime's exception-handling
probe tested the legacy encoding; it now tests try_table, every
variants.json entry carries it, and a browser without it is told which
feature it lacks instead of failing with a CompileError. There is no
switch back to -fno-exceptions: engines without exnref are old enough
not to keep a second build for them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #42, fixes#41. Build() now refuses a configuration whose transitive
dependencies hold two configurations exporting a module of the same
name, such as one project loaded along two paths with different arg
lists, before it builds anything, and names both paths. Until now both
builds ended up in the one program, which compiled cleanly and then
read garbage. Merged cleanly on top of claude/forward-project-args;
the suite, with claude/dependency-notes' fixtures, passes under the
new check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #40. GitProject and LocalProject default their args to the loading
project's own (ProjectArgs()), this tree lints with LintRules' lite
set, and the fixes from the branch's review: a project file gets its
args only for the CrafterBuildProject call, host tooling gets project
cache entries of its own, and `--` ends crafter-build's options.
The conflicts with claude/dependency-notes are all places both sides
added to:
- Platform.cppm and Platform.cpp: PragmaDependencies() beside
ProjectArgs() and LoadingHostTooling(), all three kept.
- LoadProject: the record PragmaDependencies() reads moves up with the
frame's dependencies, which this branch sets before dlopen. Left
where it was, it would have copied preDeps.configs after the vector
was moved into the frame, and recorded nothing.
- README: the pragma section's new last sentence goes before the new
"Dependency notes for agents" section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README and the spec comment said that a project two paths load with
different lists ends up in the program twice. They now say the build
stops, with an error that names both paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixture is P -> M and P -> A -> M, where M's own flag --wide
changes a struct it exports. Loaded with --wide --split, P gives M its
args without --wide while A passes them on whole: Build(P) has to fail,
name the module and both paths, and leave no bin/ anywhere. Loaded with
--wide alone, both paths give M the same list, there is one M, and the
build has to go through. Before the fix the first case built cleanly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project that two paths load with different arg lists is two
configurations, each with a variant of its own, and Build() built both.
The consumer's compile got both PCM dirs on its -fprebuilt-module-path,
and clang resolved the module by name to the first for every import,
also for the import inside the other build's BMI consumers, with no
check that it was the build they had been compiled against. With
P -> M and P -> A -> M, where the two lists change a struct M exports,
P compiled against one layout, A against the other, and the program
built cleanly, then read garbage or crashed (#41).
Each configuration now checks, before it makes a directory or builds a
dependency, that no two configurations among it and its transitive
dependencies export a module of the same name, and otherwise fails with
both paths and what tells the builds apart. Going by module name rather
than by project file also catches a checkout on one path and a clone of
the same project on the other. Per configuration, not per invocation:
two builds of one project in different programs, such as a test and a
#pragma crafter dependency tool, are fine.
Picking one of the two builds was not an option: arg lists have no merge
rule, so whichever consumer lost would silently get a different build
from the one it asked for.
Fixes#41
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README suggested giving one sibling a list without a flag only the
project reads, so that the sibling would not get a build directory of
its own. When another path loads the same sibling with the full list,
that filter is what gives it the second directory: two builds of one
project, both in the program (#41). It now says to build such a list
once and give it to every sibling. It also says the default is read
when the spec is made, so a spec made at namespace scope, on another
thread or in a script forwards nothing, and that cfg.projectArgs, the
part ApplyStandardArgs did not interpret, is no substitute: it drops
the target and the build type. The comments on ProjectArgs() and on
Configuration::projectArgs say the same.
The spec comment named --local as a flag the default carries down a
chain of siblings. Through a GitProject hop it breaks: the clone has no
checkouts beside it. The GitProject comment said the clone is keyed by
the args; only the Configuration is, and every arg list builds in the
one clone. AGENTS.md now notes that a stale harness gives no symptom
for a changed default member initialiser: project.cpp silently gets the
old default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`run -- record -- a` has to run the record script with the args `--`
and `a`: the first `--` ends the options, the second comes after the
script's name and is the script's. The project args keep only what
stood before `run`. And `run -- -x` has to look for a script called
-x, since after `--` an option-like word is an operand. Before the fix
the first `--` showed up in the project args, and `-x` joined it there
while the bare `run` listed the scripts and succeeded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The help has advertised `[options] [-- project-args...]`, but Run never
looked for `--`. It fell through to projectArgs as a word of its own,
and the words after it were still read as this tool's options and
verbs: `crafter-build -- -v` turned verbose output on. In projectArgs
`--` changed the project's variant hash, and now that siblings inherit
the args, the hash of every sibling down the chain, each building into
directories of its own that the project's clean never removes.
`--` now ends this tool's options and goes nowhere itself. Every later
word is an operand of the verb, as POSIX has it: a project arg for a
build, a glob for test, lint and format, the script's name and then its
args for run. A glob rather than a project arg after `test` because
`crafter-build test -- 'House*'` is how such a filter gets typed; a
project arg goes before the `--`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixture app has `#pragma crafter dependency "../tool"` and loads
../shared for itself, which the tool loads too; the app is loaded with
no args, the case where the two cache keys used to match. With
CRAFTER_BUILD_MARCH=x86-64-v2, the app's copy of shared has to keep
that march and no -fPIC, and the tool's copy has to be host tooling.
Before the fix the app got the tool's copy back: one Configuration,
with march native and -fPIC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ForceHostTooling rewrites a `#pragma crafter dependency` and everything
it loads in place: host target, march native, release, -fPIC. Those are
entries of the project cache like any other, keyed by project file and
args alone. A project that loaded the same sibling as its tool, with
the same args, got the rewritten entry back. That happens whenever the
project gets no args, since host tooling gets none either: a consumer
built for x86-64-v2 then imported a sibling's BMI compiled with
-march=native, and clang refused it. Any arg on the command line split
the keys and hid it.
A thread-local count of pragma loads now marks the tool's load and
every load inside it, its project file's siblings included, and
GitProject and LocalProject put that mark in the key. The clone stays
shared; only the Configuration is separate, as for two arg lists. A
module-internal function rather than a spec field, because the
siblings of a tool are loaded by the tool's own project.cpp, which
cannot know it is host tooling.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ProjectArgs gets a sibling that makes its own sibling's spec at
namespace scope. Before the fix that spec took the args the file was
first loaded with; now it takes none on every load. The checks go
through one CheckLoaded helper, which prints what each sibling was
given, so a failure says which args a wrong frame handed over.
ProjectDependency loads a fixture whose static initialiser asks for its
pragma dependency, in a child, because a throw there terminates the
process: before the fix it found none. The check that the dependency
gets none of the consumer's args moves into the first load, which now
passes the app an arg it ignores; the separate load could not fail its
own check, and its early return skipped every later one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ProjectArgs() read args that LoadProject recorded when it pushed the
frame, so they were there for everything that ran on that frame, not
only for CrafterBuildProject. Two things went wrong with that.
The pragma pre-pass runs on the frame too, so both of its specs needed
a hand-written `.args = {}` to keep the consumer's args from reaching
host tooling, and anything else built there would have needed one as
well. And dlopen runs a project file's static initialisers only on the
first load of that file: a later load gets the same mapping back. A
spec made at namespace scope therefore kept the first load's args for
every later load of the file, a silently wrong variant.
The frame now gets the args just before CrafterBuildProject is called.
The pre-pass and the static initialisers see none, on every load, so
the two opt-outs go. The dependencies the pre-pass resolved move the
other way, to just before dlopen: they are the same on every load, and
ProjectDependency() called from a static initialiser used to find none
and terminate the process.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in dependency notes: a crafter-build project that ships an
AGENTS.consumer.md beside its project.cpp has it kept in a marked
section of each consumer's AGENTS.md, refreshed by `crafter-build` and
`crafter-build test`, chosen by cfg.dependencyNotes (Transitive,
Direct, None). The new partition is Crafter.Build:Agents; the loader
records where each GitProject and LocalProject came from and what a
project file's pragmas resolved to.
The branch started before master's four external-fetch commits. Both
touch External.cpp, the Clang partition, project.cpp, README.md and
AGENTS.md, but in different places, so the merge needed no hand
reconciliation; the merged tree passes build.sh, the self-host build,
format --check and the test suite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README's section: notes are labelled by name, the file is written
in place, a project.cpp that fails to load gets no update, `test
--list` writes nothing, the section follows what the build loaded, and
tests' dependencies don't count. The AGENTS.md gotcha: an unpinned
dependency can change this file's section on any build here, and the
dirty tree then rebuilds crafter.build-lib under a new version.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each fixed finding gets a check that fails without its fix: inline code
and a list item's fence, a section removed from between two paragraphs,
a section before an unclosed fence, a CRLF file, no error when there is
nothing to write, a hard-linked and a read-only AGENTS.md, a dependency
whose project file sits in the consumer's own directory, a test's
dependency, and `test --list`. The read-only case needs an ordinary
user, since root writes through the permission bits, and is skipped
under root.
The merge cases go through a helper that counts a throw as a failure,
so one regression no longer ends the run before the cases after it. A
passing run removes its staged copy, whose builds leave a std PCM per
configuration in a tmpfs.
Reverting the inline-code rule, the own-notes exclusion, the skipping
of tests, the `test --list` guard or the paragraph-preserving removal
each fails its check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A review of the feature turned up the following, fixed here.
Writing. AGENTS.md was replaced through a temporary renamed over it,
which swaps in another file: a read-only AGENTS.md was rewritten
anyway, a hard link such as a CLAUDE.md beside it kept the old text,
ACLs and ownership were dropped, and without an fsync a power cut could
still leave it empty. The temporary was named after the PID, which
every Flatpak instance shares as PID 2. The file is now written in
place, from the first byte that differs, then truncated and synced,
under an flock on the file itself. A write that fails partway is undone
where the disk allows; with the section last, where it is appended,
only the section can be damaged. A read-only file is opened for writing
only when something would change, so it is refused then and silent
otherwise.
Reading. libc++'s filebuf reports a failed read as the end of the file,
so an I/O error came back as a short file and the section was written
over what was left. Reads go through read(2).
Code fences. ```make``` was taken for a fence, and a fence in a list
item was missed while its indented closer opened one, so markers a user
quoted in a code block could be taken for the section, or the real one
hidden and appended again. Backtick info strings, block quotes and list
items are now followed, and a block ends with its container.
Warnings with nothing to write. With no notes, nothing short of a
well-formed section is reported, so an AGENTS.md that ends in an open
code block, or a project set to None, no longer warns on every build.
An unclosed fence after a section no longer stops it being rewritten
in place.
Labels. The `from` line carried the git URL, credentials and all, or a
path relative to the consumer that changed with the machine, the cache
location and --local. It is now the repository's or the directory's
name. Notes are deduplicated by repository rather than by clone, so one
library pinned two ways is described once, as the nearer spec has it.
The walk. It no longer looks at the project's tests: a plain build
never did, and a test's dependency can point at the Configuration the
project file returned by value. It goes ring by ring over projects
rather than configurations, so what the project names itself comes
first even with the library/executable split. The project's own
AGENTS.consumer.md is excluded by path, also when a dependency's
project file sits beside the project's.
Removing a section dropped a blank line even from between two
paragraphs, joining them; only a section that ends the file takes it
now. A CRLF file gets a CRLF section.
A note was checked on its raw bytes and again, under other whitespace
rules, on the trimmed ones, so it could pass the first check and fail
the merge, which then dropped every note. It is normalised once and
checked on exactly what is merged.
`test --list` only lists, and no longer writes AGENTS.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sibling-project example dropped its `.args = depArgs`, which no
longer has to be written, and says what a sibling receives instead: the
args the project file was loaded with, or whatever `.args` is set to,
with `{}` for nothing. It also says that releases up to 2.2.0 forward
nothing without `.args`, which matters for a project.cpp that has to
load on both.
The pragma section says that a project file's own dependency is built
for the host and gets none of the project's args, the one place the
default does not apply.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crafter.LintRules now has AddLiteRules: the house rules without the ones
that only decide layout, whitespace and naming. This tree switches to it.
The full set's naming rule now wants camelCase statics and constants,
which this tree spells PascalCase throughout, so `lint` with it reported
49 findings on code that follows the documented convention; with the lite
set `lint` and `format --check` are clean.
AGENTS.md described the 17 rules as the enforced style and told
contributors to let `format` restyle their code. It now says which 8 rules
`lint` enforces, that `format` only rewrites string concatenation and
builtin integer types, and lists layout and naming as conventions kept by
hand.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crafter.Math and Crafter.Asset now get this project's args through the
GitProjectSpec default. Loading this file therefore needs a crafter-build
that exports ProjectArgs: after pulling, run ./build.sh before the
bootstrap binary loads it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ProjectArgs loads a chain app -> lib -> leaf of LocalProjects with --debug
and a project flag. lib, loaded without .args, has to see both, and so
does its own sibling, which takes the default one level further down.
Copies loaded with `.args = {}` and with an explicit list get nothing and
exactly that list. The GitProjectSpec default is checked on the spec
itself, since a GitProject would need a repository to clone. Without the
defaults, five of its checks fail.
ProjectDependency now also loads its consumer with a project flag: the
pragma dependency must still come back with no args. Without the explicit
`.args = {}` in the pragma path it gets the consumer's flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every project.cpp with a sibling dependency opened with the same line,
std::vector<std::string> depArgs(args.begin(), args.end());
and handed depArgs to each GitProject and LocalProject. CrafterBuildProject
receives a span of string_views and the specs wanted owned strings, and
without the copy the sibling got none of the project's args: it was built
for the host while the project cross-compiled, and in release while it
built --debug. GitProject then turned the copy straight back into
string_views for LoadProject.
Both specs now default their args to ProjectArgs(), the args LoadProject
was given for the project file whose CrafterBuildProject is running. It
reads the per-thread stack LoadProject already keeps for IsTopLevelProject
and ProjectDependency, whose frames now record the args. A default member
initializer rather than a new type or an optional: the member stays a
std::vector<std::string>, so every existing `.args = depArgs` keeps its
exact meaning (each of the ~50 calls across the Catcrafts projects passes
one), and `.args = {}` still forwards nothing. It is the shape of
Configuration::target = HostTarget(). A project.cpp that drops its .args
needs a crafter-build with this change: an older one compiles it and
forwards nothing.
ProjectArgs() is declared in :External, next to the defaults that call
it. :Platform, where it is defined, cannot be imported there: its
unexported `struct Configuration;` would make :External's exported one an
ill-formed redeclaration, which clang rejects.
The #pragma crafter dependency path now passes `.args = {}` explicitly.
Host tooling is built for the host whatever the consumer targets and was
loaded with no args; the default would otherwise hand it the consumer's
--target= and project flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What a library ships and where, what a consumer's AGENTS.md gets and
when it is written, the section's format, and what cfg.dependencyNotes
selects.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge on its own: the section is appended after one blank line,
merging again changes nothing, a moved section is rewritten where it
stands, and removing it gives the file its own bytes back. Markers
inside a fenced block, one quoting a fenced block of its own, are not
the section's. Unbalanced markers, a document that ends inside a code
block and notes that would break the section are refused, naming the
line.
End to end, through LoadProject on a staged fixture: app names lib
and has tool as host tooling; lib builds against base and has libtool
as its own. Transitive gives tool, lib and base in that order, and
neither app's own notes nor libtool's. A second update writes nothing,
mtime included. Direct drops base, edited notes replace the old ones
in the one section, and None restores the original bytes. A project
without an AGENTS.md does not get one, and a symlinked AGENTS.md is
written through the link, which stays a link. Through Run in process,
lint and run leave the file alone, while `test --list` and a build
that fails at the link both refresh it.
The fixture's AGENTS.md is checked in as AGENTS.md.in: an AGENTS.md in
this tree would read as instructions for the agents working on it.
Moving the CLI hook above the run and lint branches, or letting Direct
walk past the first loaded project, fails four of the checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An agent working on a crafter-build project reads its AGENTS.md, and
learns nothing there about the libraries the project builds against
unless someone copied that in by hand. This repository shows the cost:
the code style section of its AGENTS.md restates what Crafter.LintRules
enforces, and goes stale whenever the rules change.
A library now ships those notes itself, as AGENTS.consumer.md beside
its project.cpp; nothing in its project.cpp changes. When a consumer
has an AGENTS.md beside its own project.cpp, `crafter-build` and
`crafter-build test` keep a section in it, between two marker comments,
holding its dependencies' notes. The section is appended the first time
and rewritten in place after that. The file is written only when the
section changes, and an AGENTS.md is never created. lint, format, run
and clean leave it alone. The update runs after loading, which has just
fetched every dependency, and before compiling, so a build that fails
still has the notes.
cfg.dependencyNotes picks the dependencies. Transitive, the default,
follows `dependencies` all the way down from the project's own
configurations, its tests and its `#pragma crafter dependency` tooling;
Direct stops at the first loaded project on each path; None removes
the section. A dependency's own pragma tooling never counts: it is
about how that dependency is built, not how it is used, and being no
`dependencies` edge it is never reached. Nor are a dependency's tests:
nothing builds them for the consumer, and their configurations can
point at a local their project file has since moved out of.
The walk has to know which Configurations belong to another project,
and where that project lives, and cfg.path cannot say: only the
Configuration GitProject and LocalProject return is rebased, so a
dependency's internal configurations keep "./", which resolves to the
consumer. The loader records it instead, LoadedProjectOf beside the
project cache, and keeps what the root's pragmas resolved to past the
frame LoadProject pops, PragmaDependencies. Neither is exported.
The markers count only as whole lines outside fenced code blocks, so
documentation can quote them. Unbalanced markers and a document ending
inside a code block leave the file alone with a warning: guessing which
part of someone's file is generated would overwrite their text, and
missing the section would add another one on every build. A note that
would break the section when it is read back, through a marker line of
its own or a code block it never closes, is left out with a warning.
The file is replaced through a temporary renamed onto the symlink's
target, so a linked AGENTS.md stays a link and a write cut short leaves
the old file rather than part of the new one.
The code is a new partition, Crafter.Build:Agents, listed in build.sh,
CrafterBuildModules, project.cpp and the umbrella.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Say how often an unpinned ExternalDependency is pulled now that the
configurations of a build share it, and list the target in the clone's
cache key, which it is part of.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ConcurrentExternalFetch raced eight threads through GitFetch only.
BuildExternal takes the same lock on a line of its own, so that line
could go without any test noticing. Now that the configurations of one
pass share a single BuildExternal, its lock is what keeps two
crafter-builds, or two passes, out of each other's pulls.
The same threads now fetch through both, each into a clone of its own,
and both clones must end at the last round. With the lock in
BuildExternal removed, the test failed in two of three runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight tests that include a header from the same tracked branch of a
local repository go through RunTests at once, and again after the
branch has moved. GIT_TRACE logs every git process, and each fetch of
the repository, clone or pull, runs one `git upload-pack` in it, so the
trace counts the fetches: one per run rather than one per test, and the
second run still pulls and brings the clone to the new commit.
On the commit before the fix both runs fetch eight times. The tests
themselves pass there, because the lock from 653dd0b already keeps the
pulls apart. Going through RunTests, rather than Build() on threads of
the test's own, also holds RunTests to building its tests with one
depResults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`crafter-build test` builds a project's tests side by side, and a
project whose tests each copy its externalDependencies hands every one
of them the same unpinned dependency. Each test's Build() pulled the
shared clone. Before the clone had a lock, those pulls ran at the same
time and broke its .git/FETCH_HEAD, which failed every test of
Crafter.Graphics (issue #34). Since 653dd0b they wait for each other,
but every one is still a network round trip: 26 tests cost 26 pulls of
each dependency, one after another, and for a CMake dependency as many
configure checks and builds.
depResults already records one build pass, keyed by the PcmDir of every
dependency configuration built in it. An external dependency now goes
in the same map, keyed by the directory its build leaves
(ExternalBuildDir: the clone, or for CMake its build directory). The
first configuration that needs it fetches and builds it; the others
wait for that and take its outcome. Keying on the build directory
rather than the clone keeps a CMake declaration from taking the outcome
of a header-only declaration of the same clone, which only fetched it.
A fresh depResults is a fresh pass, so the next build still pulls, and
every caller of Build() gets this: RunTests, a plain build, and the
pragma dependencies LoadProject builds.
To make that possible BuildExternal is split. BuildExternal is now the
shared work, lock, fetch, configure and build, and returns the error.
ExternalOutputs computes what one declaration compiles and links with,
because two declarations of one clone can differ in includeDirs,
libDirs and libs; ExternalBuildResult loses its error field. The work
no longer stops between steps when the configuration that started it
is cancelled: others may be waiting on it, and a run cut short would
leave them a clone nobody fetched. A dependency configuration's Build()
already runs to completion for the same reason.
The promise and future the dependency threads used to build or wait for
a configuration are now one helper, OncePerPass, which both kinds use.
An exception thrown in BuildExternal used to escape its thread into
std::terminate; it is now a build error, as for a dependency.
With eight tests that share a local dependency, a test run fetched it
eight times before this and fetches it once now (SharedExternalFetch,
next commit).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master's last two runs (#368 and #370) failed in the Build step with
The state dir (/root/.cache/flatpak-builder) is not on the same
filesystem as the target dir (/workspace/Catcrafts/Crafter.Build)
while #369, the 2.2.0 tag run on the same commit with the same
flatpak-builder, passed. The difference was the runner. Since 14 October
the Hetzner box has a second privileged runner, cx33-org-flatpak, which
serves every Catcrafts repository under the label `flatpak` so their
package jobs can run crafter-build as a Flatpak. It mounts no host
volumes, deliberately, since any repository in the organisation can send
it jobs. Whenever it was idle first it took this job too, logged our four
volumes as "not a valid volume, will be ignored", and left the
flatpak-builder state on the container's overlay while the workspace sat
on a Docker volume. Even past that check it would have had no build
cache, no web root and no signing key.
The org runner's label is right for the jobs it was made for, and giving
it our volumes would hand the signing key to every repository in the
organisation. So the job asks for a label only its own runner carries:
`crafter-flatpak`, after the volumes and key directory that runner alone
allows. The runner on the host is relabelled to match
(/etc/forgejo-runner/config-flatpak.yaml, previous copy kept as
config-flatpak.yaml.bak-2026-10-31-label), so until this reaches master
a push there with the old `runs-on: flatpak` goes only to the org runner
and fails the same way.
AGENTS.md now says the label must stay unique and quotes the error a job
on the wrong runner stops with, so the next collision is found from the
log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Shaders record the files glslang opened (a depfile) and their settings
(source path, stage, entry point, defines, include directories) in the
build directory, and rebuild when an included file or a setting changes,
not only the shader's own source.
- An executable relinks when its link command changed, not only when an
object is newer: a link flag in project.cpp, or a dependency's library
that moved (an ExternalDependency built into a new cache directory). The
command is recorded beside the objects; its flags come from ordered sets,
so the order the dependency threads finish in can't change it.
- An external's newest artifact counts its libDirs and versioned .so names.
- Submodules are checked out at the revisions the fetched commit records,
from the URLs its .gitmodules gives, after every clone, switch, checkout
and pull, so a pinned commit builds against its own submodules, and one
moved by hand is put back; with no .gitmodules, git isn't asked.
- Each external clone's fetch and build hold a file lock: parallel builds
pulled into one clone at once ("cannot lock ref").
- mingw:
- -stdlib=libstdc++ always;
- the std module's source is found through libstdc++.modules.json in the
mingw toolchain's own library directories (the generic lib/ holds the
host's Linux one), asking the compiler once a run;
- a toolchain that isn't there is a build error saying where it looked,
not std::terminate;
- CRAFTER_MINGW_DIR names the prefix, the directory holding
x86_64-w64-mingw32/ and lib/gcc/;
- inside the Flatpak the host's /usr mingw is used.
- RunScript's exit-status check reads as the one range it is.
- Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules,
LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and
ToolchainResolve extended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RunScript caught std::exception only. Anything else a script threw went
on across the project.so boundary, and crafter-build ended in
std::terminate instead of reporting the script's failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local Flatpak build was one long flatpak-builder line in AGENTS.md,
with its state wherever the caller happened to stand, and installing
the result was not written down anywhere. It is not a rare chore: the
development box compiles with the distro's clang and the Flatpak with
its bundled LLVM, a major version apart at the moment, so anything
toolchain-sensitive has to be tried in the Flatpak too.
Both are now scripts in project.cpp, the first users of cfg.scripts.
`run flatpak` builds the manifest as it is, from the working tree, and
ends with `crafter-build --version` from inside the result. Its state
is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and
not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in
the sandbox, so it does not depend on the host variant, and a
variant-keyed directory would strand the LLVM cache on every host
compiler update. The manifest already skips build/ when it copies the
sources. flatpak-builder keys its cache on the manifest's path, so the
manifest is never rewritten into a copy the way CI stamps the version
in, and a local build reports 0.0.0-dev. The only argument accepted is
--jobs=<N>: anything that ends the run without a build would leave the
smoke test checking the previous one.
`run install` installs the last export as branch `local` from a user
remote of its own and makes it current, leaving the published branch
installed. It does not build first: flatpak-builder never caches a
`type: dir` module, so every build reruns build.sh and the suite.
remote-add --if-not-exists is followed by remote-modify --url, because
the remote may exist and point at another checkout.
Both refuse to run from the Flatpak build of crafter-build, which has
no flatpak to drive.
The smoke test goes through `flatpak build` and not `<builder> --run`,
which is what CI uses. flatpak-builder 1.4.9 run as the
org.flatpak.Builder app exits 1 when the command it ran succeeded and 0
when it failed (`true` gives 1, `exit 3` gives 0), so the script
reported a good build as a failure.
Measured on the 56-core box with LLVM and the WASI sysroot cached: 260
seconds, the crafter-build module only, 29 tests passing in the sandbox.
A second `run install` after a rebuild moved the installed commit.
`clean` now also drops that state, about 2 GB and a cold LLVM build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
RunScript on hand-built configurations: the named script alone runs,
with the project, the project directory and the arguments exactly as
given; its return value comes back whole from 0 to 255 and as 1 outside
that; a script that throws exits 1. A project without scripts, a
duplicate, an untypeable name, a missing callback and an unknown name
all fail without running anything. ctx.Run returns the command's exit
code and 128+signal for a killed one, and ShellQuote carries quotes,
spaces and expansions through sh intact.
Then the same through Run() with a fixture project, because the parsing
is where this goes wrong. `run record a --help -v clean --debug
--version run` must hand all seven words to the script, leave the one
project arg that stood before `run` alone, and not delete the staged
bin/ and build/. `run clean` and `run help` look for scripts of those
names. Moving the two capture branches below the verbs in the parse
loop fails this test.
The command-line half needs the installed module sources and is skipped
without them, but only when the in-process half reported no failure.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project.cpp could describe how a project is built, tested and linted,
but not the chores around it: packaging, installing, publishing. Those
stayed shell lines in a README or in someone's history.
Configuration gains `scripts`, a plain list like `defines`. A Script is
a name and a C++ callback, as a lint rule is; there is no script
language and no config file. The callback gets a ScriptContext (the
project, the project file's directory, the arguments) and returns the
exit status. ctx.Run runs a shell command with the terminal attached
and returns its status. It is only stored when project.cpp loads, which
happens for every verb and for every project that depends on this one.
`crafter-build run <name> [args...]` runs one, a bare `run` lists them.
Only the scripts of the project crafter-build was pointed at are
runnable, never a dependency's.
Everything after the script's name is handed to the script untouched.
The two branches that do this sit ahead of every other branch of the
parse loop, --version and --help included. Below the verbs,
`run package clean` would delete build/, and any other word would land
in the project's args and so in the variant hash, moving the build to
another directory. Options before the name keep their meaning, so
project args go there.
The driver is its own partition, :Script. It refuses a project with no
scripts, a name that is empty or starts with '-' (it could never be
typed), a name declared twice (running either would be a guess) and an
unknown name, each with exit 1 and without running anything. A script
that throws exits 1. A return value outside 0..255 becomes 1: an exit
status is eight bits, and a raw wait status such as 256 would otherwise
wrap to success.
Two helpers in :Platform:
- RunCommandAttached runs a command with inherited stdio. Every helper
so far captured through popen, which is right for a compile step and
useless for a command that runs for half an hour or asks a question.
It reads the wait status through DecodeWaitStatus, which
RunCommandChecked and RunCommandWithTimeout now share.
- ShellQuote is exported. It existed twice, as ShellQuote in :External
and ShellQuoteSh in :Test, with the same behaviour; both are gone. A
script that puts a path into a command needs it too.
Not included: a string shorthand for one-command scripts (a one-line
lambda covers it) and a ctx.Build() helper (Build() is already
exported).
A new partition and new exported symbols: run ./build.sh.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Drives Run() with `format clean` on a staged project whose project file
cannot compile. Both trees must survive, and the run must fail: getting
as far as loading the project is what shows it went on to format rather
than cleaning and exiting 0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every verb is a bare word that is only read as a verb while no
conflicting verb is active; after one, the same word is a glob. The
guard on `clean` checked the test and lint verbs and then itself, where
the format verb was meant. `crafter-build format clean`, which asks for
the rules matching "clean", therefore removed both trees and exited 0
without formatting anything.
The guard now names format, like the three beside it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CI takes the newest <release> as the notes of a tag and refuses a tag
that does not match it. 2.2.0 covers what changed since 2.1.0: Shader
entries with per-entry defines and a named output variant, sysroot
compiles against the sysroot's own libc++ headers, a project.so stamp
that hashes every part of the host build, and the vendored json.hpp
removed.
The variant API is what catcrafts.net's deploy needs: Crafter.Graphics
master has used it since 2026-10-06, and a stable Flatpak that lacks it
cannot build the site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GetCompileCommand is checked as text, without a compiler: the forced
include directory is present for linux-gnu and linux-musl targets with
a sysroot, and absent without a sysroot, for wasm32 and for mingw.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
clang's driver ranks the host toolchain's own libc++ headers
(<install>/../include/c++/v1) above --sysroot. BuildStdPcm has forced
the sysroot's copy for the std module for that reason, but ordinary
compiles did not. A unit that only does `import std;` never noticed. A
unit that includes a standard header textually got the host's libc++,
configured for the host's libc: cross-building Crafter.Graphics for
aarch64 musl, Crafter.Asset's vendored gdeflate died on <chrono> with
"unknown rune table for this platform". Against a glibc sysroot the
same unit would compile against the wrong libc++ release and say
nothing.
Every non-wasm, non-mingw compile with a sysroot now gets
`-nostdinc++ -isystem <sysroot>/usr/include/c++/v1`. wasm has its own
sysroot handling and mingw is a libstdc++ target.
-nostdinc++ leaves -stdlib=libc++ unused on each compile, which clang
reports once per unit, so the command also carries
-Wno-unused-command-line-argument. The cost is that the driver's
unused-argument diagnostics are silent on sysroot builds, including for
a user's own misplaced flag (a -l or -Wl, option in compileFlags).
Not addressed: external cmake dependencies are still configured
without the target or the sysroot, and objects already built in an
existing sysroot build directory are not invalidated by the new flags.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ShaderCompile registers one source twice and checks that the variant
gets its own output name, takes the other #ifdef branch and receives
the value of a NAME=VALUE define. Staleness is pinned per output: while
only the plain output exists the variant must still be due, or Build()
would never compile a variant added to a built project. Malformed
defines and a variant with a path separator must be refused by name.
ShaderVariantBuild drives the same through Build(), which is where the
entries compile in parallel into one directory: a variant added to an
already built configuration is compiled into its own file, and two
entries with one output name fail the build without touching the
existing output. Its fixture tests the macro with `#if`, which pins
that a bare define is 1 and not empty.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Crafter.Graphics has to ship each of its compute shaders in two builds
of one source: one that indexes VK_EXT_descriptor_heap and one that
reads the same slots from a descriptor set, for drivers without the
extension. A Shader was a path, an entry point and a stage, and its
output name was derived from the path alone, so one source could only
be compiled once.
Shader gains `defines`, spelled like the compiler's -D ("NAME" is
defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a
preamble, and `variant`, which names the output: "ui.comp.glsl" with
variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one
place that name is formed; Check, Compile and the forwarding of a
dependency's shaders into the consumer's bin dir all use it.
Two fields make one mistake easy: defines set, variant forgotten. Both
entries then compiled in parallel into the same file, the build
reported success with whichever finished last, and Check() called both
up to date from then on. The same has always been true of a.vert next
to a.frag. Build() now refuses a configuration in which two entries
share an output name and says which.
A define without a name, or one spanning lines, and a variant with a
path separator are rejected by name. Left to glslang the first two
surface as "-1:<line>" with no file, and the last as a failed open
after the whole compile.
Not addressed: Check() still compares only the source's mtime, so a
change to the defines, or to an included file, does not recompile
(#35).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
HostCacheSourceStamp already edits a module source with a backdated
mtime and expects the PCM cache to be rebuilt. Its own header names the
real danger, a project.so compiled against declarations that no longer
match the process it is loaded into, but it never looked at the
project.so. It now records the file's mtime before the edit and expects
a newer one after it, with project.cpp untouched, so only the stamp can
force the rebuild.
Fails without the previous commit: the stamp was the version string
alone and the project.so was reused.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The stamp that decides whether a compiled project file can be reused
was meant to cover the crafter-build version, the toolchain key, the
module-source stamp and the compile command. Only the version went in.
The four parts were joined by a format string with "\0" between the
fields, and a format string is read as a C string: it ended at the
first NUL, so std::format saw "{}" and dropped the other arguments
without a diagnostic.
A project.so is loaded into the crafter-build process and fills in
Configuration objects by their layout. With the stamp blind to the
declarations, one built against other module sources was loaded as
current whenever the version string was unchanged, which is every
working-tree build (`git describe --dirty` does not follow the tree's
contents) and every local Flatpak build. Growing a class that project
files construct then corrupts the heap: a project.so built while Shader
was 56 bytes, loaded after it had become 104, aborted with "free():
invalid size" as soon as it declared a shader. CI builds were spared
only because each carries its own version.
The parts are now appended to a std::string with NUL separators, the
way the other stamps in this file are built. Every existing project.so
carries a stamp of the version alone, so each is recompiled once.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
