RunScript on hand-built configurations: the named script alone runs,
with the project, the project directory and the arguments exactly as
given; its return value comes back whole from 0 to 255 and as 1 outside
that; a script that throws exits 1. A project without scripts, a
duplicate, an untypeable name, a missing callback and an unknown name
all fail without running anything. ctx.Run returns the command's exit
code and 128+signal for a killed one, and ShellQuote carries quotes,
spaces and expansions through sh intact.
Then the same through Run() with a fixture project, because the parsing
is where this goes wrong. `run record a --help -v clean --debug
--version run` must hand all seven words to the script, leave the one
project arg that stood before `run` alone, and not delete the staged
bin/ and build/. `run clean` and `run help` look for scripts of those
names. Moving the two capture branches below the verbs in the parse
loop fails this test.
The command-line half needs the installed module sources and is skipped
without them, but only when the in-process half reported no failure.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Drives Run() with `format clean` on a staged project whose project file
cannot compile. Both trees must survive, and the run must fail: getting
as far as loading the project is what shows it went on to format rather
than cleaning and exiting 0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GetCompileCommand is checked as text, without a compiler: the forced
include directory is present for linux-gnu and linux-musl targets with
a sysroot, and absent without a sysroot, for wasm32 and for mingw.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ShaderCompile registers one source twice and checks that the variant
gets its own output name, takes the other #ifdef branch and receives
the value of a NAME=VALUE define. Staleness is pinned per output: while
only the plain output exists the variant must still be due, or Build()
would never compile a variant added to a built project. Malformed
defines and a variant with a path separator must be refused by name.
ShaderVariantBuild drives the same through Build(), which is where the
entries compile in parallel into one directory: a variant added to an
already built configuration is compiled into its own file, and two
entries with one output name fail the build without touching the
existing output. Its fixture tests the macro with `#if`, which pins
that a bare define is 1 and not empty.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
HostCacheSourceStamp already edits a module source with a backdated
mtime and expects the PCM cache to be rebuilt. Its own header names the
real danger, a project.so compiled against declarations that no longer
match the process it is loaded into, but it never looked at the
project.so. It now records the file's mtime before the edit and expects
a newer one after it, with project.cpp untouched, so only the stamp can
force the rebuild.
Fails without the previous commit: the stamp was the version string
alone and the project.so was reused.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two toolchains that differ only in the clang release, as the Flatpak's
do across an update, must give one Configuration two variants; so must
the same release installed under another prefix. The no-argument form
is checked to be the variant under the running toolchain, and BuildDir
to be named for it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SysrootStdModule runs over staged layouts that need no compiler: a
sysroot with its own module source, one on a point release of the
toolchain's libc++, one on the next major release with a shipped
source, one nothing was shipped for, one without libc++ at all, and the
two cases where a release cannot be read. The last block drives Build()
against a sysroot claiming libc++ 99 and expects the one-line
explanation back instead of compiler output, which is the failure the
fix replaces.
ToolchainResolve additionally checks that the release is read from the
libc++ the module source comes from (the prefix's, an explicit libc++
install's, the system's) and where the shipped sources are looked up.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
packaging/flatpak/ holds the manifest for net.catcrafts.CrafterBuild and
its side files. The runtime is the Freedesktop Sdk, not the Platform: a
build tool needs glibc's headers and crt objects, gcc's libgcc_s, cmake,
ninja, git and python at run time. Everything compiler-shaped is ours
under /app, built from one LLVM 22.1.8 source tree: clang, lld, libclang,
libc++ with its std module, compiler-rt for the host and for wasm32, the
WASI sysroot (wasi-sdk 34 in sysroot-only mode, driven by the clang just
built) and wasmtime for running the result. One tree means the modules
crafter-build builds and the compiler that consumes them cannot drift
apart, and the toolchain resolver finds all of it beside the executable
with no configuration.
The crafter-build module runs the bootstrap, the self-host build and the
test suite inside the sandbox, so a Flatpak that installs has passed the
suite with the toolchain it ships. CRAFTER_BUILD_MARCH is x86-64-v2 for
the shipped binary; user projects keep -march=native.
WasmHelloRuns compiles a fixture for wasm32-wasip1 and runs it under
wasmtime. The existing wasm tests only inspect generated files, so this
is the one that proves the sysroot, the std module for wasm and wasm-ld
work together; it skips where the sysroot or wasmtime is missing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
std.pcm was rebuilt when it was older than std.cppm, and project.so when
it was older than the crafter-build executable. Both inputs are installed
files, and an OSTree deployment (the Flatpak) gives every installed file
mtime 0: std.cppm and the executable always looked older than the cache,
so a clang bump would never rebuild std.pcm and a crafter-build update
would never recompile project.so. clang crashes on a PCM from another
compiler instead of reporting it, so the failure would have been a
backtrace on the user's first build after an update.
Each artifact now has a one-line stamp beside it. std.pcm's covers the
precompile command and the module source's bytes, which also catches a
libc++ update under the same clang. project.so's covers the crafter-build
version, the toolchain key, the module source stamp and the compile
command; project.cpp's and the pre-dependency artifacts' mtimes remain
inputs, since those live next to it with real timestamps. The module
source stamp stands in for the executable: project.so resolves Crafter::
symbols from the running binary, so only declarations matter, and an
implementation-only rebuild of crafter-build no longer recompiles it.
The std.pcm stamp folds into the module cache stamp, so a rebuilt
std.pcm rebuilds the PCMs that import it.
The cache directory carries the toolchain key
(<target>-<march>-<clang version>-<hash>), so a compiler bump starts
cold instead of handing the new clang PCMs it will crash on. LoadProject
and Build() now derive the directory from the same StdPcmDir. Old
<target>-<march> directories are left behind; `clean` never touched the
cache and still does not.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Inside the Flatpak the bundled toolchain compiled every bootstrap object,
then glslang's configure failed its test link with `unable to find
library -lc++`. clang searches its per-target runtime directory
(lib/<triple>) and the system library paths for libc++, not
<prefix>/lib, so with the classic layout a toolchain outside /usr needs
the search path spelled out, just as its loader needs the rpath.
LibcxxLinkFlags and build.sh now emit both, the same pair an explicit
CRAFTER_BUILD_LIBCXX_DIR already got.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every clang++, clang and llvm-ar the tool spawned was a bare name on
PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI
sysroot /usr/share/wasi-sysroot and the mingw headers
/usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A
toolchain installed anywhere else, which is what the Flatpak ships
under /app, had no way in.
Crafter.Build:Toolchain resolves all of it once per process:
CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build
(<prefix>/bin, found through the same walk GetCrafterBuildHome does),
else PATH. What sits beside the compiler is used when present: its
libclang.so, its clang-c headers, its std module source, its WASI
sysroot. An explicit prefix without a clang++ is an error rather than a
fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already
exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other
than the compiler's own, and CRAFTER_MINGW_DIR now does what its help
line always claimed: it is passed as --sysroot for the mingw target.
Binaries linked by a toolchain outside /usr get an rpath to its lib
dir, since its libc++.so.1 is not on the loader's default path.
HostTarget() is the resolved clang's -print-target-triple. A clang built
without a vendor reports x86_64-unknown-linux-gnu where Arch's reports
x86_64-pc-linux-gnu, so target equality that decides whether a test runs
locally and which targets a bare `test` sweeps goes through SameTarget,
which ignores the vendor field. The strings themselves are never
rewritten: clang rejects a PCM whose triple differs from the importing
unit's. The examples and README stop hardcoding the host triple; the
default is already the host.
`crafter-build --version` prints the version baked in at build time
(CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the
compiler and the host triple, and is answered before any argument is
forwarded to project.cpp. The version is a define on the library, so
each version is its own library variant and relinks the executable and
tests once per commit.
build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR
and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already
there, and writes a fresh archive instead of appending to one that may
hold members from an older layout.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
crafter-build is moving to a single Flatpak distribution, and there is
no Windows host in that picture: no build.cmd, no MSVC or mingw host
build, no DLL plus launcher. Delete every host-Windows path instead of
leaving it to rot unverified: the cmd.exe command runner and quoting,
the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants
with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port
probe, the LoadLibrary libclang loader, the console-width and isatty
shims, and the preprocessor guards around the Linux bodies.
The x86_64-w64-mingw32 cross-compile target stays. It is target logic
(output names, -femulated-tls, the libstdc++ link extras, copying a
dependency DLL next to a mingw exe, wine as the runner) and did not
depend on any of the removed code. The msvc target goes with the host
that was the only thing able to use it.
Three target-path warts fixed on the way: ExpectedOutput named a mingw
static library <name>.lib while the archive step wrote lib<name>.a, so
those libraries relinked on every run; the import lib is no longer
copied next to a mingw exe, since the dependency's own bin dir is
already on the link line; and llvm-ar now writes every archive, not
only the LTO ones, so Debug and Release archives are made the same way.
CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary.
Nothing downstream used them, so they are removed rather than kept as
an empty macro nobody can verify.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Crafter.LintRules declares a test that links crafter-build, fetched
with GitProject. crafter-build's own project.cpp loads Crafter.LintRules
as a `#pragma crafter dependency`, so the chain was: crafter-build loads
LintRules, LintRules' test wiring loads crafter-build's project.cpp from
the external cache, which loads LintRules again, and the cycle guard
threw. Every command that loads project.cpp in this repository has
failed since that test was added, including the last CI run on master.
A project file had no way to tell whether it was the one the user
pointed crafter-build at or a dependency being loaded for somebody
else, so it could not skip wiring that only its own build needs.
IsTopLevelProject() answers that from the loading-frame stack
ProjectDependency() already reads: one frame is the top-level project
file, a second is only ever pushed for a dependency. Outside any project
file it is true as well, which is what a test harness constructing
Configurations expects.
The ProjectDependency test records the answer from both the app and the
greeter project files and checks each.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Brings in `#pragma crafter dependency`, the switch to Crafter.LintRules
for the house style, and the contributor guide in AGENTS.md.
The branch predates the musl host work on master, and both rewrote the
command that compiles project.so, so the merge reconciles them by hand:
- LoadProject compiles project.so for hostConfig.target (HostTarget(),
from master) with the pre-dependency module path and archives from
the branch. -fuse-ld=lld appears once, with both reasons in the
comment.
- ForceHostTooling added -fPIC only for the literal x86_64-pc-linux-gnu
triple. hostConfig now comes from HostTarget(), so on Alpine the host
is x86_64-alpine-linux-musl and the dependency archive would not have
been position-independent. It now matches any *-linux-gnu or
*-linux-musl host, as project.cpp and the cmake flags do since the
musl change, and its comment no longer argues against HostTarget().
- The ProjectDependency test compared the dependency's target against
the glibc triple, which the musl CI job would fail; it compares
against HostTarget().
- README keeps master's paragraph on the bootstrap triple. AGENTS.md
describes the test block as `if (linuxTarget)`, which is what
project.cpp has now.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
lint-rules.h moves to its own repo as the module Crafter.LintRules, which
project.cpp now imports through `#pragma crafter dependency`. Editing a
rule no longer needs `touch project.cpp`: the rebuilt dependency makes
project.so stale by itself. tests/HouseRules moves with the rules.
Building this checkout needs a crafter-build that understands the
directive (`./build.sh` bootstraps one) and the Crafter.LintRules repo on
Forgejo.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project.cpp could only import std and Crafter.Build, so anything shared
between build descriptions had to be a header every consumer copies. Now
#pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"]
names a sibling crafter-build project that project.cpp itself imports a
module from. LoadProject scans for the directive, resolves it through
GitProject (a URL) or LocalProject (a path relative to the project file),
forces the result into host-tooling mode (the host triple LoadProject's
own cache is keyed on, native march, no debug, -fPIC so the archive links
into a shared object), builds it, and puts its BMIs and archive on the
project.so command line. The dependency's own project.cpp goes through
the same loader, so its directives resolve recursively; a cycle between
project files is an error instead of unbounded recursion.
The dependency is built on every load, fresh or not: clang crashes rather
than erroring when it imports a BMI whose own imported PCM was rebuilt
underneath it, and the module depfile — which Module::Check already reads —
is what notices the host PCM cache was regenerated. Its artifact mtime is
an input to project.so's staleness check, so editing the dependency
recompiles project.cpp without touching it. The Linux link gains
-fuse-ld=lld because release archives hold ThinLTO bitcode.
ProjectDependency(name) hands the same Configuration* back from inside
CrafterBuildProject, by configuration name or exported module name, so a
test that links the module needn't spell the source twice. It reads a
thread_local frame stack that an RAII guard keeps for the whole of
LoadProject, which also provides the cycle check.
ExpectedOutput() is the link gate's artifact-path lambda hoisted into the
:Clang partition, so the staleness check and the -r runner share the one
rule. Windows branches are wired identically but unverified.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The fixture puts a #warning in each kind of source the build compiles —
module interface, module implementation, C, consumer — plus an unknown -z
value so ld.lld warns on the link step too, and asserts the build succeeds
cold (every unit compiling, every warning emitted), succeeds with nothing to
do, and succeeds again after each source is touched back into staleness.
The staged copy lives outside the repo because the whole test depends on
units compiling for the first time.
The last pass breaks a source on purpose: "never fails" would satisfy
everything above just as well, so the error path is asserted too — both that
it fails and that it still reports the compiler's diagnostic.
One header per pass, asserting both directions — the objects that include
it recompile, the ones that don't are untouched — then running the binary,
since "was recompiled" only matters if the program agrees with itself.
Covers the module interface (a macro in its global module fragment decides
an exported class's layout), the implementation unit, a C source, an idle
rebuild that must recompile nothing, and an object whose depfile is gone.
The implementation unit's header has a space in its name so the depfile
spells it escaped.
The first cut scanned the initialiser's tokens and required every one to be a
literal or an operator. That is exactly the kind of approximation this work has
been removing, and it was wrong in both directions:
const int A = sizeof(Big); missed — `sizeof` is a keyword
const int B = Base + 1; missed — `Base` is an identifier
const auto C = 5_notConstexpr; would have been reported, wrongly
clang_Cursor_Evaluate answers the question directly. Evaluating a variable
declaration evaluates its initialiser, so anything that folds is recognised and
a call result still is not. Costs nothing measurable — lint stays at ~15s.
Found one real case the token version could not see: an EShMessages fold over
three glslang enum constants in Crafter.Build-Shader.cpp. Promoting it to
constexpr then made `naming` ask for constant naming, since a constexpr
variable is a compile-time constant — so it is `Messages` now. Two rules
agreeing on the same declaration is the intended behaviour.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two rules the AST makes possible, plus the mutation analysis behind them.
const-local reports a local that is never written. It is restricted to SCALARS
— integers, bools, enums, floating types — and that restriction is what makes
the answer exact rather than a guess: a scalar has no member functions, so the
only ways to write one are assignment, ++/--, having its address taken, or
binding to a non-const reference. All four are now tracked in the walk:
- assignment and compound assignment visit their LEFT operand in a write
context, the right one normally;
- ++/-- and & write their operand;
- a call argument is checked against the callee's parameter type, so passing
to `const int&` or by value is a read while `int&` is a write;
- initialising a non-const reference writes what it binds to.
For a class type a non-const method call could mutate it, and deciding that is
the whole-program analysis clang-tidy does, so those are simply out of scope
rather than guessed at.
constexpr-constant promotes a const constant whose initialiser is made only of
literals and operators, so `const int A = 1 << 4;` qualifies and
`const int B = Compute();` does not.
On this repository const-local found 103 candidates, which was too many to be
useful, and the reason was informative: most were range-for bindings and
pointer locals. `for (T* const x : …)` and `T* const p` are not spellings
anybody writes, and the useful constness for a pointer is on the pointee, which
this rule cannot advise on. Excluding both leaves 36, all plain bool or enum
locals worth fixing — isWasm, isPe, exists, writes, isC and so on. Those 36 are
fixed in this commit; the compiler verified every one.
Both rules are report-only. The analysis is exact, but adding const is a
judgement about intent as much as mechanics, and a wrong suggestion should cost
a glance rather than a build. const-local also deliberately does not become a
transform: inserting `const` before a shared type would apply it to every
declarator in a multi-declarator statement, including any that IS written.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tokens are not enough for this one, which is worth stating because it is the
opposite of the enum-class case. Splitting a multi-declarator statement by
copying the shared type prefix is wrong in C++:
int* a, b; -> int* a; int* b; // b was int, not int*
Only per-declarator types get it right, and clang has already resolved them —
`int *` for a, plain `int` for b. A token-based splitter cannot know.
The regex it replaces bailed on `*`, `&`, `<>`, parens and quotes, so pointers,
templates and call initialisers were all left alone. All three split now, and
the fixture proves the mixed pointer case above comes out correctly.
Groups are found structurally rather than by matching a line shape: the first
declarator's extent starts at the shared type, so begin < nameOffset, while a
continuation declarator's starts at its own name, so begin == nameOffset. That
signal comes from the AST itself. nameOffset is now on LintDecl, which is also
what lets the replacement reuse each declarator's original text verbatim
instead of reconstructing it.
Replacing a byte range rather than rewriting whole lines means a comment after
the ';' is outside the edit and survives — the line-based version refused to
touch any line carrying a comment. A comment INSIDE the statement still bails,
since the rewrite would swallow it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The regex required the enum's name to follow `enum` on the same line, so a
declaration split over lines went unreported. Asking for the next token instead
makes the split and unsplit spellings read identically, and distinguishes the
`enum` KEYWORD from the same letters appearing elsewhere without needing word
boundaries to stand in for lexing.
Deliberately not moved to the AST, even though LintDecl already carries an exact
isScopedEnum. An AST is one configuration's slice, so a plain enum inside a
preprocessor branch inactive for the host would silently stop being reported;
tokens are lexed without evaluating #if, so every platform stays covered. Being
right about less is not an improvement here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Linting a march whose PCMs are not built produced 102 findings — 34 files times
three AST rules — for a single fact: the project has not been built for that
configuration. The one actionable sentence was buried.
Now recorded per file and reported once, naming the rules that could not run,
how many files were affected, one example reason, and what to do about it. Still
one error, so the run still fails; --no-ast remains the way to proceed without
building.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rewrite itself stays token-shaped — it edits type SPELLINGS, which an AST
discards — but what it must not touch now comes from the AST.
The old exemption was per LINE: `int main`, `argc`, `argv`, `extern "`. Being a
transform, a missed exemption here does not over-report, it emits code that no
longer matches the API being called, so this is the rule where guessing from
substrings mattered most. And being per-line, it also disabled the rule for
anything sharing a line with one of those words.
Now a declaration with C language linkage, or one whose initialiser binds to an
entity declared outside the project, contributes a protected byte range and
keeps its spelling. That answers the case directly: a function declared in
somebody else's header taking `unsigned int` keeps `unsigned int`, and a local
initialised from strtoul keeps `unsigned long`, because of where those are
declared rather than because of what the line says.
main is protected from the start of its declaration to the opening brace of its
body, not for its whole extent. Its signature is fixed by the language; its body
is ordinary code. Three findings on this repository came out of that
distinction, all correct:
for (int i = 1; i < argc; ++i) -> for (std::int32_t i = 1; ...)
skipped before only because `argc` appeared on the line, plus Crafter::Run's own
`int argc` and return type, which are ours rather than the language's.
All three AST rules now share one interop test instead of carrying a denylist
each, and it is the same test: whose header dictates this spelling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule was 125 lines: four std::regex, a hand-rolled {/} scope stack, a
cumulative paren-depth counter so a wrapped parameter list would not look like
a declaration, a 40-entry keyword denylist, and a "function-shaped line" guess
whose own comment conceded it was heuristic. Storage class came from
lineStr.contains("static "). It is now ~55 lines that ask clang what kind of
declaration each thing is and what encloses it.
The three regressions the old version carried special cases for — a call with
an inline lambda argument, a bare statement call, a one-liner method — need no
handling at all, because a call is not a declaration. Their tests pass
unchanged.
On this repository the exact version found 42 violations the heuristic had
never been able to see, all real:
- 37 members of the libclang function-pointer table added two commits ago
were PascalCase. The old varDecl regex could not match a declaration whose
type is decltype(&f), so they were silently skipped. Renamed to camelCase,
which mirrors clang_createIndex -> createIndex more closely anyway.
- Crafter.Build-Shader.cpp had a snake_case local, file_name_list, invisible
to the heuristic for the same reason (it declares a const char* array).
- Four extern "C" declarations of libc functions in tests were reported as
badly-named functions. Those are named by the C library, so C language
linkage is now an exemption — the same principled test no-char-pointer
uses, rather than another denylist entry.
New tests cover what the line-based version structurally could not reach: a
signature wrapped over several lines, `static` on its own line above the
declaration it applies to, and a member versus a local inside a method.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule was `\bchar\s*\*` over the text minus a substring denylist — argv,
getenv, setenv, dlerror, c_str, .data(, reinterpret_cast, extern ". Every entry
was a patch for one interop site, the list could only grow as libraries
arrived, and each entry disabled the rule for the whole LINE it appeared on.
It now walks declarations and asks the question the denylist was approximating:
whose header dictates this spelling? A declaration with C language linkage, or
one whose initialiser binds to an entity declared outside the project root, is
somebody else's API and keeps its spelling. getenv, c_str and friends are
exempt because of where they are declared, not because they are named here, so
a new external library needs no new entry.
Two bugs found while testing this, both of which had made the rule silently
pass over the entire repository:
clang_getCursorLanguage cannot be used to detect extern "C". Its default answer
is CXLanguage_C for a plain function, variable or parameter even in a C++
translation unit, so isExternC was true almost everywhere and exempted
everything. Replaced by tracking CXCursor_LinkageSpec depth during the walk,
reading the extent text to tell extern "C" from extern "C++".
Attributing any foreign reference in a subtree to the enclosing declaration was
too broad: a function that merely touched libc++ somewhere in its body would
exempt its own signature. Narrowed to initialiser contexts — a variable, field
or parameter — which is where a binding to a foreign API actually occurs.
Also: functions now carry their RESULT type rather than the whole function
type, since the parameters arrive as their own declarations and would otherwise
be reported twice. main's parameters are exempt structurally, its signature
being fixed by the language rather than chosen here.
Two sites keep an explicit lint-disable, both Crafter::Run taking main's argv
verbatim. That is two visible, reasoned suppressions in place of a denylist
that silently disabled the rule for every line mentioning one of eight tokens.
ExternalCloneDir and ExternalIncludeFlags are now exposed from :External, so a
source that includes an external dependency's headers can be parsed without
running a build to discover where they are. BuildExternal derives its own
working directory through the same function, so the two cannot drift.
Crafter.Build-Shader.cpp needed this to parse at all.
A file with no compile command — project.cpp, which LoadProject builds with its
own flags — is not a translation unit of the build graph, so AST rules skip it
the way a rule self-filters by extension. That is distinct from a file that
should have parsed and did not, which stays an error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds LintContext::Decls() — clang's view of the declarations written in the
file — plus AddAstLintRule to register a rule that reads it. No rule uses it
yet; the three that will are migrated separately.
The declarations come back as a flat vector with parent indices rather than an
opaque cursor handle: no lifetimes cross the project-DLL boundary, no callback
hops back into project.so per node, and "is this at namespace scope or inside a
function?" becomes an index lookup instead of a hand-rolled brace stack.
Three things had to be solved for this to work at all on this codebase.
libclang cannot see through `export`. A C++20 export declaration has no
CXCursorKind, so `export namespace Crafter { … }` arrives as a childless
CXCursor_UnexposedDecl and clang_visitChildren does not descend. Five of the
eleven interfaces here are written that way — 677 lines, including
Configuration and LintContext, yielding zero usable cursors. A plain
`namespace` IS descended into, so the fix is to blank the keyword before
parsing, byte-length preserving so every line and column still lands on the
original file. `export module` is left alone or the unit stops being a module
interface. Verified end-to-end against a fixture whose asserted line numbers
match the unblanked file.
PCMs are flag-locked, so each file has to parse with the flags that built it.
CollectConfigSources now records which Configuration owns each source instead
of flattening to a set, because three regimes coexist: the library, each test
(carrying its own target, defines and -march), and project.cpp, which Build
never touches and which therefore gets no command at all.
libclang resolves its builtin headers relative to its own install path, which
need not match the clang++ that wrote the PCMs. When it doesn't, every parse
dies on "'stddef.h' file not found", so -resource-dir is passed explicitly
from `clang++ -print-resource-dir`.
Two flags on each declaration replace what would otherwise become more
substring denylists: isExternC, and isForeignApi for a declaration that binds
to an entity declared outside the project root — resolved through
clang_getCursorReferenced and the same inside-the-root test the dependency
walk already uses. Parameters and fields inherit it, so an exemption covers a
whole signature rather than the one node that named the foreign entity.
Failure is never silent. A fatal diagnostic leaves a fragment that is
indistinguishable from a file declaring nothing, so it is reported as an error
instead: the rule is skipped, a finding explains why, and summary.errors makes
the run fail. --no-ast opts out deliberately and exits normally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the linter's hand-rolled character scanning with clang's lexer.
StripComments could not see raw string literals, which was documented as a
v1 limitation but was worse than that: an odd number of quotes inside a raw
string desynchronised it for the rest of the file, hiding real code and
leaving the contents of later literals standing as if they were code. There
are 33 raw strings across 6 files here, including the two largest.
The reflow guards had the same class of bug from the other direction. They
were substring probes, so Line(n).contains("//") fired on // inside a string
literal and contains("R\"") fired on the characters R" inside a literal while
missing raw strings opened on an earlier line. Both misfire on this repo:
"MARKER" ends in R". They are now LineHasComment() and
LineHasMultiLineToken(), and two wrapped call sites in tests/Lint that had
been silently unjoinable join as a result.
libclang is dlopen'd rather than linked, so the mingw and MSVC cross-builds
keep linking and the release tarballs stay self-contained. Tokens need no
PCMs, no build flags and no prior build, and they cover preprocessor branches
that are inactive for the host — which is why the layout rules belong on
tokens rather than on an AST that would only ever see one platform's slice.
No rule behaviour was intended to change beyond the two guard fixes; lint
over this repo produced byte-identical output across the CommentStripped
swap.
A primary module interface unit — `export module Widget;`, no partitions —
recorded nothing about what it imported. GetInterfacesAndImplementations
registered the Module and then erased the file from the scan list, so the
import pass only ever saw partitions, and Module had no vectors to hold an
edge anyway.
Two consequences, both reported as issue #26:
Module::Check consulted only its own .cppm and its partitions. A data
member added to an imported module left Widget.pcm, Widget.o and every
consumer object untouched while the imported library rebuilt and both
binaries relinked — one executable holding two class layouts, no
diagnostic, and a crash somewhere unrelated. Wiping build/ was the only
cure, so `crafter-build test` could not be trusted straight after an
interface edit.
Module::Compile waited on nothing. Two modules in one Configuration
compile on concurrent threads, so a primary interface importing a
sibling was a coin flip between working and "module 'Base' not found".
Partitions never had either problem — they carry the same three vectors and
Check/Compile honour them — which is why the gap only surfaced on a module
whose interface is one flat unit.
Module now carries moduleDependencies, externalModuleDependencies and
pendingImports with the same meanings as on ModulePartition; primary units
stay in the scan list so their imports land there; Check sees through them;
Compile orders itself behind a local sibling; and ResolvePendingImports
sweeps them so an edge survives dependencies being wired up afterwards.
Build() now Checks every interface before spawning any compile thread — the
`compiled` flag a waiter blocks on is raised either by a Compile that runs
or by the Check that decides none is needed, so a Check still pending while
another module's thread waits would have hung the build.
Resolves#26
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three follow-ons to the stale-build report, all cases of an identity not
capturing something that changes the output.
The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by
every crafter-build on the machine, and freshness was a per-file mtime
comparison. That cannot tell "this PCM is newer than my source" from "this PCM
was built from different sources that happen to be newer", so a package install
and a working checkout — or two checkouts of different versions — silently
compiled their project.cpp against each other's declarations. Invalidation now
keys on a stamp over the bytes of every module source, which also covers the
case one file's mtime never could: the cached PCMs import each other, so a
change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm
untouched.
Project args ApplyStandardArgs does not itself interpret are now folded into
VariantId. Such a flag typically decides what gets compiled or bundled — the
report's example is --no-webgpu dropping entries from cfg.files — and without it
both settings shared one bin dir and interleaved their outputs there, leaving a
bundle matching neither. Sorted and deduplicated so flag order doesn't split the
cache, and inherited by test Configurations.
`crafter-build clean` removes the project's bin/ and build/ trees. It
deliberately does not load project.cpp: cleaning is most often reached when
something is already wrong, and a clean that first needs the project to compile
is useless exactly then.
Adding a data member to a class in a module interface did not rebuild every
object compiled against the old layout. The build succeeded with no error or
warning and the resulting binary mixed both layouts, surfacing later as a
SIGSEGV in a destructor.
GetInterfacesAndImplementations scans a TU's `import X;` statements when the
source is declared. An import that matches neither a module in the
Configuration nor one reachable through `dependencies` was dropped on the
floor, leaving that TU with no staleness edge to the interface it consumes.
`dependencies` is frequently assigned *after* the scan — AddTest does exactly
that, resolving tests/<name>/main.cpp and only then returning a builder whose
.Dependencies() supplies the library — so consumers of a dependency's modules
routinely carried no edge at all. A layout change then rebuilt the library,
relinked the consumer, and kept the consumer's object as it was.
Unresolved names are now remembered on the partition/implementation as
pendingImports, and Configuration::ResolvePendingImports retries them against
the dependency DAG as it stands. Build() calls it immediately before comparing
mtimes, which closes the window for every caller rather than only the ones that
declare in the right order; TestBuilder::Dependencies also calls it so the
Configuration is coherent for anyone inspecting it before the build.
Resolves#27
The transforms guard themselves against comments and raw strings before
joining or rewriting a line, because pulling text up past a `//` buries it
and reflowing a multi-line literal changes the string. Those guards were
substring probes over the raw line, so they answered the wrong question:
Line(n).contains("//") fires on // inside a string literal
Line(n).contains("R\"") fires on the characters R" inside a literal, and
MISSES a raw string opened on an earlier line
Both misfire on this repo's own sources. "MARKER" ends in R", and any string
mentioning a lint-disable directive contains //. Two wrapped call sites in
tests/Lint were being left unjoined for exactly these reasons; they join now,
and the results are in this commit.
Replaced by LineHasComment() (added with the token layer) and a new
LineHasMultiLineToken(), which reports whether any token actually covering
that line spans a line boundary — a raw string or a block comment. Backed by
a per-line bitmap derived from the token cache and invalidated with it.
The guards themselves stay: joining across a real comment or a real
multi-line literal is still unsafe, and there are tests for both. What
changes is that they now fire on comments and literals rather than on the
characters that spell them.
format-concat gets narrower as a result. It used to refuse any line
containing R" and ask for a manual fix; now only a literal that genuinely
spans lines does that, because a single-line raw string reduces to R"…" in
the stripped view, fails the plain-literal test, and travels through as an
argument with its spelling intact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
StripComments hand-scanned five states over raw characters and could not see
a raw string literal, which was documented as a v1 limitation. It was worse
than "does not recognise them": an odd number of quotes inside a raw string
desynchronised the scanner for the rest of the file. Simulated on the new
fixture, the old output was
auto banner = R" "hi)" \n \n ... "MARKER text"
— the code after the raw string blanked away, and the *contents* of a later
string literal left standing as if it were code. Every rule reading
CommentStripped() saw that. There are 33 raw strings across 6 files here,
including the two largest.
The replacement projects the token stream onto a copy of the buffer, blanking
comment tokens whole and the bodies of string/character literals. Editing a
copy in place makes the length-preserving contract structural rather than
something each branch has to remember, and the token extents make raw
strings, escapes, encoding prefixes and '"' all fall out for free. A raw
string reduces to R"…" so rules that bracket a literal by counting quotes
keep working. Digit separators (1'000) are excluded by requiring the text
before the quote to be an encoding prefix.
The public contract is unchanged, so no rule needed editing, and `lint` over
this repo produces byte-identical output to the previous implementation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds LintContext::Tokens() and friends, backed by clang_tokenize, as the
substrate the rules will move onto. Nothing consumes it yet.
libclang is dlopen'd rather than linked: -lclang would break the mingw and
MSVC cross-builds at link time and would put a libclang.so.NN runtime
dependency into the otherwise self-contained release tarballs. The clang-c
header is used for its declarations only, and the function-pointer table is
typed with decltype so the signatures cannot drift from the real API.
Three properties this buys that the hand-rolled scanners could not have:
- a raw string literal or block comment is ONE token, so the documented
"raw string literals are not recognized" limitation goes away;
- `//` inside a literal is not a comment, so LineHasComment() replaces the
Line(n).contains("//") probes that false-positive on it;
- tokens cover preprocessor branches that are inactive for the host, since
clang_tokenize lexes rather than evaluates #if. Token rules therefore
keep seeing every platform's code, which an AST could not offer.
The parse backing the tokenizer is expected to fail on module units — no
PCMs, no build flags — and that is fine, because lexing has no semantic
prerequisites. Verified in the new tests.
LintSummary::Clean() now counts `errors`. It previously ignored them, so an
infrastructure failure that produced no findings reported clean and exited
0; a missing libclang would have been exactly that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Canonical LGPL-3.0-only text, GPL-3.0 companion, SPDX headers on all
first-party sources, MIT for examples/. Vendored lib/ untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The browser wasm pipeline hardcoded -msimd128 for every wasm32 target and
baked a single wasm URL into index.html, so newer codegen features that
aren't yet baseline across engines (relaxed SIMD today; threads, future SIMD
revisions later) couldn't be adopted without dropping the browsers that lack
them.
Add a general, feature-parameterized mechanism owned entirely by
Crafter.Build:
- Configuration::wasmVariants declares N codegen variants (label, extra -m
flags, runtime probes). Build() compiles the baseline plus one
outputName.<label>.wasm per variant, recompiling the whole graph (incl.
dep libs + std PCM) with the variant's flags — relaxed-SIMD is per-TU
codegen, not a link switch. wasmVariantFlags folds into VariantId so each
variant's objects/PCMs land in their own build+bin dir.
- EnableWasiBrowserRuntime emits a variants.json manifest (label -> url +
probes), preferred-first with the baseline as the universal fallback.
- The shipped runtime.js runs inlined wasm-feature-detect probes
(relaxed-simd, simd, tail-call, bulk-memory, exception-handling, threads),
picks the first variant whose probes all pass, and falls back to the single
baked CRAFTER_WASM_URL when no manifest is present (backward compatible).
- EnableWasiRelaxedSimdVariant registers the relaxed-SIMD variant — the
motivating case (Chrome 114+/Firefox 120+ enable it by default; Safari
still flag-gates it as of mid-2026).
Verified end to end: a wasm32-wasip1 build emits both wasi-hello.wasm and
wasi-hello.relaxed-simd.wasm + variants.json; Firefox selects the
relaxed-simd variant and runs it.
Resolves#24
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build() resets each Module/ModulePartition's per-build `compiled`/`checked`
flags so a reused Configuration re-evaluates mtimes. That reset recursed into
cfg.dependencies — but dependency Configurations are shared across the build
DAG and each is compiled concurrently by its own Build() call.
A parent/sibling's recursive reset could therefore clear a shared dependency's
module `compiled` atomic *after* that dependency's module-compile thread had
set it true and exited, but before an intra-config waiter (its impl, or a
dependent partition) ran compiled.wait(false). The waiter then blocked forever
on a flag nothing would re-signal: the build froze mid-compile, idle, with no
compiler process alive — exactly the hang in issue #16.
Reset only the current configuration's own modules. Every config in the tree
already gets its own Build() call (the per-PcmDir builder registered in
depResults), which resets its own state at the top of that call, sequenced
before its compile threads spawn. Cross-config module state is consulted only
via PCM file mtimes and the depResults futures, never via these flags, so the
narrower reset is correct and removes the data race entirely.
Adds ConcurrentDependencyReset: builds a static-lib dependency fully, then
builds a consumer that depends on it while the dependency is already cached in
depResults (so it is never rebuilt), and asserts the consumer build leaves the
dependency's module `compiled` flag intact. Fails deterministically on the old
recursive reset; passes with the fix.
Resolves#16
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two crafter-build invocations sharing XDG_CACHE_HOME used to clobber each
other's writes to <cache>/<target>-<march>/std.pcm and the
Crafter.Build-*.pcm modules: each LoadProject path wrote directly to the
final path, so a reader could see a half-written file and die with
"malformed or corrupted precompiled file: 'can't skip to bit X from Y'"
(issue #14). Every BuildStdPcm / EnsureCrafterBuildPcms write now goes via
<final>.tmp.<pid>.<seq> and atomic-renames into place; concurrent writers
always see either the old or the new file, never torn bytes. The mingw-on-
Linux std.cppm copy is per-PID for the same reason. Adds a regression test
(ConcurrentCacheRace) that races four LoadProject() calls against a cold
scratch cache — reproduces the race 5/5 without the fix and passes 5/5
with it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The suite had only HelloWorld, which built and exited an empty exe. Add
in-process tests covering each public surface area users actually touch:
- StaticLib / ModuleInterface / DependencyLink — Build() against
fixtures for libraries, project-local module interfaces, and
cross-config module deps with link verification (runs the built exe).
- ShaderCompile — drives Shader::Compile directly, validates SPIR-V
magic + Check() idempotency.
- StandardArgs — covers --debug, --target=, --march=, --mtune=,
--lib/--shared promotions, and ArgQuery::Has / Get.
- TestRunnerSpec — FromSpec parse rules, ForTarget routing for host,
wasm32-wasip1, aarch64-linux-gnu (+ sysroot QEMU_LD_PREFIX),
i686 → qemu-i386 rewrite, mingw → wine on Linux hosts, FromEnv.
- VariantId — confirms type / debug / sysroot / defines / compileFlags /
target / march all perturb the cache key, plus PcmDir routing.
- WasiBrowserRuntime — calls EnableWasiBrowserRuntime, asserts the
three cfg.files entries get registered and index.html had its
template placeholder substituted.
- RunSingleTestExit — drives RunSingleTest against tiny sh scripts and
pins the documented exit-code mapping (0/77/non-zero) and the
Cmd-prefix runner path.
Closes#12.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
With test.toml + ForTarget covering the cross-arch + Windows-on-Linux
cases, the env-var-driven transport runners are dead weight. This commit
removes them and the retired tests that exercised the env-var plumbing:
- TestRunner::Ssh / SshWin / Wsl factories and their copy/exec/cleanup
template machinery.
- TestRunner::Shell enum (Host/Sh/Cmd) and the ShellQuoteSh helper —
only Host shell quoting is needed once the remote shells are gone.
- TestRunner::copy / cleanup / remoteDir / argsShell fields.
- WindowsPathToWsl and the {remote_bundle}/{bin_win}/{bundle_wsl}
placeholder substitution in RunSingleTest's transport branch.
- ParseRunnerSpec narrowed from {local, cmd, ssh, sshwin, wsl} to
{local, cmd} — the override hatch is preserved, just simpler.
- tests/SshRunner, tests/WindowsViaSsh, tests/QemuUser: these tested
the CRAFTER_BUILD_RUNNER_<target> → runner plumbing that has been
replaced by ForTarget. The runner derivation is exercised every
time CrossArchAarch64 / Wasi / WindowsViaWine runs.
- tests/UnitLib: ssh/sshwin spec assertions become "throws on bogus
spec" assertions.
Refs issue #8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collapses three outer-driver tests into declarative top-level fixtures:
- CrossArchAarch64: outer + inner pair becomes main.cpp + test.toml. The
'is this really an ARM aarch64 ELF?' artifact-introspection check is
dropped — qemu-aarch64 refuses to run wrong-arch ELFs anyway, so a
silent host-arch fallback would still fail the run.
- Wasi: outer + inner pair becomes main.cpp + test.toml. The WASM
magic-byte check is dropped on the same logic (wasmtime refuses
non-WASM input).
- Defines: simple defines-propagation smoke test becomes test.toml with
[defines] CRAFTER_TEST_FOO = "42".
Adds WindowsViaWine to replace the (forthcoming-deletion) WindowsViaSsh:
declarative target=x86_64-w64-mingw32 + requires=[tool:wine,
tool:x86_64-w64-mingw32-g++]. Exercises the new Wine runner and the
ForTarget derivation end-to-end.
Diamond and CrossProjectModule had speculative --target= reading that
made them attempt cross-compilation under the multi-target sweep. They
have no sysroot/toolchain plumbing, so those cross-builds always fail.
Hardcoded them to HostTarget(); cross-arch tests live in their own
test.toml from now on.
Refs issue #8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
