-flto in a wasm configuration's compileFlags or linkFlags reached the
link, where LTO code generation runs without the backend options that
keep exception handling: every landing pad was dropped, and the build
succeeded with throws that escaped their catch blocks. -fno-exceptions
there made every `import std;` fail on the std module built with
exceptions, and a wasm shared library failed to link against the eh/
libc++, which is static archives built without -fPIC. Build() refuses
all three now, each with a line that says why, before it makes a
directory or starts a thread.
The sysroot check sat in BuildStdPcm, which the linter calls as well.
There the error only dropped the compile command, so lint ran no AST
rule on a wasm configuration and reported it clean. It also refused
static libraries, which never link, fired after the shader and
external threads had started, and called a missing sysroot, or one
without the target, "no C++ exception support". It is part of the same
early check now, which first says when the sysroot or the target's
directory is missing, and asks for libunwind only where something
links.
C sources got only the signal.h define. setjmp.h refused to compile in
C, and a C object given -wasm-enable-sjlj through compileFlags used the
legacy exception encoding next to C++ objects in the standard one, a
mix engines refuse to load. They get WasmCompileFlags now, like the C++
units, a variant pass's flags included.
The baseline probes include simd: V8 and SpiderMonkey switch SIMD off
on x86 CPUs without SSE4.1, which virtual machines often present, and
such an engine passed the exception-handling probe only to fail
compiling the module; the manifest tests expect it on every entry. The
Flatpak pins WASI_SDK_EXCEPTIONS=DUAL, which
wasi-sdk 34 defaults to only with clang 23 or newer, and checks that
the eh/ libunwind got installed: the wasm tests skip without it, so CI
would otherwise pass a Flatpak whose every wasm build fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wasm builds were compiled with -fno-exceptions. A throw did not
compile, libc++'s own throws (vector::at, std::format, filesystem)
aborted the program, and code that relies on exceptions could not be
built for the browser at all.
Every wasm unit and the std module now get -fwasm-exceptions, and the
link adds -lunwind, which code built that way calls into and the driver
does not add. The flag also selects the libc++ under the sysroot's
lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh
one. Those libraries use the standardized encoding (try_table, exnref)
while LLVM 23 still emits the legacy one by default, which wasmtime
refuses to load and which would put both in one binary, so the flags
ask for the standard form with -wasm-use-legacy-eh=false. That makes
the setjmp lowering loadable under wasmtime too.
The flags live in one constant, WasmBaselineFlags, which the compile
command and the std module both take through WasmCompileFlags: a BMI
is refused by a unit whose exception setting differs from its own.
VariantId and StdPcmDir hash it, so the first build after the update
starts fresh instead of importing PCMs built without exceptions, and
two releases sharing the Flatpak's cache don't take turns rebuilding
one std PCM. A sysroot without the exception-enabled libraries stops
the build with a line saying so, rather than at the link with "unable
to find library -lunwind".
The result needs an engine with exnref: Chrome 137, Firefox 131,
Safari 18.4, the bundled wasmtime. The runtime's exception-handling
probe tested the legacy encoding; it now tests try_table, every
variants.json entry carries it, and a browser without it is told which
feature it lacks instead of failing with a CompileError. There is no
switch back to -fno-exceptions: engines without exnref are old enough
not to keep a second build for them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #42, fixes#41. Build() now refuses a configuration whose transitive
dependencies hold two configurations exporting a module of the same
name, such as one project loaded along two paths with different arg
lists, before it builds anything, and names both paths. Until now both
builds ended up in the one program, which compiled cleanly and then
read garbage. Merged cleanly on top of claude/forward-project-args;
the suite, with claude/dependency-notes' fixtures, passes under the
new check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #40. GitProject and LocalProject default their args to the loading
project's own (ProjectArgs()), this tree lints with LintRules' lite
set, and the fixes from the branch's review: a project file gets its
args only for the CrafterBuildProject call, host tooling gets project
cache entries of its own, and `--` ends crafter-build's options.
The conflicts with claude/dependency-notes are all places both sides
added to:
- Platform.cppm and Platform.cpp: PragmaDependencies() beside
ProjectArgs() and LoadingHostTooling(), all three kept.
- LoadProject: the record PragmaDependencies() reads moves up with the
frame's dependencies, which this branch sets before dlopen. Left
where it was, it would have copied preDeps.configs after the vector
was moved into the frame, and recorded nothing.
- README: the pragma section's new last sentence goes before the new
"Dependency notes for agents" section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project that two paths load with different arg lists is two
configurations, each with a variant of its own, and Build() built both.
The consumer's compile got both PCM dirs on its -fprebuilt-module-path,
and clang resolved the module by name to the first for every import,
also for the import inside the other build's BMI consumers, with no
check that it was the build they had been compiled against. With
P -> M and P -> A -> M, where the two lists change a struct M exports,
P compiled against one layout, A against the other, and the program
built cleanly, then read garbage or crashed (#41).
Each configuration now checks, before it makes a directory or builds a
dependency, that no two configurations among it and its transitive
dependencies export a module of the same name, and otherwise fails with
both paths and what tells the builds apart. Going by module name rather
than by project file also catches a checkout on one path and a clone of
the same project on the other. Per configuration, not per invocation:
two builds of one project in different programs, such as a test and a
#pragma crafter dependency tool, are fine.
Picking one of the two builds was not an option: arg lists have no merge
rule, so whichever consumer lost would silently get a different build
from the one it asked for.
Fixes#41
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The help has advertised `[options] [-- project-args...]`, but Run never
looked for `--`. It fell through to projectArgs as a word of its own,
and the words after it were still read as this tool's options and
verbs: `crafter-build -- -v` turned verbose output on. In projectArgs
`--` changed the project's variant hash, and now that siblings inherit
the args, the hash of every sibling down the chain, each building into
directories of its own that the project's clean never removes.
`--` now ends this tool's options and goes nowhere itself. Every later
word is an operand of the verb, as POSIX has it: a project arg for a
build, a glob for test, lint and format, the script's name and then its
args for run. A glob rather than a project arg after `test` because
`crafter-build test -- 'House*'` is how such a filter gets typed; a
project arg goes before the `--`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ForceHostTooling rewrites a `#pragma crafter dependency` and everything
it loads in place: host target, march native, release, -fPIC. Those are
entries of the project cache like any other, keyed by project file and
args alone. A project that loaded the same sibling as its tool, with
the same args, got the rewritten entry back. That happens whenever the
project gets no args, since host tooling gets none either: a consumer
built for x86-64-v2 then imported a sibling's BMI compiled with
-march=native, and clang refused it. Any arg on the command line split
the keys and hid it.
A thread-local count of pragma loads now marks the tool's load and
every load inside it, its project file's siblings included, and
GitProject and LocalProject put that mark in the key. The clone stays
shared; only the Configuration is separate, as for two arg lists. A
module-internal function rather than a spec field, because the
siblings of a tool are loaded by the tool's own project.cpp, which
cannot know it is host tooling.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ProjectArgs() read args that LoadProject recorded when it pushed the
frame, so they were there for everything that ran on that frame, not
only for CrafterBuildProject. Two things went wrong with that.
The pragma pre-pass runs on the frame too, so both of its specs needed
a hand-written `.args = {}` to keep the consumer's args from reaching
host tooling, and anything else built there would have needed one as
well. And dlopen runs a project file's static initialisers only on the
first load of that file: a later load gets the same mapping back. A
spec made at namespace scope therefore kept the first load's args for
every later load of the file, a silently wrong variant.
The frame now gets the args just before CrafterBuildProject is called.
The pre-pass and the static initialisers see none, on every load, so
the two opt-outs go. The dependencies the pre-pass resolved move the
other way, to just before dlopen: they are the same on every load, and
ProjectDependency() called from a static initialiser used to find none
and terminate the process.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in dependency notes: a crafter-build project that ships an
AGENTS.consumer.md beside its project.cpp has it kept in a marked
section of each consumer's AGENTS.md, refreshed by `crafter-build` and
`crafter-build test`, chosen by cfg.dependencyNotes (Transitive,
Direct, None). The new partition is Crafter.Build:Agents; the loader
records where each GitProject and LocalProject came from and what a
project file's pragmas resolved to.
The branch started before master's four external-fetch commits. Both
touch External.cpp, the Clang partition, project.cpp, README.md and
AGENTS.md, but in different places, so the merge needed no hand
reconciliation; the merged tree passes build.sh, the self-host build,
format --check and the test suite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A review of the feature turned up the following, fixed here.
Writing. AGENTS.md was replaced through a temporary renamed over it,
which swaps in another file: a read-only AGENTS.md was rewritten
anyway, a hard link such as a CLAUDE.md beside it kept the old text,
ACLs and ownership were dropped, and without an fsync a power cut could
still leave it empty. The temporary was named after the PID, which
every Flatpak instance shares as PID 2. The file is now written in
place, from the first byte that differs, then truncated and synced,
under an flock on the file itself. A write that fails partway is undone
where the disk allows; with the section last, where it is appended,
only the section can be damaged. A read-only file is opened for writing
only when something would change, so it is refused then and silent
otherwise.
Reading. libc++'s filebuf reports a failed read as the end of the file,
so an I/O error came back as a short file and the section was written
over what was left. Reads go through read(2).
Code fences. ```make``` was taken for a fence, and a fence in a list
item was missed while its indented closer opened one, so markers a user
quoted in a code block could be taken for the section, or the real one
hidden and appended again. Backtick info strings, block quotes and list
items are now followed, and a block ends with its container.
Warnings with nothing to write. With no notes, nothing short of a
well-formed section is reported, so an AGENTS.md that ends in an open
code block, or a project set to None, no longer warns on every build.
An unclosed fence after a section no longer stops it being rewritten
in place.
Labels. The `from` line carried the git URL, credentials and all, or a
path relative to the consumer that changed with the machine, the cache
location and --local. It is now the repository's or the directory's
name. Notes are deduplicated by repository rather than by clone, so one
library pinned two ways is described once, as the nearer spec has it.
The walk. It no longer looks at the project's tests: a plain build
never did, and a test's dependency can point at the Configuration the
project file returned by value. It goes ring by ring over projects
rather than configurations, so what the project names itself comes
first even with the library/executable split. The project's own
AGENTS.consumer.md is excluded by path, also when a dependency's
project file sits beside the project's.
Removing a section dropped a blank line even from between two
paragraphs, joining them; only a section that ends the file takes it
now. A CRLF file gets a CRLF section.
A note was checked on its raw bytes and again, under other whitespace
rules, on the trimmed ones, so it could pass the first check and fail
the merge, which then dropped every note. It is normalised once and
checked on exactly what is merged.
`test --list` only lists, and no longer writes AGENTS.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every project.cpp with a sibling dependency opened with the same line,
std::vector<std::string> depArgs(args.begin(), args.end());
and handed depArgs to each GitProject and LocalProject. CrafterBuildProject
receives a span of string_views and the specs wanted owned strings, and
without the copy the sibling got none of the project's args: it was built
for the host while the project cross-compiled, and in release while it
built --debug. GitProject then turned the copy straight back into
string_views for LoadProject.
Both specs now default their args to ProjectArgs(), the args LoadProject
was given for the project file whose CrafterBuildProject is running. It
reads the per-thread stack LoadProject already keeps for IsTopLevelProject
and ProjectDependency, whose frames now record the args. A default member
initializer rather than a new type or an optional: the member stays a
std::vector<std::string>, so every existing `.args = depArgs` keeps its
exact meaning (each of the ~50 calls across the Catcrafts projects passes
one), and `.args = {}` still forwards nothing. It is the shape of
Configuration::target = HostTarget(). A project.cpp that drops its .args
needs a crafter-build with this change: an older one compiles it and
forwards nothing.
ProjectArgs() is declared in :External, next to the defaults that call
it. :Platform, where it is defined, cannot be imported there: its
unexported `struct Configuration;` would make :External's exported one an
ill-formed redeclaration, which clang rejects.
The #pragma crafter dependency path now passes `.args = {}` explicitly.
Host tooling is built for the host whatever the consumer targets and was
loaded with no args; the default would otherwise hand it the consumer's
--target= and project flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An agent working on a crafter-build project reads its AGENTS.md, and
learns nothing there about the libraries the project builds against
unless someone copied that in by hand. This repository shows the cost:
the code style section of its AGENTS.md restates what Crafter.LintRules
enforces, and goes stale whenever the rules change.
A library now ships those notes itself, as AGENTS.consumer.md beside
its project.cpp; nothing in its project.cpp changes. When a consumer
has an AGENTS.md beside its own project.cpp, `crafter-build` and
`crafter-build test` keep a section in it, between two marker comments,
holding its dependencies' notes. The section is appended the first time
and rewritten in place after that. The file is written only when the
section changes, and an AGENTS.md is never created. lint, format, run
and clean leave it alone. The update runs after loading, which has just
fetched every dependency, and before compiling, so a build that fails
still has the notes.
cfg.dependencyNotes picks the dependencies. Transitive, the default,
follows `dependencies` all the way down from the project's own
configurations, its tests and its `#pragma crafter dependency` tooling;
Direct stops at the first loaded project on each path; None removes
the section. A dependency's own pragma tooling never counts: it is
about how that dependency is built, not how it is used, and being no
`dependencies` edge it is never reached. Nor are a dependency's tests:
nothing builds them for the consumer, and their configurations can
point at a local their project file has since moved out of.
The walk has to know which Configurations belong to another project,
and where that project lives, and cfg.path cannot say: only the
Configuration GitProject and LocalProject return is rebased, so a
dependency's internal configurations keep "./", which resolves to the
consumer. The loader records it instead, LoadedProjectOf beside the
project cache, and keeps what the root's pragmas resolved to past the
frame LoadProject pops, PragmaDependencies. Neither is exported.
The markers count only as whole lines outside fenced code blocks, so
documentation can quote them. Unbalanced markers and a document ending
inside a code block leave the file alone with a warning: guessing which
part of someone's file is generated would overwrite their text, and
missing the section would add another one on every build. A note that
would break the section when it is read back, through a marker line of
its own or a code block it never closes, is left out with a warning.
The file is replaced through a temporary renamed onto the symlink's
target, so a linked AGENTS.md stays a link and a write cut short leaves
the old file rather than part of the new one.
The code is a new partition, Crafter.Build:Agents, listed in build.sh,
CrafterBuildModules, project.cpp and the umbrella.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`crafter-build test` builds a project's tests side by side, and a
project whose tests each copy its externalDependencies hands every one
of them the same unpinned dependency. Each test's Build() pulled the
shared clone. Before the clone had a lock, those pulls ran at the same
time and broke its .git/FETCH_HEAD, which failed every test of
Crafter.Graphics (issue #34). Since 653dd0b they wait for each other,
but every one is still a network round trip: 26 tests cost 26 pulls of
each dependency, one after another, and for a CMake dependency as many
configure checks and builds.
depResults already records one build pass, keyed by the PcmDir of every
dependency configuration built in it. An external dependency now goes
in the same map, keyed by the directory its build leaves
(ExternalBuildDir: the clone, or for CMake its build directory). The
first configuration that needs it fetches and builds it; the others
wait for that and take its outcome. Keying on the build directory
rather than the clone keeps a CMake declaration from taking the outcome
of a header-only declaration of the same clone, which only fetched it.
A fresh depResults is a fresh pass, so the next build still pulls, and
every caller of Build() gets this: RunTests, a plain build, and the
pragma dependencies LoadProject builds.
To make that possible BuildExternal is split. BuildExternal is now the
shared work, lock, fetch, configure and build, and returns the error.
ExternalOutputs computes what one declaration compiles and links with,
because two declarations of one clone can differ in includeDirs,
libDirs and libs; ExternalBuildResult loses its error field. The work
no longer stops between steps when the configuration that started it
is cancelled: others may be waiting on it, and a run cut short would
leave them a clone nobody fetched. A dependency configuration's Build()
already runs to completion for the same reason.
The promise and future the dependency threads used to build or wait for
a configuration are now one helper, OncePerPass, which both kinds use.
An exception thrown in BuildExternal used to escape its thread into
std::terminate; it is now a build error, as for a dependency.
With eight tests that share a local dependency, a test run fetched it
eight times before this and fetches it once now (SharedExternalFetch,
next commit).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Shaders record the files glslang opened (a depfile) and their settings
(source path, stage, entry point, defines, include directories) in the
build directory, and rebuild when an included file or a setting changes,
not only the shader's own source.
- An executable relinks when its link command changed, not only when an
object is newer: a link flag in project.cpp, or a dependency's library
that moved (an ExternalDependency built into a new cache directory). The
command is recorded beside the objects; its flags come from ordered sets,
so the order the dependency threads finish in can't change it.
- An external's newest artifact counts its libDirs and versioned .so names.
- Submodules are checked out at the revisions the fetched commit records,
from the URLs its .gitmodules gives, after every clone, switch, checkout
and pull, so a pinned commit builds against its own submodules, and one
moved by hand is put back; with no .gitmodules, git isn't asked.
- Each external clone's fetch and build hold a file lock: parallel builds
pulled into one clone at once ("cannot lock ref").
- mingw:
- -stdlib=libstdc++ always;
- the std module's source is found through libstdc++.modules.json in the
mingw toolchain's own library directories (the generic lib/ holds the
host's Linux one), asking the compiler once a run;
- a toolchain that isn't there is a build error saying where it looked,
not std::terminate;
- CRAFTER_MINGW_DIR names the prefix, the directory holding
x86_64-w64-mingw32/ and lib/gcc/;
- inside the Flatpak the host's /usr mingw is used.
- RunScript's exit-status check reads as the one range it is.
- Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules,
LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and
ToolchainResolve extended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RunScript caught std::exception only. Anything else a script threw went
on across the project.so boundary, and crafter-build ended in
std::terminate instead of reporting the script's failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project.cpp could describe how a project is built, tested and linted,
but not the chores around it: packaging, installing, publishing. Those
stayed shell lines in a README or in someone's history.
Configuration gains `scripts`, a plain list like `defines`. A Script is
a name and a C++ callback, as a lint rule is; there is no script
language and no config file. The callback gets a ScriptContext (the
project, the project file's directory, the arguments) and returns the
exit status. ctx.Run runs a shell command with the terminal attached
and returns its status. It is only stored when project.cpp loads, which
happens for every verb and for every project that depends on this one.
`crafter-build run <name> [args...]` runs one, a bare `run` lists them.
Only the scripts of the project crafter-build was pointed at are
runnable, never a dependency's.
Everything after the script's name is handed to the script untouched.
The two branches that do this sit ahead of every other branch of the
parse loop, --version and --help included. Below the verbs,
`run package clean` would delete build/, and any other word would land
in the project's args and so in the variant hash, moving the build to
another directory. Options before the name keep their meaning, so
project args go there.
The driver is its own partition, :Script. It refuses a project with no
scripts, a name that is empty or starts with '-' (it could never be
typed), a name declared twice (running either would be a guess) and an
unknown name, each with exit 1 and without running anything. A script
that throws exits 1. A return value outside 0..255 becomes 1: an exit
status is eight bits, and a raw wait status such as 256 would otherwise
wrap to success.
Two helpers in :Platform:
- RunCommandAttached runs a command with inherited stdio. Every helper
so far captured through popen, which is right for a compile step and
useless for a command that runs for half an hour or asks a question.
It reads the wait status through DecodeWaitStatus, which
RunCommandChecked and RunCommandWithTimeout now share.
- ShellQuote is exported. It existed twice, as ShellQuote in :External
and ShellQuoteSh in :Test, with the same behaviour; both are gone. A
script that puts a path into a command needs it too.
Not included: a string shorthand for one-command scripts (a one-line
lambda covers it) and a ctx.Build() helper (Build() is already
exported).
A new partition and new exported symbols: run ./build.sh.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every verb is a bare word that is only read as a verb while no
conflicting verb is active; after one, the same word is a glob. The
guard on `clean` checked the test and lint verbs and then itself, where
the format verb was meant. `crafter-build format clean`, which asks for
the rules matching "clean", therefore removed both trees and exited 0
without formatting anything.
The guard now names format, like the three beside it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
clang's driver ranks the host toolchain's own libc++ headers
(<install>/../include/c++/v1) above --sysroot. BuildStdPcm has forced
the sysroot's copy for the std module for that reason, but ordinary
compiles did not. A unit that only does `import std;` never noticed. A
unit that includes a standard header textually got the host's libc++,
configured for the host's libc: cross-building Crafter.Graphics for
aarch64 musl, Crafter.Asset's vendored gdeflate died on <chrono> with
"unknown rune table for this platform". Against a glibc sysroot the
same unit would compile against the wrong libc++ release and say
nothing.
Every non-wasm, non-mingw compile with a sysroot now gets
`-nostdinc++ -isystem <sysroot>/usr/include/c++/v1`. wasm has its own
sysroot handling and mingw is a libstdc++ target.
-nostdinc++ leaves -stdlib=libc++ unused on each compile, which clang
reports once per unit, so the command also carries
-Wno-unused-command-line-argument. The cost is that the driver's
unused-argument diagnostics are silent on sysroot builds, including for
a user's own misplaced flag (a -l or -Wl, option in compileFlags).
Not addressed: external cmake dependencies are still configured
without the target or the sysroot, and objects already built in an
existing sysroot build directory are not invalidated by the new flags.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Crafter.Graphics has to ship each of its compute shaders in two builds
of one source: one that indexes VK_EXT_descriptor_heap and one that
reads the same slots from a descriptor set, for drivers without the
extension. A Shader was a path, an entry point and a stage, and its
output name was derived from the path alone, so one source could only
be compiled once.
Shader gains `defines`, spelled like the compiler's -D ("NAME" is
defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a
preamble, and `variant`, which names the output: "ui.comp.glsl" with
variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one
place that name is formed; Check, Compile and the forwarding of a
dependency's shaders into the consumer's bin dir all use it.
Two fields make one mistake easy: defines set, variant forgotten. Both
entries then compiled in parallel into the same file, the build
reported success with whichever finished last, and Check() called both
up to date from then on. The same has always been true of a.vert next
to a.frag. Build() now refuses a configuration in which two entries
share an output name and says which.
A define without a name, or one spanning lines, and a variant with a
path separator are rejected by name. Left to glslang the first two
surface as "-1:<line>" with no file, and the last as a failed open
after the whole compile.
Not addressed: Check() still compares only the source's mtime, so a
change to the defines, or to an included file, does not recompile
(#35).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The stamp that decides whether a compiled project file can be reused
was meant to cover the crafter-build version, the toolchain key, the
module-source stamp and the compile command. Only the version went in.
The four parts were joined by a format string with "\0" between the
fields, and a format string is read as a C string: it ended at the
first NUL, so std::format saw "{}" and dropped the other arguments
without a diagnostic.
A project.so is loaded into the crafter-build process and fills in
Configuration objects by their layout. With the stamp blind to the
declarations, one built against other module sources was loaded as
current whenever the version string was unchanged, which is every
working-tree build (`git describe --dirty` does not follow the tree's
contents) and every local Flatpak build. Growing a class that project
files construct then corrupts the heap: a project.so built while Shader
was 56 bytes, loaded after it had become 104, aborted with "free():
invalid size" as soon as it declared a shader. CI builds were spared
only because each carries its own version.
The parts are now appended to a std::string with NUL separators, the
way the other stamps in this file are built. Every existing project.so
carries a stamp of the version alone, so each is recompiled once.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A cross build against a sysroot whose libc++ differs from the
toolchain's failed while precompiling the std module, with a page of
"no member named 'runtime_format' in namespace 'std'". The module source
is a list of using-declarations for what one libc++ release declares, so
it only compiles against that release's headers. BuildStdPcm wants the
sysroot's own <sysroot>/usr/share/libc++/v1/std.cppm, but Alpine
packages the headers without it, and the fallback was the toolchain's
copy whatever its release. That held while the compiler came from the
same distro as the sysroot; with the Flatpak's libc++ 22.1.8 and Alpine
edge on 23.1.2 it no longer does.
The release is now read from _LIBCPP_VERSION in the sysroot's and the
toolchain's <__config>, and SysrootStdModule picks in this order: the
sysroot's own source, the toolchain's when both are the same major
release (a release branch only takes fixes), the copy crafter-build
ships for that release under share/crafter-build/libc++-modules/<major>.
When none fits the build stops with one line naming both releases and
the directory the matching source belongs in, before clang is run. If
either release cannot be read the toolchain's source is tried as
before, since there is nothing to compare.
Matching sources live in crafter-build's own share directory rather
than the LLVM prefix: they are data this tool ships for sysroots, not
part of any one compiler install, and "sources ship" already describes
that directory. `--version` prints the toolchain's libc++ release and
the releases sources are shipped for, the first thing to look at when
such a build fails.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
std.pcm was rebuilt when it was older than std.cppm, and project.so when
it was older than the crafter-build executable. Both inputs are installed
files, and an OSTree deployment (the Flatpak) gives every installed file
mtime 0: std.cppm and the executable always looked older than the cache,
so a clang bump would never rebuild std.pcm and a crafter-build update
would never recompile project.so. clang crashes on a PCM from another
compiler instead of reporting it, so the failure would have been a
backtrace on the user's first build after an update.
Each artifact now has a one-line stamp beside it. std.pcm's covers the
precompile command and the module source's bytes, which also catches a
libc++ update under the same clang. project.so's covers the crafter-build
version, the toolchain key, the module source stamp and the compile
command; project.cpp's and the pre-dependency artifacts' mtimes remain
inputs, since those live next to it with real timestamps. The module
source stamp stands in for the executable: project.so resolves Crafter::
symbols from the running binary, so only declarations matter, and an
implementation-only rebuild of crafter-build no longer recompiles it.
The std.pcm stamp folds into the module cache stamp, so a rebuilt
std.pcm rebuilds the PCMs that import it.
The cache directory carries the toolchain key
(<target>-<march>-<clang version>-<hash>), so a compiler bump starts
cold instead of handing the new clang PCMs it will crash on. LoadProject
and Build() now derive the directory from the same StdPcmDir. Old
<target>-<march> directories are left behind; `clean` never touched the
cache and still does not.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Inside the Flatpak the bundled toolchain compiled every bootstrap object,
then glslang's configure failed its test link with `unable to find
library -lc++`. clang searches its per-target runtime directory
(lib/<triple>) and the system library paths for libc++, not
<prefix>/lib, so with the classic layout a toolchain outside /usr needs
the search path spelled out, just as its loader needs the rpath.
LibcxxLinkFlags and build.sh now emit both, the same pair an explicit
CRAFTER_BUILD_LIBCXX_DIR already got.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every clang++, clang and llvm-ar the tool spawned was a bare name on
PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI
sysroot /usr/share/wasi-sysroot and the mingw headers
/usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A
toolchain installed anywhere else, which is what the Flatpak ships
under /app, had no way in.
Crafter.Build:Toolchain resolves all of it once per process:
CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build
(<prefix>/bin, found through the same walk GetCrafterBuildHome does),
else PATH. What sits beside the compiler is used when present: its
libclang.so, its clang-c headers, its std module source, its WASI
sysroot. An explicit prefix without a clang++ is an error rather than a
fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already
exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other
than the compiler's own, and CRAFTER_MINGW_DIR now does what its help
line always claimed: it is passed as --sysroot for the mingw target.
Binaries linked by a toolchain outside /usr get an rpath to its lib
dir, since its libc++.so.1 is not on the loader's default path.
HostTarget() is the resolved clang's -print-target-triple. A clang built
without a vendor reports x86_64-unknown-linux-gnu where Arch's reports
x86_64-pc-linux-gnu, so target equality that decides whether a test runs
locally and which targets a bare `test` sweeps goes through SameTarget,
which ignores the vendor field. The strings themselves are never
rewritten: clang rejects a PCM whose triple differs from the importing
unit's. The examples and README stop hardcoding the host triple; the
default is already the host.
`crafter-build --version` prints the version baked in at build time
(CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the
compiler and the host triple, and is answered before any argument is
forwarded to project.cpp. The version is a define on the library, so
each version is its own library variant and relinks the executable and
tests once per commit.
build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR
and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already
there, and writes a fresh archive instead of appending to one that may
hold members from an older layout.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
crafter-build is moving to a single Flatpak distribution, and there is
no Windows host in that picture: no build.cmd, no MSVC or mingw host
build, no DLL plus launcher. Delete every host-Windows path instead of
leaving it to rot unverified: the cmd.exe command runner and quoting,
the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants
with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port
probe, the LoadLibrary libclang loader, the console-width and isatty
shims, and the preprocessor guards around the Linux bodies.
The x86_64-w64-mingw32 cross-compile target stays. It is target logic
(output names, -femulated-tls, the libstdc++ link extras, copying a
dependency DLL next to a mingw exe, wine as the runner) and did not
depend on any of the removed code. The msvc target goes with the host
that was the only thing able to use it.
Three target-path warts fixed on the way: ExpectedOutput named a mingw
static library <name>.lib while the archive step wrote lib<name>.a, so
those libraries relinked on every run; the import lib is no longer
copied next to a mingw exe, since the dependency's own bin dir is
already on the link line; and llvm-ar now writes every archive, not
only the LTO ones, so Debug and Release archives are made the same way.
CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary.
Nothing downstream used them, so they are removed rather than kept as
an empty macro nobody can verify.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Crafter.LintRules declares a test that links crafter-build, fetched
with GitProject. crafter-build's own project.cpp loads Crafter.LintRules
as a `#pragma crafter dependency`, so the chain was: crafter-build loads
LintRules, LintRules' test wiring loads crafter-build's project.cpp from
the external cache, which loads LintRules again, and the cycle guard
threw. Every command that loads project.cpp in this repository has
failed since that test was added, including the last CI run on master.
A project file had no way to tell whether it was the one the user
pointed crafter-build at or a dependency being loaded for somebody
else, so it could not skip wiring that only its own build needs.
IsTopLevelProject() answers that from the loading-frame stack
ProjectDependency() already reads: one frame is the top-level project
file, a second is only ever pushed for a dependency. Outside any project
file it is true as well, which is what a test harness constructing
Configurations expects.
The ProjectDependency test records the answer from both the app and the
greeter project files and checks each.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Brings in `#pragma crafter dependency`, the switch to Crafter.LintRules
for the house style, and the contributor guide in AGENTS.md.
The branch predates the musl host work on master, and both rewrote the
command that compiles project.so, so the merge reconciles them by hand:
- LoadProject compiles project.so for hostConfig.target (HostTarget(),
from master) with the pre-dependency module path and archives from
the branch. -fuse-ld=lld appears once, with both reasons in the
comment.
- ForceHostTooling added -fPIC only for the literal x86_64-pc-linux-gnu
triple. hostConfig now comes from HostTarget(), so on Alpine the host
is x86_64-alpine-linux-musl and the dependency archive would not have
been position-independent. It now matches any *-linux-gnu or
*-linux-musl host, as project.cpp and the cmake flags do since the
musl change, and its comment no longer argues against HostTarget().
- The ProjectDependency test compared the dependency's target against
the glibc triple, which the musl CI job would fail; it compares
against HostTarget().
- README keeps master's paragraph on the bootstrap triple. AGENTS.md
describes the test block as `if (linuxTarget)`, which is what
project.cpp has now.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project.cpp could only import std and Crafter.Build, so anything shared
between build descriptions had to be a header every consumer copies. Now
#pragma crafter dependency "<source>" [branch "<b>" | commit "<sha>"] [project "<file>"]
names a sibling crafter-build project that project.cpp itself imports a
module from. LoadProject scans for the directive, resolves it through
GitProject (a URL) or LocalProject (a path relative to the project file),
forces the result into host-tooling mode (the host triple LoadProject's
own cache is keyed on, native march, no debug, -fPIC so the archive links
into a shared object), builds it, and puts its BMIs and archive on the
project.so command line. The dependency's own project.cpp goes through
the same loader, so its directives resolve recursively; a cycle between
project files is an error instead of unbounded recursion.
The dependency is built on every load, fresh or not: clang crashes rather
than erroring when it imports a BMI whose own imported PCM was rebuilt
underneath it, and the module depfile — which Module::Check already reads —
is what notices the host PCM cache was regenerated. Its artifact mtime is
an input to project.so's staleness check, so editing the dependency
recompiles project.cpp without touching it. The Linux link gains
-fuse-ld=lld because release archives hold ThinLTO bitcode.
ProjectDependency(name) hands the same Configuration* back from inside
CrafterBuildProject, by configuration name or exported module name, so a
test that links the module needn't spell the source twice. It reads a
thread_local frame stack that an RAII guard keeps for the whole of
LoadProject, which also provides the cycle check.
ExpectedOutput() is the link gate's artifact-path lambda hoisted into the
:Clang partition, so the staleness check and the -r runner share the one
rule. Windows branches are wired identically but unverified.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Linux host code paths were gated on the x86_64_pc_linux_gnu target
define alone, and LoadProject hardcoded --target=x86_64-pc-linux-gnu when
compiling project.cpp and the module PCMs. Both break on musl hosts: the
glibc release launcher cannot run there at all (no glibc loader; gcompat
lacks libgcc_s and the __isoc23_* symbols), and a musl-built launcher
would still compile project.cpp for the wrong libc.
- The host gates test clang's __linux__ instead of one triple's define:
they mean "Linux host", whatever the libc (or, later, the CPU).
- LoadProject derives the host triple from HostTarget() (clang's
-print-target-triple), which is also more correct on glibc distros
whose triple isn't x86_64-pc-linux-gnu.
- project.cpp / the cmake libc++ flags treat any *-linux-gnu or
*-linux-musl target as Linux.
- build.sh targets the host toolchain's triple (CRAFTER_BUILD_TARGET
overrides) and derives the per-target define from it. On Arch this is
byte-for-byte the previous behaviour.
- CI: a release-musl job bootstraps on alpine:edge, runs the tests, and
attaches crafter-build-linux-x86_64-musl-v2.tar.gz to the rolling
latest release (via the API, so the glibc assets survive). Dynamic
against musl libc++: the launcher dlopens project.so, so it cannot be
fully static.
First consumer: imsd's package CI, which cross-compiles for aarch64 from
an Alpine container.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RunCommand merged stderr into stdout, dropped pclose's status and returned
the text; every compile, link and archive site then read "printed something"
as "failed". A warning is printing something, so a translation unit that
warned failed the build — but only on the run that actually recompiled it,
since warnings aren't re-emitted for an object that's already up to date.
The same unchanged source therefore passed or failed depending on the state
of the build tree: flaky-looking tests locally, and a cold CI checkout
surfacing every latent warning in a project at once as unrelated failures.
RunCommand is gone, replaced by RunBuildCommand: it goes through
RunCommandChecked, returns "" when the command exited 0 (so every caller's
`if (!result.empty())` error path is unchanged) and hands any warnings to
the new Progress::Diagnostic instead of to the error path. That also closes
the quiet half of the bug — a compiler killed by the OOM killer prints
nothing, so it used to read as success and leave the build carrying on with
a missing object; it now reports the signal that killed it.
Warnings are now shown rather than swallowed, which they weren't in either
direction before: invisible on an incremental build, fatal on a cold one.
Failing on them stays a project's choice, via -Werror in compileFlags.
The C compile loop sets no repack flag, and the "is any object newer than
the archive" sweep only walked interfaces and implementations. So editing
a .c recompiled its object and then linked nothing: the archive and the
executable kept the previous one. Surfaced by the header-change test,
where a rebuilt counter.c produced a binary still printing the old value.
The staleness check compared an artifact against its own source and its
module imports. Headers were in neither set: the scanner reads `import`
lines, and a .cppm or .cpp keeps its mtime when a header it includes is
edited — so the build reported nothing to do and left objects compiled
against the previous contents. Same silent mixed-layout binary as issue
27, reached through #include instead of import.
Ask the compiler what it actually opened. Every C++ and C compile now
passes -MD -MF <artifact>.d, and Check reads that depfile back through
NewestPrerequisite, comparing every prerequisite's mtime against the
artifact. A missing depfile (an object from a crafter-build that wrote
none) or a prerequisite that no longer exists reads as "rebuild": neither
is evidence of freshness.
The parser unescapes make syntax rather than splitting on whitespace,
since clang wraps depfiles onto continuation lines and escapes spaces in
filenames.
The first cut scanned the initialiser's tokens and required every one to be a
literal or an operator. That is exactly the kind of approximation this work has
been removing, and it was wrong in both directions:
const int A = sizeof(Big); missed — `sizeof` is a keyword
const int B = Base + 1; missed — `Base` is an identifier
const auto C = 5_notConstexpr; would have been reported, wrongly
clang_Cursor_Evaluate answers the question directly. Evaluating a variable
declaration evaluates its initialiser, so anything that folds is recognised and
a call result still is not. Costs nothing measurable — lint stays at ~15s.
Found one real case the token version could not see: an EShMessages fold over
three glslang enum constants in Crafter.Build-Shader.cpp. Promoting it to
constexpr then made `naming` ask for constant naming, since a constexpr
variable is a compile-time constant — so it is `Messages` now. Two rules
agreeing on the same declaration is the intended behaviour.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two rules the AST makes possible, plus the mutation analysis behind them.
const-local reports a local that is never written. It is restricted to SCALARS
— integers, bools, enums, floating types — and that restriction is what makes
the answer exact rather than a guess: a scalar has no member functions, so the
only ways to write one are assignment, ++/--, having its address taken, or
binding to a non-const reference. All four are now tracked in the walk:
- assignment and compound assignment visit their LEFT operand in a write
context, the right one normally;
- ++/-- and & write their operand;
- a call argument is checked against the callee's parameter type, so passing
to `const int&` or by value is a read while `int&` is a write;
- initialising a non-const reference writes what it binds to.
For a class type a non-const method call could mutate it, and deciding that is
the whole-program analysis clang-tidy does, so those are simply out of scope
rather than guessed at.
constexpr-constant promotes a const constant whose initialiser is made only of
literals and operators, so `const int A = 1 << 4;` qualifies and
`const int B = Compute();` does not.
On this repository const-local found 103 candidates, which was too many to be
useful, and the reason was informative: most were range-for bindings and
pointer locals. `for (T* const x : …)` and `T* const p` are not spellings
anybody writes, and the useful constness for a pointer is on the pointee, which
this rule cannot advise on. Excluding both leaves 36, all plain bool or enum
locals worth fixing — isWasm, isPe, exists, writes, isC and so on. Those 36 are
fixed in this commit; the compiler verified every one.
Both rules are report-only. The analysis is exact, but adding const is a
judgement about intent as much as mechanics, and a wrong suggestion should cost
a glance rather than a build. const-local also deliberately does not become a
transform: inserting `const` before a shared type would apply it to every
declarator in a multi-declarator statement, including any that IS written.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tokens are not enough for this one, which is worth stating because it is the
opposite of the enum-class case. Splitting a multi-declarator statement by
copying the shared type prefix is wrong in C++:
int* a, b; -> int* a; int* b; // b was int, not int*
Only per-declarator types get it right, and clang has already resolved them —
`int *` for a, plain `int` for b. A token-based splitter cannot know.
The regex it replaces bailed on `*`, `&`, `<>`, parens and quotes, so pointers,
templates and call initialisers were all left alone. All three split now, and
the fixture proves the mixed pointer case above comes out correctly.
Groups are found structurally rather than by matching a line shape: the first
declarator's extent starts at the shared type, so begin < nameOffset, while a
continuation declarator's starts at its own name, so begin == nameOffset. That
signal comes from the AST itself. nameOffset is now on LintDecl, which is also
what lets the replacement reuse each declarator's original text verbatim
instead of reconstructing it.
Replacing a byte range rather than rewriting whole lines means a comment after
the ';' is outside the edit and survives — the line-based version refused to
touch any line carrying a comment. A comment INSIDE the statement still bails,
since the rewrite would swallow it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Linting a march whose PCMs are not built produced 102 findings — 34 files times
three AST rules — for a single fact: the project has not been built for that
configuration. The one actionable sentence was buried.
Now recorded per file and reported once, naming the rules that could not run,
how many files were affected, one example reason, and what to do about it. Still
one error, so the run still fails; --no-ast remains the way to proceed without
building.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rewrite itself stays token-shaped — it edits type SPELLINGS, which an AST
discards — but what it must not touch now comes from the AST.
The old exemption was per LINE: `int main`, `argc`, `argv`, `extern "`. Being a
transform, a missed exemption here does not over-report, it emits code that no
longer matches the API being called, so this is the rule where guessing from
substrings mattered most. And being per-line, it also disabled the rule for
anything sharing a line with one of those words.
Now a declaration with C language linkage, or one whose initialiser binds to an
entity declared outside the project, contributes a protected byte range and
keeps its spelling. That answers the case directly: a function declared in
somebody else's header taking `unsigned int` keeps `unsigned int`, and a local
initialised from strtoul keeps `unsigned long`, because of where those are
declared rather than because of what the line says.
main is protected from the start of its declaration to the opening brace of its
body, not for its whole extent. Its signature is fixed by the language; its body
is ordinary code. Three findings on this repository came out of that
distinction, all correct:
for (int i = 1; i < argc; ++i) -> for (std::int32_t i = 1; ...)
skipped before only because `argc` appeared on the line, plus Crafter::Run's own
`int argc` and return type, which are ours rather than the language's.
All three AST rules now share one interop test instead of carrying a denylist
each, and it is the same test: whose header dictates this spelling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule was 125 lines: four std::regex, a hand-rolled {/} scope stack, a
cumulative paren-depth counter so a wrapped parameter list would not look like
a declaration, a 40-entry keyword denylist, and a "function-shaped line" guess
whose own comment conceded it was heuristic. Storage class came from
lineStr.contains("static "). It is now ~55 lines that ask clang what kind of
declaration each thing is and what encloses it.
The three regressions the old version carried special cases for — a call with
an inline lambda argument, a bare statement call, a one-liner method — need no
handling at all, because a call is not a declaration. Their tests pass
unchanged.
On this repository the exact version found 42 violations the heuristic had
never been able to see, all real:
- 37 members of the libclang function-pointer table added two commits ago
were PascalCase. The old varDecl regex could not match a declaration whose
type is decltype(&f), so they were silently skipped. Renamed to camelCase,
which mirrors clang_createIndex -> createIndex more closely anyway.
- Crafter.Build-Shader.cpp had a snake_case local, file_name_list, invisible
to the heuristic for the same reason (it declares a const char* array).
- Four extern "C" declarations of libc functions in tests were reported as
badly-named functions. Those are named by the C library, so C language
linkage is now an exemption — the same principled test no-char-pointer
uses, rather than another denylist entry.
New tests cover what the line-based version structurally could not reach: a
signature wrapped over several lines, `static` on its own line above the
declaration it applies to, and a member versus a local inside a method.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule was `\bchar\s*\*` over the text minus a substring denylist — argv,
getenv, setenv, dlerror, c_str, .data(, reinterpret_cast, extern ". Every entry
was a patch for one interop site, the list could only grow as libraries
arrived, and each entry disabled the rule for the whole LINE it appeared on.
It now walks declarations and asks the question the denylist was approximating:
whose header dictates this spelling? A declaration with C language linkage, or
one whose initialiser binds to an entity declared outside the project root, is
somebody else's API and keeps its spelling. getenv, c_str and friends are
exempt because of where they are declared, not because they are named here, so
a new external library needs no new entry.
Two bugs found while testing this, both of which had made the rule silently
pass over the entire repository:
clang_getCursorLanguage cannot be used to detect extern "C". Its default answer
is CXLanguage_C for a plain function, variable or parameter even in a C++
translation unit, so isExternC was true almost everywhere and exempted
everything. Replaced by tracking CXCursor_LinkageSpec depth during the walk,
reading the extent text to tell extern "C" from extern "C++".
Attributing any foreign reference in a subtree to the enclosing declaration was
too broad: a function that merely touched libc++ somewhere in its body would
exempt its own signature. Narrowed to initialiser contexts — a variable, field
or parameter — which is where a binding to a foreign API actually occurs.
Also: functions now carry their RESULT type rather than the whole function
type, since the parameters arrive as their own declarations and would otherwise
be reported twice. main's parameters are exempt structurally, its signature
being fixed by the language rather than chosen here.
Two sites keep an explicit lint-disable, both Crafter::Run taking main's argv
verbatim. That is two visible, reasoned suppressions in place of a denylist
that silently disabled the rule for every line mentioning one of eight tokens.
ExternalCloneDir and ExternalIncludeFlags are now exposed from :External, so a
source that includes an external dependency's headers can be parsed without
running a build to discover where they are. BuildExternal derives its own
working directory through the same function, so the two cannot drift.
Crafter.Build-Shader.cpp needed this to parse at all.
A file with no compile command — project.cpp, which LoadProject builds with its
own flags — is not a translation unit of the build graph, so AST rules skip it
the way a rule self-filters by extension. That is distinct from a file that
should have parsed and did not, which stays an error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds LintContext::Decls() — clang's view of the declarations written in the
file — plus AddAstLintRule to register a rule that reads it. No rule uses it
yet; the three that will are migrated separately.
The declarations come back as a flat vector with parent indices rather than an
opaque cursor handle: no lifetimes cross the project-DLL boundary, no callback
hops back into project.so per node, and "is this at namespace scope or inside a
function?" becomes an index lookup instead of a hand-rolled brace stack.
Three things had to be solved for this to work at all on this codebase.
libclang cannot see through `export`. A C++20 export declaration has no
CXCursorKind, so `export namespace Crafter { … }` arrives as a childless
CXCursor_UnexposedDecl and clang_visitChildren does not descend. Five of the
eleven interfaces here are written that way — 677 lines, including
Configuration and LintContext, yielding zero usable cursors. A plain
`namespace` IS descended into, so the fix is to blank the keyword before
parsing, byte-length preserving so every line and column still lands on the
original file. `export module` is left alone or the unit stops being a module
interface. Verified end-to-end against a fixture whose asserted line numbers
match the unblanked file.
PCMs are flag-locked, so each file has to parse with the flags that built it.
CollectConfigSources now records which Configuration owns each source instead
of flattening to a set, because three regimes coexist: the library, each test
(carrying its own target, defines and -march), and project.cpp, which Build
never touches and which therefore gets no command at all.
libclang resolves its builtin headers relative to its own install path, which
need not match the clang++ that wrote the PCMs. When it doesn't, every parse
dies on "'stddef.h' file not found", so -resource-dir is passed explicitly
from `clang++ -print-resource-dir`.
Two flags on each declaration replace what would otherwise become more
substring denylists: isExternC, and isForeignApi for a declaration that binds
to an entity declared outside the project root — resolved through
clang_getCursorReferenced and the same inside-the-root test the dependency
walk already uses. Parameters and fields inherit it, so an exemption covers a
whole signature rather than the one node that named the foreign entity.
Failure is never silent. A fatal diagnostic leaves a fragment that is
indistinguishable from a file declaring nothing, so it is reported as an error
instead: the rule is skipped, a finding explains why, and summary.errors makes
the run fail. --no-ast opts out deliberately and exits normally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The clang invocation was assembled inline across four regions of Build,
interleaved with the dependency-graph walk, so nothing else could ask "what
flags does this Configuration compile with". The linter's AST layer needs
exactly that, and it cannot approximate it: a precompiled module is rejected
outright by a translation unit whose target features differ from the one that
wrote it. Dropping just -march=native produces hundreds of "compiled with the
target feature '+avx512bw' but the current translation unit is not" errors and
no usable parse, so reconstructed flags fail hard rather than degrade.
GetCompileCommand is the config-pure part: target, arch, standard,
configuration defines, module search paths, includes, user compileFlags,
optimisation and LTO. Build appends only what depends on work having happened
— dependency public flags and external dependency flags. The sub-strings it
also needs on their own (includes, defines, user flags, LTO) come back as
struct members, so the .c compile path is unchanged.
Verified by probing `command` at the equivalent point before and after and
diffing: byte-identical across all 23 configurations exercised by a full build
plus the test suite.
Two incidental simplifications fell out. pcmDir was recomputing what
Configuration::PcmDir() already returns, and cmakeBuildType is now a
one-liner. GetCompileCommand is also most of what a compile_commands.json
would need, which this repo lacks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the linter's hand-rolled character scanning with clang's lexer.
StripComments could not see raw string literals, which was documented as a
v1 limitation but was worse than that: an odd number of quotes inside a raw
string desynchronised it for the rest of the file, hiding real code and
leaving the contents of later literals standing as if they were code. There
are 33 raw strings across 6 files here, including the two largest.
The reflow guards had the same class of bug from the other direction. They
were substring probes, so Line(n).contains("//") fired on // inside a string
literal and contains("R\"") fired on the characters R" inside a literal while
missing raw strings opened on an earlier line. Both misfire on this repo:
"MARKER" ends in R". They are now LineHasComment() and
LineHasMultiLineToken(), and two wrapped call sites in tests/Lint that had
been silently unjoinable join as a result.
libclang is dlopen'd rather than linked, so the mingw and MSVC cross-builds
keep linking and the release tarballs stay self-contained. Tokens need no
PCMs, no build flags and no prior build, and they cover preprocessor branches
that are inactive for the host — which is why the layout rules belong on
tokens rather than on an AST that would only ever see one platform's slice.
No rule behaviour was intended to change beyond the two guard fixes; lint
over this repo produced byte-identical output across the CommentStripped
swap.
A primary module interface unit — `export module Widget;`, no partitions —
recorded nothing about what it imported. GetInterfacesAndImplementations
registered the Module and then erased the file from the scan list, so the
import pass only ever saw partitions, and Module had no vectors to hold an
edge anyway.
Two consequences, both reported as issue #26:
Module::Check consulted only its own .cppm and its partitions. A data
member added to an imported module left Widget.pcm, Widget.o and every
consumer object untouched while the imported library rebuilt and both
binaries relinked — one executable holding two class layouts, no
diagnostic, and a crash somewhere unrelated. Wiping build/ was the only
cure, so `crafter-build test` could not be trusted straight after an
interface edit.
Module::Compile waited on nothing. Two modules in one Configuration
compile on concurrent threads, so a primary interface importing a
sibling was a coin flip between working and "module 'Base' not found".
Partitions never had either problem — they carry the same three vectors and
Check/Compile honour them — which is why the gap only surfaced on a module
whose interface is one flat unit.
Module now carries moduleDependencies, externalModuleDependencies and
pendingImports with the same meanings as on ModulePartition; primary units
stay in the scan list so their imports land there; Check sees through them;
Compile orders itself behind a local sibling; and ResolvePendingImports
sweeps them so an edge survives dependencies being wired up afterwards.
Build() now Checks every interface before spawning any compile thread — the
`compiled` flag a waiter blocks on is raised either by a Compile that runs
or by the Check that decides none is needed, so a Check still pending while
another module's thread waits would have hung the build.
Resolves#26
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three follow-ons to the stale-build report, all cases of an identity not
capturing something that changes the output.
The host PCM cache under <cache>/crafter.build/<target>-<march>/ is shared by
every crafter-build on the machine, and freshness was a per-file mtime
comparison. That cannot tell "this PCM is newer than my source" from "this PCM
was built from different sources that happen to be newer", so a package install
and a working checkout — or two checkouts of different versions — silently
compiled their project.cpp against each other's declarations. Invalidation now
keys on a stamp over the bytes of every module source, which also covers the
case one file's mtime never could: the cached PCMs import each other, so a
change to :Interface invalidates :Clang's PCM with Crafter.Build-Clang.cppm
untouched.
Project args ApplyStandardArgs does not itself interpret are now folded into
VariantId. Such a flag typically decides what gets compiled or bundled — the
report's example is --no-webgpu dropping entries from cfg.files — and without it
both settings shared one bin dir and interleaved their outputs there, leaving a
bundle matching neither. Sorted and deduplicated so flag order doesn't split the
cache, and inherited by test Configurations.
`crafter-build clean` removes the project's bin/ and build/ trees. It
deliberately does not load project.cpp: cleaning is most often reached when
something is already wrong, and a clean that first needs the project to compile
is useless exactly then.
Adding a data member to a class in a module interface did not rebuild every
object compiled against the old layout. The build succeeded with no error or
warning and the resulting binary mixed both layouts, surfacing later as a
SIGSEGV in a destructor.
GetInterfacesAndImplementations scans a TU's `import X;` statements when the
source is declared. An import that matches neither a module in the
Configuration nor one reachable through `dependencies` was dropped on the
floor, leaving that TU with no staleness edge to the interface it consumes.
`dependencies` is frequently assigned *after* the scan — AddTest does exactly
that, resolving tests/<name>/main.cpp and only then returning a builder whose
.Dependencies() supplies the library — so consumers of a dependency's modules
routinely carried no edge at all. A layout change then rebuilt the library,
relinked the consumer, and kept the consumer's object as it was.
Unresolved names are now remembered on the partition/implementation as
pendingImports, and Configuration::ResolvePendingImports retries them against
the dependency DAG as it stands. Build() calls it immediately before comparing
mtimes, which closes the window for every caller rather than only the ones that
declare in the right order; TestBuilder::Dependencies also calls it so the
Configuration is coherent for anyone inspecting it before the build.
Resolves#27
The transforms guard themselves against comments and raw strings before
joining or rewriting a line, because pulling text up past a `//` buries it
and reflowing a multi-line literal changes the string. Those guards were
substring probes over the raw line, so they answered the wrong question:
Line(n).contains("//") fires on // inside a string literal
Line(n).contains("R\"") fires on the characters R" inside a literal, and
MISSES a raw string opened on an earlier line
Both misfire on this repo's own sources. "MARKER" ends in R", and any string
mentioning a lint-disable directive contains //. Two wrapped call sites in
tests/Lint were being left unjoined for exactly these reasons; they join now,
and the results are in this commit.
Replaced by LineHasComment() (added with the token layer) and a new
LineHasMultiLineToken(), which reports whether any token actually covering
that line spans a line boundary — a raw string or a block comment. Backed by
a per-line bitmap derived from the token cache and invalidated with it.
The guards themselves stay: joining across a real comment or a real
multi-line literal is still unsafe, and there are tests for both. What
changes is that they now fire on comments and literals rather than on the
characters that spell them.
format-concat gets narrower as a result. It used to refuse any line
containing R" and ask for a manual fix; now only a literal that genuinely
spans lines does that, because a single-line raw string reduces to R"…" in
the stripped view, fails the plain-literal test, and travels through as an
argument with its spelling intact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
StripComments hand-scanned five states over raw characters and could not see
a raw string literal, which was documented as a v1 limitation. It was worse
than "does not recognise them": an odd number of quotes inside a raw string
desynchronised the scanner for the rest of the file. Simulated on the new
fixture, the old output was
auto banner = R" "hi)" \n \n ... "MARKER text"
— the code after the raw string blanked away, and the *contents* of a later
string literal left standing as if it were code. Every rule reading
CommentStripped() saw that. There are 33 raw strings across 6 files here,
including the two largest.
The replacement projects the token stream onto a copy of the buffer, blanking
comment tokens whole and the bodies of string/character literals. Editing a
copy in place makes the length-preserving contract structural rather than
something each branch has to remember, and the token extents make raw
strings, escapes, encoding prefixes and '"' all fall out for free. A raw
string reduces to R"…" so rules that bracket a literal by counting quotes
keep working. Digit separators (1'000) are excluded by requiring the text
before the quote to be an encoding prefix.
The public contract is unchanged, so no rule needed editing, and `lint` over
this repo produces byte-identical output to the previous implementation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds LintContext::Tokens() and friends, backed by clang_tokenize, as the
substrate the rules will move onto. Nothing consumes it yet.
libclang is dlopen'd rather than linked: -lclang would break the mingw and
MSVC cross-builds at link time and would put a libclang.so.NN runtime
dependency into the otherwise self-contained release tarballs. The clang-c
header is used for its declarations only, and the function-pointer table is
typed with decltype so the signatures cannot drift from the real API.
Three properties this buys that the hand-rolled scanners could not have:
- a raw string literal or block comment is ONE token, so the documented
"raw string literals are not recognized" limitation goes away;
- `//` inside a literal is not a comment, so LineHasComment() replaces the
Line(n).contains("//") probes that false-positive on it;
- tokens cover preprocessor branches that are inactive for the host, since
clang_tokenize lexes rather than evaluates #if. Token rules therefore
keep seeing every platform's code, which an AST could not offer.
The parse backing the tokenizer is expected to fail on module units — no
PCMs, no build flags — and that is fine, because lexing has no semantic
prerequisites. Verified in the new tests.
LintSummary::Clean() now counts `errors`. It previously ignored them, so an
infrastructure failure that produced no findings reported clean and exited
0; a missing libclang would have been exactly that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
