Each fixed finding gets a check that fails without its fix: inline code
and a list item's fence, a section removed from between two paragraphs,
a section before an unclosed fence, a CRLF file, no error when there is
nothing to write, a hard-linked and a read-only AGENTS.md, a dependency
whose project file sits in the consumer's own directory, a test's
dependency, and `test --list`. The read-only case needs an ordinary
user, since root writes through the permission bits, and is skipped
under root.
The merge cases go through a helper that counts a throw as a failure,
so one regression no longer ends the run before the cases after it. A
passing run removes its staged copy, whose builds leave a std PCM per
configuration in a tmpfs.
Reverting the inline-code rule, the own-notes exclusion, the skipping
of tests, the `test --list` guard or the paragraph-preserving removal
each fails its check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A review of the feature turned up the following, fixed here.
Writing. AGENTS.md was replaced through a temporary renamed over it,
which swaps in another file: a read-only AGENTS.md was rewritten
anyway, a hard link such as a CLAUDE.md beside it kept the old text,
ACLs and ownership were dropped, and without an fsync a power cut could
still leave it empty. The temporary was named after the PID, which
every Flatpak instance shares as PID 2. The file is now written in
place, from the first byte that differs, then truncated and synced,
under an flock on the file itself. A write that fails partway is undone
where the disk allows; with the section last, where it is appended,
only the section can be damaged. A read-only file is opened for writing
only when something would change, so it is refused then and silent
otherwise.
Reading. libc++'s filebuf reports a failed read as the end of the file,
so an I/O error came back as a short file and the section was written
over what was left. Reads go through read(2).
Code fences. ```make``` was taken for a fence, and a fence in a list
item was missed while its indented closer opened one, so markers a user
quoted in a code block could be taken for the section, or the real one
hidden and appended again. Backtick info strings, block quotes and list
items are now followed, and a block ends with its container.
Warnings with nothing to write. With no notes, nothing short of a
well-formed section is reported, so an AGENTS.md that ends in an open
code block, or a project set to None, no longer warns on every build.
An unclosed fence after a section no longer stops it being rewritten
in place.
Labels. The `from` line carried the git URL, credentials and all, or a
path relative to the consumer that changed with the machine, the cache
location and --local. It is now the repository's or the directory's
name. Notes are deduplicated by repository rather than by clone, so one
library pinned two ways is described once, as the nearer spec has it.
The walk. It no longer looks at the project's tests: a plain build
never did, and a test's dependency can point at the Configuration the
project file returned by value. It goes ring by ring over projects
rather than configurations, so what the project names itself comes
first even with the library/executable split. The project's own
AGENTS.consumer.md is excluded by path, also when a dependency's
project file sits beside the project's.
Removing a section dropped a blank line even from between two
paragraphs, joining them; only a section that ends the file takes it
now. A CRLF file gets a CRLF section.
A note was checked on its raw bytes and again, under other whitespace
rules, on the trimmed ones, so it could pass the first check and fail
the merge, which then dropped every note. It is normalised once and
checked on exactly what is merged.
`test --list` only lists, and no longer writes AGENTS.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sibling-project example dropped its `.args = depArgs`, which no
longer has to be written, and says what a sibling receives instead: the
args the project file was loaded with, or whatever `.args` is set to,
with `{}` for nothing. It also says that releases up to 2.2.0 forward
nothing without `.args`, which matters for a project.cpp that has to
load on both.
The pragma section says that a project file's own dependency is built
for the host and gets none of the project's args, the one place the
default does not apply.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crafter.LintRules now has AddLiteRules: the house rules without the ones
that only decide layout, whitespace and naming. This tree switches to it.
The full set's naming rule now wants camelCase statics and constants,
which this tree spells PascalCase throughout, so `lint` with it reported
49 findings on code that follows the documented convention; with the lite
set `lint` and `format --check` are clean.
AGENTS.md described the 17 rules as the enforced style and told
contributors to let `format` restyle their code. It now says which 8 rules
`lint` enforces, that `format` only rewrites string concatenation and
builtin integer types, and lists layout and naming as conventions kept by
hand.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crafter.Math and Crafter.Asset now get this project's args through the
GitProjectSpec default. Loading this file therefore needs a crafter-build
that exports ProjectArgs: after pulling, run ./build.sh before the
bootstrap binary loads it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ProjectArgs loads a chain app -> lib -> leaf of LocalProjects with --debug
and a project flag. lib, loaded without .args, has to see both, and so
does its own sibling, which takes the default one level further down.
Copies loaded with `.args = {}` and with an explicit list get nothing and
exactly that list. The GitProjectSpec default is checked on the spec
itself, since a GitProject would need a repository to clone. Without the
defaults, five of its checks fail.
ProjectDependency now also loads its consumer with a project flag: the
pragma dependency must still come back with no args. Without the explicit
`.args = {}` in the pragma path it gets the consumer's flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every project.cpp with a sibling dependency opened with the same line,
std::vector<std::string> depArgs(args.begin(), args.end());
and handed depArgs to each GitProject and LocalProject. CrafterBuildProject
receives a span of string_views and the specs wanted owned strings, and
without the copy the sibling got none of the project's args: it was built
for the host while the project cross-compiled, and in release while it
built --debug. GitProject then turned the copy straight back into
string_views for LoadProject.
Both specs now default their args to ProjectArgs(), the args LoadProject
was given for the project file whose CrafterBuildProject is running. It
reads the per-thread stack LoadProject already keeps for IsTopLevelProject
and ProjectDependency, whose frames now record the args. A default member
initializer rather than a new type or an optional: the member stays a
std::vector<std::string>, so every existing `.args = depArgs` keeps its
exact meaning (each of the ~50 calls across the Catcrafts projects passes
one), and `.args = {}` still forwards nothing. It is the shape of
Configuration::target = HostTarget(). A project.cpp that drops its .args
needs a crafter-build with this change: an older one compiles it and
forwards nothing.
ProjectArgs() is declared in :External, next to the defaults that call
it. :Platform, where it is defined, cannot be imported there: its
unexported `struct Configuration;` would make :External's exported one an
ill-formed redeclaration, which clang rejects.
The #pragma crafter dependency path now passes `.args = {}` explicitly.
Host tooling is built for the host whatever the consumer targets and was
loaded with no args; the default would otherwise hand it the consumer's
--target= and project flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What a library ships and where, what a consumer's AGENTS.md gets and
when it is written, the section's format, and what cfg.dependencyNotes
selects.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge on its own: the section is appended after one blank line,
merging again changes nothing, a moved section is rewritten where it
stands, and removing it gives the file its own bytes back. Markers
inside a fenced block, one quoting a fenced block of its own, are not
the section's. Unbalanced markers, a document that ends inside a code
block and notes that would break the section are refused, naming the
line.
End to end, through LoadProject on a staged fixture: app names lib
and has tool as host tooling; lib builds against base and has libtool
as its own. Transitive gives tool, lib and base in that order, and
neither app's own notes nor libtool's. A second update writes nothing,
mtime included. Direct drops base, edited notes replace the old ones
in the one section, and None restores the original bytes. A project
without an AGENTS.md does not get one, and a symlinked AGENTS.md is
written through the link, which stays a link. Through Run in process,
lint and run leave the file alone, while `test --list` and a build
that fails at the link both refresh it.
The fixture's AGENTS.md is checked in as AGENTS.md.in: an AGENTS.md in
this tree would read as instructions for the agents working on it.
Moving the CLI hook above the run and lint branches, or letting Direct
walk past the first loaded project, fails four of the checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An agent working on a crafter-build project reads its AGENTS.md, and
learns nothing there about the libraries the project builds against
unless someone copied that in by hand. This repository shows the cost:
the code style section of its AGENTS.md restates what Crafter.LintRules
enforces, and goes stale whenever the rules change.
A library now ships those notes itself, as AGENTS.consumer.md beside
its project.cpp; nothing in its project.cpp changes. When a consumer
has an AGENTS.md beside its own project.cpp, `crafter-build` and
`crafter-build test` keep a section in it, between two marker comments,
holding its dependencies' notes. The section is appended the first time
and rewritten in place after that. The file is written only when the
section changes, and an AGENTS.md is never created. lint, format, run
and clean leave it alone. The update runs after loading, which has just
fetched every dependency, and before compiling, so a build that fails
still has the notes.
cfg.dependencyNotes picks the dependencies. Transitive, the default,
follows `dependencies` all the way down from the project's own
configurations, its tests and its `#pragma crafter dependency` tooling;
Direct stops at the first loaded project on each path; None removes
the section. A dependency's own pragma tooling never counts: it is
about how that dependency is built, not how it is used, and being no
`dependencies` edge it is never reached. Nor are a dependency's tests:
nothing builds them for the consumer, and their configurations can
point at a local their project file has since moved out of.
The walk has to know which Configurations belong to another project,
and where that project lives, and cfg.path cannot say: only the
Configuration GitProject and LocalProject return is rebased, so a
dependency's internal configurations keep "./", which resolves to the
consumer. The loader records it instead, LoadedProjectOf beside the
project cache, and keeps what the root's pragmas resolved to past the
frame LoadProject pops, PragmaDependencies. Neither is exported.
The markers count only as whole lines outside fenced code blocks, so
documentation can quote them. Unbalanced markers and a document ending
inside a code block leave the file alone with a warning: guessing which
part of someone's file is generated would overwrite their text, and
missing the section would add another one on every build. A note that
would break the section when it is read back, through a marker line of
its own or a code block it never closes, is left out with a warning.
The file is replaced through a temporary renamed onto the symlink's
target, so a linked AGENTS.md stays a link and a write cut short leaves
the old file rather than part of the new one.
The code is a new partition, Crafter.Build:Agents, listed in build.sh,
CrafterBuildModules, project.cpp and the umbrella.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Say how often an unpinned ExternalDependency is pulled now that the
configurations of a build share it, and list the target in the clone's
cache key, which it is part of.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ConcurrentExternalFetch raced eight threads through GitFetch only.
BuildExternal takes the same lock on a line of its own, so that line
could go without any test noticing. Now that the configurations of one
pass share a single BuildExternal, its lock is what keeps two
crafter-builds, or two passes, out of each other's pulls.
The same threads now fetch through both, each into a clone of its own,
and both clones must end at the last round. With the lock in
BuildExternal removed, the test failed in two of three runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight tests that include a header from the same tracked branch of a
local repository go through RunTests at once, and again after the
branch has moved. GIT_TRACE logs every git process, and each fetch of
the repository, clone or pull, runs one `git upload-pack` in it, so the
trace counts the fetches: one per run rather than one per test, and the
second run still pulls and brings the clone to the new commit.
On the commit before the fix both runs fetch eight times. The tests
themselves pass there, because the lock from 653dd0b already keeps the
pulls apart. Going through RunTests, rather than Build() on threads of
the test's own, also holds RunTests to building its tests with one
depResults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`crafter-build test` builds a project's tests side by side, and a
project whose tests each copy its externalDependencies hands every one
of them the same unpinned dependency. Each test's Build() pulled the
shared clone. Before the clone had a lock, those pulls ran at the same
time and broke its .git/FETCH_HEAD, which failed every test of
Crafter.Graphics (issue #34). Since 653dd0b they wait for each other,
but every one is still a network round trip: 26 tests cost 26 pulls of
each dependency, one after another, and for a CMake dependency as many
configure checks and builds.
depResults already records one build pass, keyed by the PcmDir of every
dependency configuration built in it. An external dependency now goes
in the same map, keyed by the directory its build leaves
(ExternalBuildDir: the clone, or for CMake its build directory). The
first configuration that needs it fetches and builds it; the others
wait for that and take its outcome. Keying on the build directory
rather than the clone keeps a CMake declaration from taking the outcome
of a header-only declaration of the same clone, which only fetched it.
A fresh depResults is a fresh pass, so the next build still pulls, and
every caller of Build() gets this: RunTests, a plain build, and the
pragma dependencies LoadProject builds.
To make that possible BuildExternal is split. BuildExternal is now the
shared work, lock, fetch, configure and build, and returns the error.
ExternalOutputs computes what one declaration compiles and links with,
because two declarations of one clone can differ in includeDirs,
libDirs and libs; ExternalBuildResult loses its error field. The work
no longer stops between steps when the configuration that started it
is cancelled: others may be waiting on it, and a run cut short would
leave them a clone nobody fetched. A dependency configuration's Build()
already runs to completion for the same reason.
The promise and future the dependency threads used to build or wait for
a configuration are now one helper, OncePerPass, which both kinds use.
An exception thrown in BuildExternal used to escape its thread into
std::terminate; it is now a build error, as for a dependency.
With eight tests that share a local dependency, a test run fetched it
eight times before this and fetches it once now (SharedExternalFetch,
next commit).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master's last two runs (#368 and #370) failed in the Build step with
The state dir (/root/.cache/flatpak-builder) is not on the same
filesystem as the target dir (/workspace/Catcrafts/Crafter.Build)
while #369, the 2.2.0 tag run on the same commit with the same
flatpak-builder, passed. The difference was the runner. Since 14 October
the Hetzner box has a second privileged runner, cx33-org-flatpak, which
serves every Catcrafts repository under the label `flatpak` so their
package jobs can run crafter-build as a Flatpak. It mounts no host
volumes, deliberately, since any repository in the organisation can send
it jobs. Whenever it was idle first it took this job too, logged our four
volumes as "not a valid volume, will be ignored", and left the
flatpak-builder state on the container's overlay while the workspace sat
on a Docker volume. Even past that check it would have had no build
cache, no web root and no signing key.
The org runner's label is right for the jobs it was made for, and giving
it our volumes would hand the signing key to every repository in the
organisation. So the job asks for a label only its own runner carries:
`crafter-flatpak`, after the volumes and key directory that runner alone
allows. The runner on the host is relabelled to match
(/etc/forgejo-runner/config-flatpak.yaml, previous copy kept as
config-flatpak.yaml.bak-2026-10-31-label), so until this reaches master
a push there with the old `runs-on: flatpak` goes only to the org runner
and fails the same way.
AGENTS.md now says the label must stay unique and quotes the error a job
on the wrong runner stops with, so the next collision is found from the
log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Shaders record the files glslang opened (a depfile) and their settings
(source path, stage, entry point, defines, include directories) in the
build directory, and rebuild when an included file or a setting changes,
not only the shader's own source.
- An executable relinks when its link command changed, not only when an
object is newer: a link flag in project.cpp, or a dependency's library
that moved (an ExternalDependency built into a new cache directory). The
command is recorded beside the objects; its flags come from ordered sets,
so the order the dependency threads finish in can't change it.
- An external's newest artifact counts its libDirs and versioned .so names.
- Submodules are checked out at the revisions the fetched commit records,
from the URLs its .gitmodules gives, after every clone, switch, checkout
and pull, so a pinned commit builds against its own submodules, and one
moved by hand is put back; with no .gitmodules, git isn't asked.
- Each external clone's fetch and build hold a file lock: parallel builds
pulled into one clone at once ("cannot lock ref").
- mingw:
- -stdlib=libstdc++ always;
- the std module's source is found through libstdc++.modules.json in the
mingw toolchain's own library directories (the generic lib/ holds the
host's Linux one), asking the compiler once a run;
- a toolchain that isn't there is a build error saying where it looked,
not std::terminate;
- CRAFTER_MINGW_DIR names the prefix, the directory holding
x86_64-w64-mingw32/ and lib/gcc/;
- inside the Flatpak the host's /usr mingw is used.
- RunScript's exit-status check reads as the one range it is.
- Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules,
LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and
ToolchainResolve extended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RunScript caught std::exception only. Anything else a script threw went
on across the project.so boundary, and crafter-build ended in
std::terminate instead of reporting the script's failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local Flatpak build was one long flatpak-builder line in AGENTS.md,
with its state wherever the caller happened to stand, and installing
the result was not written down anywhere. It is not a rare chore: the
development box compiles with the distro's clang and the Flatpak with
its bundled LLVM, a major version apart at the moment, so anything
toolchain-sensitive has to be tried in the Flatpak too.
Both are now scripts in project.cpp, the first users of cfg.scripts.
`run flatpak` builds the manifest as it is, from the working tree, and
ends with `crafter-build --version` from inside the result. Its state
is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and
not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in
the sandbox, so it does not depend on the host variant, and a
variant-keyed directory would strand the LLVM cache on every host
compiler update. The manifest already skips build/ when it copies the
sources. flatpak-builder keys its cache on the manifest's path, so the
manifest is never rewritten into a copy the way CI stamps the version
in, and a local build reports 0.0.0-dev. The only argument accepted is
--jobs=<N>: anything that ends the run without a build would leave the
smoke test checking the previous one.
`run install` installs the last export as branch `local` from a user
remote of its own and makes it current, leaving the published branch
installed. It does not build first: flatpak-builder never caches a
`type: dir` module, so every build reruns build.sh and the suite.
remote-add --if-not-exists is followed by remote-modify --url, because
the remote may exist and point at another checkout.
Both refuse to run from the Flatpak build of crafter-build, which has
no flatpak to drive.
The smoke test goes through `flatpak build` and not `<builder> --run`,
which is what CI uses. flatpak-builder 1.4.9 run as the
org.flatpak.Builder app exits 1 when the command it ran succeeded and 0
when it failed (`true` gives 1, `exit 3` gives 0), so the script
reported a good build as a failure.
Measured on the 56-core box with LLVM and the WASI sysroot cached: 260
seconds, the crafter-build module only, 29 tests passing in the sandbox.
A second `run install` after a rebuild moved the installed commit.
`clean` now also drops that state, about 2 GB and a cold LLVM build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
RunScript on hand-built configurations: the named script alone runs,
with the project, the project directory and the arguments exactly as
given; its return value comes back whole from 0 to 255 and as 1 outside
that; a script that throws exits 1. A project without scripts, a
duplicate, an untypeable name, a missing callback and an unknown name
all fail without running anything. ctx.Run returns the command's exit
code and 128+signal for a killed one, and ShellQuote carries quotes,
spaces and expansions through sh intact.
Then the same through Run() with a fixture project, because the parsing
is where this goes wrong. `run record a --help -v clean --debug
--version run` must hand all seven words to the script, leave the one
project arg that stood before `run` alone, and not delete the staged
bin/ and build/. `run clean` and `run help` look for scripts of those
names. Moving the two capture branches below the verbs in the parse
loop fails this test.
The command-line half needs the installed module sources and is skipped
without them, but only when the in-process half reported no failure.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project.cpp could describe how a project is built, tested and linted,
but not the chores around it: packaging, installing, publishing. Those
stayed shell lines in a README or in someone's history.
Configuration gains `scripts`, a plain list like `defines`. A Script is
a name and a C++ callback, as a lint rule is; there is no script
language and no config file. The callback gets a ScriptContext (the
project, the project file's directory, the arguments) and returns the
exit status. ctx.Run runs a shell command with the terminal attached
and returns its status. It is only stored when project.cpp loads, which
happens for every verb and for every project that depends on this one.
`crafter-build run <name> [args...]` runs one, a bare `run` lists them.
Only the scripts of the project crafter-build was pointed at are
runnable, never a dependency's.
Everything after the script's name is handed to the script untouched.
The two branches that do this sit ahead of every other branch of the
parse loop, --version and --help included. Below the verbs,
`run package clean` would delete build/, and any other word would land
in the project's args and so in the variant hash, moving the build to
another directory. Options before the name keep their meaning, so
project args go there.
The driver is its own partition, :Script. It refuses a project with no
scripts, a name that is empty or starts with '-' (it could never be
typed), a name declared twice (running either would be a guess) and an
unknown name, each with exit 1 and without running anything. A script
that throws exits 1. A return value outside 0..255 becomes 1: an exit
status is eight bits, and a raw wait status such as 256 would otherwise
wrap to success.
Two helpers in :Platform:
- RunCommandAttached runs a command with inherited stdio. Every helper
so far captured through popen, which is right for a compile step and
useless for a command that runs for half an hour or asks a question.
It reads the wait status through DecodeWaitStatus, which
RunCommandChecked and RunCommandWithTimeout now share.
- ShellQuote is exported. It existed twice, as ShellQuote in :External
and ShellQuoteSh in :Test, with the same behaviour; both are gone. A
script that puts a path into a command needs it too.
Not included: a string shorthand for one-command scripts (a one-line
lambda covers it) and a ctx.Build() helper (Build() is already
exported).
A new partition and new exported symbols: run ./build.sh.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Drives Run() with `format clean` on a staged project whose project file
cannot compile. Both trees must survive, and the run must fail: getting
as far as loading the project is what shows it went on to format rather
than cleaning and exiting 0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every verb is a bare word that is only read as a verb while no
conflicting verb is active; after one, the same word is a glob. The
guard on `clean` checked the test and lint verbs and then itself, where
the format verb was meant. `crafter-build format clean`, which asks for
the rules matching "clean", therefore removed both trees and exited 0
without formatting anything.
The guard now names format, like the three beside it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CI takes the newest <release> as the notes of a tag and refuses a tag
that does not match it. 2.2.0 covers what changed since 2.1.0: Shader
entries with per-entry defines and a named output variant, sysroot
compiles against the sysroot's own libc++ headers, a project.so stamp
that hashes every part of the host build, and the vendored json.hpp
removed.
The variant API is what catcrafts.net's deploy needs: Crafter.Graphics
master has used it since 2026-10-06, and a stable Flatpak that lacks it
cannot build the site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GetCompileCommand is checked as text, without a compiler: the forced
include directory is present for linux-gnu and linux-musl targets with
a sysroot, and absent without a sysroot, for wasm32 and for mingw.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
clang's driver ranks the host toolchain's own libc++ headers
(<install>/../include/c++/v1) above --sysroot. BuildStdPcm has forced
the sysroot's copy for the std module for that reason, but ordinary
compiles did not. A unit that only does `import std;` never noticed. A
unit that includes a standard header textually got the host's libc++,
configured for the host's libc: cross-building Crafter.Graphics for
aarch64 musl, Crafter.Asset's vendored gdeflate died on <chrono> with
"unknown rune table for this platform". Against a glibc sysroot the
same unit would compile against the wrong libc++ release and say
nothing.
Every non-wasm, non-mingw compile with a sysroot now gets
`-nostdinc++ -isystem <sysroot>/usr/include/c++/v1`. wasm has its own
sysroot handling and mingw is a libstdc++ target.
-nostdinc++ leaves -stdlib=libc++ unused on each compile, which clang
reports once per unit, so the command also carries
-Wno-unused-command-line-argument. The cost is that the driver's
unused-argument diagnostics are silent on sysroot builds, including for
a user's own misplaced flag (a -l or -Wl, option in compileFlags).
Not addressed: external cmake dependencies are still configured
without the target or the sysroot, and objects already built in an
existing sysroot build directory are not invalidated by the new flags.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ShaderCompile registers one source twice and checks that the variant
gets its own output name, takes the other #ifdef branch and receives
the value of a NAME=VALUE define. Staleness is pinned per output: while
only the plain output exists the variant must still be due, or Build()
would never compile a variant added to a built project. Malformed
defines and a variant with a path separator must be refused by name.
ShaderVariantBuild drives the same through Build(), which is where the
entries compile in parallel into one directory: a variant added to an
already built configuration is compiled into its own file, and two
entries with one output name fail the build without touching the
existing output. Its fixture tests the macro with `#if`, which pins
that a bare define is 1 and not empty.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Crafter.Graphics has to ship each of its compute shaders in two builds
of one source: one that indexes VK_EXT_descriptor_heap and one that
reads the same slots from a descriptor set, for drivers without the
extension. A Shader was a path, an entry point and a stage, and its
output name was derived from the path alone, so one source could only
be compiled once.
Shader gains `defines`, spelled like the compiler's -D ("NAME" is
defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a
preamble, and `variant`, which names the output: "ui.comp.glsl" with
variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one
place that name is formed; Check, Compile and the forwarding of a
dependency's shaders into the consumer's bin dir all use it.
Two fields make one mistake easy: defines set, variant forgotten. Both
entries then compiled in parallel into the same file, the build
reported success with whichever finished last, and Check() called both
up to date from then on. The same has always been true of a.vert next
to a.frag. Build() now refuses a configuration in which two entries
share an output name and says which.
A define without a name, or one spanning lines, and a variant with a
path separator are rejected by name. Left to glslang the first two
surface as "-1:<line>" with no file, and the last as a failed open
after the whole compile.
Not addressed: Check() still compares only the source's mtime, so a
change to the defines, or to an included file, does not recompile
(#35).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
HostCacheSourceStamp already edits a module source with a backdated
mtime and expects the PCM cache to be rebuilt. Its own header names the
real danger, a project.so compiled against declarations that no longer
match the process it is loaded into, but it never looked at the
project.so. It now records the file's mtime before the edit and expects
a newer one after it, with project.cpp untouched, so only the stamp can
force the rebuild.
Fails without the previous commit: the stamp was the version string
alone and the project.so was reused.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The stamp that decides whether a compiled project file can be reused
was meant to cover the crafter-build version, the toolchain key, the
module-source stamp and the compile command. Only the version went in.
The four parts were joined by a format string with "\0" between the
fields, and a format string is read as a C string: it ended at the
first NUL, so std::format saw "{}" and dropped the other arguments
without a diagnostic.
A project.so is loaded into the crafter-build process and fills in
Configuration objects by their layout. With the stamp blind to the
declarations, one built against other module sources was loaded as
current whenever the version string was unchanged, which is every
working-tree build (`git describe --dirty` does not follow the tree's
contents) and every local Flatpak build. Growing a class that project
files construct then corrupts the heap: a project.so built while Shader
was 56 bytes, loaded after it had become 104, aborted with "free():
invalid size" as soon as it declared a shader. CI builds were spared
only because each carries its own version.
The parts are now appended to a std::string with NUL separators, the
way the other stamps in this file are built. Every existing project.so
carries a stamp of the version alone, so each is recompiled once.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CI takes the newest <release> as the notes of a tag and refuses a tag
that does not match it. 2.1.0 covers what changed since 2.0.0: the
bundled LLVM 23.1.2, build outputs kept per compiler, and the std
module source chosen by the sysroot's libc++ release.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README still gave the output path as bin/<name>-<target>-<march>/,
which has not been its shape since the variant hash was introduced. It
now shows the real one and says what the hash stands for, including
that a compiler update starts a new directory and `clean` removes the
old. AGENTS.md lists the toolchain among VariantId()'s inputs and notes
that a glob over variant directories matches two after an update.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two toolchains that differ only in the clang release, as the Flatpak's
do across an update, must give one Configuration two variants; so must
the same release installed under another prefix. The no-argument form
is checked to be the variant under the running toolchain, and BuildDir
to be named for it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project built with one clang kept its outputs when crafter-build
started using another. The output directory is named by VariantId(),
which hashed every input that changes what is compiled except the
compiler itself, and staleness is decided by mtimes, which a compiler
update leaves alone. A PCM can only be read by the clang release that
wrote it, so after the bundled LLVM moved from 22 to 23 the first build
found nothing stale, and the first edit failed with "module file
Greeter.pcm uses an older format that is no longer supported" until the
project was cleaned by hand. A project with a `#pragma crafter
dependency` did not get that far: its project file imports the
dependency's PCM, so it failed to load, and for a git dependency that
PCM lives in the cache, where `crafter-build clean` does not reach. The
same happens natively when the distro updates clang.
The toolchain's key now goes into the hash, as the host PCM cache was
already keyed on it. An update starts an empty variant directory and
rebuilds into it; the previous one stays until `clean`, as after any
other change of variant, and two crafter-build versions can work in one
checkout without disturbing each other. VariantId() gains an overload
taking the toolchain so the dependence can be tested in-process, the
way ResolveToolchain and SysrootStdModule take theirs.
External cmake builds are not covered: their directory is fixed per
clone and keyed on the compiler's path, not its release. Objects of two
clang releases link together on the host, so nothing fails there, but
they are not rebuilt either.
Verified by replaying the update: three projects built with the 2.0.0
Flatpak, then built, edited and run with this one without cleaning,
then built with 2.0.0 again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The previous commit made the LLVM tree inside the wasi-sdk checkout a
git repository with `git init`, so that wasi-sdk's patch step would
apply. That was half the fix: wasi-sdk also runs `git rev-parse HEAD`
in that tree to write the sysroot's VERSION file, which on an empty
repository raises a traceback during configure and leaves VERSION
without its llvm lines. The build went on regardless, so the suite
passed and the published beta carries the truncated file.
Fetching the tree as a git source pinned to the release tag gives
wasi-sdk what it has upstream, a submodule checkout at a commit, and
removes the shell step instead of stacking a second workaround on it.
The llvm module keeps the release tarball: it is the same commit, and a
tarball is the cheaper download for a tree that is only compiled.
flatpak-builder keeps a mirror of the clone in its state directory, so
the fetch costs the cold run about 300 MB once.
Verified locally: no traceback, VERSION lists llvm 85ac56026243 and
llvm-version 23.1.2, the eight patch steps apply, the suite passes in
the sandbox and a wasm32-wasip1 program runs under the bundled wasmtime.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README said a cross build precompiles the sysroot's own std.cppm,
which was never the whole story for sysroots that do not ship one. It
now states the order (the sysroot's, the toolchain's for the same major
release, the sources crafter-build carries for other releases), what
the failure looks like when nothing matches and how to resolve it, and
lists the new directory in the install layout.
AGENTS.md records the rule for the next LLVM bump: the release the
bundled toolchain leaves behind stays available as a module-source
module, because stable distros remain on it long after.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With LLVM 23 bundled, a sysroot still on libc++ 22 without its own
module source (Alpine v3.24) is the mirror image of the case that
started this: the toolchain's source is a release ahead of the
sysroot's headers. The new libcxx-22-modules module installs libc++
22.1.8's module source under share/crafter-build/libc++-modules/22,
where SysrootStdModule looks for the release a sysroot is on.
It only configures libc++'s runtimes build and installs the cxx-modules
component: std.cppm as configure generates it and the .inc files from
the source tree, nothing compiled, about 600 KB installed. The
component is staged and copied because it also writes
lib/libc++.modules.json, which belongs to the bundled libc++.
Verified in the built Flatpak: `crafter-build --version` lists release
22, and a cross build against an Alpine v3.24 aarch64 sysroot (libc++
22.1.3) precompiles that source with clang 23, links and runs under
qemu.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LLVM 23 has been the stable series since September, and the distros
crafter-build cross-builds for have moved to it: Alpine edge ships
libc++ 23.1.2, which is what surfaced the std module mismatch fixed in
the previous commits. wasi-sdk 34 already builds against a newer LLVM
than 22 upstream, so the WASI sysroot loses nothing by following.
The bump exposed a patch step that never ran. wasi-sdk patches its LLVM
tree with `git apply` in src/llvm-project, which upstream is a
submodule. Here that directory is a tarball unpacked inside the
wasi-sdk checkout, and git, finding the outer repository, treats every
patched path as outside the current directory, prints "Skipped patch"
and exits 0. LLVM 22 built without the three patches; libc++ 23 does
not compile for WASI without the one that keeps the z/OS locale shim
out of other platforms' builds (strtod_l redeclared with an abi_tag).
`git init` on the unpacked tree makes it a repository of its own, so
wasi-sdk's step applies whatever patch list the pinned tag carries, as
it does upstream, and fails loudly when a patch no longer fits.
Built locally: the whole manifest, the suite inside the sandbox (26
passed, WasmHelloRuns under the bundled wasmtime included), and aarch64
musl cross builds against Alpine edge and v3.24 sysroots that run under
qemu. The native development loop is unaffected and stays on whatever
clang the machine has.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SysrootStdModule runs over staged layouts that need no compiler: a
sysroot with its own module source, one on a point release of the
toolchain's libc++, one on the next major release with a shipped
source, one nothing was shipped for, one without libc++ at all, and the
two cases where a release cannot be read. The last block drives Build()
against a sysroot claiming libc++ 99 and expects the one-line
explanation back instead of compiler output, which is the failure the
fix replaces.
ToolchainResolve additionally checks that the release is read from the
libc++ the module source comes from (the prefix's, an explicit libc++
install's, the system's) and where the shipped sources are looked up.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A cross build against a sysroot whose libc++ differs from the
toolchain's failed while precompiling the std module, with a page of
"no member named 'runtime_format' in namespace 'std'". The module source
is a list of using-declarations for what one libc++ release declares, so
it only compiles against that release's headers. BuildStdPcm wants the
sysroot's own <sysroot>/usr/share/libc++/v1/std.cppm, but Alpine
packages the headers without it, and the fallback was the toolchain's
copy whatever its release. That held while the compiler came from the
same distro as the sysroot; with the Flatpak's libc++ 22.1.8 and Alpine
edge on 23.1.2 it no longer does.
The release is now read from _LIBCPP_VERSION in the sysroot's and the
toolchain's <__config>, and SysrootStdModule picks in this order: the
sysroot's own source, the toolchain's when both are the same major
release (a release branch only takes fixes), the copy crafter-build
ships for that release under share/crafter-build/libc++-modules/<major>.
When none fits the build stops with one line naming both releases and
the directory the matching source belongs in, before clang is run. If
either release cannot be read the toolchain's source is tried as
before, since there is nothing to compare.
Matching sources live in crafter-build's own share directory rather
than the LLVM prefix: they are data this tool ships for sysroots, not
part of any one compiler install, and "sources ship" already describes
that directory. `--version` prints the toolchain's libc++ release and
the releases sources are shipped for, the first thing to look at when
such a build fails.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
flatpak-builder printed "Building module llvm" on every CI run instead
of "Cache hit". The build environment, CRAFTER_BUILD_VERSION included,
sat in the manifest's top-level build options, and those are part of
every module's cache key; the version changes with each commit, so no
module ever hit. Runs still took twelve minutes rather than eighty only
because ccache made the LLVM rebuild cheap.
The variables only matter to crafter-build's own module, so they move
there, and CI rewrites the version on that module. LLVM, the WASI
sysroot and wasmtime are now keyed on nothing that changes between
commits.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every non-tag build called itself 0.0.0-dev+<hash>, a placeholder that
said nothing about which release it came after and put a `+` in the
bundle's download URL. A master build now uses `git describe`, as a
local checkout already does: 2.0.0-3-g21746b1 is three commits past
2.0.0. The placeholder remains only for history that has no tag yet.
Three more things the tag path needed before its first run. The
forgejo-release action takes the notes text in `release-notes` and a
path in `release-notes-file`; the workflow passed a path to the former,
so the release body would have been the file name. The tag filter is
spelled with globs so it reads the same as a workflow pattern or a plain
glob. Python, used to extract the notes from the metainfo, was only in
the container as somebody else's dependency and is now listed. The
2.0.0 entry carries the date it is tagged on.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The publish step created the OSTree repo only when the directory was
missing. On the runner host the web root exists before the first
publish, as an empty directory, so the init was skipped and
build-commit-from failed with `opendir(objects)`. The check now looks
for the repo's config file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The privileged runner that already existed is registered to another
repository, so its labels cannot serve this one. CI runs on a separate
privileged runner registered to Crafter.Build only, with four volumes
rather than three now that the signing key is mounted from the host.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The runner is our own machine, so the repo signing key does not need to
pass through the forge as a secret. It is generated once under
/etc/crafter-flatpak on the host and bind-mounted read-only into the job
container, where each run imports it into a throwaway keyring. One less
secret to manage, and the private key never leaves the box that uses it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README leads with the Flatpak install, keeps the native loop as the
way to build from source, and drops the tarball, musl and makepkg text
that no longer describes anything CI produces. The install layout shows
the /app tree and the per-app cache directory.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
