Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
0cc699139c
docs(packaging): add the 6.0.0 release entry
All checks were successful
Flatpak / flatpak (push) Successful in 12m12s
The release of this push and of everything on master since 5.0.0: lldb,
the Khronos validation layer and the SPIRV-Tools programs in the
Flatpak, --die-with-parent in the README's alias, and the release
.flatpakref with the repository's key in it. Its Breaking item, the end
of beta and of the rolling release, makes it a major.
2026-11-02 17:17:02 +01:00
54aaf078b2
docs: a release is a signed tag pushed with master
The README offered master builds as the `beta` branch, AGENTS.template.md
described the `latest` tag CI force-pushed, a release as a tag pushed
after master, and CI as running on every push to master, and the
repo-better-solution lens told reviewers that users get master through
`beta`.

The README now says that releases go to `stable`, the branch its install
commands pick even while the repo still holds the old `beta` branch.
AGENTS.template.md says that every pushed range adds a release entry and
goes out in one push with a signed tag on the commit master receives,
its last commit or, on the pull-request path, the merge, how to write
the entry, what every run checks before building and what a tag run also
checks, publishes and creates, what follows when one fails, what CI runs
for, and what becomes of `beta`; the lens says users get master through
releases on `stable`. The entry's rules are the owner's: at most three
sections, New, Fixes and Breaking, the most important item first and
nothing after them, and a version that is a major for a Breaking
section, a minor for New items and a patch for Fixes alone. The release
paragraph also gets the blank line that keeps it apart from the one on
local builds.
2026-11-02 17:17:02 +01:00
be81fc098e
feat(ci): drop beta and latest, release signed tags on master
A push to master published the `beta` branch and moved the rolling
`latest` tag and its "Latest master build" prerelease, and a pushed
X.Y.Z tag made the release and published `stable`. The owner wants a
release for every push to master, made by the signed tag that goes with
it, and nothing for a push without one. So `beta`, `latest` and the
branch trigger are gone: the workflow runs for pushed tags, pull
requests and manual runs only, and the `latest` tag and its prerelease
have been deleted from the forge.

`beta` gets no more builds, and its installations would not hear of it:
the release notes ship in stable builds only. The Publish step marks
beta's last build end-of-life, once, with a reason that names stable,
and the next `flatpak update` of a beta installation prints it. The
reason also names the branch, because a non-interactive update prints
only the app's name with it.

Before building, a tag run now checks the tag: it has to be annotated
and signed by the owner as Forgejo verifies it, its commit has to be on
master, its version has to be X.Y.Z and newer than every release the
forge has published, its own included (a later tag whose run still
queues does not count), and the metainfo's newest <release> entry has to
be that version. The tag run checked only the last of these. It reads
the tag from the forge's API, because the checkout fetches a pushed tag
again as a plain ref to its commit, so the job's clone never has the
annotated tag, and Forgejo verifies a signed tag's own signature but
falls back to the commit's for an unsigned one, so the step also
requires that the verdict covers the tag's payload. The metainfo's entry
is read once, as the first <release> element: the Version step takes its
version and renders its notes into release-notes.md with the renderer
the Release notes step had, and stops the run when they come out empty,
so a comment or another attribute order cannot give a release one
entry's version and another's notes. It does so on every run, pull
requests and manual runs included, so a broken entry shows before a
release.

Release runs share a concurrency group, so they queue one behind
another; a pull request still cancels its own outdated run. A failed
release run can be re-run while no newer release exists, since its tag
is still on its commit; a release that an attempt left half made has to
be deleted first. The release step runs without `override`, which would
delete a tag that has moved off the run's commit since the Version step
and tag the old commit instead.

A manual run no longer publishes anything; it used to publish `beta`
and move `latest` from whichever branch it was started on. No step
pushes to git, so the checkout keeps no credentials. git describe in
build.sh, project.cpp and the workflow has no `latest` tag left to
exclude.
2026-11-02 17:17:02 +01:00
2babe94a6a
fix(ci): publish under the lock Crafter.Review's workflow takes
Crafter.Review's workflow publishes its Flatpak into the same repo, the
web root's repo/, and holds an flock on .publish.lock in the web root
while it commits and updates the summary. Its comment says that this
protects both only once Crafter.Build's workflow takes the lock too, and
ours never did, so a publish of each could interleave: a
build-commit-from and a build-update-repo --generate-static-deltas
--prune of each workflow on one repo at the same time.

The Publish step now holds the same lock from its commit to the ref
files it writes into the web root, and the job installs util-linux for
flock, as Crafter.Review's jobs do.

AGENTS.template.md says that the repo is shared and the lock taken.
Neither it nor the workflow's header comment says any longer that the
web root and the signing key are this runner's alone, or that the
organisation-wide `flatpak` runner mounts none of them. That runner,
which serves every Catcrafts repository and refused all of this job's
volumes when it last took it, in October, now mounts the web root and
the key for Crafter.Review's publish job, and the volumes a runner
allows are open to every job it runs. The owner has ruled the sharing
intended. The header also stops saying that a job of ours that lands
there fails before building: that held while the runner refused every
volume, and AGENTS.template.md keeps it as the October history.
2026-11-02 14:56:41 +01:00
2447d6ec4f
fix(ci): attach the flatpakref with the repo's key to releases
The Bundle step copied packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref
into dist/ as the checkout has it, with GPGKey=GPG_KEY_BASE64_PLACEHOLDER,
so all six releases, 2.0.0 to 5.0.0, attach a .flatpakref that does not
install: `flatpak install --from` it stops with "GPG: Unable to export
keys: GPGME: No data" (Flatpak 1.18.4) before it adds the remote. The
copy the publish step writes to the web root has the key filled in, and
installing from it goes ahead.

The key is known only from the publish step on, so the release's copy,
and SHA256SUMS over it, are made there from the web root's file instead
of in the Bundle step.
2026-11-02 14:56:41 +01:00
a1bc96a323
fix(review): name each lens section, as crafter-review 1.0 requires
crafter-review 1.0.1, the released build installed on this machine, and
the newer build installed beside it as //local refuse the config key
lensSections: a lens's repository section is named per lens, as
lenses.<name>.section, instead of being found by listing a directory.
With the key in place `crafter-review config` reports it as a problem,
and freeze and run stop at "preflight: config: unknown key lensSections".

The three files in .review-gates/lenses/ are named under lenses, one
entry each, the migration the Crafter.Review session gave; with them
`crafter-review config` reports no problem.
2026-11-02 14:56:41 +01:00
00f07dae88
docs: push reviewed ranges straight to master
All checks were successful
Flatpak / flatpak (push) Successful in 30m58s
AGENTS.template.md had every change go through a pull request against
master. The owner has made pushing straight to master the default.

The commit and push guidelines say so: an approved range is pushed to
master as a fast-forward, with no merge commit, and a pull request is
opened only when the owner asks for one; when master has moved since
the freeze, the range is rebased, frozen and reviewed again. What
follows from that: the README changes in the same push; without a pull
request CI does not run on a range before it reaches master unless
someone starts it by hand, so the review's gate0 is the check before the
push; and the freeze exception for a merge commit applies only to a pull
request. Design rule 8, the testing note and the release steps lose
their pull-request wording, and the testing note no longer speaks of a
commented-out lint step: ci.yaml has no lint step at all.
2026-11-01 23:21:35 +01:00
03951ff458
docs: update the first local Flatpak build's time and size
AGENTS.template.md and project.cpp gave the first run flatpak as about
25 minutes on a 56-core machine and its state as about 2 GB, figures
written on 2026-10-06, before lldb and the Vulkan and SPIR-V modules
were in the manifest. A first run of this commit's manifest, in a fresh
worktree, took 30 min 36 s with 56 CPU threads, 14 min of it for LLVM
with lldb, and left build/flatpak at 4.1 GB, 3.3 GB of it state/.

The figures become those, and the line on later runs says that they
rebuild only the crafter-build module while the manifest is unchanged.
2026-11-01 22:28:08 +01:00
40dcb18a1e
feat(flatpak): ship the SPIRV-Tools programs
Crafter.Graphics' PushConstantRewrite and UIFusedShader tests require
tool:spirv-val, and the Sdk has no spirv-val, so inside the Flatpak they
are skipped. The spirv-tools module the validation layer needs built
only the libraries, and removed them afterwards.

It now builds the programs as well, spirv-val, spirv-opt, spirv-dis and
the rest, which link SPIRV-Tools statically, and ships them. The
headers, the static libraries, the CMake and pkg-config files and
libSPIRV-Tools-shared.so, which nothing here links, are still removed,
and a post-install step runs spirv-val --version.
2026-11-01 22:28:08 +01:00
c8517a41cf
docs(readme): --die-with-parent in the install alias
flatpak run becomes bubblewrap, and without --die-with-parent the
sandbox outlives it. With crafter-build started through the alias in an
interactive bash or zsh on a terminal, SIGKILL to its flatpak run
process alone left a -r program running and a build writing into bin/
and build/. With the flag, both ended with it.

bubblewrap kills the sandbox when flatpak run or its parent dies, and
for crafter-build typed at the prompt, in the foreground or with &, that
parent is the shell: with the flag, the sandbox also ended when the
shell was SIGKILLed, or exited after a disown. In a (crafter-build -r &)
subshell, flatpak run outlives the subshell and is adopted by another
process, so its sandbox outlived the shell with the flag as well. Ctrl-C
ended the program either way.

The README's alias gets the flag, and the paragraph after it says what
that changes: a program started through the alias, as crafter-build ...
or crafter-build ... & at a bash or zsh prompt, ends with its shell,
even after a disown.
2026-11-01 22:28:08 +01:00
a32b8c898a
docs: debugging inside the Flatpak
The README gets a Debugging section: the debugging tools the sandbox
carries, that a program to debug needs cfg.debug (which --debug sets
where the project calls ApplyStandardArgs), how to start the tools with
--command, why a debugger in one flatpak run cannot see a program that
another one started (each has process IDs of its own), and flatpak
enter as the way into a running program's sandbox, with the ptrace
opt-in that takes where ptrace_scope is 1. It warns off lldb's
gdb-remote from a second sandbox: when a process in lldb's own sandbox
has the remote program's process ID, lldb takes that process's
executable for the program's. And it shows which variables turn the
layer and GPU-AV on from the environment, and that VK_LAYER_PATH and
VK_ADD_LAYER_PATH do not reach the sandbox.

AGENTS.template.md gets the module layout and the rule that the layer
follows the Sdk's Vulkan loader. Its CI section says that a change to a
module's sources or build-commands builds it again and every module
after it, so that a new version of swig or libedit, which come before
LLVM, now rebuilds it too, and loses the cold-run time it gave, written
on 2026-10-03, before any of these modules were in the manifest.
2026-11-01 22:28:08 +01:00
16db37bbe6
feat(flatpak): ship lldb and the Khronos validation layers
Programs that -r and the test runner start run inside the Flatpak's
sandbox, and the Sdk it runs on has gdb but neither lldb nor the Khronos
validation layer: in the sandbox of beta 5.0.0, a vkCreateInstance that
asks for VK_LAYER_KHRONOS_validation returns VK_ERROR_LAYER_NOT_PRESENT.
flatpak run clears VK_LAYER_PATH and VK_ADD_LAYER_PATH, so the layer the
host has cannot stand in.

lldb becomes a project of the llvm module and brings lldb-server,
lldb-dap and lldb-mcp. Its Python, libedit, curses, libxml2 and lzma
support are set ON instead of being left to detection, so a dependency
that goes missing fails the configure step rather than shipping an lldb
without it. The Sdk has neither SWIG nor libedit, so both become modules
before llvm, SWIG as build-only.

The layer is Vulkan-ValidationLayers vulkan-sdk-1.4.357.0, the Vulkan
SDK release of the Sdk's loader (libvulkan.so.1.4.357), built from
Vulkan-Headers, Vulkan-Utility-Libraries, SPIRV-Headers and SPIRV-Tools
at the revisions that release's scripts/known_good.json names. Those
four are built into /app at those revisions and removed afterwards. All
five come after the LLVM and WASI modules, so that bumping them reuses
the cached LLVM and WASI builds. UPDATE_DEPS is turned off: it defaults
to ON, which has update_deps.py clone the dependencies, and of all the
modules only crafter-build's build-args share the network.
2026-11-01 19:20:22 +01:00
e2bd0b14ee
feat(review): set up Crafter.Review's pre-push review
All checks were successful
Flatpak / flatpak (push) Successful in 13m7s
Crafter.Review reviews a frozen commit range before it is pushed: the
approach, a static proof of the code's bugs, and every claim in the
messages and in the documentation the range changes, one claude process
per lens and unit, or per whole-range pass, and the owner approves the
exact frozen hash before the push. This sets it up for this repository.

.review-gates/config.json turns on three lenses in the commits mode, at
xhigh: repo-better-solution, repo-bug-prover and repo-facts. The
functional lens is off: it exercises a web application through a
browser, and this repository's product is a command-line tool. The
security lens is off by the owner's ruling: crafter-build stores no
secret of its own, and while its Flatpak runs with the user's SSH agent
(--socket=ssh-auth) and the whole host filesystem (--filesystem=host),
everything it runs is code its user chose to run. The rules CI keeps
around its token and signing key go to the approach lens's section
instead.

gate0 lists the commands run by hand before a freeze. build.sh comes
first, because bin/crafter-build compiles project.cpp against the
interfaces it was bootstrapped with, and lint comes after the self-host
build, because lint's AST rules parse with the module PCMs a build
leaves and lint builds only the std PCM itself.

.review-gates/lenses/ holds this repository's section of each lens's
charter, appended to the text Crafter.Review ships: that AGENTS.md is
absent from the frozen tree, the layers and the consumers of
interfaces/, the execution contexts a bug can hide in, where libc++ and
clang are readable from the review's sandbox, and what a "passes the
suite" claim traces to.

The /prepush skill that crafter-review init writes stays untracked. It
is generated from the installed Crafter.Review and refreshed by every
init, so a committed copy would lag the binary it describes. One init
per checkout restores it, which AGENTS.template.md now says, together
with the rules freeze holds every commit of a range to: signed
(requireSigned), with the owner's address as its author and committer
(authors), and no AI attribution trailer (aiTrailers forbid).
2026-11-01 17:33:27 +01:00
a3fe1bdda7
docs(packaging): add the 5.0.0 release entry
All checks were successful
Flatpak / flatpak (push) Successful in 11m53s
The fix for #43, a mingw build that saw the host's glibc headers and
Linux libraries through a prefix like /usr as clang's sysroot, and what
the review of it brought along: units that recompile after a std module
rebuilt in place, CMake externals built again when their configure
inputs change, std modules keyed on every flag they are compiled with,
and lint that fails rather than passing files it could not parse. A
major, because Toolchain::mingwSysroot is now mingwPrefix and every
build directory moves once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 23:55:28 +01:00
76831d6733 Merge branch 'claude/issue-43': mingw builds without the host's headers, std modules keyed on their flags
All checks were successful
Flatpak / flatpak (push) Successful in 11m41s
Brings in the fix for #43. The mingw target no longer gets a system
prefix (the host's /usr in the Flatpak, or any prefix whose include/
has a C library's features.h) as clang's --sysroot, but a view of its
mingw-w64 parts, made in the cache or in $XDG_RUNTIME_DIR; a prefix of
mingw-w64 alone stays the sysroot whole. From the review of the first
version: the std module's directory is keyed on every flag it is
compiled with and VariantId on that key, so every std PCM and build
directory moves once; units record the BMIs they import; a CMake
external whose configure inputs changed starts from an empty build
directory; lint fails a configuration that has no compile command.
Toolchain::mingwSysroot is now mingwPrefix.

The MingwSysroot test writes its stand-in std module through
std::format, because the module scanner reads string literals as code.
#45 replaces the scanner's regexes with clang-scan-deps.

The branch starts at master's tip, so the merge is a tree identical to
the branch's; that tree passes format, lint and the test suite, natively
and inside the Flatpak sandbox, and CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:35:49 +00:00
c56d98abba
docs: which mingw prefixes are viewed, and what keys the std module
All checks were successful
Flatpak / flatpak (pull_request) Successful in 7m28s
The README said every prefix reached clang as a view of its
x86_64-w64-mingw32/ and lib/gcc/, in ~/.cache. It now says a prefix is
the sysroot as it is unless it also holds a Linux system, names
lib64/gcc/ as well, gives the view's place inside the Flatpak and the
XDG_RUNTIME_DIR fallback, and says messages name headers by their path
in the prefix.

AGENTS.template.md describes MingwSysroot's two outcomes and
MingwViewToPrefix, has the std PCM keyed on its flags in the cache
rule, lists StdPcmKey among what VariantId hashes, and gains two
gotchas: anything that changes how the std module is compiled goes into
StdPcmFlags, and a CMake external whose recorded configure inputs
differ is built again from an empty directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:59:47 +01:00
55ad128b79
test: lint fails when a configuration's std module cannot be built
A host configuration pointed at a sysroot without libc++ cannot get its
std module built, so its files cannot be parsed. With an AST rule
registered, the run has to skip the rule and fail instead of reporting
the files clean, as it did before the fix. The cache is redirected
around this run only: the std module's directory is made before its
build fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:59:38 +01:00
40dcf2271d
fix(lint): fail the files whose configuration has no compile command
When GetCompileCommand threw or the std module failed to build, the
lint driver cleared the configuration's command, and an empty command
means "not a translation unit" (project.cpp, a header): every rule that
needs an AST skipped the files quietly and the run passed them as clean.
MingwSysroot made that reachable for every mingw configuration whose
view cannot be made. The reason is now kept per configuration and
reported as an AST failure for each of its files, which fails the run
and prints it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:59:37 +01:00
0e69e71b11
test: a CMake external configured with other inputs is built from nothing
ExternalReconfigure builds a one-file C library from a local repository
as a CMake external, then plants a file in its build directory to stand
for what an old configure left. Built again with the same inputs, the
directory is kept. With its recorded inputs rewritten the way another
crafter-build's configure would have left them, the next build has to
start from an empty directory and build the library again. Before the
fix the planted file survived the reconfigure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:59:23 +01:00
56b44f616a
fix(external): start a CMake build over when its configure inputs change
ConfigureCMake records the flags it injects (compiler, libc++, and for
the mingw target CMAKE_SYSROOT) and reconfigures when they differ, but
it did so in place. CMake then keeps what the old flags compiled: the
Makefiles generator writes --sysroot into build.make, which make does
not compare, so nothing was recompiled, and the check results the old
configure cached stay too. A mingw external built by a release that gave
clang the host's /usr as the sysroot kept its objects reading
__libc_single_threaded after the fix for #43, and only deleting
<cache>/external/<name>-<hash>/build by hand got rid of them.

A build directory with a CMakeCache.txt whose recorded inputs differ is
now removed before the configure. It keeps its name, since projects
point include directories at headers generated in it. The two
crafter-builds the Flatpak's branches are, sharing one cache, rebuild
such an external from nothing each time they take turns, where they
used to reuse each other's objects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:59:22 +01:00
8e759a8410
test: a std module rebuilt in place recompiles the units importing it
StdPcmRebuild builds a project whose main.cpp imports std, then dates
the source before the object and the object before std.pcm, as a std
module rebuilt after the object leaves them, so that nothing else makes
the object stale. The next build has to compile main.cpp again, and the
idle build before it must compile nothing. Without -fmodule-file-deps
the object was only relinked. The test redirects the cache, since
redating a std PCM the real cache shares would rebuild its other users.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:59:06 +01:00
27697d06aa
fix: recompile the units whose std module was rebuilt in place
A .cpp unit's depfile listed its source and headers, never the BMIs it
imports: clang's driver adds -module-file-deps to --precompile only. A
std module rebuilt in its directory, which happens whenever its source
changes under one compiler and one set of flags (a libc++ update, or
mingw-w64's under the same prefix), left every object that imports it
as it was, and the next build only relinked them. Implementation units
are now compiled with -fmodule-file-deps, so the std PCM and the
project's own PCMs are prerequisites of their objects like headers.

Depfiles written before this lack the edge, so a std module that moves
directories still needs VariantId to move with it (StdPcmKey).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:59:05 +01:00
5d25495621
fix: key the std module's directory on everything it is compiled with
StdPcmDir named the directory by target, march and toolchain, then
grew a case per target for the rest: the wasm flags, and with the first
fix for #43 the mingw prefix. Every other flag BuildStdPcm passes went
unkeyed, so configurations that differed only there took turns
rebuilding one std PCM in place, under the PCMs built against it: a
project with -mtune=generic against project.cpp's -mtune=native on every
run, or two sysroots for one target. And the mingw prefix was keyed only
when there was one, so a build without a prefix landed in the directory
an older release had filled with a CRAFTER_MINGW_DIR=/usr build, and
relinked its objects.

StdPcmFlags is now the one source of the flags BuildStdPcm compiles
with, and StdPcmKey names the directory by target, march and toolchain
followed by a hash of those flags and, for the mingw target, of the
prefix behind its --sysroot. VariantId hashes StdPcmKey in place of the
mingw prefix: units do not notice a std module that moved directories,
so a build has to move with it. Every std PCM directory and build
directory moves once, as after a compiler update; the old ones stay
until `clean` and the cache is cleared by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:58:44 +01:00
06b61e84f9
fix(mingw): view only a prefix that holds a Linux system
The first fix showed clang every mingw prefix through a view of its
mingw-w64 parts. That broke a prefix that holds mingw-w64 alone, where
Windows libraries installed with --prefix land in include/ and lib/:
with CRAFTER_MINGW_DIR=/opt/mingw, <SDL2/SDL.h> and -lSDL2 were no
longer found. Such a prefix is clang's sysroot whole again. Only a
prefix that also holds a Linux system gets the view
(Toolchain::mingwPrefixIsSystem): the host's /usr inside the Flatpak,
whatever its include/ holds, and a prefix whose include/ has a C
library's features.h, which every glibc and musl header includes and
mingw-w64 has no use for, as CRAFTER_MINGW_DIR=/usr has.

MakeMingwSysroot makes the view in the cache, or in
$XDG_RUNTIME_DIR/crafter.build when the cache's filesystem takes no
symbolic links (vfat, exFAT, CIFS without unix extensions): the XDG
spec promises them there, and nothing else in the cache needs them.
Its path is absolute, because CMake runs the compiler from directories
of its own and a relative cache did not resolve from there. The prefix
is resolved once, absolute and lexically normal, so /opt/mingw/ and
/opt/mingw are one prefix and not two builds.

clang names a header by the path it found it at, so every diagnostic
from a viewed mingw-w64 header named a hashed cache directory.
RunBuildCommand now writes paths into the view as paths into the
prefix, as BuildStdPcm did for its manifest messages already.

Also from the review of PR #44: SysrootCompileFlags builds a mingw
command, which can make a view, so it redirects the cache like the
other tests; the manifest filter's comment names what it still keeps
out; the C path asks MingwSysroot() once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:58:30 +01:00
5fdbf2baee
docs: the mingw sysroot view in the README and AGENTS.template.md
All checks were successful
Flatpak / flatpak (pull_request) Successful in 7m2s
The README says which directories of a mingw-w64 prefix clang sees, and
why the rest of a prefix like /usr is kept out of a Windows build.
AGENTS.template.md names MingwSysroot in the :Toolchain partition, with
the reason the prefix itself is never a sysroot, counts the views among
what goes in the cache, and adds the mingw prefix to what VariantId
hashes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
b162cc0552
test: a mingw build sees no header from its prefix's include/
MingwSysroot stages a prefix shaped like a Linux system's /usr with a
mingw-w64 cross toolchain in it, without needing a real one: glibc's
<sys/single_threaded.h> in include/, gcc's per-target directory, and a
stand-in mingw-w64 tree whose libstdc++ has an empty std module. A
static library for x86_64-w64-mingw32, so that nothing is linked,
compiles a C++ and a C source that stop with #error when they see the
prefix's include/ or miss mingw-w64's or libstdc++'s headers. Against
the library before the fix, the C++ source stopped on
<sys/single_threaded.h>. The test also checks that the std module is
not cached in the directory older releases built it in against the
prefix itself. It needs only clang, so it runs in CI too, where no
mingw-w64 is installed.

The stand-in std module's source is written through std::format: the
build's import scanner reads string literals as code, and the literal
"export module std;" made the test itself a unit of a module std.

VariantId checks that a prefix gives a mingw configuration a variant of
its own, that two prefixes give two, and that other targets ignore it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
ab998081e1
fix: give the mingw target a sysroot that holds mingw-w64 alone
Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was
clang's --sysroot as is. clang's mingw driver takes a sysroot for a tree
that holds mingw-w64 alone, and adds its include/ to the include path
and its lib/ to the library path. For /usr those are the Linux system's
own: every header mingw-w64 lacks came from glibc, and libstdc++'s
<ext/atomicity.h>, finding <sys/single_threaded.h>, had shared_ptr read
__libc_single_threaded, which nothing on Windows defines. Every program
that used one failed to link, among them every Crafter.Graphics Win32
executable, through Crafter.Asset's GLB loader. CRAFTER_MINGW_DIR=/usr
did the same outside the Flatpak (#43).

The sysroot is now MingwSysroot(): a directory in the cache, one per
prefix, holding only links to the prefix's x86_64-w64-mingw32/, lib/gcc/
and lib64/gcc/ (where openSUSE keeps gcc). Through it clang finds
mingw-w64, gcc's version and libstdc++ exactly as it does through the
mingw gcc on PATH, and nothing else. It is made by the first mingw
compile, link or CMake configure of a process, so no other command
writes to the cache for it and a failure to make it is that build's
error. The Toolchain field is now mingwPrefix, since clang never gets
the prefix itself, and it is made absolute when the toolchain is
resolved, as it ends up in link targets and directory names. Messages
keep naming paths under the prefix rather than under the view.

The prefix is also part of the mingw target's VariantId and of its std
PCM directory. Units do not track the std PCM they import: when the
fixed release built in a directory a broken build had left, the std
module was rebuilt and the objects compiled against glibc's headers
were only relinked, failing as before. As with a compiler update, a new
directory is what gets them rebuilt. It also keeps two releases that
share a cache, as the Flatpak's branches do, from rebuilding one std PCM
under each other's PCMs, and two prefixes apart. Without a prefix both
stay as they were: clang's own discovery has not changed.

Two shorter ways do not work: --sysroot=<prefix>/x86_64-w64-mingw32
loses the directory gcc's version names, so <string> is not found, and
clang ignores --gcc-toolchain and --gcc-install-dir for this target.

Fixes #43

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:48:06 +01:00
909e027698
feat(ci): leave the install commands out of the release notes
All checks were successful
Flatpak / flatpak (push) Successful in 11m5s
Every release page ended in the two flatpak commands that install
crafter-build, which the top of the README already gives. The notes
now stop at their last section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 17:36:32 +01:00
7dd9785058
docs(packaging): add the 4.0.0 release entry
Some checks failed
Flatpak / flatpak (push) Has been cancelled
AGENTS.md written from a tracked AGENTS.template.md, so the dependency
notes no longer change a file the project commits. A major, because a
project that kept 3.0.0's notes section in its committed AGENTS.md
stops getting it updated until it moves to the template: the rule is a
major whenever the notes have a Breaking section, even one that no
build fails over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 17:27:31 +01:00
7678286f0d
Merge branch 'claude/agents-template': AGENTS.md written from a tracked AGENTS.template.md
All checks were successful
Flatpak / flatpak (push) Successful in 11m17s
Brings in the template: a project commits AGENTS.template.md, and
`crafter-build` and `crafter-build test` write a gitignored AGENTS.md
from it with the dependency notes merged in, so a change to a
dependency's notes no longer shows up in the consumer's diff. An
AGENTS.md no build wrote is never replaced, and without a template
AGENTS.md is left alone, with a warning when it still holds a 3.0.0
section. This repository moves its own AGENTS.md to the template.

The branch starts at master's tip, so the merge is a tree identical to
the branch's; that tree passes format --check, lint and the test suite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 16:56:27 +01:00
5c3e9a874b
docs: keep this repository's agent notes in AGENTS.template.md
This repository now uses the template itself. AGENTS.md is written by
the self-host build and `test` and is gitignored, so the notes
Crafter.Math ships reach the agents working here, and a change to them
no longer leaves a tracked file modified, which also made every build
afterwards a dirty-tree version, a new crafter.build-lib variant and a
full rebuild.

The gotcha that described the old arrangement now says where to edit,
that a fresh clone or worktree has no AGENTS.md until its first build
(./build.sh does not write it), and what an older harness does to the
generated file. The paths table and the never-commit list name both
files. The CI error for a missing signing key and a manifest comment
point at the template, since the forge no longer shows an AGENTS.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 16:56:11 +01:00
2a5f541485
docs: AGENTS.template.md in the README
The dependency notes section now starts from the template: what to
commit and what to ignore, the two commands that move a project with a
committed AGENTS.md over, what a build writes, where the section goes,
and what becomes of an AGENTS.md no build wrote and of one that still
holds a section from 3.0.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 16:53:42 +01:00
74e8370b2b
feat(agents): write AGENTS.md from a tracked AGENTS.template.md
The dependency notes were kept in a section of the AGENTS.md beside
project.cpp, the file a project commits, so a build could change a
tracked file at any time: a dependency editing its AGENTS.consumer.md,
an unpinned one moving on, a flag pulling a dependency in or out, each
showed up in the consumer's diff as a change nobody made. Users
reported it as the feature clobbering their diffs.

A project now commits its instructions as AGENTS.template.md, and a
build and a test run write AGENTS.md from it: a first line saying the
file is generated and where to edit, the template's text as written,
then the notes section, after it or where the template's marker lines
put it. AGENTS.md is build output and goes into .gitignore. It is
created when missing, since a fresh clone has none, and otherwise
written as before: only when its text changes, in place and under a
lock, so links, ownership and a read-only file are respected.

The first line is also how a build tells its own AGENTS.md from one
someone wrote. Beside a template the second is easily there, as a
committed AGENTS.md nobody has removed yet; it is never replaced, and
each build warns until it is deleted. An empty file counts as a
build's, since that is what a write cut short leaves of a new one. An
AGENTS.md linked to the template is refused: writing it would write
the committed file again. A template the notes cannot be placed in
leaves no empty AGENTS.md behind.

The in-place mode is removed rather than kept as the fallback for a
project without a template. It is the default that put generated text
into tracked files unasked, the moment any dependency started shipping
notes. Without a template a build no longer touches AGENTS.md. One
that still holds a section from 3.0.0 would go stale silently, under a
preamble promising that every build rewrites it, so it gets a warning
naming the move instead.

MergeDependencyNotes keeps its rules and now runs on the template. The
test changes in this commit rather than its own, because it pinned the
in-place behaviour this removes: it now covers the generated file's
shape, edits and markers in the template, the ownership check, a CRLF
template, a link to the template and the cases without one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 16:53:42 +01:00
b26807f5aa
docs(packaging): add the 3.0.0 release entry
All checks were successful
Flatpak / flatpak (push) Successful in 12m53s
C++ exceptions in wasm builds, scripts, dependency notes, the browser
runtime's exit and park, and mingw found where it is; the fixes since
2.2.0; and the two changes that need a look when upgrading: the wasm
sysroot and engine requirements, and siblings inheriting the loading
project's args. A major, because a wasm project.cpp with
-fno-exceptions or an older sysroot, and a project.cpp whose siblings
relied on getting no args, build differently or not at all on this
release.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 06:54:33 +01:00
1d52350b31
feat(ci): render the release notes as Markdown with headings and lists
The release page got the newest <release> block as the plain text of
its paragraphs, which fit two paragraphs of prose and nothing more.
From 3.0.0 the notes group the changes under New, Fixes and Breaking,
each in descending order of importance, and AppStream's description
markup has lists but no headings to carry that.

The metainfo stays the one source of the notes, since the Flatpak
installs it and the tag check reads it. A category is written there as
a paragraph that introduces a list, <p>New</p><ul>...</ul>, which is
how AppStream notes lead into a list anyway, and the CI step renders
such a paragraph as a Markdown heading over its items. A paragraph on
its own stays a paragraph, so the older entries render as before. The
step parses the XML with ElementTree instead of two regexes: entities
are decoded, <em> and <code> get their Markdown spelling, and a code
span keeps text like lib/<triple>/eh from being read as an HTML tag by
the forge.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-31 06:54:33 +01:00
70ff11d0a1
feat(wasm): link libsetjmp, so setjmp needs no flag
All checks were successful
Flatpak / flatpak (push) Successful in 17m58s
-mllvm -wasm-enable-sjlj, in every wasm compile because setjmp.h does
not compile without it, turns setjmp and longjmp into calls to
__wasm_setjmp, __wasm_setjmp_test and __wasm_longjmp and a throw of
the __c_longjmp tag, all of which live in wasi-libc's libsetjmp. The
driver does not link it, so a program that used setjmp, or a C library
that handles its errors that way, failed to link with undefined
__wasm_setjmp until the project put -lsetjmp in its linkFlags.

Build() adds it to every wasm link, next to -lunwind. A static archive
contributes only the members something references, so a program
without setjmp gets nothing from it, and every sysroot that passes the
exception check ships it. WasmExceptions drops its own -lsetjmp, and
its C fixture now fails to link without this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:29:52 +01:00
3e125d83f0
docs: wasm exceptions as the review left them
The README called an exception escaping main a trap. It is an uncaught
WebAssembly exception, and nothing prints its type or what(); the
README says so now. It also lists what a wasm build refuses and why,
says that setjmp needs -lsetjmp and that a longjmp across a noexcept
function ends the program, that a wasmtime older than 47 needs
-W exceptions=y, and that an old wasm-opt on PATH breaks release links.
It documents exit and park in a page from EnableWasiBrowserRuntime,
with the declaration of the park import.

The wasi example still sent Arch users to wasi-libc++, which has no
exception-enabled libraries. It asks for a wasi-sdk 33 sysroot or the
Flatpak now, and names the browsers a page needs. AGENTS.md notes that
the runtime test needs node, so it runs in the native loop and not in
CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:10:41 +01:00
2a05b8e6fb
test(wasm): the cases the review found
WasmBuildRefusals checks that -flto in compileFlags and in linkFlags,
-fno-exceptions and a shared library are each refused with their
cause, and that none of them leaves bin/ or build/ behind. With the
check disabled, each of them failed on a missing std.cppm instead,
after the build directories had been made.

WasiSysrootWithoutExceptions also takes its sysroot through missing
and without the target's directory, each with a message of its own,
checks that a refused build leaves nothing behind, and that a static
library passes without libunwind.

WasmExceptions catches an exception thrown inside libc++.a (stoi).
vector::at throws from a header compiled into the fixture, so the
fixture also passed against the libc++ built without exceptions. Its C
source uses setjmp, which the old C flags did not compile. An
exception that escapes main has to end the run as wasmtime's uncaught
exception, where any nonzero exit used to pass, a trap or a timeout
included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:10:41 +01:00
45b9f5cbb7
fix(wasm): what the review found
-flto in a wasm configuration's compileFlags or linkFlags reached the
link, where LTO code generation runs without the backend options that
keep exception handling: every landing pad was dropped, and the build
succeeded with throws that escaped their catch blocks. -fno-exceptions
there made every `import std;` fail on the std module built with
exceptions, and a wasm shared library failed to link against the eh/
libc++, which is static archives built without -fPIC. Build() refuses
all three now, each with a line that says why, before it makes a
directory or starts a thread.

The sysroot check sat in BuildStdPcm, which the linter calls as well.
There the error only dropped the compile command, so lint ran no AST
rule on a wasm configuration and reported it clean. It also refused
static libraries, which never link, fired after the shader and
external threads had started, and called a missing sysroot, or one
without the target, "no C++ exception support". It is part of the same
early check now, which first says when the sysroot or the target's
directory is missing, and asks for libunwind only where something
links.

C sources got only the signal.h define. setjmp.h refused to compile in
C, and a C object given -wasm-enable-sjlj through compileFlags used the
legacy exception encoding next to C++ objects in the standard one, a
mix engines refuse to load. They get WasmCompileFlags now, like the C++
units, a variant pass's flags included.

The baseline probes include simd: V8 and SpiderMonkey switch SIMD off
on x86 CPUs without SSE4.1, which virtual machines often present, and
such an engine passed the exception-handling probe only to fail
compiling the module; the manifest tests expect it on every entry. The
Flatpak pins WASI_SDK_EXCEPTIONS=DUAL, which
wasi-sdk 34 defaults to only with clang 23 or newer, and checks that
the eh/ libunwind got installed: the wasm tests skip without it, so CI
would otherwise pass a Flatpak whose every wasm build fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 06:10:41 +01:00
fec206839e
test(wasi-runtime): exit and park, under node
runtime.js had no test that runs it. WasiBrowserRuntimeExit runs the
real file under node, once per scenario, against a module assembled in
the harness. Between _start and the call that ends it the module has
the frames that caught the old exit: a destructor (catch_all_ref, then
throw_ref), a catch (...) and a noexcept function, each reporting when
it runs. An exit has to run none of them and record its status; a park
has to run none of them, nor the code after it, and leave the program
callable with its state intact; an exit from a later call has to end
the program too, a return from main has to give status 0, and a crash
or an exception nothing caught has to stay an error.

With the JS throw put back into proc_exit, the exit from main and the
exit from a later call both end in the noexcept frame's terminate.

It skips without node, or with a node whose engine lacks the
standardized exception handling. The Flatpak has no node, so CI does
not run it; the native loop does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:55:32 +01:00
0dabce0b74
fix(wasi-runtime): end exit and park without running the program
runtime.js implemented proc_exit by throwing a JavaScript error through
the wasm stack, and the switch to -fwasm-exceptions made that error
catchable: every cleanup pad, catch (...) and noexcept terminate pad
catches a foreign exception. An exit therefore ran the destructors of
every frame it left, a catch (...) swallowed it and carried on, and
under a noexcept frame it ended in std::terminate. Crafter.Graphics
calls std::_Exit(0) in Window::StartSync to hand the thread to
requestAnimationFrame; once the exit's destructors had torn down
main's locals, every later frame read destroyed state.

proc_exit is now a function of a small helper module, bound directly
as the program's import, that reports to the runtime and then executes
unreachable. A trap is the one thing wasm code cannot catch, and raised
in wasm rather than from JS it does not depend on each engine keeping
a trap uncatchable when it passes back through a JS frame. Nothing of
the program runs on the way out and its shadow stack stays as it was,
which is what exit and _Exit require. window.crafter_wasi.exitCode
holds the status, also after a return from main, so that bridges which
call into the program on their own can stop.

Handing the thread to the browser for good is a separate import,
crafter_runtime.park. It ends the call the same way but leaves the
program alive, and main's locals valid for the calls that follow, so
StartSync keeps its contract that no caller code after it runs. Until
Crafter.Graphics moves to park, its _Exit behaves as before minus the
destructors: the runtime records an exit but cannot stop a bridge that
calls exports itself.

A WASI reactor build, where main returns and state lives in statics,
was the alternative. It breaks StartSync's promise never to return,
which the native targets rely on. JSPI would keep the whole stack
alive, but Safari does not have it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:55:21 +01:00
18413bfae4
docs: wasm targets and their C++ exceptions
The README described the mingw target and said nothing about wasm. It
now says where the WASI sysroot comes from, that C++ exceptions work as
on the other targets and what that takes (-fwasm-exceptions in the
standardized form, the sysroot's exception-enabled libc++, -lunwind
added to the link), which engines run the result, and that an exception
escaping main ends the program without calling std::terminate.

AGENTS.md says where the wasm flags live, that VariantId and the std
PCM directory hash them, and that the wasm tests skip where the
distro's sysroot lacks the eh/ libraries, so they run in the Flatpak.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:55:00 +01:00
8862092b63
test(wasm): exceptions in a wasm build, and a sysroot without them
WasmExceptions builds a fixture for wasm32-wasip1 and runs it under
wasmtime. It throws through four frames whose destructors have to run,
catches the exception, catches one that vector::at throws, and with an
argument lets one escape main, which has to fail the run. With the
previous flags the fixture does not compile ("cannot use 'throw' with
exceptions disabled"), and each part of turning exceptions on fails it
in its own way, from a BMI built without them to the legacy encoding
wasmtime refuses.

WasiSysrootWithoutExceptions takes a scratch sysroot through three
states, no libunwind, libunwind under eh/ and libunwind in the
triple's own directory, and checks that only the first stops the
build, with the message naming the library it looked for. It needs no
working wasm toolchain, so it also runs where the others skip.

WasmHelloRuns and WasmExceptions treat a sysroot without the
exception-enabled libraries as missing and skip. The manifest tests
expect the exception-handling probe on every entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:48 +01:00
63c4040000
feat(wasm): compile C++ with WebAssembly exception handling
wasm builds were compiled with -fno-exceptions. A throw did not
compile, libc++'s own throws (vector::at, std::format, filesystem)
aborted the program, and code that relies on exceptions could not be
built for the browser at all.

Every wasm unit and the std module now get -fwasm-exceptions, and the
link adds -lunwind, which code built that way calls into and the driver
does not add. The flag also selects the libc++ under the sysroot's
lib/<triple>/eh, which wasi-sdk 33 and later ship next to the noeh
one. Those libraries use the standardized encoding (try_table, exnref)
while LLVM 23 still emits the legacy one by default, which wasmtime
refuses to load and which would put both in one binary, so the flags
ask for the standard form with -wasm-use-legacy-eh=false. That makes
the setjmp lowering loadable under wasmtime too.

The flags live in one constant, WasmBaselineFlags, which the compile
command and the std module both take through WasmCompileFlags: a BMI
is refused by a unit whose exception setting differs from its own.
VariantId and StdPcmDir hash it, so the first build after the update
starts fresh instead of importing PCMs built without exceptions, and
two releases sharing the Flatpak's cache don't take turns rebuilding
one std PCM. A sysroot without the exception-enabled libraries stops
the build with a line saying so, rather than at the link with "unable
to find library -lunwind".

The result needs an engine with exnref: Chrome 137, Firefox 131,
Safari 18.4, the bundled wasmtime. The runtime's exception-handling
probe tested the legacy encoding; it now tests try_table, every
variants.json entry carries it, and a browser without it is told which
feature it lacks instead of failing with a CompileError. There is no
switch back to -fno-exceptions: engines without exnref are old enough
not to keep a second build for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:54:39 +01:00
eefd1df3eb
Merge branch 'claude/issue-41': refuse a build that holds two builds of one module
All checks were successful
Flatpak / flatpak (push) Successful in 12m49s
PR #42, fixes #41. Build() now refuses a configuration whose transitive
dependencies hold two configurations exporting a module of the same
name, such as one project loaded along two paths with different arg
lists, before it builds anything, and names both paths. Until now both
builds ended up in the one program, which compiled cleanly and then
read garbage. Merged cleanly on top of claude/forward-project-args;
the suite, with claude/dependency-notes' fixtures, passes under the
new check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:30:36 +01:00
75142276ab
Merge branch 'claude/forward-project-args': sibling projects inherit the args, lint with the lite rules
PR #40. GitProject and LocalProject default their args to the loading
project's own (ProjectArgs()), this tree lints with LintRules' lite
set, and the fixes from the branch's review: a project file gets its
args only for the CrafterBuildProject call, host tooling gets project
cache entries of its own, and `--` ends crafter-build's options.

The conflicts with claude/dependency-notes are all places both sides
added to:
- Platform.cppm and Platform.cpp: PragmaDependencies() beside
  ProjectArgs() and LoadingHostTooling(), all three kept.
- LoadProject: the record PragmaDependencies() reads moves up with the
  frame's dependencies, which this branch sets before dlopen. Left
  where it was, it would have copied preDeps.configs after the vector
  was moved into the frame, and recorded nothing.
- README: the pragma section's new last sentence goes before the new
  "Dependency notes for agents" section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:27:37 +01:00
cc0a6aed8c
Merge branch 'claude/dependency-notes': a library's AGENTS.consumer.md in its consumers' AGENTS.md
All checks were successful
Flatpak / flatpak (push) Successful in 12m18s
Brings in dependency notes: a crafter-build project that ships an
AGENTS.consumer.md beside its project.cpp has it kept in a marked
section of each consumer's AGENTS.md, refreshed by `crafter-build` and
`crafter-build test`, chosen by cfg.dependencyNotes (Transitive,
Direct, None). The new partition is Crafter.Build:Agents; the loader
records where each GitProject and LocalProject came from and what a
project file's pragmas resolved to.

The branch started before master's four external-fetch commits. Both
touch External.cpp, the Clang partition, project.cpp, README.md and
AGENTS.md, but in different places, so the merge needed no hand
reconciliation; the merged tree passes build.sh, the self-host build,
format --check and the test suite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:08:02 +01:00
ae5e3b6b32
docs: dependency notes as the review left them
The README's section: notes are labelled by name, the file is written
in place, a project.cpp that fails to load gets no update, `test
--list` writes nothing, the section follows what the build loaded, and
tests' dependencies don't count. The AGENTS.md gotcha: an unpinned
dependency can change this file's section on any build here, and the
dirty tree then rebuilds crafter.build-lib under a new version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:02:36 +01:00
ec9790589f
test(agents): the cases the review found
Each fixed finding gets a check that fails without its fix: inline code
and a list item's fence, a section removed from between two paragraphs,
a section before an unclosed fence, a CRLF file, no error when there is
nothing to write, a hard-linked and a read-only AGENTS.md, a dependency
whose project file sits in the consumer's own directory, a test's
dependency, and `test --list`. The read-only case needs an ordinary
user, since root writes through the permission bits, and is skipped
under root.

The merge cases go through a helper that counts a throw as a failure,
so one regression no longer ends the run before the cases after it. A
passing run removes its staged copy, whose builds leave a std PCM per
configuration in a tmpfs.

Reverting the inline-code rule, the own-notes exclusion, the skipping
of tests, the `test --list` guard or the paragraph-preserving removal
each fails its check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:02:36 +01:00
7815856d60
fix(agents): what the review of dependency notes found
A review of the feature turned up the following, fixed here.

Writing. AGENTS.md was replaced through a temporary renamed over it,
which swaps in another file: a read-only AGENTS.md was rewritten
anyway, a hard link such as a CLAUDE.md beside it kept the old text,
ACLs and ownership were dropped, and without an fsync a power cut could
still leave it empty. The temporary was named after the PID, which
every Flatpak instance shares as PID 2. The file is now written in
place, from the first byte that differs, then truncated and synced,
under an flock on the file itself. A write that fails partway is undone
where the disk allows; with the section last, where it is appended,
only the section can be damaged. A read-only file is opened for writing
only when something would change, so it is refused then and silent
otherwise.

Reading. libc++'s filebuf reports a failed read as the end of the file,
so an I/O error came back as a short file and the section was written
over what was left. Reads go through read(2).

Code fences. ```make``` was taken for a fence, and a fence in a list
item was missed while its indented closer opened one, so markers a user
quoted in a code block could be taken for the section, or the real one
hidden and appended again. Backtick info strings, block quotes and list
items are now followed, and a block ends with its container.

Warnings with nothing to write. With no notes, nothing short of a
well-formed section is reported, so an AGENTS.md that ends in an open
code block, or a project set to None, no longer warns on every build.
An unclosed fence after a section no longer stops it being rewritten
in place.

Labels. The `from` line carried the git URL, credentials and all, or a
path relative to the consumer that changed with the machine, the cache
location and --local. It is now the repository's or the directory's
name. Notes are deduplicated by repository rather than by clone, so one
library pinned two ways is described once, as the nearer spec has it.

The walk. It no longer looks at the project's tests: a plain build
never did, and a test's dependency can point at the Configuration the
project file returned by value. It goes ring by ring over projects
rather than configurations, so what the project names itself comes
first even with the library/executable split. The project's own
AGENTS.consumer.md is excluded by path, also when a dependency's
project file sits beside the project's.

Removing a section dropped a blank line even from between two
paragraphs, joining them; only a section that ends the file takes it
now. A CRLF file gets a CRLF section.

A note was checked on its raw bytes and again, under other whitespace
rules, on the trimmed ones, so it could pass the first check and fail
the merge, which then dropped every note. It is normalised once and
checked on exactly what is merged.

`test --list` only lists, and no longer writes AGENTS.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:02:28 +01:00
c2e8bf0a93
docs: dependency notes in the README
What a library ships and where, what a consumer's AGENTS.md gets and
when it is written, the section's format, and what cfg.dependencyNotes
selects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:26 +01:00
eb1c3a4b1d
test(agents): what a consumer's AGENTS.md gets, and what stays put
The merge on its own: the section is appended after one blank line,
merging again changes nothing, a moved section is rewritten where it
stands, and removing it gives the file its own bytes back. Markers
inside a fenced block, one quoting a fenced block of its own, are not
the section's. Unbalanced markers, a document that ends inside a code
block and notes that would break the section are refused, naming the
line.

End to end, through LoadProject on a staged fixture: app names lib
and has tool as host tooling; lib builds against base and has libtool
as its own. Transitive gives tool, lib and base in that order, and
neither app's own notes nor libtool's. A second update writes nothing,
mtime included. Direct drops base, edited notes replace the old ones
in the one section, and None restores the original bytes. A project
without an AGENTS.md does not get one, and a symlinked AGENTS.md is
written through the link, which stays a link. Through Run in process,
lint and run leave the file alone, while `test --list` and a build
that fails at the link both refresh it.

The fixture's AGENTS.md is checked in as AGENTS.md.in: an AGENTS.md in
this tree would read as instructions for the agents working on it.

Moving the CLI hook above the run and lint branches, or letting Direct
walk past the first loaded project, fails four of the checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:21 +01:00
e87fb0cc12
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md
An agent working on a crafter-build project reads its AGENTS.md, and
learns nothing there about the libraries the project builds against
unless someone copied that in by hand. This repository shows the cost:
the code style section of its AGENTS.md restates what Crafter.LintRules
enforces, and goes stale whenever the rules change.

A library now ships those notes itself, as AGENTS.consumer.md beside
its project.cpp; nothing in its project.cpp changes. When a consumer
has an AGENTS.md beside its own project.cpp, `crafter-build` and
`crafter-build test` keep a section in it, between two marker comments,
holding its dependencies' notes. The section is appended the first time
and rewritten in place after that. The file is written only when the
section changes, and an AGENTS.md is never created. lint, format, run
and clean leave it alone. The update runs after loading, which has just
fetched every dependency, and before compiling, so a build that fails
still has the notes.

cfg.dependencyNotes picks the dependencies. Transitive, the default,
follows `dependencies` all the way down from the project's own
configurations, its tests and its `#pragma crafter dependency` tooling;
Direct stops at the first loaded project on each path; None removes
the section. A dependency's own pragma tooling never counts: it is
about how that dependency is built, not how it is used, and being no
`dependencies` edge it is never reached. Nor are a dependency's tests:
nothing builds them for the consumer, and their configurations can
point at a local their project file has since moved out of.

The walk has to know which Configurations belong to another project,
and where that project lives, and cfg.path cannot say: only the
Configuration GitProject and LocalProject return is rebased, so a
dependency's internal configurations keep "./", which resolves to the
consumer. The loader records it instead, LoadedProjectOf beside the
project cache, and keeps what the root's pragmas resolved to past the
frame LoadProject pops, PragmaDependencies. Neither is exported.

The markers count only as whole lines outside fenced code blocks, so
documentation can quote them. Unbalanced markers and a document ending
inside a code block leave the file alone with a warning: guessing which
part of someone's file is generated would overwrite their text, and
missing the section would add another one on every build. A note that
would break the section when it is read back, through a marker line of
its own or a code block it never closes, is left out with a warning.
The file is replaced through a temporary renamed onto the symlink's
target, so a linked AGENTS.md stays a link and a write cut short leaves
the old file rather than part of the new one.

The code is a new partition, Crafter.Build:Agents, listed in build.sh,
CrafterBuildModules, project.cpp and the umbrella.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
66 changed files with 3726 additions and 274 deletions

View file

@ -1,37 +1,52 @@
name: Flatpak name: Flatpak
# One job, three outcomes: # One job, two outcomes:
# pull request -> build the Flatpak and run the suite inside it # pull request, manual run -> build the Flatpak and run the suite inside it
# push master -> also publish the result as the `beta` branch of the # pushed tag X.Y.Z -> also release the build: publish it as the
# Catcrafts repo and refresh the rolling `latest` release # `stable` branch of the Catcrafts repo and
# tag X.Y.Z -> publish as `stable` and create release X.Y.Z # create release X.Y.Z
#
# A release is a tag that the owner makes and signs locally and pushes with
# the master it releases; a push to master without one runs nothing. The tag
# names the version, and the metainfo's newest <release> entry has to be that
# version: its description is the release notes.
# #
# The `crafter-flatpak` runner label is the privileged runner on the Hetzner # The `crafter-flatpak` runner label is the privileged runner on the Hetzner
# box (bubblewrap needs it) with four volumes allowed: the Sdk install, the # box (bubblewrap needs it) with four volumes allowed: the Sdk install, the
# flatpak-builder state (ccache and the LLVM module, so only crafter-build # flatpak-builder state (ccache and the LLVM module, so only crafter-build
# itself rebuilds on a normal run), the published repo's web root, and the # itself rebuilds on a normal run), the published repo's web root, and the
# repo signing key, which lives on the runner host and never in the forge. # repo signing key, which lives on the runner host and never in the forge.
# No other runner carries that label. Plain `flatpak` also matches the # No other runner carries that label. Plain `flatpak` is the label of the
# organisation's runner, which mounts none of those volumes; a job it picks # organisation's runner, which serves every Catcrafts repository and also
# up fails before building anything. # mounts the web root and the key, for Crafter.Review's publish job, so any
# job it runs can mount them (AGENTS.template.md, CI and publishing).
on: on:
pull_request: pull_request:
branches: [master] branches: [master]
push: push:
branches: [master]
# Bare semver tags. Spelled with globs so it means the same whether the # Bare semver tags. Spelled with globs so it means the same whether the
# forge reads it as a workflow pattern or as a plain glob; `latest`, the # forge reads it as a workflow pattern or as a plain glob.
# rolling tag this workflow moves itself, does not start with a digit.
tags: ['[0-9]*.[0-9]*.[0-9]*'] tags: ['[0-9]*.[0-9]*.[0-9]*']
workflow_dispatch: workflow_dispatch:
# Pushes are tags only, so every release run is in one group, and a release
# starts only once the one before it has finished. A pull request's group
# also holds its number, which only a pull request's event carries, so a
# new push to it cancels its own outdated run and no other.
concurrency:
group: flatpak-${{ github.event_name }}-${{ github.event.pull_request.number }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env: env:
APP_ID: net.catcrafts.CrafterBuild APP_ID: net.catcrafts.CrafterBuild
BRANCH: stable
MANIFEST: packaging/flatpak/net.catcrafts.CrafterBuild.yaml MANIFEST: packaging/flatpak/net.catcrafts.CrafterBuild.yaml
CI_MANIFEST: packaging/flatpak/ci-manifest.yaml CI_MANIFEST: packaging/flatpak/ci-manifest.yaml
RUNTIME: org.freedesktop.Sdk//26.08 RUNTIME: org.freedesktop.Sdk//26.08
REPO_DIR: /srv/flatpak/repo REPO_DIR: /srv/flatpak/repo
WEB_DIR: /srv/flatpak WEB_DIR: /srv/flatpak
# A release tag carries a signature Forgejo verifies for this address.
RELEASE_SIGNER: jorijnvdgraaf@catcrafts.net
jobs: jobs:
flatpak: flatpak:
@ -45,11 +60,12 @@ jobs:
-v /etc/crafter-flatpak:/run/crafter-flatpak:ro -v /etc/crafter-flatpak:/run/crafter-flatpak:ro
steps: steps:
- name: Install tools - name: Install tools
# util-linux: flock, for the Publish step.
run: | run: |
pacman-key --init pacman-key --init
pacman-key --populate archlinux pacman-key --populate archlinux
pacman -Sy --noconfirm --needed archlinux-keyring pacman -Sy --noconfirm --needed archlinux-keyring
pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python pacman -Syu --noconfirm --needed flatpak flatpak-builder ccache git nodejs jq gnupg go-yq ostree python util-linux
git config --global --add safe.directory '*' git config --global --add safe.directory '*'
- name: Sdk - name: Sdk
@ -60,40 +76,137 @@ jobs:
flatpak update -y --noninteractive flatpak update -y --noninteractive
- name: Checkout - name: Checkout
# Every branch and tag, for git describe and the release checks; no
# step pushes, so the token is not kept in the checkout.
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: true persist-credentials: false
- name: Version and channel - name: Version
id: meta id: meta
run: | run: |
set -eu set -eu
if [ "${GITHUB_REF_TYPE}" = tag ]; then # The metainfo's newest entry is its first <release> element, the
version="$GITHUB_REF_NAME" # one parse that gives both its version and the release notes,
branch=stable # before anything is built: a comment or another attribute order
# The metainfo's newest release is the release notes and must be # would mislead a text search, and an entry whose notes render
# the version being tagged. # empty stops the run here. The notes go to release-notes.md, for
top=$(grep -m1 -oP '<release version="\K[^"]+' packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml) # the release page, as Markdown. AppStream has no headings, so a
if [ "$top" != "$version" ]; then # paragraph that introduces a list is one: <p>New</p><ul>...</ul>
echo "metainfo lists $top as the newest release, but the tag is $version" >&2 # comes out as "## New" over its items, the shape the notes take
# since 3.0.0. A paragraph on its own stays a paragraph, as the
# older entries are written. <em> and <code> get their Markdown
# spelling; a code span also keeps text such as lib/<triple>/eh
# from being read as an HTML tag by the forge.
newest=$(python3 - <<'EOF'
import sys
import xml.etree.ElementTree as ET
release = ET.parse("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").getroot().find("releases/release")
desc = None if release is None else release.find("description")
def text(el):
parts = [el.text or ""]
for child in el:
inner = text(child)
parts.append({"em": f"*{inner}*", "code": f"`{inner}`"}.get(child.tag, inner))
parts.append(child.tail or "")
return " ".join("".join(parts).split())
blocks = [] if desc is None else list(desc)
out = []
for i, el in enumerate(blocks):
heads_list = i + 1 < len(blocks) and blocks[i + 1].tag in ("ul", "ol")
if el.tag == "p":
out.append(f"## {text(el)}" if heads_list else text(el))
elif el.tag == "ul":
out.append("\n".join(f"- {text(li)}" for li in el))
elif el.tag == "ol":
out.append("\n".join(f"{n}. {text(li)}" for n, li in enumerate(el, 1)))
notes = "\n\n".join(out)
if not notes.strip():
sys.exit("the metainfo needs a <release> at the top of <releases> whose <description> renders release notes")
with open("release-notes.md", "w") as f:
print(notes, file=f)
print(release.get("version", ""))
EOF
)
if [ "$GITHUB_EVENT_NAME" = push ]; then
# A release: the only pushes this workflow runs for are tags.
# The tag is the version; it has to be annotated, signed by the
# owner as Forgejo verifies it against the keys its users added,
# on master, newer than every published release, and the version
# of the metainfo's newest entry. The tag itself is read from the
# forge: the checkout fetches a pushed tag again as a plain ref
# to its commit, so the local one is never the annotated tag.
version=$GITHUB_REF_NAME
if ! printf '%s\n' "$version" | grep -qxE '[0-9]+\.[0-9]+\.[0-9]+'; then
echo "the tag $version is not a version of the form X.Y.Z" >&2
exit 1
fi
api="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY"
tag=$(curl -sSf "$api/tags/$version")
object=$(printf '%s' "$tag" | jq -r .id)
commit=$(printf '%s' "$tag" | jq -r .commit.sha)
if [ "$commit" != "$GITHUB_SHA" ]; then
echo "the forge has the tag $version on $commit, not on this run's commit $GITHUB_SHA" >&2
exit 1
fi
if [ "$object" = "$commit" ]; then
echo "$version is a lightweight tag; a release tag is annotated and signed: git tag -s $version -m 'crafter-build $version'" >&2
exit 1
fi
# Forgejo verifies a signed tag's own signature, and for an
# unsigned one falls back to its commit's, so the verdict also
# has to cover the tag's payload, not the commit's.
signed=$(curl -sSf "$api/git/tags/$object" |
jq -r --arg c "$commit" --arg v "$version" '"\(.verification.verified) \(.verification.signer.email // "") \(.verification.payload // "" | startswith("object \($c)\ntype commit\ntag \($v)\n"))"')
if [ "$signed" != "true $RELEASE_SIGNER true" ]; then
echo "Forgejo does not verify the tag $version itself as signed by $RELEASE_SIGNER (verified, signer, over the tag's payload: $signed)" >&2
exit 1
fi
if ! git merge-base --is-ancestor "$GITHUB_SHA" origin/master; then
echo "the tag $version is on $(git rev-parse --short "$GITHUB_SHA"), which is not on master: push the tag with the master it releases" >&2
exit 1
fi
# Newer than every release the forge has published, its own
# included, and not every tag: a later tag whose run still
# queues behind this one has no release yet, and a tag that is
# no X.Y.Z version never gets one.
for t in $(git tag --list | awk '/^[0-9]+\.[0-9]+\.[0-9]+$/' | sort -V); do
[ "$(printf '%s\n%s\n' "$t" "$version" | sort -V | tail -n 1)" = "$t" ] || continue
code=$(curl -s -o /dev/null -w '%{http_code}' "$api/releases/tags/$t")
case "$code" in
404) ;;
200)
if [ "$t" = "$version" ]; then
echo "$version is released already; to re-run after a release left half made, delete that release first" >&2
else
echo "$version is not newer than the release $t" >&2
fi
exit 1 ;;
*) echo "the forge answered $code when asked whether $t is released" >&2; exit 1 ;;
esac
done
if [ "$newest" != "$version" ]; then
echo "the metainfo's newest release is \"$newest\", but the tag is $version: the tagged commit needs the <release> entry for $version at the top of packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml" >&2
exit 1 exit 1
fi fi
else else
# A master build names itself after the release it follows, the # Nothing is published. The build names itself after the
# way a local checkout does: 2.0.0-3-g21746b1 is three commits # release it follows, the way a local checkout does:
# past 2.0.0. Before the first tag there is nothing to describe # 5.0.0-3-g21746b1 is three commits past 5.0.0. Before the first
# from and the bare hash gets a placeholder in front. # tag there is nothing to describe from and the bare hash gets a
described=$(git describe --tags --always --exclude=latest) # placeholder in front.
described=$(git describe --tags --always)
case "$described" in case "$described" in
[0-9]*.[0-9]*.[0-9]*) version="$described" ;; [0-9]*.[0-9]*.[0-9]*) version="$described" ;;
*) version="0.0.0-dev+$described" ;; *) version="0.0.0-dev+$described" ;;
esac esac
branch=beta
fi fi
echo "version=$version" >> "$GITHUB_OUTPUT" echo "version=$version" >> "$GITHUB_OUTPUT"
echo "branch=$branch" >> "$GITHUB_OUTPUT" echo "version $version"
echo "version $version, flatpak branch $branch"
- name: Build - name: Build
# The manifest is rewritten beside the original so its relative # The manifest is rewritten beside the original so its relative
@ -106,7 +219,7 @@ jobs:
grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST" grep -n 'CRAFTER_BUILD_VERSION' "$CI_MANIFEST"
flatpak-builder --force-clean --ccache --disable-rofiles-fuse \ flatpak-builder --force-clean --ccache --disable-rofiles-fuse \
--state-dir=/root/.cache/flatpak-builder \ --state-dir=/root/.cache/flatpak-builder \
--repo=repo --default-branch="${{ steps.meta.outputs.branch }}" \ --repo=repo --default-branch="$BRANCH" \
build-dir "$CI_MANIFEST" build-dir "$CI_MANIFEST"
- name: Smoke test - name: Smoke test
@ -114,18 +227,16 @@ jobs:
flatpak-builder --run build-dir "$CI_MANIFEST" crafter-build --version flatpak-builder --run build-dir "$CI_MANIFEST" crafter-build --version
flatpak-builder --run build-dir "$CI_MANIFEST" sh -c 'cd examples/hello-world && crafter-build && ls bin/*/hello*' flatpak-builder --run build-dir "$CI_MANIFEST" sh -c 'cd examples/hello-world && crafter-build && ls bin/*/hello*'
# Everything below publishes and only runs on master and on tags. # Everything below releases the build and runs only for a pushed tag.
- name: Bundle - name: Bundle
if: github.event_name != 'pull_request' if: github.event_name == 'push'
run: | run: |
mkdir -p dist mkdir -p dist
flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "${{ steps.meta.outputs.branch }}" flatpak build-bundle repo "dist/crafter-build-${{ steps.meta.outputs.version }}.flatpak" "$APP_ID" "$BRANCH"
cp packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref dist/
(cd dist && sha256sum * > SHA256SUMS)
ls -la dist ls -la dist
- name: Upload bundle - name: Upload bundle
if: github.event_name != 'pull_request' if: github.event_name == 'push'
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: crafter-build-${{ steps.meta.outputs.version }} name: crafter-build-${{ steps.meta.outputs.version }}
@ -133,19 +244,24 @@ jobs:
if-no-files-found: error if-no-files-found: error
- name: Publish to the Catcrafts repo - name: Publish to the Catcrafts repo
if: github.event_name != 'pull_request' if: github.event_name == 'push'
run: | run: |
set -eu set -eu
# The signing key is generated once on the runner host # The signing key is generated once on the runner host
# (/etc/crafter-flatpak: signing-key.asc and key-id) and mounted # (/etc/crafter-flatpak: signing-key.asc and key-id) and mounted
# read-only; it is imported into a throwaway keyring per run. # read-only; it is imported into a throwaway keyring per run.
if [ ! -s /run/crafter-flatpak/signing-key.asc ] || [ ! -s /run/crafter-flatpak/key-id ]; then if [ ! -s /run/crafter-flatpak/signing-key.asc ] || [ ! -s /run/crafter-flatpak/key-id ]; then
echo "no signing key under /etc/crafter-flatpak on the runner host; generate it as described in AGENTS.md (CI and publishing)" >&2 echo "no signing key under /etc/crafter-flatpak on the runner host; generate it as described in AGENTS.template.md (CI and publishing)" >&2
exit 1 exit 1
fi fi
export GNUPGHOME=$(mktemp -d) export GNUPGHOME=$(mktemp -d)
gpg --batch --quiet --import /run/crafter-flatpak/signing-key.asc gpg --batch --quiet --import /run/crafter-flatpak/signing-key.asc
FLATPAK_GPG_KEY_ID=$(cat /run/crafter-flatpak/key-id) FLATPAK_GPG_KEY_ID=$(cat /run/crafter-flatpak/key-id)
# Crafter.Review's workflow publishes into the same repo and holds
# this lock while it does, so a commit and the summary update of
# one workflow never interleave with the other's.
exec 9> "$WEB_DIR/.publish.lock"
flock 9
# The live repo is the canonical copy; builds are promoted into it, # The live repo is the canonical copy; builds are promoted into it,
# never built into it. # never built into it.
# Keyed on the repo's config file, not the directory: the web root # Keyed on the repo's config file, not the directory: the web root
@ -153,56 +269,49 @@ jobs:
[ -f "$REPO_DIR/config" ] || ostree init --repo="$REPO_DIR" --mode=archive [ -f "$REPO_DIR/config" ] || ostree init --repo="$REPO_DIR" --mode=archive
flatpak build-commit-from --src-repo=repo \ flatpak build-commit-from --src-repo=repo \
--gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
"$REPO_DIR" "app/$APP_ID/x86_64/${{ steps.meta.outputs.branch }}" "$REPO_DIR" "app/$APP_ID/x86_64/$BRANCH"
# Every push to master used to be published as `beta`. Its last
# build is marked end-of-life, once, so that the next
# `flatpak update` of a beta installation says why nothing follows.
# The reason names the branch: with --noninteractive, flatpak's
# message names only the app.
beta="app/$APP_ID/x86_64/beta"
if ostree rev-parse --repo="$REPO_DIR" "$beta" >/dev/null 2>&1 &&
! ostree show --repo="$REPO_DIR" --print-metadata-key=ostree.endoflife "$beta" >/dev/null 2>&1; then
flatpak build-commit-from --src-ref="$beta" \
--end-of-life="The beta branch gets no more updates: every change now comes as a release on the stable branch. Install stable with flatpak install catcrafts net.catcrafts.CrafterBuild//stable, then remove beta with flatpak uninstall net.catcrafts.CrafterBuild//beta." \
--gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
"$REPO_DIR" "$beta"
fi
flatpak build-update-repo --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \ flatpak build-update-repo --gpg-sign="$FLATPAK_GPG_KEY_ID" --gpg-homedir="$GNUPGHOME" \
--generate-static-deltas --prune "$REPO_DIR" --generate-static-deltas --prune "$REPO_DIR"
# The repo description and the app ref carry the public key. # The repo description and the app ref carry the public key.
key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0) key=$(gpg --export "$FLATPAK_GPG_KEY_ID" | base64 --wrap=0)
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/catcrafts.flatpakrepo > "$WEB_DIR/catcrafts.flatpakrepo"
sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" sed "s|GPGKey=.*|GPGKey=$key|" packaging/flatpak/net.catcrafts.CrafterBuild.flatpakref > "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref"
flock -u 9
# The release attaches this .flatpakref, not the one in the
# checkout: that one has a placeholder where the key goes, and
# flatpak cannot install from it.
cp "$WEB_DIR/net.catcrafts.CrafterBuild.flatpakref" dist/
(cd dist && sha256sum * > SHA256SUMS)
ls -la dist
- name: Rolling 'latest' release
if: github.event_name != 'pull_request' && github.ref_type != 'tag'
run: |
git config user.email "ci@catcrafts.net"
git config user.name "Crafter Build CI"
git tag -f latest
git push origin latest --force
- name: Publish rolling 'latest' release
if: github.event_name != 'pull_request' && github.ref_type != 'tag'
uses: https://code.forgejo.org/actions/forgejo-release@v2
with:
direction: upload
url: ${{ github.server_url }}
repo: ${{ github.repository }}
tag: latest
title: Latest master build
prerelease: true
override: true
release-dir: dist
token: ${{ secrets.GITHUB_TOKEN }}
- name: Release notes
if: github.ref_type == 'tag'
run: |
# The newest <release> block's description, as plain text.
python3 - > release-notes.md <<'EOF'
import re, html
text = open("packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml").read()
block = re.search(r"<release [^>]*>(.*?)</release>", text, re.S).group(1)
paras = re.findall(r"<p>(.*?)</p>", block, re.S)
print("\n\n".join(html.unescape(" ".join(p.split())) for p in paras))
print("\nInstall:\n\n flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo\n flatpak install catcrafts net.catcrafts.CrafterBuild\n")
EOF
- name: Publish release - name: Publish release
if: github.ref_type == 'tag' if: github.event_name == 'push'
# The notes are release-notes.md, which the Version step wrote. No
# `override`: it would delete a tag that has moved off this commit
# since the Version step and tag the old commit instead. Without it
# the action refuses such a tag, and a release that an earlier
# attempt left half made, such as a draft whose upload failed, has
# to be deleted before a re-run.
uses: https://code.forgejo.org/actions/forgejo-release@v2 uses: https://code.forgejo.org/actions/forgejo-release@v2
with: with:
direction: upload direction: upload
url: ${{ github.server_url }} url: ${{ github.server_url }}
repo: ${{ github.repository }} repo: ${{ github.repository }}
tag: ${{ github.ref_name }} tag: ${{ steps.meta.outputs.version }}
title: crafter-build ${{ github.ref_name }} title: crafter-build ${{ steps.meta.outputs.version }}
release-notes-file: release-notes.md release-notes-file: release-notes.md
release-dir: dist release-dir: dist
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}

4
.gitignore vendored
View file

@ -5,3 +5,7 @@ share/crafter-build/
.claude .claude
# written by CI next to the manifest so relative source paths still resolve # written by CI next to the manifest so relative source paths still resolve
packaging/flatpak/ci-manifest.yaml packaging/flatpak/ci-manifest.yaml
# written by every build and test run from AGENTS.template.md
/AGENTS.md
.review-gates/push/
.review-gates/trees/

38
.review-gates/config.json Normal file
View file

@ -0,0 +1,38 @@
{
"base": "origin/master",
"rangeFrom": "upstream",
"authors": [
"jorijnvdgraaf@catcrafts.net"
],
"aiTrailers": "forbid",
"requireSigned": true,
"lenses": {
"repo-better-solution": {
"section": ".review-gates/lenses/repo-better-solution.md"
},
"repo-bug-prover": {
"section": ".review-gates/lenses/repo-bug-prover.md"
},
"repo-facts": {
"section": ".review-gates/lenses/repo-facts.md"
}
},
"modes": {
"commits": {
"kind": "commits",
"lenses": [
"repo-better-solution",
"repo-bug-prover",
"repo-facts"
],
"effort": "xhigh"
}
},
"gate0": [
"./build.sh",
"./bin/crafter-build",
"./bin/crafter-build test",
"./bin/crafter-build format --check",
"./bin/crafter-build lint"
]
}

View file

@ -0,0 +1,42 @@
## This repository: Crafter.Build
A C++26 modules build system that builds itself. The stated design is
`README.md` (the manual) and `AGENTS.template.md` (the contributor guide;
`AGENTS.md` is written from it by a crafter-build build or test run, not
by `build.sh`, and is not in the tree).
Its "Design rules" and "Security considerations" are the owner's
decisions: a range that departs from one without saying so is a finding;
whether one should change is the owner's call, not a defect of the range.
Among them, for CI, which holds the workflow's token and the signing key
its runner mounts: no secret is added to the workflow, and the
environment is never echoed in it.
The layers: the library (`interfaces/`, `implementations/`), which the
executable wraps and every `project.cpp` is compiled against; this
repository's own `project.cpp`; `build.sh`, the bootstrap that builds the
tool without the tool; the Flatpak (`packaging/flatpak/`), which bundles
the LLVM users build with; CI (`.forgejo/workflows/ci.yaml`). A target
difference branches on `cfg.target` at run time, never on the host; an
output path goes through `BinDir()`, `BuildDir()` or `PcmDir()`; whatever
changes what gets compiled or bundled reaches `VariantId()`.
Search first: the partitions' own helpers (`RunBuildCommand`,
`RunCommandChecked`, `Progress::Diagnostic`, `ApplyStandardArgs`,
`HostTarget`, `GitProject`, `LocalProject`); the Crafter.* libraries the
tree already uses, cloned under `~/.cache/crafter.build/external/`; libc++
and clang as the host installs them (`/run/host/usr/include/c++/v1`,
`/run/host/usr/lib/clang/`); git, cmake and flatpak-builder.
Commit shape here: a fix, its test and its README change are separate
commits when each stands on its own, in that order, and a bug fix ships
with a test that fails before it. That split is not a later commit
repairing an earlier one.
Compatibility: the `project.cpp` of every sibling repository beside this
checkout (`../Crafter.*/project.cpp`) and of every user compiles against
what `interfaces/` exports and is loaded by the binary; the CLI, the
`CRAFTER_BUILD_*` variables, the cache an older version left under
`~/.cache/crafter.build/` and the `bin/` layout CI globs are contracts too.
Users get master through releases on the Flatpak's `stable` branch. A
change an existing `project.cpp` does not survive is a
finding unless the range documents, in the README, what users must change.

View file

@ -0,0 +1,46 @@
## This repository: Crafter.Build
The execution contexts to enumerate, beside the charter's:
- `project.cpp` is compiled to a shared object and loaded into the running
crafter-build, resolving `Crafter::` symbols from the executable
(`--export-dynamic`); its static initialisers run at load. Its sibling
projects (`GitProject`, `LocalProject`) load the same way into the same
process, and its `#pragma crafter dependency` targets are built for the
host first, on every load.
- The clang++, clang and llvm-ar the tool spawns, and the git and cmake it
runs, are shell commands (`popen`, `std::system` in `:Platform`), many in
parallel; such a step fails on its exit code, never because it printed.
Running a test or a built artifact is a shell command too (`:Test`,
`:Clang`). A path, file name or flag that reaches one with a space or a
shell metacharacter in it is an input to attack. Other steps run inside
the process, among them glslang's shader compiles, on a thread of their
own per stale shader (`:Shader`), and lint's parse through the libclang
the linter dlopens (`:Lint`).
- Several crafter-build processes share `~/.cache/crafter.build/` at once:
the std and Crafter.Build PCMs per target and march (the
`ConcurrentCacheRace` test), and the external clones with their `.lock`
files.
- Three builds of the tool: the bootstrap from `build.sh` (without
`CRAFTER_BUILD_HAS_ASSET`), the self-hosted binary on the host's LLVM, and
the Flatpak on its own LLVM under `/app`, in a sandbox whose `/usr` is the
Freedesktop SDK's. Their toolchains can be a release apart.
- Targets: the host triple, a sysroot (aarch64, musl), `x86_64-w64-mingw32`
under wine, `wasm32-wasi*` under wasmtime (or wasmer for `-r`), and
browser builds through the runtime in `wasi-runtime/`.
- Incremental builds: a first build, a rebuild after an edit, a rebuild
after a toolchain or flag change and a build after `clean` start from
different states; whatever changes what gets compiled has to reach
`VariantId()`, or a stale artifact is reused.
- Tests: each its own subprocess, in parallel, with a 60 s timeout. A test
that edits or touches files stages a copy under
`fs::temp_directory_path()`; one that only builds its fixture builds it in
place, into the fixture's own `bin/` and `build/` in the checkout.
Semantics to read on disk rather than recall: libc++ and clang as the host
installs them (`/run/host/usr/include/c++/v1`,
`/run/host/usr/share/libc++/v1/std.cppm`,
`/run/host/usr/lib/clang/<major>/include`), the dependencies as cloned
(`~/.cache/crafter.build/external/<Name>-<hash>/`), the vendored
`lib/DirStackFileIncluder.h`. The Flatpak's own LLVM is not readable from
here: a semantic that may differ between LLVM releases is unverified for
the Flatpak build.

View file

@ -0,0 +1,18 @@
## This repository: Crafter.Build
- A claim that a tree passes `build.sh`, the self-host build, the test
suite, `format --check` or `lint` traces to the Gate 0 log among the
evidence paths INPUTS names: each command with its exit code and the sha
it ran at. A claim of a run anywhere else (the Flatpak through
`run flatpak`, CI, wine, qemu, node, a browser) traces to a capture of
that run that names the commit, or is unverifiable.
- A test a message names exists as a `cfg.AddTest("<Name>")` line in
`project.cpp` and a `tests/<Name>/main.cpp` whose checks pin what the
message says.
- `AGENTS.md` is generated from `AGENTS.template.md` and the dependencies'
`AGENTS.consumer.md` and is not in the tree: a claim about it is checked
against those sources.
- A `<release>` entry in
`packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` is the
release notes users read: every sentence in it is a claim about the code
since the previous tag.

View file

@ -11,6 +11,7 @@ crafter-build is a C++26 modules build system. A project is described by a `proj
| `interfaces/Crafter.Build-<Partition>.cppm` | Module interface partitions. `Crafter.Build.cppm` is the umbrella that re-exports them. | | `interfaces/Crafter.Build-<Partition>.cppm` | Module interface partitions. `Crafter.Build.cppm` is the umbrella that re-exports them. |
| `implementations/Crafter.Build-<Partition>.cpp` | Implementation units, one per partition, named `Crafter.Build:<Partition>_impl`. `main.cpp` is a thin wrapper around `Crafter::Run`. | | `implementations/Crafter.Build-<Partition>.cpp` | Implementation units, one per partition, named `Crafter.Build:<Partition>_impl`. `main.cpp` is a thin wrapper around `Crafter::Run`. |
| `project.cpp` | The self-host build description: a `crafter.build-lib` library, a `crafter.build-exe` executable, the test declarations, the lint rules and the `flatpak` and `install` scripts. | | `project.cpp` | The self-host build description: a `crafter.build-lib` library, a `crafter.build-exe` executable, the test declarations, the lint rules and the `flatpak` and `install` scripts. |
| `AGENTS.template.md` | The source of AGENTS.md. Every build and test run writes AGENTS.md from it, with the notes the dependencies ship; AGENTS.md is gitignored, so changes go here. |
| `tests/<Name>/main.cpp` | One self-test per directory; `tests/<Name>/fixture/` holds the sample project it builds. | | `tests/<Name>/main.cpp` | One self-test per directory; `tests/<Name>/fixture/` holds the sample project it builds. |
| `examples/` | Standalone user-facing projects, MIT licensed. | | `examples/` | Standalone user-facing projects, MIT licensed. |
| `build.sh` | Bootstrap script. It compiles everything by hand, without crafter-build. | | `build.sh` | Bootstrap script. It compiles everything by hand, without crafter-build. |
@ -18,10 +19,11 @@ crafter-build is a C++26 modules build system. A project is described by a `proj
| `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. | | `lib/` | Vendored third-party headers (nlohmann/json, glslang's DirStackFileIncluder). Do not edit. |
| `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. | | `share/crafter-build/`, `bin/`, `build/`, `lib/*.a` | Generated and gitignored. |
| `packaging/flatpak/`, `.forgejo/workflows/ci.yaml` | The Flatpak manifest with its metainfo, repo and ref files, and the Forgejo Actions CI that builds and publishes it. | | `packaging/flatpak/`, `.forgejo/workflows/ci.yaml` | The Flatpak manifest with its metainfo, repo and ref files, and the Forgejo Actions CI that builds and publishes it. |
| `.review-gates/` | The pre-push review's `config.json` and this repository's lens sections (`lenses/`), read by Crafter.Review. Its `push/` and `trees/` are generated and gitignored. |
Partitions and what they own: Partitions and what they own:
- `:Toolchain` resolves the compiler once per process (`GetToolchain`): clang++, clang, llvm-ar, libclang, libc++'s std module source and the default sysroots, from `CRAFTER_BUILD_LLVM_DIR`, else a toolchain installed beside crafter-build, else PATH. It also holds the version baked in at build time. `SysrootStdModule` picks the std module source for a sysroot build by the sysroot's libc++ release. - `:Toolchain` resolves the compiler once per process (`GetToolchain`): clang++, clang, llvm-ar, libclang, libc++'s std module source and the default sysroots, from `CRAFTER_BUILD_LLVM_DIR`, else a toolchain installed beside crafter-build, else PATH. It also holds the version baked in at build time. `SysrootStdModule` picks the std module source for a sysroot build by the sysroot's libc++ release. `MingwSysroot` settles the mingw target's `--sysroot`: the mingw-w64 prefix itself, or, for a prefix that also holds a Linux system (the host's `/usr`, anything whose `include/` has `features.h`), a view in the cache of its mingw-w64 directories, since clang would search its `include/` and `lib/` too (issue #43). `MingwViewToPrefix` turns paths into the view back into paths into the prefix for messages.
- `:Clang` holds `Configuration`, `Build()`, `VariantId()`, `ApplyStandardArgs`, `HostTarget`, the CLI entry point `Crafter::Run`, and the lint, test and script data types. - `:Clang` holds `Configuration`, `Build()`, `VariantId()`, `ApplyStandardArgs`, `HostTarget`, the CLI entry point `Crafter::Run`, and the lint, test and script data types.
- `:Platform` runs commands, implements `LoadProject` (compile and load `project.cpp`), the host PCM cache, `#pragma crafter dependency`, and all OS-specific code. - `:Platform` runs commands, implements `LoadProject` (compile and load `project.cpp`), the host PCM cache, `#pragma crafter dependency`, and all OS-specific code.
- `:Interface` and `:Implementation` model module units, their staleness checks and their compiles. - `:Interface` and `:Implementation` model module units, their staleness checks and their compiles.
@ -29,6 +31,7 @@ Partitions and what they own:
- `:Test` holds `AddTest`, `RunTests` and `TestRunner`. - `:Test` holds `AddTest`, `RunTests` and `TestRunner`.
- `:Lint` is the lint/format driver plus the libclang token and AST layer. - `:Lint` is the lint/format driver plus the libclang token and AST layer.
- `:Script` is the driver behind `crafter-build run`: it looks a name up in `cfg.scripts`, calls the callback and turns what it returns into the exit code. - `:Script` is the driver behind `crafter-build run`: it looks a name up in `cfg.scripts`, calls the callback and turns what it returns into the exit code.
- `:Agents` writes a project's AGENTS.md from its AGENTS.template.md: it walks the dependencies `cfg.dependencyNotes` selects, reads each one's `AGENTS.consumer.md` and merges them into the template's dependency notes section. Which Configurations belong to another project, and where it lives, comes from the loader (`LoadedProjectOf` in `:External`, `PragmaDependencies` in `:Platform`), because only the Configuration `GitProject` and `LocalProject` return has its `path` rebased.
- `:Progress` is console output. `:Shader` compiles shaders through glslang. `:Asset` is the hook into Crafter.Asset. - `:Progress` is console output. `:Shader` compiles shaders through glslang. `:Asset` is the hook into Crafter.Asset.
## Dev environment ## Dev environment
@ -48,7 +51,7 @@ Three things to know about this loop:
- Changes under `implementations/` are picked up by the self-host build and by the tests, which link the local library. Changes under `interfaces/` are not: the harness compiles against the copies in `share/crafter-build/` and the PCM cache built from them, so a new or changed declaration is invisible until you run `./build.sh` again. The symptom is `use of undeclared identifier` for the symbol you just added. A changed default member initialiser has no symptom at all: `project.cpp` compiles against the old one and quietly gets the old default, as `project.cpp`'s sibling loads do with a harness that predates `GitProjectSpec::args = ProjectArgs()`. - Changes under `implementations/` are picked up by the self-host build and by the tests, which link the local library. Changes under `interfaces/` are not: the harness compiles against the copies in `share/crafter-build/` and the PCM cache built from them, so a new or changed declaration is invisible until you run `./build.sh` again. The symptom is `use of undeclared identifier` for the symbol you just added. A changed default member initialiser has no symptom at all: `project.cpp` compiles against the old one and quietly gets the old default, as `project.cpp`'s sibling loads do with a harness that predates `GitProjectSpec::args = ProjectArgs()`.
- A new exported symbol that a `project.cpp` reaches on its default path also needs `./build.sh`: the old binary does not export it, and `LoadProject` fails with an undefined symbol. - A new exported symbol that a `project.cpp` reaches on its default path also needs `./build.sh`: the old binary does not export it, and `LoadProject` fails with an undefined symbol.
`./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. That takes `bin/crafter-build` itself with it, and `build/flatpak`, the state of the local Flatpak build (see "CI and publishing"): about 2 GB, and the next `run flatpak` rebuilds LLVM. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`. `./build.sh` clones and builds glslang on first run and takes several minutes; later runs are faster. `./bin/crafter-build clean` deletes `bin/` and `build/` without loading `project.cpp`. That takes `bin/crafter-build` itself with it, and `build/flatpak`, the state of the local Flatpak build (see "CI and publishing"): about 4 GB after a first run, and the next `run flatpak` rebuilds LLVM. Set `CRAFTER_BUILD_HOME` only when running a binary from outside this tree; `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` default to `native`.
## Build, test and style commands ## Build, test and style commands
@ -66,7 +69,7 @@ Three things to know about this loop:
./bin/crafter-build clean ./bin/crafter-build clean
``` ```
Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` on every PR; its lint step is currently commented out, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/<name>.log`. Before you call a change done, all three must exit 0: `format` (then re-read your diff), `lint`, `test`. CI runs `./build.sh` and `crafter-build test` for every release tag, every pull request into `master` and every manual run, and has no lint step, so lint is enforced only by you. Failed, crashed and timed-out tests leave their output in `build/test-logs/<name>.log`.
## Testing instructions ## Testing instructions
@ -116,31 +119,34 @@ Beyond the rules:
- There is no host-platform preprocessor split: the host is Linux. Target differences (mingw, wasm) branch on `cfg.target` strings at runtime, in the library as well as in `project.cpp` and tests. - There is no host-platform preprocessor split: the host is Linux. Target differences (mingw, wasm) branch on `cfg.target` strings at runtime, in the library as well as in `project.cpp` and tests.
- A build step fails when its command exits nonzero, never because it printed. Use `RunBuildCommand` or `RunCommandChecked`; warnings go to `Progress::Diagnostic`. - A build step fails when its command exits nonzero, never because it printed. Use `RunBuildCommand` or `RunCommandChecked`; warnings go to `Progress::Diagnostic`.
- Every output path goes through `Configuration::BinDir()`, `BuildDir()` and `PcmDir()`. Hand-rolled `path / "bin" / ...` clobbered variants before; do not add new ones. - Every output path goes through `Configuration::BinDir()`, `BuildDir()` and `PcmDir()`. Hand-rolled `path / "bin" / ...` clobbered variants before; do not add new ones.
- Anything that changes what gets compiled or bundled belongs in `VariantId()`. It already hashes type, debug, sysroot, defines, compile flags, wasm variant flags, uninterpreted project args and the toolchain (`Toolchain::Key()`). - Anything that changes what gets compiled or bundled belongs in `VariantId()`. It already hashes type, debug, sysroot, defines, compile flags, the wasm baseline and variant flags, the std module the units import (`StdPcmKey`), uninterpreted project args and the toolchain (`Toolchain::Key()`).
## Design rules ## Design rules
These were decided deliberately. Do not undo them without the maintainer's say-so. These were decided deliberately. Do not undo them without the maintainer's say-so.
1. **Configuration is C++.** No JSON, YAML, TOML or DSL for project, runner or lint configuration. Lint rules are C++ callbacks; there is no clang-format or clang-tidy integration and no separate "format rule" type. A rule that calls `ctx.SetContent` is a transform, and that is the whole mechanism. 1. **Configuration is C++.** No JSON, YAML, TOML or DSL for project, runner or lint configuration. Lint rules are C++ callbacks; there is no clang-format or clang-tidy integration and no separate "format rule" type. A rule that calls `ctx.SetContent` is a transform, and that is the whole mechanism.
2. **Per-project artifacts go in `<project>/build` and `<project>/bin`. Only cross-project things go in `~/.cache/crafter.build/`:** the std PCM and Crafter.Build PCMs per target and march, and external clones keyed by their spec hash. 2. **Per-project artifacts go in `<project>/build` and `<project>/bin`. Only cross-project things go in `~/.cache/crafter.build/`:** the std PCM and Crafter.Build PCMs per target, march, toolchain and std module flags, external clones keyed by their spec hash, and the mingw sysroot views keyed by prefix.
3. **Sources ship, PCMs build locally.** The Flatpak installs `*.cppm` under `/app/share/crafter-build/`. Nothing target-baked ships except the executable and the toolchain beside it, and the executable is built for a portable `-march`. 3. **Sources ship, PCMs build locally.** The Flatpak installs `*.cppm` under `/app/share/crafter-build/`. Nothing target-baked ships except the executable and the toolchain beside it, and the executable is built for a portable `-march`.
4. **crafter-build is a monolithic executable with `--export-dynamic`.** `project.so` resolves `Crafter::` symbols from the running binary at dlopen time. There is no Windows host; `x86_64-w64-mingw32` is a cross-compile target for user projects only. 4. **crafter-build is a monolithic executable with `--export-dynamic`.** `project.so` resolves `Crafter::` symbols from the running binary at dlopen time. There is no Windows host; `x86_64-w64-mingw32` is a cross-compile target for user projects only.
5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point. 5. **Convention over configuration.** Lead with the zero-config path. Env var, then CLI flag, then full `project.cpp` are the escalation, not the entry point.
6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch. 6. **One skip mechanism** (runner probe plus exit 77) and **one `TestRunner` shape** (template fields with `{bin}`, `{bundle}` and similar placeholders; `Cmd(...)` covers any wrapper). Do not add a special-case runner class or a hardcoded "skip on this OS" branch.
7. **The house-style rules stay in their own repo.** They are not library API of crafter-build and are not forced on downstream projects. 7. **The house-style rules stay in their own repo.** They are not library API of crafter-build and are not forced on downstream projects.
8. **Fix warts inline; do not stack workarounds.** The previous version of this tool died of accumulated hacks. If a change needs a shortcut, say so in the PR rather than hiding it. 8. **Fix warts inline; do not stack workarounds.** The previous version of this tool died of accumulated hacks. If a change needs a shortcut, say so in its commit message rather than hiding it.
9. **`project.cpp` keeps the library/executable split.** `crafter.build-lib` owns the sources, dependencies and glslang; `crafter.build-exe` is `main.cpp` plus link flags. 9. **`project.cpp` keeps the library/executable split.** `crafter.build-lib` owns the sources, dependencies and glslang; `crafter.build-exe` is `main.cpp` plus link flags.
10. **Host-tooling builds** (the target of a `#pragma crafter dependency`) get `-fPIC` and the host triple scoped to that build. Keep it scoped. 10. **Host-tooling builds** (the target of a `#pragma crafter dependency`) get `-fPIC` and the host triple scoped to that build. Keep it scoped.
## Commit and PR guidelines ## Commit and push guidelines
- Branch from `master` and open the PR against `master` on the project's Forgejo at `forgejo.catcrafts.net/Catcrafts/Crafter.Build`. Issue work uses `claude/issue-<n>` branch names. Pushes go over SSH; HTTPS push is disabled on the server. - Branch from `master`; issue work uses `claude/issue-<n>` branch names. Once the owner approves its frozen hash, a range goes straight to `master` on the project's Forgejo at `forgejo.catcrafts.net/Catcrafts/Crafter.Build`, as a fast-forward push with no merge commit and no pull request; open a pull request against `master` only when the owner asks for one. If `master` has moved since the freeze, rebase onto it, then freeze and review the rebased range again. Pushes go over SSH; HTTPS push is disabled on the server.
- Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice. - Subject line: `type(scope): imperative summary`, with `feat`, `fix`, `refactor`, `docs` or `test`; the scope (`lint`, `incremental`, ...) is optional. Body: what was wrong, why, and why the fix has this shape, wrapped at about 75 columns. Read `git log` for the voice.
- Separate commits for the fix, its test and the README change when each stands on its own. - Separate commits for the fix, its test and the README change when each stands on its own.
- Update the README in the same PR when user-visible behaviour changes. - Update the README in the same push when user-visible behaviour changes.
- `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it on PRs, publishes the `beta` branch of the Catcrafts repo on every push to `master`, and the `stable` branch plus a release on an `X.Y.Z` tag. - `format`, `lint` and `test` green before pushing. CI builds the Flatpak and runs the suite inside it for every release tag, every pull request into `master` and every manual run. Without a pull request, CI does not run on a range before it reaches `master` unless someone starts it by hand (`workflow_dispatch`), so the review's gate0, the commands in `.review-gates/config.json`, is the check before the push.
- Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`. - Every push to `master` is a release, made by the signed tag pushed with it (see "CI and publishing"). Every pushed range adds a `<release>` entry for its version at the top of `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, in a commit of its own (`docs(packaging): add the X.Y.Z release entry`), dated the day of the push in UTC. Its notes have up to three sections, `<p>New</p>`, `<p>Fixes</p>` and `<p>Breaking</p>` in that order, each followed by a `<ul>` of its items, the most important first, and nothing after the last section. The version follows from them: a Breaking section makes it a major, New items a minor, and Fixes alone a patch. Once the owner has approved the frozen hash, the commit `master` is to receive gets the signed tag of that version, made locally, and both go out in one push: `git tag -s X.Y.Z -m 'crafter-build X.Y.Z'`, then `git push --atomic origin HEAD:master X.Y.Z`. That commit is the range's last one, or, when the owner asks for a pull request instead, the merge of the reviewed branch into `master`.
- Every push of new commits goes through Crafter.Review's pre-push review first, and goes out only once the owner approves the frozen hash. Run `flatpak run net.catcrafts.CrafterReview init` once in a fresh checkout or worktree: it writes the `/prepush` skill that drives the review, and the skill stays untracked. `freeze` refuses merge commits: a range reaches `master` as a fast-forward, and when the owner asks for a pull request instead, the merge of the reviewed branch into `master` is pushed without a freeze of its own.
- Every commit of a pushed range is GPG-signed, has `jorijnvdgraaf@catcrafts.net` as its author and committer, and carries no AI attribution trailer in its message, neither `Co-Authored-By:` nor `Assisted-by:`. `freeze` refuses a range with a commit that breaks any of these (`requireSigned`, `authors` and `aiTrailers` in `.review-gates/config.json`).
- Never commit `bin/`, `build/`, `share/crafter-build/`, `lib/*.a`, `AGENTS.md` or fixture build outputs (all gitignored), and never edit the vendored files in `lib/`.
- Keep `CRAFTER_BUILD_MARCH=x86-64-v2` in the manifest for the shipped binary: it has to run on any x86_64 machine. User projects still default to `native`. - Keep `CRAFTER_BUILD_MARCH=x86-64-v2` in the manifest for the shipped binary: it has to run on any x86_64 machine. User projects still default to `native`.
## Security considerations ## Security considerations
@ -150,14 +156,14 @@ These were decided deliberately. Do not undo them without the maintainer's say-s
- `--runner=cmd:<command>` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests. - `--runner=cmd:<command>` executes the named wrapper around every test binary. Wrappers come from `--runner=` or `CRAFTER_BUILD_RUNNER_*`; do not hardcode hosts, wrappers or credentials in `project.cpp` or tests.
- `crafter-build run <name>` executes whatever the script does, with your permissions. Only the scripts of the project crafter-build was pointed at can be run, never a dependency's; keep it that way. - `crafter-build run <name>` executes whatever the script does, with your permissions. Only the scripts of the project crafter-build was pointed at can be run, never a dependency's; keep it that way.
- The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library. - The linter loads `libclang` at runtime by name. `CRAFTER_BUILD_LIBCLANG` pins an exact path and must fail loudly when wrong, never fall back to another library.
- CI force-pushes the rolling `latest` tag and publishes a prerelease on every push to `master` with the workflow's own token, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it. - CI creates a release for each release tag with the workflow's own token, pushes nothing to git, and signs the Flatpak repo with the key mounted from the runner host. Do not add secrets to the workflow or echo the environment in it.
- Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way. - Tests write only under `fs::temp_directory_path()` and the fixture's own `bin/` and `build/`. Keep it that way.
## CI and publishing ## CI and publishing
`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, so the LLVM and WASI modules are rebuilt only when their sources change; a cold run takes about 80 minutes on 8 cores, a warm one a fraction of that), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. `.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and no other repository's job runs on it. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The web root and the key are not this runner's alone: the organisation-wide `flatpak` runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository, mounts both for Crafter.Review's publish job, and the volumes a runner allows (`valid_volumes` in its config) are open to every job it runs, so a job of any Catcrafts repository that runs there can mount them as well. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it. Crafter.Review's workflow publishes into the same repo, and each workflow holds an flock on `/var/www/flatpak/.publish.lock` while it commits and updates the summary.
Keep the label unique to that runner. Plain `flatpak` also belongs to the organisation-wide runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository and on purpose allows no volumes. Whenever this job lands there, flatpak-builder stops with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of the job log shows which runner took it and lists every volume it ignored. Keep the label unique to that runner. Plain `flatpak` belongs to the organisation-wide runner. When this job still asked for `flatpak` and that runner took it (runs #368 and #370, in October 2026), the runner allowed none of the job's volumes, and flatpak-builder stopped with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of such a job's log shows which runner took it and lists every volume it ignored.
The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing: The key is made once, as root on the runner host, and the step fails with a message pointing here when it is missing:
@ -169,7 +175,8 @@ gpg --batch --pinentry-mode loopback --passphrase '' --export-secret-keys --armo
printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id printf '%s\n' "$KEYID" > /etc/crafter-flatpak/key-id && chmod 600 /etc/crafter-flatpak/signing-key.asc /etc/crafter-flatpak/key-id
``` ```
To release: add a `<release>` to `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml` (its description becomes the release notes), merge, and push the tag `X.Y.Z`. CI refuses a tag that does not match the newest entry. A release is a tag `X.Y.Z`, annotated and signed, pushed with the `master` it releases; a push to `master` without one runs nothing. Before building, every run renders the release notes from the newest `<release>` entry in `packaging/flatpak/net.catcrafts.CrafterBuild.metainfo.xml`, and stops when the metainfo does not parse or the notes come out empty. A tag run also checks that Forgejo verifies the tag as signed by the address the workflow's `RELEASE_SIGNER` names, that the tag's commit is on `master`, that its version has the form `X.Y.Z` and is newer than every release the forge has published, its own included, and that the newest entry is that version. Once the build and the suite inside it pass, the tag run publishes `stable` and creates the release, with the notes, the bundle, the `.flatpakref` and `SHA256SUMS` attached. Release runs queue one behind another, and a failed one can be re-run while no newer release exists, once a release it left half made is deleted; when the fix needs a commit of its own, that push is the next version's release. Pull requests and manual runs publish nothing. The `beta` branch, which every push to `master` used to go to, gets no more builds: the publish step marks its last build end-of-life, once, and the next `flatpak update` of a beta installation prints the reason.
Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive. Local builds are two scripts in `project.cpp`. Run them with the native binary: the Flatpak build of crafter-build has no `flatpak` to drive.
```bash ```bash
@ -177,7 +184,7 @@ Local builds are two scripts in `project.cpp`. Run them with the native binary:
./bin/crafter-build run install # install that build as branch `local` and make it current ./bin/crafter-build run install # install that build as branch `local` and make it current
``` ```
`run flatpak` needs `flatpak-builder` or the `org.flatpak.Builder` app, and the Sdk (`flatpak install flathub org.freedesktop.Sdk//26.08`). It builds the working tree, uncommitted changes included, runs the whole suite inside the sandbox and reports version `0.0.0-dev`. Its state is `build/flatpak/` (`state/`, `repo/`, `build-dir/`). The first run builds LLVM and the WASI sysroot, about 25 minutes on a 56-core machine; later runs rebuild only the crafter-build module, a few minutes, because flatpak-builder never caches a `type: dir` source. The checkout must not be under `/tmp`, which `org.flatpak.Builder` cannot see. Use it for anything toolchain-sensitive: the native loop compiles with the distro's clang and the Flatpak with its bundled LLVM, and the two can be a major version apart. `run flatpak` needs `flatpak-builder` or the `org.flatpak.Builder` app, and the Sdk (`flatpak install flathub org.freedesktop.Sdk//26.08`). It builds the working tree, uncommitted changes included, runs the whole suite inside the sandbox and reports version `0.0.0-dev`. Its state is `build/flatpak/` (`state/`, `repo/`, `build-dir/`). The first run builds every module, LLVM with lldb, the WASI sysroot and the Vulkan validation layer among them, in about 30 minutes with 56 CPU threads; later runs rebuild only the crafter-build module while the manifest is unchanged, a few minutes, because flatpak-builder never caches a `type: dir` source. The checkout must not be under `/tmp`, which `org.flatpak.Builder` cannot see. Use it for anything toolchain-sensitive: the native loop compiles with the distro's clang and the Flatpak with its bundled LLVM, and the two can be a major version apart.
`run install` does not build; it installs what `run flatpak` last exported. It points the user remote `crafter-build-local` at `build/flatpak/repo`, installs `net.catcrafts.CrafterBuild//local` from it next to whichever published branch is installed, and makes it the branch `flatpak run net.catcrafts.CrafterBuild` starts. `flatpak make-current --user net.catcrafts.CrafterBuild <branch>` switches back. Before `crafter-build clean`, or before deleting the checkout, run `flatpak uninstall --user net.catcrafts.CrafterBuild//local` and `flatpak remote-delete --user crafter-build-local`; otherwise the remote is left pointing at a repo that is gone. `run install` does not build; it installs what `run flatpak` last exported. It points the user remote `crafter-build-local` at `build/flatpak/repo`, installs `net.catcrafts.CrafterBuild//local` from it next to whichever published branch is installed, and makes it the branch `flatpak run net.catcrafts.CrafterBuild` starts. `flatpak make-current --user net.catcrafts.CrafterBuild <branch>` switches back. Before `crafter-build clean`, or before deleting the checkout, run `flatpak uninstall --user net.catcrafts.CrafterBuild//local` and `flatpak remote-delete --user crafter-build-local`; otherwise the remote is left pointing at a repo that is gone.
@ -187,7 +194,13 @@ Local builds are two scripts in `project.cpp`. Run them with the native binary:
- A `#pragma crafter dependency` target is rebuilt on every load on purpose: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the depfile check is what catches a regenerated host cache. Do not optimise the rebuild away. - A `#pragma crafter dependency` target is rebuilt on every load on purpose: clang crashes rather than erroring when it imports a BMI whose own imported PCM was rebuilt underneath it, and the depfile check is what catches a regenerated host cache. Do not optimise the rebuild away.
- A host-tooling library such as Crafter.LintRules may `import Crafter.Build;` without declaring it as a dependency, because `LoadProject` fills the cache it resolves from. Listing Crafter.Build in its `cfg.dependencies` would clone and build all of crafter-build for every consumer. Its test does link crafter-build, so that test is wired behind `IsTopLevelProject()`: without the guard, crafter-build loading the library as a dependency would load its own `project.cpp` again through the test's `GitProject` and hit the cycle guard. - A host-tooling library such as Crafter.LintRules may `import Crafter.Build;` without declaring it as a dependency, because `LoadProject` fills the cache it resolves from. Listing Crafter.Build in its `cfg.dependencies` would clone and build all of crafter-build for every consumer. Its test does link crafter-build, so that test is wired behind `IsTopLevelProject()`: without the guard, crafter-build loading the library as a dependency would load its own `project.cpp` again through the test's `GitProject` and hit the cycle guard.
- `crafter-build clean` leaves a pragma dependency's build alone; it lives with the clone in the cache. - `crafter-build clean` leaves a pragma dependency's build alone; it lives with the clone in the cache.
- Units see a std module rebuilt in place, because their depfiles list the BMIs they import (`-fmodule-file-deps`), but not one that moved to another directory: their old depfiles still name the old one. So anything that changes how the std module is compiled goes into `StdPcmFlags`, which `BuildStdPcm` compiles with and `StdPcmKey` hashes. `VariantId()` hashes `StdPcmKey`, so the build moves with it. A flag added to the command anywhere else leaves both directories where they are.
- A CMake external whose recorded configure inputs (`<build>/.crafter-options`) differ from this build's is built again from an empty build directory, not reconfigured in place: CMake keeps what an old compiler or `--sysroot` compiled, and the check results it cached.
- The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop. - The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop.
- A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone. - A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone.
- A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules/<major>/`, one `libcxx-<major>-modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot. - A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules/<major>/`, one `libcxx-<major>-modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot.
- The Flatpak carries the debugging tools for what it builds, because those programs run in its sandbox: lldb is a project of the `llvm` module, with `LLDB_ENABLE_PYTHON`, `LIBEDIT`, `CURSES`, `LIBXML2` and `LZMA` set ON so that a missing dependency fails the configure step (SWIG through Python, which needs it), and the `swig` and `libedit` modules come before it. The Khronos validation layer is the `vulkan-validation-layers` module plus four modules before it, build-only except for SPIRV-Tools' programs (`spirv-val` and the rest), all after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. The layer follows the Sdk's Vulkan loader: when `runtime-version` moves to a runtime with a newer loader, move the layer to the Vulkan SDK release of that loader and take the other four revisions from that release's `scripts/known_good.json`.
- The wasm compile flags live in one place, `WasmBaselineFlags` in `Crafter.Build-Clang.cppm`: the std module and every unit have to agree on them, and `VariantId()` and the std PCM directory hash them, so a change rebuilds instead of mixing old and new BMIs. They enable C++ exceptions, so the link takes the exception-enabled libc++ and libunwind from the sysroot's `lib/<triple>/eh/` (wasi-sdk 33 and later). The Flatpak's sysroot has them and a distro's may not; where they are missing the wasm tests skip, so run those through the Flatpak. A browser page checks the engine with the `WasmBaselineProbes` named in `variants.json`.
- `WasiBrowserRuntimeExit` runs `wasi-runtime/runtime.js` under node and skips where there is none. The Flatpak has no node, so CI never runs it: after touching `runtime.js`, run the suite in the native loop with node on `PATH`.
- `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both. - `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both.
- AGENTS.md is build output: the self-host build and `test` write it from `AGENTS.template.md`, with the `AGENTS.consumer.md` of each dependency that ships one (Crafter.Math, today) merged in, and it is gitignored. Edit the template; an edit made in AGENTS.md is gone after the next build. A fresh clone or worktree has no AGENTS.md until its first `./bin/crafter-build` or `test`, since `./build.sh` does not write it. A harness from before the template (3.0.0 and older) never creates one and keeps its section in one it finds; the next build with a current harness writes it over.

View file

@ -4,15 +4,17 @@ A C++26 modules build system. Project descriptions are written in C++ with no JS
## Install ## Install
crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler: crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler, and lldb and the Vulkan validation layers to debug the result (see [Debugging](#debugging)):
```bash ```bash
flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo
flatpak install catcrafts net.catcrafts.CrafterBuild flatpak install catcrafts net.catcrafts.CrafterBuild
alias crafter-build='flatpak run net.catcrafts.CrafterBuild' alias crafter-build='flatpak run --die-with-parent net.catcrafts.CrafterBuild'
``` ```
Pre-release builds of `master` are the `beta` branch (`flatpak install catcrafts net.catcrafts.CrafterBuild//beta`), and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-<version>.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host. `--die-with-parent` ends the sandbox with the `flatpak run` that started it. Without the flag, killing only the `flatpak run` process leaves the sandbox running: a build carries on writing into `bin/` and `build/`, and a program started with `-r` keeps running. With it, bubblewrap kills everything in the sandbox with SIGKILL when the `flatpak run` process dies or when that process's parent does. For `crafter-build …` or `crafter-build … &` typed at a bash or zsh prompt, that parent is the shell, so the sandbox ends with the shell, even after a `disown`. Ctrl-C reaches the program either way.
Releases are published to the `stable` branch that the commands above install, and every release also attaches a single-file bundle on the Forgejo releases page (`flatpak install --user crafter-build-<version>.flatpak`). The sandbox sees the whole host filesystem, so projects can live anywhere. Binaries built for the host link the bundled `libc++.so.1` through an rpath into the Flatpak: they run inside it (`-r`, the test runner), and are not meant to be copied out onto the host.
## Writing a project.cpp ## Writing a project.cpp
@ -99,6 +101,55 @@ import Crafter.LintRules;
Such a dependency is built for the host, whatever the project targets, and its `project.cpp` gets none of the project's args. Such a dependency is built for the host, whatever the project targets, and its `project.cpp` gets none of the project's args.
### Dependency notes for agents
An agent working on a project reads its `AGENTS.md`, which says nothing about the libraries the project builds against unless someone copied that in by hand. A library ships it instead: an `AGENTS.consumer.md` beside its `project.cpp`, written for whoever uses the library rather than whoever works on it.
A consumer that wants its dependencies' notes writes its own instructions in an `AGENTS.template.md` beside its `project.cpp`, and commits that instead of an `AGENTS.md`. `crafter-build` and `crafter-build test` then write `AGENTS.md` from the template with the notes added, so `AGENTS.md` is build output and stays out of version control: a dependency that changes its notes changes nothing a diff shows. A project with a committed `AGENTS.md` moves over from the directory of its `project.cpp`:
```bash
git mv AGENTS.md AGENTS.template.md
echo /AGENTS.md >> .gitignore
```
What a build then writes:
```markdown
<!-- Written by crafter-build from AGENTS.template.md, and rewritten on every build. Edit that file, not this one. -->
# The template's text
...
<!-- crafter-build: dependency notes begin -->
<!-- Written by crafter-build from the AGENTS.consumer.md each dependency ships, and rewritten on every build. Edit those files, not this section; cfg.dependencyNotes in project.cpp picks which dependencies are included. -->
<!-- from Crafter.Math -->
## Crafter.Math
...
<!-- crafter-build: dependency notes end -->
```
- Each file goes in as written, under a line naming the dependency: its repository's name, or a `LocalProject`'s directory name. That reads the same on every machine, and for a `--local` checkout and its git clone alike. The file lands among the consumer's own headings, so start it with a `##` heading that names the library.
- The section goes after the template's text and a blank line. To have it elsewhere, put the begin and end marker lines in the template where it should go: whatever they enclose is replaced. A template moved over from a committed `AGENTS.md` has them where its section was, around notes that can be deleted.
- `AGENTS.md` is written only when its text changes, in place, so a `CLAUDE.md` linked to it, its ownership and a read-only `AGENTS.md` are respected. An edit made in it is gone after the next build, and a fresh clone has none until its first build. The template is never written.
- An `AGENTS.md` that no build wrote, such as a committed one still next to a new template, is never replaced: each build warns instead, until it is deleted. Without a template, `AGENTS.md` is never touched. One that still holds a section from crafter-build 3.0.0, which kept the notes in a committed `AGENTS.md`, gets a warning: move it over as above, or delete the section.
- The update runs once `project.cpp` has loaded and before the project compiles, so a build that fails still has the notes. A `project.cpp` that fails to load gets no update. `test --list`, `lint`, `format`, `run` and `clean` leave the file alone.
- The section follows what the build loaded: a dependency that only some flags pull in (a `--target=`, a flag of the project's own) comes and goes with them. An unpinned dependency's notes follow its default branch, as its code does, so the section can change on any build. Pin a `commit` to decide when.
- A library reached through two specs, pinned at one revision here and another further down, is described once, as the nearer one has it.
- When there are notes to write, unbalanced markers in the template leave `AGENTS.md` as it is, with a warning. A notes file that would break the section (a code block it never closes, a marker line of its own) is left out, also with a warning. Markers quoted inside a fenced code block don't count.
`cfg.dependencyNotes` picks the dependencies:
| Value | Notes from |
|---|---|
| `DependencyNotes::Transitive` (default) | everything the project builds against, all the way down: its `dependencies` and its own `#pragma crafter dependency` tooling |
| `DependencyNotes::Direct` | only the projects the project's own configurations and `#pragma crafter dependency` lines name |
| `DependencyNotes::None` | none: `AGENTS.md` holds the template's text alone |
Three things never count: the project's own `AGENTS.consumer.md`; its tests' dependencies, since a plain build never looks at the tests; and a dependency's `#pragma crafter dependency` tooling, which is about how the dependency is built, not how it is used.
## Tests ## Tests
Tests live under `tests/<Name>/`. The simplest case is a single C++ file: Tests live under `tests/<Name>/`. The simplest case is a single C++ file:
@ -152,7 +203,17 @@ For elaborate setups (rsync, ssh ControlMaster, env vars on the spawn, `QEMU_LD_
For cross-arch builds, set `cfg.sysroot` on the `Configuration` to point at a sysroot for the target. `BuildStdPcm` and the compile/link commands both consume it: `--sysroot=<path>` is passed to clang, and the std module is precompiled against the sysroot's libc++ headers. Ordinary compiles are pointed at the same headers (`-nostdinc++ -isystem <sysroot>/usr/include/c++/v1`), so a unit that `#include`s a standard header gets the sysroot's libc++ and not the host toolchain's. The module source has to come from the same libc++ release as those headers, so it is taken from `<sysroot>/usr/share/libc++/v1/std.cppm` when the sysroot ships it, else from the toolchain when both are the same major release, else from the sources crafter-build carries for other releases (`share/crafter-build/libc++-modules/<major>/`, `crafter-build --version` lists them). Alpine is the common case for the last two: its `libc++-dev` has the headers and no module source. When nothing matches, the build stops with one line naming both releases, installing that release's `share/libc++/v1` directory into the sysroot resolves it. The cheapest aarch64 sysroot is an extracted [Arch Linux ARM rootfs](http://os.archlinuxarm.org/) plus `libc++` + `libc++abi` + `gcc` installed into it via `pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy …` (point pacman at ALARM's mirrors via explicit config to avoid pulling host-arch packages). For cross-arch builds, set `cfg.sysroot` on the `Configuration` to point at a sysroot for the target. `BuildStdPcm` and the compile/link commands both consume it: `--sysroot=<path>` is passed to clang, and the std module is precompiled against the sysroot's libc++ headers. Ordinary compiles are pointed at the same headers (`-nostdinc++ -isystem <sysroot>/usr/include/c++/v1`), so a unit that `#include`s a standard header gets the sysroot's libc++ and not the host toolchain's. The module source has to come from the same libc++ release as those headers, so it is taken from `<sysroot>/usr/share/libc++/v1/std.cppm` when the sysroot ships it, else from the toolchain when both are the same major release, else from the sources crafter-build carries for other releases (`share/crafter-build/libc++-modules/<major>/`, `crafter-build --version` lists them). Alpine is the common case for the last two: its `libc++-dev` has the headers and no module source. When nothing matches, the build stops with one line naming both releases, installing that release's `share/libc++/v1` directory into the sysroot resolves it. The cheapest aarch64 sysroot is an extracted [Arch Linux ARM rootfs](http://os.archlinuxarm.org/) plus `libc++` + `libc++abi` + `gcc` installed into it via `pacman --root <rootfs> --config <alarm-pacman.conf> --dbpath <rootfs>/var/lib/pacman -Sy …` (point pacman at ALARM's mirrors via explicit config to avoid pulling host-arch packages).
For `--target=x86_64-w64-mingw32` (Windows from Linux), the pipeline cross-compiles as long as `mingw-w64` is installed, with libstdc++ 15 or newer for its std module. clang finds it by the `x86_64-w64-mingw32-gcc` on `PATH`; `CRAFTER_MINGW_DIR` names another prefix (the directory holding `x86_64-w64-mingw32/` and `lib/gcc/`), and inside the Flatpak the host's `/usr` is used when it has mingw-w64. Without one the build stops with a message saying where it looked. Pair it with `--runner=cmd:wine` to run the result. Only the MinGW ABI is supported, there is no MSVC target and no Windows host. `-fuse-ld=lld` is auto-injected on every link, and `-lstdc++exp` is auto-linked for the mingw target so user projects don't need to declare those flags. For `--target=x86_64-w64-mingw32` (Windows from Linux), the pipeline cross-compiles as long as `mingw-w64` is installed, with libstdc++ 15 or newer for its std module. clang finds it by the `x86_64-w64-mingw32-gcc` on `PATH`; `CRAFTER_MINGW_DIR` names another prefix (the directory holding `x86_64-w64-mingw32/` and `lib/gcc/`), and inside the Flatpak the host's `/usr` is used when it has mingw-w64. Without one the build stops with a message saying where it looked. The prefix is clang's sysroot as it is, so Windows libraries installed into its `include/` and `lib/` are found, unless it also holds a Linux system: the host's `/usr` in the Flatpak, or any prefix whose `include/` has a C library's `features.h`, as `/usr` does. clang searches a sysroot's `include/` and `lib/`, and in such a prefix those hold glibc's headers and Linux libraries, so clang is shown only its `x86_64-w64-mingw32/`, `lib/gcc/` and `lib64/gcc/`, as links in `mingw-sysroot/` in the cache (`~/.cache/crafter.build/`, in the Flatpak `~/.var/app/net.catcrafts.CrafterBuild/cache/crafter.build/`, or `$XDG_RUNTIME_DIR/crafter.build/` when the cache's filesystem takes no symbolic links). Compiler messages still name those headers by their paths in the prefix. Pair it with `--runner=cmd:wine` to run the result. Only the MinGW ABI is supported, there is no MSVC target and no Windows host. `-fuse-ld=lld` is auto-injected on every link, and `-lstdc++exp` is auto-linked for the mingw target so user projects don't need to declare those flags.
For `--target=wasm32-wasip1`, the WASI sysroot is the one beside the toolchain (the Flatpak bundles one) unless `CRAFTER_BUILD_WASI_SYSROOT` names another, and tests run under `wasmtime`. C++ exceptions work as on the other targets: every unit and the std module are compiled with `-fwasm-exceptions` in the standardized form of WebAssembly exception handling (exnref), against the exception-enabled libc++ in the sysroot's `lib/<triple>/eh/`, and `-lunwind` is linked automatically. A sysroot without that libc++ (wasi-sdk before 33) stops the build with one line saying so. The result runs on engines with that proposal: Chrome 137, Firefox 131 and Safari 18.4 or newer, and the wasmtime the Flatpak bundles. A page from `EnableWasiBrowserRuntime` checks first and names what an older browser lacks in the console, instead of failing to compile the module. An exception that escapes `main` leaves the program as an uncaught WebAssembly exception (wasmtime reports `thrown Wasm exception` and exits 1, a page gets a `WebAssembly.Exception`) without calling `std::terminate`, so a `std::set_terminate` handler does not run and nothing prints the exception's type or `what()`: wasm exception handling unwinds in a single phase, and nothing finds out that no handler is left until the stack is gone. Catch it in `main` to report it.
A wasm build refuses three things before it starts, each with a message that says why: `-flto` in `compileFlags` or `linkFlags` (LTO code generation drops the exception handling, [WebAssembly/wasi-sdk#629](https://github.com/WebAssembly/wasi-sdk/issues/629)), `-fno-exceptions` (the std module is built with exceptions, and a unit without them cannot import it) and a shared library (the exception-enabled libc++ is static archives only). `setjmp` and `longjmp` work in C and C++ sources, with `-lsetjmp` linked automatically like `-lunwind`; a `longjmp` across a `noexcept` function ends the program, because it is a wasm exception and the function's terminate handler catches it. Outside the Flatpak, a wasmtime older than 47 has exception handling off and needs `-W exceptions=y`, and a `wasm-opt` on `PATH` from a binaryen that cannot read the standardized encoding breaks release links, since clang runs it on optimized wasm links when it finds one.
In a page from `EnableWasiBrowserRuntime`, `exit` and `_Exit` end the program as they would anywhere else: none of its code runs on the way out, no destructor and no `catch (...)`, its status is in `window.crafter_wasi.exitCode`, and bridges that call into it on their own are expected to stop. A program that hands the thread to the browser for good and goes on serving calls, as Crafter.Graphics's `Window::StartSync()` does, parks instead of exiting. Parking never returns, runs nothing of the program either, and leaves `main`'s locals valid for the calls that follow:
```cpp
extern "C" [[noreturn]] __attribute__((import_module("crafter_runtime"), import_name("park"))) void Park();
```
### Configuring runners ### Configuring runners
@ -305,6 +366,32 @@ Things to know:
- Inside the Flatpak, commands run in its sandbox: the host's files are visible, but the tools on `PATH` are the ones the Flatpak ships (`git`, `cmake`, `python3`, `tar`, ...), not everything installed on the host. - Inside the Flatpak, commands run in its sandbox: the host's files are visible, but the tools on `PATH` are the ones the Flatpak ships (`git`, `cmake`, `python3`, `tar`, ...), not everything installed on the host.
- A return value outside 0 to 255 is not an exit status and becomes 1. So does a script that throws, with the exception's message printed. - A return value outside 0 to 255 is not an exit status and becomes 1. So does a script that throws, with the exception's message printed.
## Debugging
Programs that `-r` and the test runner start run inside the Flatpak's sandbox, and so do the tools to debug them: lldb, with `lldb-server`, `lldb-dap` and `lldb-mcp`, from the same LLVM build as the compiler; `gdb`, `gdbserver`, `valgrind` and `strace` from the Freedesktop Sdk; the Khronos Vulkan validation layer; and the SPIRV-Tools programs (`spirv-val`, `spirv-opt`, `spirv-dis`, ...). A default build is optimised (`-O3`, and ThinLTO except for wasm) and carries no debug information: set `cfg.debug = true` for a program you want to debug, which compiles its C++ units with `-g` and links it without LTO; its C and CUDA sources (`cfg.cFiles`, `cfg.cuda`) stay at `-O3` without debug information. A `project.cpp` that calls `ApplyStandardArgs` sets it from the `--debug` argument, as in `crafter-build --debug -r`. Start the tools with `--command`:
```bash
flatpak run --die-with-parent --command=lldb net.catcrafts.CrafterBuild ./bin/myapp-<variant>/myapp
flatpak run --die-with-parent --command=valgrind net.catcrafts.CrafterBuild ./bin/myapp-<variant>/myapp
```
Every `flatpak run` is a sandbox of its own, with process IDs of its own, so a debugger in one cannot see a program that another one, such as `crafter-build -r`, started. Either start the program under the debugger, or run the debugger inside that program's sandbox with `flatpak enter`, giving the instance `flatpak ps` lists for net.catcrafts.CrafterBuild and the debugger's full path:
```bash
flatpak ps --columns=instance,application
flatpak enter <instance> /app/bin/lldb -n myapp
```
Where `/proc/sys/kernel/yama/ptrace_scope` is 1, a debugger in the sandbox, where processes hold no capabilities, can attach to a program that is not its descendant only when the program allowed it with `prctl(PR_SET_PTRACER, …)`: with the debugger's process ID, or with `PR_SET_PTRACER_ANY` for any process. Otherwise the attach fails. A program started under the debugger is its descendant. Prefer `flatpak enter` to `lldb-server gdbserver` in one sandbox and lldb's `gdb-remote` from another: when a process in lldb's own sandbox has the remote program's process ID, lldb takes that process's executable for the program's: symbols come out wrong, and a breakpoint in the program's source finds no location.
The validation layer is enabled the usual ways, by the program itself or from the environment: `flatpak run` passes `VK_INSTANCE_LAYERS`, `VK_LOADER_LAYERS_ENABLE` and the layer's `VK_KHRONOS_VALIDATION_*` settings into the sandbox. `VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1` adds GPU-assisted validation (GPU-AV):
```bash
VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1 crafter-build -r
```
The layer is the Vulkan SDK release of the runtime's own Vulkan loader. `VK_LAYER_PATH` and `VK_ADD_LAYER_PATH` are cleared on the way into the sandbox, so a layer installed on the host is out of reach.
## Examples ## Examples
[examples/](examples/) contains progressively larger self-contained projects: [examples/](examples/) contains progressively larger self-contained projects:

View file

@ -36,10 +36,9 @@ else
STD_CPPM=/usr/share/libc++/v1/std.cppm STD_CPPM=/usr/share/libc++/v1/std.cppm
fi fi
# The version baked into the binary: the environment (CI sets it from the # The version baked into the binary: the environment (CI sets it), else the
# tag), else the nearest tag, else a development marker. `latest` is the # nearest tag, else a development marker.
# rolling CI tag and never a version. VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo 0.0.0-dev)}"
VERSION="${CRAFTER_BUILD_VERSION:-$(git describe --tags --always --dirty --exclude=latest 2>/dev/null || echo 0.0.0-dev)}"
mkdir -p build bin lib share/crafter-build share/crafter-build/wasi-runtime mkdir -p build bin lib share/crafter-build share/crafter-build/wasi-runtime
@ -48,7 +47,7 @@ cp wasi-runtime/index.html.in share/crafter-build/wasi-runtime/
# Partition order is the import order: every unit is listed before the # Partition order is the import order: every unit is listed before the
# units that import it, and the umbrella last. # units that import it, and the umbrella last.
PARTITIONS="Toolchain Shader Platform Interface Implementation External Clang Test Lint Script Progress Asset" PARTITIONS="Toolchain Shader Platform Interface Implementation External Agents Clang Test Lint Script Progress Asset"
cp interfaces/Crafter.Build.cppm share/crafter-build/ cp interfaces/Crafter.Build.cppm share/crafter-build/
for p in $PARTITIONS; do for p in $PARTITIONS; do

View file

@ -4,9 +4,14 @@ Build a C++26 program for the `wasm32-wasi` target and run it in a browser.
## Prerequisites ## Prerequisites
- `wasi-libc`, `wasi-libc++`, `wasi-libc++abi` (Arch packages — provide - A WASI sysroot with the exception-enabled libc++, `lib/wasm32-wasip1/eh/`
`/usr/share/wasi-sysroot/`, which `crafter-build` finds automatically). from wasi-sdk 33 or later: wasm builds compile C++ exceptions and link those
Other distros: install the WASI SDK and set `cfg.sysroot` in `project.cpp`. libraries. The Flatpak bundles one, which `crafter-build` uses as it is.
Elsewhere, point `CRAFTER_BUILD_WASI_SYSROOT` at a wasi-sdk sysroot, or set
`cfg.sysroot` in `project.cpp`. Arch's `wasi-libc++` packages have no such
libraries, and the build stops with a message saying so.
- For the browser: WebAssembly exception handling, in Chrome 137, Firefox 131,
Safari 18.4 or newer.
## Build ## Build

View file

@ -0,0 +1,591 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
module;
#include <errno.h>
#include <fcntl.h>
#include <sys/file.h>
#include <unistd.h>
module Crafter.Build:Agents_impl;
import std;
import :Agents;
import :Clang;
import :External;
import :Platform;
import :Progress;
namespace fs = std::filesystem;
using namespace Crafter;
namespace {
constexpr std::string_view BeginMarker = "<!-- crafter-build: dependency notes begin -->";
constexpr std::string_view EndMarker = "<!-- crafter-build: dependency notes end -->";
// Right under the begin marker, for whoever opens the file: an edit made
// inside the section is gone after the next build.
constexpr std::string_view Preamble = "<!-- Written by crafter-build from the AGENTS.consumer.md each dependency ships, and rewritten on every build. Edit those files, not this section; cfg.dependencyNotes in project.cpp picks which dependencies are included. -->";
// The first line of every AGENTS.md a build writes. It tells whoever
// opens the file, an agent asked to change it included, where an edit
// has to be made to outlive the next build, and it is how a build tells
// its own AGENTS.md from one someone wrote, which it never replaces. A
// changed word would make every AGENTS.md written before look like
// someone's own.
constexpr std::string_view GeneratedLine = "<!-- Written by crafter-build from AGENTS.template.md, and rewritten on every build. Edit that file, not this one. -->";
constexpr std::string_view NotesFileName = "AGENTS.consumer.md";
constexpr std::string_view TemplateFileName = "AGENTS.template.md";
constexpr std::string_view AgentsFileName = "AGENTS.md";
void Warn(std::string_view message) {
Progress::Diagnostic(std::format("warning: {}", message));
}
// The line ending `text` uses, so a CRLF file does not end up mixed.
std::string_view LineEnd(std::string_view text) {
return text.contains("\r\n") ? "\r\n" : "\n";
}
std::string ErrnoMessage() {
return std::error_code(errno, std::generic_category()).message();
}
// Closes on the way out of scope, which also drops an flock taken
// through it.
class Descriptor {
public:
explicit Descriptor(std::int32_t fd) : fd_(fd) {}
~Descriptor() {
if (fd_ >= 0) ::close(fd_);
}
Descriptor(const Descriptor&) = delete;
Descriptor& operator=(const Descriptor&) = delete;
std::int32_t Get() const { return fd_; }
bool Valid() const { return fd_ >= 0; }
private:
std::int32_t fd_;
};
// Everything `fd` reads, into `out`; empty on success, otherwise the
// error. Through read(2) rather than a stream: libc++'s filebuf reports a
// failed read as the end of the file, so an I/O error came back as a
// short read that looked successful, and the section was then written
// over a truncated AGENTS.md.
std::string ReadAll(std::int32_t fd, std::string& out) {
out.clear();
std::array<char, 16384> buffer;
for (;;) {
const auto got = ::read(fd, buffer.data(), buffer.size());
if (got > 0) {
out.append(buffer.data(), static_cast<std::size_t>(got));
} else if (got == 0) {
return "";
} else if (errno != EINTR) {
return ErrnoMessage();
}
}
}
std::string ReadFile(const fs::path& file, std::string& out) {
const Descriptor in(::open(file.c_str(), O_RDONLY | O_CLOEXEC));
if (!in.Valid()) return ErrnoMessage();
return ReadAll(in.Get(), out);
}
struct Line {
std::string_view text; // without the newline and a '\r' before it
std::size_t begin = 0; // offset of the line's first byte
std::size_t next = 0; // offset of the line after it
};
std::vector<Line> SplitLines(std::string_view text) {
std::vector<Line> lines;
std::size_t pos = 0;
while (pos < text.size()) {
const std::size_t newline = text.find('\n', pos);
const std::size_t end = newline == std::string_view::npos ? text.size() : newline;
std::string_view line = text.substr(pos, end - pos);
if (line.ends_with('\r')) line.remove_suffix(1);
const std::size_t next = newline == std::string_view::npos ? text.size() : newline + 1;
lines.push_back({ .text = line, .begin = pos, .next = next });
pos = next;
}
return lines;
}
std::string_view TrimEnd(std::string_view text) {
const std::size_t last = text.find_last_not_of(" \t");
return last == std::string_view::npos ? std::string_view() : text.substr(0, last + 1);
}
// A line with the block quote markers in front of it taken off.
struct Shape {
std::size_t quotes = 0; // `>` markers
std::size_t indent = 0; // columns of indentation after them
std::string_view rest; // the line from its first other character
};
Shape ShapeOf(std::string_view line) {
Shape shape;
std::size_t pos = 0;
for (;;) {
std::size_t probe = pos;
while (probe < line.size() && probe - pos < 3 && line[probe] == ' ') ++probe;
if (probe >= line.size() || line[probe] != '>') break;
pos = probe + 1;
if (pos < line.size() && line[pos] == ' ') ++pos;
++shape.quotes;
}
for (; pos < line.size() && (line[pos] == ' ' || line[pos] == '\t'); ++pos) {
shape.indent += line[pos] == '\t' ? 4 - shape.indent % 4 : 1;
}
shape.rest = line.substr(pos);
return shape;
}
// The columns a list item marker at the start of `text` takes, with the
// spaces after it: `-`, `+` or `*`, or up to nine digits and `.` or `)`,
// then at least one space. Zero when `text` starts with none.
std::size_t ListMarker(std::string_view text) {
std::size_t pos = 0;
if (!text.empty() && (text[0] == '-' || text[0] == '+' || text[0] == '*')) {
pos = 1;
} else {
while (pos < text.size() && pos < 9 && text[pos] >= '0' && text[pos] <= '9') ++pos;
if (pos == 0 || pos >= text.size() || (text[pos] != '.' && text[pos] != ')')) return 0;
++pos;
}
const std::size_t marker = pos;
while (pos < text.size() && text[pos] == ' ') ++pos;
return pos == marker ? 0 : pos;
}
struct Fence {
char ch = '\0';
std::size_t length = 0;
std::string_view after; // the rest of the line behind the run
};
// The run of three or more backticks or tildes `text` starts with.
std::optional<Fence> FenceRun(std::string_view text) {
if (text.empty() || (text[0] != '`' && text[0] != '~')) return std::nullopt;
std::size_t length = 0;
while (length < text.size() && text[length] == text[0]) ++length;
if (length < 3) return std::nullopt;
return Fence{ .ch = text[0], .length = length, .after = text.substr(length) };
}
// Fenced code blocks, close to how CommonMark reads them. A run of three
// or more backticks or tildes opens one after at most three spaces of
// indentation; a backtick run only when no backtick follows it on the
// line, since ```make``` is inline code. A run of the same character at
// least as long, with nothing behind it but spaces, closes it. Keeping
// the character and the length is what lets a ~~~ block quote a ``` one.
//
// Block quotes and list items are followed as far as fences need: the
// `>` markers come off every line, list markers off the line that opens
// a block, and a block ends with the container it sits in, at a line with
// fewer `>` or one indented less than its list item's content. Read
// without them, `- ```bash` was missed while its indented closer was
// taken for an opener, and the rest of the file came out inside out. Not
// followed: indented code blocks, and the continuation lines of a list
// item before a fence that opens there.
class CodeFences {
public:
// Feed the lines in order. True for a fence and for every line
// inside a block.
bool InCode(std::string_view line) {
const Shape shape = ShapeOf(line);
if (open_ != '\0') {
const bool blank = TrimEnd(shape.rest).empty();
const bool left = !blank && (shape.quotes < quotes_ || shape.indent < content_);
if (!left) {
const std::optional<Fence> fence = FenceRun(shape.rest);
if (fence && fence->ch == open_ && fence->length >= length_ && TrimEnd(fence->after).empty() && shape.indent <= content_ + 3) open_ = '\0';
return true;
}
// The container ended, and the block with it; the line is
// read again as one outside any block.
open_ = '\0';
}
if (shape.indent > 3) return false;
std::size_t column = shape.indent;
std::string_view text = shape.rest;
std::size_t content = 0;
for (std::size_t marker = ListMarker(text); marker > 0; marker = ListMarker(text)) {
column += marker;
text.remove_prefix(marker);
content = column;
}
const std::optional<Fence> fence = FenceRun(text);
if (!fence || (fence->ch == '`' && fence->after.contains('`'))) return false;
open_ = fence->ch;
length_ = fence->length;
quotes_ = shape.quotes;
content_ = content;
++opened_;
return true;
}
bool Open() const { return open_ != '\0'; }
// How many blocks have opened so far.
std::size_t Opened() const { return opened_; }
private:
char open_ = '\0';
std::size_t length_ = 0;
std::size_t quotes_ = 0; // the block quote depth it opened at
std::size_t content_ = 0; // its list item's content column; 0 outside a list
std::size_t opened_ = 0;
};
// Where a text's marker lines are, outside code blocks, as indices into
// `lines`.
struct MarkerScan {
std::vector<Line> lines;
std::vector<std::size_t> begins;
std::vector<std::size_t> ends;
// The line a code block that never closes opens on.
std::optional<std::size_t> unclosedBlock;
};
MarkerScan ScanMarkers(std::string_view text) {
MarkerScan scan;
scan.lines = SplitLines(text);
CodeFences fences;
for (std::size_t i = 0; i < scan.lines.size(); ++i) {
const std::size_t opened = fences.Opened();
const bool code = fences.InCode(scan.lines[i].text);
if (fences.Opened() != opened) scan.unclosedBlock = i;
if (code) continue;
const std::string_view line = TrimEnd(scan.lines[i].text);
if (line == BeginMarker) scan.begins.push_back(i);
if (line == EndMarker) scan.ends.push_back(i);
}
if (!fences.Open()) scan.unclosedBlock.reset();
return scan;
}
// Why `text` cannot go into the section as it is; empty when it can. A
// marker line of its own would end or restart the section for whoever
// reads it, a template copied from the result included, and a code block
// left open would run on over the notes after it and the end marker.
// Lines are numbered from `firstLine`.
std::string NoteProblem(std::string_view text, std::size_t firstLine) {
const MarkerScan scan = ScanMarkers(text);
std::optional<std::size_t> marker;
if (!scan.begins.empty()) marker = scan.begins[0];
if (!scan.ends.empty()) marker = std::min(marker.value_or(scan.ends[0]), scan.ends[0]);
if (marker) return std::format("line {} is a dependency notes marker", *marker + firstLine);
if (scan.unclosedBlock) return std::format("line {} opens a code block that never closes", *scan.unclosedBlock + firstLine);
return "";
}
struct NormalizedNote {
std::string text;
// The line of the file `text` starts on.
std::size_t firstLine = 1;
};
// A notes file as it goes into the section: LF line ends, no blank lines
// in front and no whitespace behind, so the section spaces its notes the
// same whatever the files end with.
NormalizedNote NormalizeNote(std::string_view raw) {
std::string text;
text.reserve(raw.size());
for (std::size_t i = 0; i < raw.size(); ++i) {
if (raw[i] == '\r' && i + 1 < raw.size() && raw[i + 1] == '\n') continue;
text.push_back(raw[i]);
}
const std::size_t last = text.find_last_not_of(" \t\r\n");
if (last == std::string::npos) return {};
const std::size_t first = text.find_first_not_of(" \t\r\n");
const std::size_t newline = text.rfind('\n', first);
const std::size_t begin = newline == std::string::npos ? 0 : newline + 1;
const auto skipped = std::count(text.begin(), text.begin() + static_cast<std::ptrdiff_t>(begin), '\n');
return { .text = text.substr(begin, last + 1 - begin), .firstLine = static_cast<std::size_t>(skipped) + 1 };
}
// `text` with `eol` ending its lines, whether they ended in LF or CRLF.
std::string WithLineEnds(std::string_view text, std::string_view eol) {
std::string out;
out.reserve(text.size());
for (std::size_t i = 0; i < text.size(); ++i) {
if (text[i] == '\r' && i + 1 < text.size() && text[i + 1] == '\n') continue;
if (text[i] == '\n') {
out += eol;
} else {
out.push_back(text[i]);
}
}
return out;
}
std::string RenderSection(std::span<const DependencyNote> notes, std::string_view eol) {
std::string section = std::format("{}{}{}{}", BeginMarker, eol, Preamble, eol);
for (const DependencyNote& note : notes) section += std::format("{0}<!-- from {1} -->{0}{2}{0}", eol, note.name, WithLineEnds(note.text, eol));
section += std::format("{0}{1}{0}", eol, EndMarker);
return section;
}
// The note beside a dependency's project file, or nullopt when there is
// none or it cannot be used.
std::optional<DependencyNote> ReadNote(const fs::path& file, const std::string& name) {
std::error_code ec;
if (!fs::is_regular_file(file, ec)) return std::nullopt;
std::string content;
if (const std::string error = ReadFile(file, content); !error.empty()) {
Warn(std::format("{} could not be read ({}), so its notes are left out", file.string(), error));
return std::nullopt;
}
NormalizedNote note = NormalizeNote(content);
if (note.text.empty()) return std::nullopt;
// On the very bytes the merge checks, so the two can never disagree,
// and numbered as the author's file is.
if (const std::string problem = NoteProblem(note.text, note.firstLine); !problem.empty()) {
Warn(std::format("{}: {}, so its notes are left out", file.string(), problem));
return std::nullopt;
}
return DependencyNote{ .name = name, .text = std::move(note.text) };
}
// The loaded projects `configs` lead to without passing through another
// loaded project, in the order the configurations name them: one ring
// further out. `walked` spans the whole collection, so each
// Configuration is looked at once however many paths reach it.
std::vector<const Configuration*> NextRing(std::span<const Configuration* const> configs, std::unordered_set<const Configuration*>& walked) {
std::vector<const Configuration*> ring;
std::vector<const Configuration*> pending(configs.rbegin(), configs.rend());
while (!pending.empty()) {
const Configuration* cfg = pending.back();
pending.pop_back();
if (!cfg || !walked.insert(cfg).second) continue;
if (LoadedProjectOf(*cfg)) {
ring.push_back(cfg);
continue;
}
for (auto dep = cfg->dependencies.rbegin(); dep != cfg->dependencies.rend(); ++dep) pending.push_back(*dep);
}
return ring;
}
// The AGENTS.consumer.md of every project `project` depends on, as
// project.dependencyNotes selects them, ring by ring: first the loaded
// projects (the ones GitProject and LocalProject returned) that its
// `#pragma crafter dependency` lines and its configurations name, in
// that order, then the ones those depend on. Its own configurations, and
// a dependency's internal ones, are only walked through. Direct stops
// after the first ring.
//
// Never walked: the project's tests, since a plain build never touches
// them and a test's dependency can point at the Configuration the project
// file returned by value, which has since moved; and a dependency's own
// pragma tooling, which is about how the dependency is built, not how it
// is used, and is no `dependencies` edge to begin with.
std::vector<DependencyNote> CollectNotes(const Configuration& project, const fs::path& projectFile, const fs::path& projectDir) {
std::unordered_set<const Configuration*> walked = { &project };
// The project's own notes are for its consumers, so they never go
// into its own AGENTS.md, whatever dependency they sit beside. Both
// places count for a symlinked project file.
std::unordered_set<fs::path> files = { projectDir / NotesFileName };
std::error_code ec;
if (const fs::path canonical = fs::canonical(projectFile, ec); !ec) files.insert(canonical.parent_path() / NotesFileName);
std::unordered_set<std::string> identities;
std::vector<DependencyNote> notes;
std::vector<const Configuration*> start;
for (const Configuration* dep : PragmaDependencies(projectFile)) start.push_back(dep);
start.insert(start.end(), project.dependencies.begin(), project.dependencies.end());
for (std::vector<const Configuration*> ring = NextRing(start, walked); !ring.empty();) {
std::vector<const Configuration*> next;
for (const Configuration* cfg : ring) {
const std::optional<LoadedProject> loaded = LoadedProjectOf(*cfg);
const fs::path file = loaded->projectFile.parent_path() / NotesFileName;
// One library under two specs (a pin here, another revision
// further down) is described once, as the ring nearest the
// project has it.
if (identities.insert(loaded->identity).second && files.insert(file).second) {
if (std::optional<DependencyNote> note = ReadNote(file, loaded->name)) notes.push_back(std::move(*note));
}
next.insert(next.end(), cfg->dependencies.begin(), cfg->dependencies.end());
}
if (project.dependencyNotes == DependencyNotes::Direct) break;
ring = NextRing(next, walked);
}
return notes;
}
// Whether a build may replace `agentsMd`, the AGENTS.md beside a
// template: one a build wrote, or one with no text at all, which is what
// a write cut short leaves of a new file. Anything else someone wrote,
// and beside a template that is easily the case: an AGENTS.md committed
// before the template was, or one an agent kept its own notes in.
bool Replaceable(std::string_view agentsMd) {
if (agentsMd.find_first_not_of(" \t\r\n") == std::string_view::npos) return true;
std::string_view first = agentsMd.substr(0, agentsMd.find('\n'));
if (first.ends_with('\r')) first.remove_suffix(1);
return first == GeneratedLine;
}
// An AGENTS.md with no template beside it is someone's own, and no build
// writes it. One that still holds a section from the builds that kept the
// notes in a committed AGENTS.md would go stale without a word, under a
// preamble saying every build rewrites it, so each build says so until
// the file has moved to a template or the section is gone.
void WarnStaleSection(const fs::path& agents) {
std::error_code ec;
if (!fs::is_regular_file(agents, ec)) return;
std::string text;
if (!ReadFile(agents, text).empty()) return;
const MarkerScan scan = ScanMarkers(text);
if (scan.begins.empty() && scan.ends.empty()) return;
Warn(std::format("{} has a dependency notes section, which builds no longer update; rename the file to {} and keep {} out of version control to have every build write it with current notes, or delete the section", agents.string(), TemplateFileName, AgentsFileName));
}
// Bring `fd`, which holds `current`, to `wanted`: from the first byte the
// two differ at, so the text in front of the section is never rewritten,
// then cut to length and synced. Empty on success, otherwise the error.
//
// In place rather than through a temporary renamed over the file: a
// rename swaps in another file, which drops whatever belongs to the old
// one (a hard link such as a CLAUDE.md beside it, ACLs, ownership, being
// read-only) and needs fsyncs to be crash safe at all. The price is that
// a reader can catch the file half written, and that a write failing
// partway, on a full disk say, leaves it so until the next build writes
// it again.
std::string WriteChanged(std::int32_t fd, std::string_view current, std::string_view wanted) {
const auto diverge = std::ranges::mismatch(current, wanted);
for (auto done = static_cast<std::size_t>(diverge.in2 - wanted.begin()); done < wanted.size();) {
const auto wrote = ::pwrite(fd, wanted.data() + done, wanted.size() - done, static_cast<off_t>(done));
if (wrote > 0) {
done += static_cast<std::size_t>(wrote);
} else if (wrote == 0) {
return "the write made no progress";
} else if (errno != EINTR) {
return ErrnoMessage();
}
}
if (::ftruncate(fd, static_cast<off_t>(wanted.size())) != 0) return ErrnoMessage();
if (::fsync(fd) != 0) return ErrnoMessage();
return "";
}
}
std::string Crafter::MergeDependencyNotes(std::string_view templateMd, std::span<const DependencyNote> notes) {
for (const DependencyNote& note : notes) {
if (const std::string problem = NoteProblem(note.text, 1); !problem.empty()) throw std::runtime_error(std::format("the notes from {}: {}", note.name, problem));
}
const MarkerScan scan = ScanMarkers(templateMd);
const bool section = scan.begins.size() == 1 && scan.ends.size() == 1 && scan.begins[0] < scan.ends[0];
const std::string_view eol = LineEnd(templateMd);
if (notes.empty()) {
// With nothing to place, nothing can land in the wrong place, so
// anything short of a well-formed section is left as it was written.
if (!section) return std::string(templateMd);
std::string_view before = templateMd.substr(0, scan.lines[scan.begins[0]].begin);
const std::string_view after = templateMd.substr(scan.lines[scan.ends[0]].next);
// A section that ends the template takes the blank line in front of
// it, which an appended one gets. One placed elsewhere keeps the text
// around it apart.
if (after.empty() && before.ends_with(std::format("{0}{0}", eol))) before.remove_suffix(eol.size());
return std::format("{}{}", before, after);
}
if (scan.begins.size() > 1) throw std::runtime_error(std::format("line {}: a second dependency notes begin marker (the first is on line {})", scan.begins[1] + 1, scan.begins[0] + 1));
if (scan.ends.size() > 1) throw std::runtime_error(std::format("line {}: a second dependency notes end marker (the first is on line {})", scan.ends[1] + 1, scan.ends[0] + 1));
if (!scan.ends.empty() && (scan.begins.empty() || scan.ends[0] < scan.begins[0])) throw std::runtime_error(std::format("line {}: a dependency notes end marker with no begin marker before it", scan.ends[0] + 1));
if (!scan.begins.empty() && scan.ends.empty()) throw std::runtime_error(std::format("line {}: a dependency notes begin marker with no end marker after it", scan.begins[0] + 1));
const std::string rendered = RenderSection(notes, eol);
if (section) return std::format("{}{}{}", templateMd.substr(0, scan.lines[scan.begins[0]].begin), rendered, templateMd.substr(scan.lines[scan.ends[0]].next));
// Appended, the section would sit inside that block and read as code.
if (scan.unclosedBlock) throw std::runtime_error(std::format("line {}: a code block opens here and never closes", *scan.unclosedBlock + 1));
if (templateMd.empty()) return rendered;
// On a line of its own, after a blank one.
return std::format("{}{}{}{}", templateMd, templateMd.ends_with('\n') ? std::string_view() : eol, eol, rendered);
}
bool Crafter::UpdateAgentsMd(const Configuration& project, const fs::path& projectFile) {
std::error_code ec;
// Beside the project file as it was named, so a symlinked project.cpp
// keeps the AGENTS.md beside the link.
const fs::path projectDir = fs::absolute(projectFile, ec).lexically_normal().parent_path();
if (ec) return false;
const fs::path agents = projectDir / AgentsFileName;
const fs::path templateFile = projectDir / TemplateFileName;
if (!fs::is_regular_file(templateFile, ec)) {
WarnStaleSection(agents);
return false;
}
// Linked to the template, AGENTS.md would have every build write the
// file that is committed, which is what the template is there to stop.
if (fs::equivalent(agents, templateFile, ec)) {
Warn(std::format("{} and {} are the same file, so {} was not written", agents.string(), templateFile.string(), AgentsFileName));
return false;
}
std::string templateMd;
if (const std::string error = ReadFile(templateFile, templateMd); !error.empty()) {
Warn(std::format("{} could not be read ({}), so {} was not written", templateFile.string(), error, agents.string()));
return false;
}
const std::vector<DependencyNote> notes = project.dependencyNotes == DependencyNotes::None ? std::vector<DependencyNote>() : CollectNotes(project, projectFile, projectDir);
std::string wanted;
try {
wanted = std::format("{}{}{}", GeneratedLine, LineEnd(templateMd), MergeDependencyNotes(templateMd, notes));
} catch (const std::exception& e) {
Warn(std::format("{}: {}, so {} was not written", templateFile.string(), e.what(), agents.string()));
return false;
}
// Created when missing, since a fresh clone has no build output, and only
// now, with the text in hand, so a template the notes cannot be placed
// in leaves no empty AGENTS.md behind. Locked from the read through the
// write, so two builds never interleave their writes, and on the file
// itself, which also covers two projects that link one AGENTS.md. A
// read-only descriptor takes the lock as well, so a read-only AGENTS.md
// is still read without complaint.
const Descriptor reading(::open(agents.c_str(), O_RDONLY | O_CREAT | O_CLOEXEC, 0666));
if (!reading.Valid()) {
Warn(std::format("{} could not be opened ({}), so it was not written", agents.string(), ErrnoMessage()));
return false;
}
while (::flock(reading.Get(), LOCK_EX) != 0 && errno == EINTR) {}
std::string current;
if (const std::string error = ReadAll(reading.Get(), current); !error.empty()) {
Warn(std::format("{} could not be read ({}), so it was not written", agents.string(), error));
return false;
}
if (current == wanted) return false;
if (!Replaceable(current)) {
Warn(std::format("{} was not written by a build, so it was not replaced by one written from {}; once the template holds what it says, delete it", agents.string(), TemplateFileName));
return false;
}
// Only now, with something to write: a read-only AGENTS.md is refused
// here, and not on every build that leaves it as it is.
const Descriptor writing(::open(agents.c_str(), O_WRONLY | O_CLOEXEC));
if (!writing.Valid()) {
Warn(std::format("{} could not be opened for writing ({}), so it was not written", agents.string(), ErrnoMessage()));
return false;
}
if (const std::string error = WriteChanged(writing.Get(), current, wanted); !error.empty()) {
// Put back what was there. A full disk that refused the new bytes
// usually still takes the old ones, whose space was just freed.
if (WriteChanged(writing.Get(), wanted, current).empty()) {
Warn(std::format("{} could not be written ({}), so it was left as it was", agents.string(), error));
} else {
Warn(std::format("{} could not be written ({}) and may now be half written; the next build writes it again", agents.string(), error));
}
return false;
}
// A file that appears beside project.cpp, or changes there, unasked,
// says where it came from.
if (Progress::GetVerbosity() != Progress::Verbosity::Quiet) {
Progress::Clear();
if (notes.empty()) {
std::println("Wrote {} from {}", agents.string(), TemplateFileName);
} else {
std::println("Wrote {} from {} and the notes of {} {}", agents.string(), TemplateFileName, notes.size(), notes.size() == 1 ? "dependency" : "dependencies");
}
}
return true;
}

View file

@ -13,6 +13,7 @@ import :Toolchain;
import :Test; import :Test;
import :Lint; import :Lint;
import :Script; import :Script;
import :Agents;
import :Progress; import :Progress;
import :Asset; import :Asset;
namespace fs = std::filesystem; namespace fs = std::filesystem;
@ -246,20 +247,57 @@ void Configuration::GetInterfacesAndImplementations(std::span<fs::path> interfac
} }
} }
fs::path Crafter::StdPcmDir(const Configuration& config) { std::string Crafter::StdPcmFlags(const Configuration& config) {
// The std PCM is cached per target+march+toolchain: a PCM is only valid if (config.target == "x86_64-w64-mingw32") {
// for the exact clang that wrote it, so a compiler bump starts a cold // -femulated-tls keeps PCM TLS access matching libstdc++'s emutls
// directory instead of handing the new clang a PCM it will crash on. A // definitions; a mismatch surfaces as undefined std::__once_callable.
// wasm variant pass compiles it with extra codegen flags (e.g. return std::format(" --target={} -march={} -mtune={} -femulated-tls -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier", config.target, config.march, config.mtune);
// -mrelaxed-simd) — its BMI must not be shared with the baseline's, or
// the consuming TUs see a target-feature mismatch. Suffix the cache dir
// with the variant flags when present.
std::string stdPcmKey = std::format("{}-{}-{}", config.target, config.march, GetToolchain().Key());
for (const std::string& f : config.wasmVariantFlags) {
stdPcmKey += "+";
for (char c : f) stdPcmKey += (c == '/' || c == '\\') ? '_' : c;
} }
return GetCacheDir()/stdPcmKey; const bool isWasm = config.target.starts_with("wasm32");
const std::string sysrootFlag = config.sysroot.empty() ? std::string() : std::format(" --sysroot={}", config.sysroot);
// wasm32 rejects -march, and the std module has to be compiled the way
// the units importing it are, a variant pass's flags (e.g.
// -mrelaxed-simd) included, or the BMI is not feature-compatible.
std::string archFlags = isWasm ? WasmCompileFlags(config) : std::format(" -march={} -mtune={}", config.march, config.mtune);
// non-wasm cross builds: the driver ranks the host toolchain's own
// libc++ headers (<install>/../include/c++/v1) above --sysroot, so the
// PCM would mix host libc++ with the foreign libc. Force the sysroot's
// copy.
if (!isWasm && !config.sysroot.empty()) {
archFlags += std::format(" -nostdinc++ -isystem {}/usr/include/c++/v1", config.sysroot);
}
return std::format(" --target={} -std=c++26{}{} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier", config.target, sysrootFlag, archFlags);
}
std::string Crafter::StdPcmKey(const Configuration& config, const Toolchain& toolchain) {
// A PCM is only valid for the clang that wrote it and the flags it was
// written with, so a compiler bump starts a cold directory instead of
// handing the new clang a PCM it will crash on, and so does any flag.
// With only some of them in the name, configurations that differed in
// the rest took turns rebuilding one std PCM in place: -mtune=generic
// against project.cpp's -mtune=native on every run, two sysroots, or two
// releases sharing a cache, as the Flatpak's branches do, each time under
// the PCMs built against the previous one, which clang crashes on rather
// than rejecting.
std::string compiled = StdPcmFlags(config);
// The base command's part: the toolchain's key covers the compiler and
// libc++, and for the mingw target what clang finds mingw-w64 through is
// the prefix, given whole or through a view (MingwSysroot), named here by
// the prefix rather than by the view's own path.
if (config.target == "x86_64-w64-mingw32" && !toolchain.mingwPrefix.empty()) {
compiled += std::format("|mingw:{}|{}", toolchain.mingwPrefix.string(), toolchain.mingwPrefixIsSystem ? "view" : "whole");
}
return std::format("{}-{}-{}-{:08x}", config.target, config.march, toolchain.Key(), static_cast<std::uint32_t>(std::hash<std::string>{}(compiled)));
}
fs::path Crafter::StdPcmDir(const Configuration& config) {
return GetCacheDir() / StdPcmKey(config, GetToolchain());
}
std::string Crafter::WasmCompileFlags(const Configuration& config) {
std::string flags = std::format(" {}", WasmBaselineFlags);
for (const std::string& f : config.wasmVariantFlags) flags += std::format(" {}", f);
return flags;
} }
fs::path Crafter::ExpectedOutput(const Configuration& config) { fs::path Crafter::ExpectedOutput(const Configuration& config) {
@ -316,15 +354,11 @@ CompileCommand Crafter::GetCompileCommand(const Configuration& config) {
if (isWasm) { if (isWasm) {
// -mllvm is consumed by codegen but not the link driver, which is the // -mllvm is consumed by codegen but not the link driver, which is the
// same command line; quiet the unused-flag warning rather than split // same command line; quiet the unused-flag warning rather than split
// compile and link commands. // compile and link commands. An active variant pass (see the variant
out.command += " -fno-exceptions -msimd128 -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL -Wno-unused-command-line-argument"; // driver at the end of Build) adds its codegen flags (e.g.
// Active variant pass (see the variant driver at the end of Build): // -mrelaxed-simd) to every TU; they are part of VariantId, so those
// extra codegen flags (e.g. -mrelaxed-simd) applied to every TU. Empty // objects/PCMs land in their own dir.
// for the baseline build. Part of VariantId, so these objects/PCMs out.command += std::format("{} -Wno-unused-command-line-argument", WasmCompileFlags(config));
// land in their own dir.
for (const std::string& f : config.wasmVariantFlags) {
out.command += std::format(" {}", f);
}
} }
if (config.target == "x86_64-w64-mingw32") { if (config.target == "x86_64-w64-mingw32") {
// mingw libstdc++ defines TLS via __emutls_v.* (emulated TLS); without // mingw libstdc++ defines TLS via __emutls_v.* (emulated TLS); without
@ -404,12 +438,14 @@ CompileCommand Crafter::GetCompileCommand(const Configuration& config) {
// (cross-TU inlining) plus dead-section GC and safe identical-code folding. // (cross-TU inlining) plus dead-section GC and safe identical-code folding.
// Default-on in Release, never Debug (keeps builds fast and debuggable). // Default-on in Release, never Debug (keeps builds fast and debuggable).
// Excluded for wasm32 — its -mllvm/sjlj codegen flags don't compose with // Excluded for wasm32 — its -mllvm/sjlj codegen flags don't compose with
// LTO here — and for nvcc .cu objects below (they can't emit LLVM bitcode, // LTO here, and wasi-sdk lists exceptions under LTO as broken
// so they link in as ordinary objects alongside the bitcode ones). ThinLTO // (WebAssembly/wasi-sdk#629) — and for nvcc .cu objects below (they can't
// rather than monolithic -flto so link time and memory scale with arbitrary // emit LLVM bitcode, so they link in as ordinary objects alongside the
// user project sizes. Compile-side flags (-flto, -ffunction/-fdata-sections) // bitcode ones). ThinLTO rather than monolithic -flto so link time and
// ride on `command`, which is reused as the link base; the link-only -Wl // memory scale with arbitrary user project sizes. Compile-side flags
// flags go on linkExtras so they don't warn on each -c compile. // (-flto, -ffunction/-fdata-sections) ride on `command`, which is reused
// as the link base; the link-only -Wl flags go on linkExtras so they
// don't warn on each -c compile.
out.useLto = !config.debug && !isWasm; out.useLto = !config.debug && !isWasm;
out.ltoCompileFlags = out.useLto ? " -flto=thin -ffunction-sections -fdata-sections" : ""; out.ltoCompileFlags = out.useLto ? " -flto=thin -ffunction-sections -fdata-sections" : "";
out.ltoLinkFlags = out.useLto ? " -flto=thin -Wl,--gc-sections -Wl,--icf=safe" : ""; out.ltoLinkFlags = out.useLto ? " -flto=thin -Wl,--gc-sections -Wl,--icf=safe" : "";
@ -479,6 +515,54 @@ namespace {
walk(config, config.name); walk(config, config.name);
return error; return error;
} }
// What a wasm32 configuration cannot be built with now that every unit
// has C++ exceptions, and what its sysroot has to have for that; empty
// for any other target and when nothing is wrong. Each would otherwise
// fail at the first compile or link with a message that does not say
// why, or not fail at all.
std::string WasmBuildError(const Configuration& config) {
if (!config.target.starts_with("wasm32")) return "";
for (const std::vector<std::string>* flags : { &config.compileFlags, &config.linkFlags }) {
for (const std::string& flag : *flags) {
// The compile has the backend options that keep exception
// handling; LTO code generation at the link does not, so it
// drops every landing pad and the build succeeds with throws
// that escape their catch blocks (WebAssembly/wasi-sdk#629).
if (flag.starts_with("-flto")) {
return std::format("{}: {} is not supported on wasm32: LTO code generation drops C++ exception handling (WebAssembly/wasi-sdk#629), so every throw would escape its catch", config.name, flag);
}
// The std module is built with exceptions, and a unit built
// without them cannot import it.
if (flag == "-fno-exceptions" || flag == "-fno-cxx-exceptions") {
return std::format("{}: {} is not supported on wasm32: wasm builds have C++ exceptions, the std module included, and a unit compiled without them cannot import std", config.name, flag);
}
}
}
// wasi-sdk's exception support does not cover shared libraries: the
// libc++ under lib/<triple>/eh is static archives built without -fPIC.
if (config.type == ConfigurationType::LibraryDynamic) {
return std::format("{}: wasm32 shared libraries are not supported: the exception-enabled libc++ they would link has static archives only", config.name);
}
const fs::path sysroot = config.sysroot;
if (!fs::is_directory(sysroot)) {
return std::format("no WASI sysroot at {}: the Flatpak bundles one, and CRAFTER_BUILD_WASI_SYSROOT selects another", sysroot.string());
}
const fs::path libDir = sysroot / "lib" / NormalizeTarget(config.target);
if (!fs::is_directory(libDir)) {
return std::format("the WASI sysroot at {} has nothing for {} (no {})", sysroot.string(), config.target, libDir.string());
}
// Only a link takes the exception-enabled libraries: a static
// library's objects compile against the headers alone. The driver
// looks in eh/ first and then in the triple's own directory, where a
// sysroot built with exceptions only (WASI_SDK_EXCEPTIONS=ON) has them.
// Without them the first link fails with "unable to find library
// -lunwind", which says nothing about why it was asked for.
if (config.type == ConfigurationType::Executable && !fs::exists(libDir / "eh" / "libunwind.a") && !fs::exists(libDir / "libunwind.a")) {
return std::format("the WASI sysroot at {} has no C++ exception support (no {}): wasm builds use WebAssembly exception handling and need the exception-enabled libc++ and libunwind that wasi-sdk 33 and later ship. The Flatpak bundles such a sysroot; CRAFTER_BUILD_WASI_SYSROOT selects another", sysroot.string(), (libDir / "eh" / "libunwind.a").string());
}
return "";
}
} }
BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex) { BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex) {
@ -539,6 +623,11 @@ BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, s
if (config.sysroot.empty() && config.target.starts_with("wasm32")) { if (config.sysroot.empty() && config.target.starts_with("wasm32")) {
config.sysroot = GetToolchain().wasiSysroot.string(); config.sysroot = GetToolchain().wasiSysroot.string();
} }
// As early as the duplicate-module check, so a refused build leaves no
// directory behind and starts no thread, but after the sysroot is known.
if (std::string refused = WasmBuildError(config); !refused.empty()) {
return {std::move(refused), false, {}};
}
fs::path buildDir = config.BuildDir(); fs::path buildDir = config.BuildDir();
fs::path outputDir = config.BinDir(); fs::path outputDir = config.BinDir();
@ -856,21 +945,22 @@ BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, s
// Same target-aware setup as the C++ compile path (line 459-): wasm32 // Same target-aware setup as the C++ compile path (line 459-): wasm32
// rejects -march, silently ignores -mtune, and needs --sysroot to find // rejects -march, silently ignores -mtune, and needs --sysroot to find
// wasi-libc headers. Build the prefix once so all C compiles share it. // wasi-libc headers. Build the prefix once so all C compiles share it.
const bool cIsWasm = config.target.starts_with("wasm32"); std::string cArchFlags = isWasm
std::string cArchFlags = cIsWasm
? std::string() ? std::string()
: std::format(" -march={} -mtune={}", config.march, config.mtune); : std::format(" -march={} -mtune={}", config.march, config.mtune);
if (!config.sysroot.empty()) { if (!config.sysroot.empty()) {
cArchFlags += std::format(" --sysroot={}", config.sysroot); cArchFlags += std::format(" --sysroot={}", config.sysroot);
} else if (config.target == "x86_64-w64-mingw32" && !GetToolchain().mingwSysroot.empty()) { } else if (config.target == "x86_64-w64-mingw32") {
// The C++ side gets it through GetBaseCommand. // The C++ side gets it through GetBaseCommand.
cArchFlags += std::format(" --sysroot={}", GetToolchain().mingwSysroot.string()); if (const fs::path& sysroot = MingwSysroot(); !sysroot.empty()) cArchFlags += std::format(" --sysroot={}", sysroot.string());
} }
if (cIsWasm) { if (isWasm) {
// Matches the C++ path's wasi flag set so any libc shim defines // The C++ path's wasm flags, all of them, a variant pass's included.
// (e.g. _WASI_EMULATED_SIGNAL → signal.h shims) are visible to // With only the signal.h define, wasi-libc's setjmp.h refused to
// C dependencies that drag in signal.h transitively. // compile in C, and a C object given -wasm-enable-sjlj through
cArchFlags += " -D_WASI_EMULATED_SIGNAL"; // compileFlags used LLVM's legacy exception encoding next to C++
// objects in the standard one, a mix engines refuse to load.
cArchFlags += WasmCompileFlags(config);
} }
for (const fs::path& cFile : config.cFiles) { for (const fs::path& cFile : config.cFiles) {
files += std::format(" {}_source.o ", (buildDir / cFile.filename()).string()); files += std::format(" {}_source.o ", (buildDir / cFile.filename()).string());
@ -1138,6 +1228,17 @@ BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, s
// compile so __once_callable et al resolve in static libstdc++. // compile so __once_callable et al resolve in static libstdc++.
linkExtras += " -lstdc++exp -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread -Wl,-Bdynamic"; linkExtras += " -lstdc++exp -static-libstdc++ -static-libgcc -Wl,-Bstatic -lpthread -Wl,-Bdynamic";
} }
// Code built with -fwasm-exceptions calls into libunwind (the
// __cpp_exception tag, the landing-pad context, _Unwind_RaiseException),
// and -wasm-enable-sjlj turns setjmp and longjmp into calls into
// wasi-libc's libsetjmp (__wasm_setjmp, __wasm_longjmp, the __c_longjmp
// tag). The driver adds neither to the link the way it adds libc++. A
// static archive contributes only what something references, so a
// program without setjmp gets nothing from libsetjmp, and every sysroot
// that passes WasmBuildError ships both.
if (isWasm) {
linkExtras += " -lunwind -lsetjmp";
}
// Force a relink if the expected output is missing or older than any dep // Force a relink if the expected output is missing or older than any dep
// artifact. Missing covers: previous build produced a different outputName, // artifact. Missing covers: previous build produced a different outputName,
@ -1481,10 +1582,13 @@ void Crafter::EnableWasiBrowserRuntime(Configuration& cfg) {
// Variant manifest. runtime.js fetches this, runs each entry's probes in // Variant manifest. runtime.js fetches this, runs each entry's probes in
// order, and instantiates the first whose probes all pass — so the // order, and instantiates the first whose probes all pass — so the
// preferred (most feature-rich) variants come first and the baseline last // preferred (most feature-rich) variants come first and the baseline last
// as the universal fallback (empty probes always pass). A variant with a // as the fallback. Every entry carries WasmBaselineProbes besides its own:
// non-empty label maps to outputName.<label>.wasm (produced by Build's // every file is built with the baseline flags, so an engine without
// variant driver); the baseline maps to outputName.wasm. Even with no // exception handling can run none of them, and runtime.js tells it so
// declared variants we still emit a one-entry manifest so the runtime has a // instead of failing to compile the module. A variant with a non-empty
// label maps to outputName.<label>.wasm (produced by Build's variant
// driver); the baseline maps to outputName.wasm. Even with no declared
// variants we still emit a one-entry manifest so the runtime has a
// single, uniform code path. // single, uniform code path.
auto jsonEscape = [](std::string_view s) { auto jsonEscape = [](std::string_view s) {
std::string out; std::string out;
@ -1499,12 +1603,14 @@ void Crafter::EnableWasiBrowserRuntime(Configuration& cfg) {
std::ofstream v(variantsPath); std::ofstream v(variantsPath);
v << "["; v << "[";
bool first = true; bool first = true;
auto emit = [&](const std::string& label, const std::vector<std::string>& probes) { auto emit = [&](const std::string& label, const std::vector<std::string>& ownProbes) {
if (!first) v << ","; if (!first) v << ",";
first = false; first = false;
std::string url = label.empty() std::string url = label.empty()
? std::format("{}.wasm", cfg.outputName) ? std::format("{}.wasm", cfg.outputName)
: std::format("{}.{}.wasm", cfg.outputName, label); : std::format("{}.{}.wasm", cfg.outputName, label);
std::vector<std::string> probes = ownProbes;
probes.insert(probes.end(), WasmBaselineProbes.begin(), WasmBaselineProbes.end());
v << "{\"label\":\"" << jsonEscape(label) << "\"" v << "{\"label\":\"" << jsonEscape(label) << "\""
<< ",\"url\":\"" << jsonEscape(url) << "\"" << ",\"url\":\"" << jsonEscape(url) << "\""
<< ",\"probes\":["; << ",\"probes\":[";
@ -1755,6 +1861,15 @@ Clean (after the `clean` subcommand):
so it works when the project itself no longer compiles. Every target, variant so it works when the project itself no longer compiles. Every target, variant
and dependency artifact under those trees goes with it. and dependency artifact under those trees goes with it.
AGENTS.md:
When an AGENTS.template.md sits beside the project file, a build and a test
run write AGENTS.md from it, with a section holding the AGENTS.consumer.md
each dependency ships beside its project.cpp, and only when the text changes.
Commit the template and keep AGENTS.md out of version control. An AGENTS.md
no build wrote is never replaced, and without a template AGENTS.md is left
alone. cfg.dependencyNotes picks the dependencies: Transitive (the default),
Direct, or None.
Project args: Project args:
Any flag not consumed above is forwarded verbatim to CrafterBuildProject as Any flag not consumed above is forwarded verbatim to CrafterBuildProject as
part of its `args` span. Project-specific flags (e.g. --target=, custom part of its `args` span. Project-specific flags (e.g. --target=, custom
@ -1775,9 +1890,11 @@ Environment:
CRAFTER_BUILD_LIBCLANG Exact libclang to load for lint. CRAFTER_BUILD_LIBCLANG Exact libclang to load for lint.
CRAFTER_BUILD_WASI_SYSROOT WASI sysroot for wasm32 targets. CRAFTER_BUILD_WASI_SYSROOT WASI sysroot for wasm32 targets.
CRAFTER_MINGW_DIR mingw-w64 prefix (holding x86_64-w64-mingw32/ CRAFTER_MINGW_DIR mingw-w64 prefix (holding x86_64-w64-mingw32/
and lib/gcc/), passed as --sysroot. Default: and lib/gcc/). Default: the mingw gcc on
the mingw gcc on PATH; in the Flatpak, the PATH; in the Flatpak, the host's /usr when
host's /usr when it has mingw-w64. it has mingw-w64. Of a prefix that holds a
Linux system too, such as /usr, clang sees
only those directories.
CRAFTER_BUILD_HOME Directory holding the module sources. CRAFTER_BUILD_HOME Directory holding the module sources.
Exit status: Exit status:
@ -1940,6 +2057,14 @@ std::int32_t Crafter::Run(std::int32_t argc, char** argv) {
return lintOpts.mode == LintMode::Check && !lintSummary.changedFiles.empty() ? 1 : 0; return lintOpts.mode == LintMode::Check && !lintSummary.changedFiles.empty() ? 1 : 0;
} }
// Builds and test runs only: lint and `format --check` promise to
// write nothing, `format` rewrites the project's sources and nothing
// else, `run` is for chores and `test --list` only lists. Here,
// before compiling, because loading has just fetched every
// dependency, so the notes are current, and an agent fixing a build
// that fails needs them.
if (!testOpts.listOnly) UpdateAgentsMd(config, projectFile);
if (runTests) { if (runTests) {
TestSummary summary = RunTests(config, testOpts, projectArgs); TestSummary summary = RunTests(config, testOpts, projectArgs);
return summary.AllPassed() ? 0 : 1; return summary.AllPassed() ? 0 : 1;

View file

@ -149,7 +149,7 @@ std::string BuildInjectedCMakeFlags(const fs::path& cmakeBuildDir, std::string_v
// The libstdc++ and the mingw-w64 install the project's own compiles // The libstdc++ and the mingw-w64 install the project's own compiles
// use (GetBaseCommand). // use (GetBaseCommand).
out += " -DCMAKE_CXX_FLAGS=-stdlib=libstdc++"; out += " -DCMAKE_CXX_FLAGS=-stdlib=libstdc++";
if (!tc.mingwSysroot.empty()) out += std::format(" -DCMAKE_SYSROOT={}", ShellQuote(tc.mingwSysroot.string())); if (const fs::path& sysroot = MingwSysroot(); !sysroot.empty()) out += std::format(" -DCMAKE_SYSROOT={}", ShellQuote(sysroot.string()));
} }
fs::path absBuildDir = fs::absolute(cmakeBuildDir); fs::path absBuildDir = fs::absolute(cmakeBuildDir);
out += std::format(" -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY={}", ShellQuote(absBuildDir.string())); out += std::format(" -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY={}", ShellQuote(absBuildDir.string()));
@ -179,6 +179,19 @@ std::string ConfigureCMake(const fs::path& cloneDir, const fs::path& cmakeBuildD
if (!needsConfigure) return ""; if (!needsConfigure) return "";
// A build directory configured with other inputs starts over instead of
// being reconfigured. CMake reconfigures in place without recompiling
// what an old compiler or --sysroot compiled (the Makefiles generator
// writes --sysroot into build.make, which make does not compare) and
// keeps the check results the old configure cached. A mingw external
// built against the host's /usr as the sysroot (issue #43) kept its
// objects reading __libc_single_threaded after the fix that way. The
// directory keeps its name, which projects point generated headers at.
if (fs::exists(cmakeBuildDir / "CMakeCache.txt")) {
std::error_code ec;
fs::remove_all(cmakeBuildDir, ec);
if (ec) return std::format("cannot clear {} for a fresh configure: {}", cmakeBuildDir.string(), ec.message());
}
if (!fs::exists(cmakeBuildDir)) fs::create_directories(cmakeBuildDir); if (!fs::exists(cmakeBuildDir)) fs::create_directories(cmakeBuildDir);
std::string cmd = std::format("cmake -S {} -B {}{}", ShellQuote(fs::absolute(cloneDir).string()), ShellQuote(fs::absolute(cmakeBuildDir).string()), injected); std::string cmd = std::format("cmake -S {} -B {}{}", ShellQuote(fs::absolute(cloneDir).string()), ShellQuote(fs::absolute(cmakeBuildDir).string()), injected);
@ -324,6 +337,10 @@ ExternalBuildResult Crafter::ExternalOutputs(const ExternalDependency& dep, std:
namespace { namespace {
std::mutex ProjectCacheMutex; std::mutex ProjectCacheMutex;
std::unordered_map<std::string, std::unique_ptr<Configuration>> ProjectCache; std::unordered_map<std::string, std::unique_ptr<Configuration>> ProjectCache;
// Keyed by the Configurations ProjectCache owns, which live as long as
// the process does, so the pointers never dangle. Guarded by
// ProjectCacheMutex as well.
std::unordered_map<const Configuration*, LoadedProject> LoadedProjects;
// Run the dep's project.cpp from its own directory so its relative paths // Run the dep's project.cpp from its own directory so its relative paths
// (cfg.path = "./", interfaces/, lib/, ...) resolve against the dep // (cfg.path = "./", interfaces/, lib/, ...) resolve against the dep
@ -353,6 +370,36 @@ namespace {
for (Shader& s : cfg.shaders) s.path = abs(s.path); for (Shader& s : cfg.shaders) s.path = abs(s.path);
return cfg; return cfg;
} }
// Into ProjectCache under `fullHash`, with `loaded` recorded beside it.
// When another thread stored the same spec first, its Configuration
// wins and `cfg` is dropped.
Configuration* StoreProject(std::string fullHash, Configuration cfg, LoadedProject loaded) {
std::lock_guard lock(ProjectCacheMutex);
if (auto it = ProjectCache.find(fullHash); it != ProjectCache.end()) {
return it->second.get();
}
auto stored = std::make_unique<Configuration>(std::move(cfg));
Configuration* ptr = stored.get();
ProjectCache.emplace(std::move(fullHash), std::move(stored));
LoadedProjects.emplace(ptr, std::move(loaded));
return ptr;
}
// `url` as it names a repository: without user info, which is where a
// token goes, and without trailing slashes or ".git", so specs that only
// spell the repository differently name the same one.
std::string RepositoryKey(std::string_view url) {
std::string key(url);
if (const std::size_t scheme = key.find("://"); scheme != std::string::npos) {
const std::size_t host = scheme + 3;
const std::size_t at = key.rfind('@', key.find('/', host));
if (at != std::string::npos && at >= host) key.erase(host, at + 1 - host);
}
while (key.ends_with('/')) key.pop_back();
if (key.ends_with(".git")) key.resize(key.size() - 4);
return key;
}
} }
Configuration* Crafter::GitProject(const GitProjectSpec& spec) { Configuration* Crafter::GitProject(const GitProjectSpec& spec) {
@ -406,14 +453,13 @@ Configuration* Crafter::GitProject(const GitProjectSpec& spec) {
std::vector<std::string_view> argViews(spec.args.begin(), spec.args.end()); std::vector<std::string_view> argViews(spec.args.begin(), spec.args.end());
Configuration cfg = LoadProjectFromRoot(projectPath, cloneDir, argViews); Configuration cfg = LoadProjectFromRoot(projectPath, cloneDir, argViews);
std::lock_guard lock(ProjectCacheMutex); const fs::path projectInRepo = spec.projectFile.lexically_normal();
if (auto it = ProjectCache.find(fullHash); it != ProjectCache.end()) { const fs::path below = projectInRepo.parent_path();
return it->second.get(); return StoreProject(std::move(fullHash), std::move(cfg), {
} .projectFile = projectPath.lexically_normal(),
auto stored = std::make_unique<Configuration>(std::move(cfg)); .name = below.empty() ? name : std::format("{}/{}", name, below.generic_string()),
Configuration* ptr = stored.get(); .identity = std::format("git|{}|{}", RepositoryKey(spec.source.url), projectInRepo.generic_string()),
ProjectCache.emplace(std::move(fullHash), std::move(stored)); });
return ptr;
} }
fs::path Crafter::GitFetch(const GitSource& source) { fs::path Crafter::GitFetch(const GitSource& source) {
@ -468,12 +514,15 @@ Configuration* Crafter::LocalProject(const LocalProjectSpec& spec) {
std::vector<std::string_view> argViews(spec.args.begin(), spec.args.end()); std::vector<std::string_view> argViews(spec.args.begin(), spec.args.end());
Configuration cfg = LoadProjectFromRoot(canonical, depRoot, argViews); Configuration cfg = LoadProjectFromRoot(canonical, depRoot, argViews);
return StoreProject(std::move(fullHash), std::move(cfg), {
.projectFile = canonical,
.name = depRoot.filename().string(),
.identity = std::format("local|{}", canonical.string()),
});
}
std::optional<LoadedProject> Crafter::LoadedProjectOf(const Configuration& cfg) {
std::lock_guard lock(ProjectCacheMutex); std::lock_guard lock(ProjectCacheMutex);
if (auto it = ProjectCache.find(fullHash); it != ProjectCache.end()) { if (auto it = LoadedProjects.find(&cfg); it != LoadedProjects.end()) return it->second;
return it->second.get(); return std::nullopt;
}
auto stored = std::make_unique<Configuration>(std::move(cfg));
Configuration* ptr = stored.get();
ProjectCache.emplace(std::move(fullHash), std::move(stored));
return ptr;
} }

View file

@ -51,8 +51,13 @@ namespace Crafter {
return; return;
} }
// -MD leaves <name>_impl.o.d beside the object for Check to read. // -MD leaves <name>_impl.o.d beside the object for Check to read, and
std::string result = RunBuildCommand(std::format("{0} {1}.cpp -c -MD -MF {2}_impl.o.d -o {2}_impl.o", clang, path.string(), (buildDir/path.filename()).string())); // -fmodule-file-deps puts the BMIs the unit imported in it with the
// headers. The driver adds that for --precompile only, so a std
// module rebuilt in place, for a mingw-w64 or libc++ update under
// the same compiler, left the objects importing it as they were, and
// the next build only relinked them.
std::string result = RunBuildCommand(std::format("{0} {1}.cpp -c -MD -MF {2}_impl.o.d -fmodule-file-deps -o {2}_impl.o", clang, path.string(), (buildDir/path.filename()).string()));
bool expected = false; bool expected = false;
if(!result.empty() && buildCancelled.compare_exchange_strong(expected, true)) { if(!result.empty() && buildCancelled.compare_exchange_strong(expected, true)) {

View file

@ -1149,6 +1149,10 @@ format` applies it to disk, and `crafter-build lint` reports where it would.
// command per configuration — GetCompileCommand walks the dependency tree. // command per configuration — GetCompileCommand walks the dependency tree.
const bool anyRuleNeedsAst = std::ranges::any_of(rules, [](const LintRule* r) { return r->needsAst; }); const bool anyRuleNeedsAst = std::ranges::any_of(rules, [](const LintRule* r) { return r->needsAst; });
std::unordered_map<const Configuration*, std::string> commands; std::unordered_map<const Configuration*, std::string> commands;
// Why a configuration of the build graph has no command: its files
// cannot be parsed, which fails the run. Cleared to nothing, the command
// read as "not a translation unit" and every semantic rule passed them.
std::unordered_map<const Configuration*, std::string> commandErrors;
if (anyRuleNeedsAst && !opts.noAst) { if (anyRuleNeedsAst && !opts.noAst) {
for (Configuration* c : localConfigs) { for (Configuration* c : localConfigs) {
auto ensure = [&](const Configuration& cfg) { auto ensure = [&](const Configuration& cfg) {
@ -1166,10 +1170,14 @@ format` applies it to disk, and `crafter-build lint` reports where it would.
Progress::Task task(std::format("Building std PCM ({}-{})", cfg.target, cfg.march)); Progress::Task task(std::format("Building std PCM ({}-{})", cfg.target, cfg.march));
fs::create_directories(assembled.stdPcmDir); fs::create_directories(assembled.stdPcmDir);
std::string error = BuildStdPcm(cfg, assembled.stdPcmDir/"std.pcm"); std::string error = BuildStdPcm(cfg, assembled.stdPcmDir/"std.pcm");
if (!error.empty()) command.clear(); if (!error.empty()) {
command.clear();
commandErrors.emplace(&cfg, std::format("building the std module for {}: {}", cfg.target, error));
} }
} catch (const std::exception&) { }
command.clear(); // surfaced per file as an AST reason } catch (const std::exception& e) {
command.clear();
commandErrors.emplace(&cfg, std::format("assembling the compile command for {}: {}", cfg.target, e.what()));
} }
commands.emplace(&cfg, std::move(command)); commands.emplace(&cfg, std::move(command));
}; };
@ -1200,6 +1208,11 @@ format` applies it to disk, and `crafter-build lint` reports where it would.
// decides what is safe to touch. Skip it and make the run fail. // decides what is safe to touch. Skip it and make the run fail.
if (rule->needsAst) { if (rule->needsAst) {
if (opts.noAst) continue; if (opts.noAst) continue;
if (auto failed = commandErrors.find(owner); failed != commandErrors.end()) {
astFailures.emplace(file, failed->second);
skippedAstRules.insert(rule->name);
continue;
}
// No compile command means this file is not a translation unit // No compile command means this file is not a translation unit
// of the build graph — project.cpp, which LoadProject compiles // of the build graph — project.cpp, which LoadProject compiles
// with its own flags, or a header. A semantic rule does not // with its own flags, or a header. A semantic rule does not

View file

@ -281,6 +281,9 @@ bool Crafter::MatchAny(std::span<const std::string> globs, std::string_view name
std::string Crafter::RunBuildCommand(std::string_view cmd) { std::string Crafter::RunBuildCommand(std::string_view cmd) {
Progress::EchoCommand(cmd); Progress::EchoCommand(cmd);
CommandResult r = RunCommandChecked(cmd); CommandResult r = RunCommandChecked(cmd);
// A mingw build against a system prefix sees mingw-w64 through a view in
// the cache, and clang names every header by the path it found it at.
r.output = MingwViewToPrefix(std::move(r.output));
if (r.exitCode == 0) { if (r.exitCode == 0) {
// Warnings from a step that succeeded. Show them — until now they were // Warnings from a step that succeeded. Show them — until now they were
@ -427,6 +430,13 @@ namespace {
// the stack is per thread. ProjectDependency() reads the top frame. // the stack is per thread. ProjectDependency() reads the top frame.
thread_local std::vector<ProjectFileFrame> LoadingFrames; thread_local std::vector<ProjectFileFrame> LoadingFrames;
// A frame's dependencies, kept past the frame for PragmaDependencies().
// Process-wide rather than per thread: whoever asks need not be the
// thread that loaded the file. The Configurations live in the project
// cache, which outlives every load.
std::mutex PragmaDependenciesMutex;
std::unordered_map<fs::path, std::vector<Configuration*>> PragmaDependenciesByProject;
// Held for the whole of one LoadProject call. Pushing a file that is // Held for the whole of one LoadProject call. Pushing a file that is
// already on the stack is a cycle, which used to recurse forever. // already on the stack is a cycle, which used to recurse forever.
class ProjectFileGuard { class ProjectFileGuard {
@ -613,6 +623,15 @@ bool Crafter::IsTopLevelProject() {
return LoadingFrames.size() <= 1; return LoadingFrames.size() <= 1;
} }
std::vector<Configuration*> Crafter::PragmaDependencies(const fs::path& projectFile) {
std::error_code ec;
const fs::path absProject = fs::canonical(projectFile, ec);
if (ec) return {};
std::lock_guard lock(PragmaDependenciesMutex);
if (auto it = PragmaDependenciesByProject.find(absProject); it != PragmaDependenciesByProject.end()) return it->second;
return {};
}
std::vector<std::string> Crafter::ProjectArgs() { std::vector<std::string> Crafter::ProjectArgs() {
if (LoadingFrames.empty()) return {}; if (LoadingFrames.empty()) return {};
const std::span<const std::string_view> args = LoadingFrames.back().args; const std::span<const std::string_view> args = LoadingFrames.back().args;
@ -721,14 +740,17 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
// library, naming the std module's source relative to itself. It is // library, naming the std module's source relative to itself. It is
// looked for in the library directories the compiler searches for // looked for in the library directories the compiler searches for
// this target, as it would look for libstdc++ itself, but only in // this target, as it would look for libstdc++ itself, but only in
// those that belong to the mingw toolchain: the list also has the // those that belong to the mingw toolchain. The list also holds
// generic <prefix>/lib, where the host's own (Linux) libstdc++ keeps // clang's resource directory, the sysroot's generic lib/, where a
// its manifest. A toolchain that isn't there is a build error, not // prefix given whole can keep a libstdc++ built for another target,
// an exception: Build() runs this with its worker threads started. // and, when clang finds no gcc directory, an empty entry that would
// name a manifest in the working directory. A toolchain that isn't
// there is a build error, not an exception: Build() runs this with
// its worker threads started.
const Toolchain& tc = GetToolchain(); const Toolchain& tc = GetToolchain();
const std::string where = tc.mingwSysroot.empty() const std::string where = tc.mingwPrefix.empty()
? std::string("where clang finds it by the x86_64-w64-mingw32 gcc on PATH") ? std::string("where clang finds it by the x86_64-w64-mingw32 gcc on PATH")
: std::format("under {}", tc.mingwSysroot.string()); : std::format("under {}", tc.mingwPrefix.string());
const std::string missing = std::format("no mingw-w64 libstdc++ with the std module (libstdc++ 15 or newer) {}. Install mingw-w64, or point CRAFTER_MINGW_DIR at its prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/", where); const std::string missing = std::format("no mingw-w64 libstdc++ with the std module (libstdc++ 15 or newer) {}. Install mingw-w64, or point CRAFTER_MINGW_DIR at its prefix, the directory holding x86_64-w64-mingw32/ and lib/gcc/", where);
// Asked once a run for each compiler and target: every mingw // Asked once a run for each compiler and target: every mingw
// configuration comes through here, up to date or not, and the answer // configuration comes through here, up to date or not, and the answer
@ -759,17 +781,19 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
} }
} }
if (manifestPath.empty()) return missing; if (manifestPath.empty()) return missing;
// Found through the view MingwSysroot() may have made of the prefix,
// and named from here on by its place in the prefix, the directory
// the user knows and can go and look at.
manifestPath = fs::path(MingwViewToPrefix(manifestPath.string()));
const std::optional<std::string> source = StdModuleSourcePath(FileBytes(manifestPath)); const std::optional<std::string> source = StdModuleSourcePath(FileBytes(manifestPath));
if (!source) return std::format("{} names no std module", manifestPath.string()); if (!source) return std::format("{} names no std module", manifestPath.string());
const fs::path stdCc = (manifestPath.parent_path() / *source).lexically_normal(); const fs::path stdCc = (manifestPath.parent_path() / *source).lexically_normal();
if (!fs::exists(stdCc)) return std::format("{}, which {} names as the std module, is missing", stdCc.string(), manifestPath.string()); if (!fs::exists(stdCc)) return std::format("{}, which {} names as the std module, is missing", stdCc.string(), manifestPath.string());
// -femulated-tls keeps PCM TLS access matching libstdc++'s emutls
// definitions; mismatch surfaces as undefined std::__once_callable.
// The std.cc → std.cppm copy uses a plain filename: held under the // The std.cc → std.cppm copy uses a plain filename: held under the
// cache lock, no other builder writes this path concurrently. // cache lock, no other builder writes this path concurrently.
fs::path stdCppm = stdPcm.parent_path() / "std.cppm"; fs::path stdCppm = stdPcm.parent_path() / "std.cppm";
std::string command = std::format("{} --target={} -march={} -mtune={} -femulated-tls -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, config.march, config.mtune, stdCppm.string(), stdPcm.string()); std::string command = std::format("{}{} --precompile {} -o {}", GetBaseCommand(config), StdPcmFlags(config), stdCppm.string(), stdPcm.string());
std::string stamp = HashStamp(command + '\0' + FileBytes(stdCc)); std::string stamp = HashStamp(command + '\0' + FileBytes(stdCc));
if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return ""; if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return "";
std::error_code copyEc; std::error_code copyEc;
@ -799,34 +823,11 @@ std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
if (!source.error.empty()) return source.error; if (!source.error.empty()) return source.error;
stdCppm = source.stdCppm.string(); stdCppm = source.stdCppm.string();
} }
std::string sysrootFlag = config.sysroot.empty()
? std::string()
: std::format(" --sysroot={}", config.sysroot);
// wasm32 rejects -march. wasi-libc++ headers require these baselines
// to compile: setjmp.h needs -mllvm -wasm-enable-sjlj (lowered to wasm
// EH); signal.h requires the emulation define; and EH itself isn't
// wired up so -fno-exceptions stays.
std::string archFlags = isWasm
? std::string(" -fno-exceptions -msimd128 -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL")
: std::format(" -march={} -mtune={}", config.march, config.mtune);
// Match the variant codegen flags the consuming TUs are built with
// (e.g. -mrelaxed-simd) so the std BMI is feature-compatible. wasm only;
// wasmVariantFlags is empty on every other path.
if (isWasm) {
for (const std::string& f : config.wasmVariantFlags) archFlags += std::format(" {}", f);
}
// non-wasm cross builds: the driver ranks the host toolchain's own
// libc++ headers (<install>/../include/c++/v1) above --sysroot, so
// the PCM would mix host libc++ with the foreign libc — force the
// sysroot's copy.
if (!isWasm && !config.sysroot.empty()) {
archFlags += std::format(" -nostdinc++ -isystem {}/usr/include/c++/v1", config.sysroot);
}
// Keyed on the command and the module source's bytes, never on // Keyed on the command and the module source's bytes, never on
// mtimes: an installed std.cppm can be dated at the epoch (OSTree), // mtimes: an installed std.cppm can be dated at the epoch (OSTree),
// and a libc++ update under the same clang changes the bytes but // and a libc++ update under the same clang changes the bytes but
// not the path. // not the path.
std::string command = std::format("{} --target={} -std=c++26{}{} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {} -o {}", GetBaseCommand(config), config.target, sysrootFlag, archFlags, stdCppm, stdPcm.string()); std::string command = std::format("{}{} --precompile {} -o {}", GetBaseCommand(config), StdPcmFlags(config), stdCppm, stdPcm.string());
std::string stamp = HashStamp(command + '\0' + FileBytes(stdCppm)); std::string stamp = HashStamp(command + '\0' + FileBytes(stdCppm));
if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return ""; if (fs::exists(stdPcm) && ReadStamp(StampPath(stdPcm)) == stamp) return "";
std::string result = RunBuildCommand(command); std::string result = RunBuildCommand(command);
@ -853,9 +854,9 @@ std::string Crafter::GetBaseCommand(const Configuration& config) {
if (config.target == "x86_64-w64-mingw32") { if (config.target == "x86_64-w64-mingw32") {
// libstdc++ from the mingw toolchain, said outright: a clang built to // libstdc++ from the mingw toolchain, said outright: a clang built to
// default to libc++ (the Flatpak's) would otherwise reach for that. // default to libc++ (the Flatpak's) would otherwise reach for that.
// Found by clang's own mingw discovery unless there is a sysroot. // Found by clang's own mingw discovery unless there is a prefix.
command += " -stdlib=libstdc++"; command += " -stdlib=libstdc++";
if (!tc.mingwSysroot.empty()) command += std::format(" --sysroot={}", tc.mingwSysroot.string()); if (const fs::path& sysroot = MingwSysroot(); !sysroot.empty()) command += std::format(" --sysroot={}", sysroot.string());
return command; return command;
} }
command += " -stdlib=libc++"; command += " -stdlib=libc++";
@ -866,13 +867,14 @@ std::string Crafter::GetBaseCommand(const Configuration& config) {
} }
namespace { namespace {
constexpr std::array<std::string_view, 13> CrafterBuildModules = { constexpr std::array<std::string_view, 14> CrafterBuildModules = {
"Crafter.Build-Toolchain", "Crafter.Build-Toolchain",
"Crafter.Build-Shader", "Crafter.Build-Shader",
"Crafter.Build-Platform", "Crafter.Build-Platform",
"Crafter.Build-Interface", "Crafter.Build-Interface",
"Crafter.Build-Implementation", "Crafter.Build-Implementation",
"Crafter.Build-External", "Crafter.Build-External",
"Crafter.Build-Agents",
"Crafter.Build-Clang", "Crafter.Build-Clang",
"Crafter.Build-Test", "Crafter.Build-Test",
"Crafter.Build-Lint", "Crafter.Build-Lint",
@ -999,6 +1001,10 @@ Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const
// Before dlopen, so the file's static initialisers can already call // Before dlopen, so the file's static initialisers can already call
// ProjectDependency(): its dependencies are the same on every load. // ProjectDependency(): its dependencies are the same on every load.
{
std::lock_guard lock(PragmaDependenciesMutex);
PragmaDependenciesByProject[absProject] = preDeps.configs;
}
guard.Frame().dependencies = std::move(preDeps.configs); guard.Frame().dependencies = std::move(preDeps.configs);
void* handle = dlopen(soPath.c_str(), RTLD_NOW | RTLD_GLOBAL); void* handle = dlopen(soPath.c_str(), RTLD_NOW | RTLD_GLOBAL);
if (!handle) { if (!handle) {

View file

@ -43,6 +43,37 @@ namespace {
while (!s.empty() && (s.back() == '\n' || s.back() == '\r' || s.back() == ' ')) s.pop_back(); while (!s.empty() && (s.back() == '\n' || s.back() == '\r' || s.back() == ' ')) s.pop_back();
return s; return s;
} }
// `link` made a symbolic link to `target`, or found one already. Another
// crafter-build may make the same link at the same moment, which is as
// good as making it; a link to anything else is an error.
void EnsureSymlink(const fs::path& target, const fs::path& link) {
std::error_code ec;
if (fs::read_symlink(link, ec) == target) return;
fs::create_directories(link.parent_path());
fs::create_directory_symlink(target, link, ec);
std::error_code ignored;
if (ec && fs::read_symlink(link, ignored) != target) {
throw fs::filesystem_error("cannot make a link", target, link, ec);
}
}
// `dir` made absolute and spelled one way, once, when the toolchain is
// resolved. The mingw prefix ends up as the target of links and in the
// names of build and cache directories: a relative path read later from
// another working directory would name something else, and /opt/mingw/
// next to /opt/mingw would rebuild everything for the same directory.
fs::path PrefixPath(const fs::path& dir) {
fs::path normal = fs::absolute(dir).lexically_normal();
if (!normal.has_filename() && normal.has_relative_path()) normal = normal.parent_path();
return normal;
}
// The view MingwSysroot() made and the prefix it shows, for
// MingwViewToPrefix. Written once, before ViewMade is set.
fs::path MadeView;
fs::path ViewedPrefix;
std::atomic<bool> ViewMade = false;
} }
std::string Toolchain::LibcxxCompileFlags() const { std::string Toolchain::LibcxxCompileFlags() const {
@ -136,11 +167,19 @@ Toolchain Crafter::ResolveToolchain(const ToolchainInputs& in) {
tc.wasiSysroot = "/usr/share/wasi-sysroot"; tc.wasiSysroot = "/usr/share/wasi-sysroot";
} }
if (std::string_view dir = Env(in, "CRAFTER_MINGW_DIR"); !dir.empty()) { if (std::string_view dir = Env(in, "CRAFTER_MINGW_DIR"); !dir.empty()) {
tc.mingwSysroot = fs::path(dir); tc.mingwPrefix = PrefixPath(fs::path(dir));
// A prefix that holds a Linux system as well as mingw-w64, /usr
// itself, has that system's C library in include/: features.h, which
// every glibc and musl header includes and mingw-w64 has no use for.
// One that holds mingw-w64 alone stays the sysroot, so Windows
// libraries installed into its include/ and lib/ stay visible.
tc.mingwPrefixIsSystem = fs::exists(tc.mingwPrefix / "include" / "features.h");
} else if (!in.hostRoot.empty() && fs::exists(in.hostRoot / "usr" / "x86_64-w64-mingw32")) { } else if (!in.hostRoot.empty() && fs::exists(in.hostRoot / "usr" / "x86_64-w64-mingw32")) {
// The sandbox has no mingw-w64 of its own, and no mingw gcc on PATH // The sandbox has no mingw-w64 of its own, and no mingw gcc on PATH
// for clang to find one by. // for clang to find one by. The host's /usr is the host's system,
tc.mingwSysroot = in.hostRoot / "usr"; // whether or not its C library's headers are installed.
tc.mingwPrefix = PrefixPath(in.hostRoot / "usr");
tc.mingwPrefixIsSystem = true;
} }
return tc; return tc;
} }
@ -186,6 +225,69 @@ const Toolchain& Crafter::RequireToolchain() {
return tc; return tc;
} }
fs::path Crafter::MakeMingwSysroot(const fs::path& prefix, std::span<const fs::path> parents) {
std::string reasons;
for (const fs::path& parent : parents) {
// One view per prefix: two shells with different CRAFTER_MINGW_DIRs
// share one cache and would otherwise rewrite each other's.
const fs::path sysroot = parent / "mingw-sysroot" / std::format("{:016x}", std::hash<std::string>{}(prefix.string()));
try {
// All clang's mingw driver takes from a sysroot other than its
// include/ and the rest of its lib/: the mingw-w64 tree (headers,
// libraries, and libstdc++ where a distro's cross gcc installs
// it) and gcc's per-target directory (crtbegin.o, libgcc, and the
// version that names libstdc++'s header directory), which
// openSUSE keeps under lib64. A link to nothing is skipped like a
// missing directory.
for (std::string_view entry : {"x86_64-w64-mingw32", "lib/gcc", "lib64/gcc"}) {
EnsureSymlink(prefix / entry, sysroot / entry);
}
return sysroot;
} catch (const std::exception& e) {
reasons += std::format("; {}", e.what());
}
}
throw std::runtime_error(std::format("cannot make a sysroot that shows clang only the mingw-w64 parts of {}{}", prefix.string(), reasons));
}
const fs::path& Crafter::MingwSysroot() {
// Made on first use rather than by GetToolchain(), so that only what
// compiles for the mingw target, a build or an AST lint of one, writes
// anything for it, and a failure surfaces as that command's error
// instead of on every command.
static const fs::path Sysroot = []() -> fs::path {
const Toolchain& tc = GetToolchain();
if (!tc.mingwPrefixIsSystem) return tc.mingwPrefix;
// In the cache, or where the XDG spec promises symbolic links when
// the cache's filesystem takes none (vfat, exFAT, CIFS without unix
// extensions), which costs nothing else crafter-build caches.
// Absolute, because CMake runs the compiler from directories of its
// own.
std::vector<fs::path> parents = { fs::absolute(GetCacheDir()) };
if (const char* runtime = std::getenv("XDG_RUNTIME_DIR"); runtime && *runtime) {
parents.push_back(fs::absolute(fs::path(runtime) / "crafter.build"));
}
fs::path view = MakeMingwSysroot(tc.mingwPrefix, parents);
MadeView = view;
ViewedPrefix = tc.mingwPrefix;
ViewMade.store(true, std::memory_order_release);
return view;
}();
return Sysroot;
}
std::string Crafter::MingwViewToPrefix(std::string text) {
if (!ViewMade.load(std::memory_order_acquire)) return text;
// With the separator, so a --sysroot= naming the view itself, in a
// command line the output quotes, stays what was run.
const std::string view = (MadeView / "").string();
const std::string prefix = (ViewedPrefix / "").string();
for (std::size_t at = text.find(view); at != std::string::npos; at = text.find(view, at + prefix.size())) {
text.replace(at, view.size(), prefix);
}
return text;
}
std::int32_t Crafter::LibcxxVersion(const fs::path& includeDir) { std::int32_t Crafter::LibcxxVersion(const fs::path& includeDir) {
// "# define _LIBCPP_VERSION 230102", the one place libc++ states its // "# define _LIBCPP_VERSION 230102", the one place libc++ states its
// release in a form the preprocessor does not have to be run for. // release in a form the preprocessor does not have to be run for.

View file

@ -0,0 +1,84 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
export module Crafter.Build:Agents;
import std;
namespace fs = std::filesystem;
// Dependency notes. A crafter-build project ships an AGENTS.consumer.md beside
// its project.cpp, written for the agents of the projects that depend on it,
// and crafter-build merges those files into each consumer's AGENTS.md. Without
// that, an agent working on a consumer knows nothing about the libraries it
// builds against unless someone copied the knowledge in by hand, and a copy
// goes stale the moment the library changes.
//
// The consumer writes its own instructions in an AGENTS.template.md beside its
// project.cpp, and every build writes AGENTS.md from that, so AGENTS.md is
// build output and stays out of version control. Kept in a committed
// AGENTS.md instead, as they once were, the notes changed that file whenever
// a dependency changed its own, on builds that changed nothing else, and
// every diff and rebase carried changes nobody had made.
export namespace Crafter {
struct Configuration;
// Which dependencies' notes a project's AGENTS.md carries: the value of
// Configuration::dependencyNotes.
enum class DependencyNotes {
// The projects this one builds against, and the projects those build
// against, all the way down.
Transitive,
// Only the projects this one's own configurations and
// `#pragma crafter dependency` lines name.
Direct,
// None: AGENTS.md holds the template's text alone.
None,
};
// One dependency's AGENTS.consumer.md, ready to merge.
struct DependencyNote {
// What the comment above the note calls the dependency: its
// repository's name, or a local project's directory name.
std::string name;
// The file's content, without leading blank lines or trailing
// whitespace. Lines end in LF; the merge writes the line ending the
// template uses.
std::string text;
};
// `templateMd` with its dependency notes section holding `notes`, in
// order: AGENTS.md as it reads below the line that says a build wrote it.
// A template without a section gets one appended after a blank line; one
// with a section, which is how a template places it, has it rewritten
// where it stands, whatever it held; and empty `notes` remove it.
// Removing a section that ends the template takes the blank line in front
// of it too, so the result ends as the text before it did. The section
// uses the template's line ending, so a CRLF file stays CRLF.
//
// The two marker lines count only as whole lines outside fenced code
// blocks, which is what lets a document quote them. Throws, naming the
// line, when there are notes to write and the markers are unbalanced, or
// when the section would have to be appended to a template that ends
// inside a code block: a lone or doubled marker is a slip in the
// template, and any guess at what it meant would put the notes, or cut
// the text, where its author did not, while notes appended inside a code
// block would read as code. With no notes it never throws, since nothing
// has to be placed. Also throws for a note that would break the section
// for whoever reads it: one with a marker line of its own, or a code
// block it never closes.
std::string MergeDependencyNotes(std::string_view templateMd, std::span<const DependencyNote> notes);
// Write the AGENTS.md beside `projectFile` from the AGENTS.template.md
// beside it, with the AGENTS.consumer.md of every dependency
// project.dependencyNotes selects merged in. `project` is the
// Configuration that file returned, after LoadProject: loading is what
// found the dependencies. Creates AGENTS.md when it is missing, and
// replaces only one a build wrote: an AGENTS.md someone wrote is left
// alone, with a warning. Without a template it writes nothing, and only
// warns about a section an earlier crafter-build left in AGENTS.md.
// Writes only when the text changes, in place, so hard links, ownership
// and a read-only AGENTS.md are respected; returns whether it wrote. A
// read or write that fails, a template the notes cannot be placed in and
// a dependency's unusable notes are warnings: the notes never fail a
// build, and nothing is reported when nothing would have been written.
bool UpdateAgentsMd(const Configuration& project, const fs::path& projectFile);
}

View file

@ -8,8 +8,38 @@ import :Interface;
import :Implementation; import :Implementation;
import :External; import :External;
import :Toolchain; import :Toolchain;
import :Agents;
namespace fs = std::filesystem; namespace fs = std::filesystem;
namespace Crafter {
// What every wasm32 translation unit is compiled with, C or C++, and the
// std module the C++ ones import. A BMI cannot be loaded by a unit whose
// options differ from the ones that wrote it, so GetCompileCommand and
// BuildStdPcm both take the set from here (through WasmCompileFlags), and
// VariantId and StdPcmDir hash it so that a crafter-build which changes
// it never reuses what the old set compiled.
//
// -fwasm-exceptions compiles C++ exceptions to WebAssembly exception
// handling. It also selects the sysroot's include/<triple>/eh and
// lib/<triple>/eh, the libc++ built with exceptions that wasi-sdk 33 and
// later ship next to the noeh/ one, so it has to reach the link too; the
// link reuses this command line, so it does. Those libraries use the
// standardized form of the proposal (try_table, exnref), while LLVM 23
// still defaults to the legacy one: wasmtime refuses to load that, and it
// would mix two encodings in one binary. -wasm-use-legacy-eh=false asks
// for the standard form, for setjmp as well. wasi-libc's setjmp.h does
// not compile without -wasm-enable-sjlj, which lowers setjmp to the same
// exception handling, and signal.h needs the emulation define.
inline constexpr std::string_view WasmBaselineFlags = "-fwasm-exceptions -mllvm -wasm-use-legacy-eh=false -msimd128 -fno-c++-static-destructors -mllvm -wasm-enable-sjlj -D_WASI_EMULATED_SIGNAL";
// The wasi-runtime/runtime.js probes an engine has to pass to run anything
// built with WasmBaselineFlags; EnableWasiBrowserRuntime puts them on every
// entry of variants.json. Exception handling with exnref is the newest
// feature those flags need (Chrome 137, Firefox 131, Safari 18.4). SIMD128
// is older, but V8 and SpiderMonkey switch it off on x86 CPUs without
// SSE4.1, which virtual machines often present.
inline constexpr auto WasmBaselineProbes = std::to_array<std::string_view>({"simd", "exception-handling"});
}
export namespace Crafter { export namespace Crafter {
struct BuildResult { struct BuildResult {
std::string result; std::string result;
@ -37,8 +67,10 @@ export namespace Crafter {
// `label` emits an additional `outputName.<label>.wasm` alongside the // `label` emits an additional `outputName.<label>.wasm` alongside the
// baseline `outputName.wasm`; `probes` names the runtime feature-detect // baseline `outputName.wasm`; `probes` names the runtime feature-detect
// checks (see wasi-runtime/runtime.js's probe registry) that must all pass // checks (see wasi-runtime/runtime.js's probe registry) that must all pass
// for a browser to be served this variant. The first variant whose probes // for a browser to be served this variant, on top of WasmBaselineProbes,
// pass wins; the baseline is the universal fallback. // which every build needs. The first variant whose probes pass wins; the
// baseline is the fallback, and an engine that fails even its probes gets
// an error naming them.
struct WasmVariant { struct WasmVariant {
std::string label; std::string label;
std::vector<std::string> flags; std::vector<std::string> flags;
@ -360,6 +392,15 @@ export namespace Crafter {
}; };
struct Configuration; struct Configuration;
// The name StdPcmDir gives the directory of this configuration's std
// module under `toolchain`: target, march and the toolchain's key, which
// say what it is, then a hash of everything else it is compiled with,
// each flag BuildStdPcm passes (StdPcmFlags) and for the mingw target the
// prefix behind its --sysroot. Two configurations that would compile the
// std module differently never share a directory, so neither rebuilds it
// under the PCMs the other compiled against it. Declared ahead of
// Configuration, whose VariantId keys on it.
std::string StdPcmKey(const Configuration& config, const Toolchain& toolchain);
// What a Script's callback is handed when `crafter-build run <name>` // What a Script's callback is handed when `crafter-build run <name>`
// starts it. // starts it.
@ -496,6 +537,15 @@ export namespace Crafter {
// }}); // }});
// Not part of VariantId: a script changes nothing that is compiled. // Not part of VariantId: a script changes nothing that is compiled.
std::vector<Script> scripts; std::vector<Script> scripts;
// Which dependencies' AGENTS.consumer.md `crafter-build` and
// `crafter-build test` merge into the AGENTS.md they write from the
// AGENTS.template.md beside project.cpp, when there is one (see
// Crafter.Build:Agents):
// cfg.dependencyNotes = DependencyNotes::Direct;
// Only read on the Configuration project.cpp returns; a dependency's
// setting decides nothing for its consumers. Not part of VariantId:
// the notes change nothing that is compiled.
DependencyNotes dependencyNotes = DependencyNotes::Transitive;
void GetInterfacesAndImplementations(std::span<fs::path> interfaces, std::span<fs::path> implementations); void GetInterfacesAndImplementations(std::span<fs::path> interfaces, std::span<fs::path> implementations);
// Retry the `import X;` names GetInterfacesAndImplementations could not // Retry the `import X;` names GetInterfacesAndImplementations could not
// place, against the dependency DAG as it stands now. Sources are // place, against the dependency DAG as it stands now. Sources are
@ -539,7 +589,8 @@ export namespace Crafter {
void AddAstLintRule(std::string name, std::function<void(LintContext&)> check); void AddAstLintRule(std::string name, std::function<void(LintContext&)> check);
// Suffix that uniquely identifies this Configuration's compile state. // Suffix that uniquely identifies this Configuration's compile state.
// target+march+mtune are spelled out for readability; the rest // target+march+mtune are spelled out for readability; the rest
// (type, debug, sysroot, defines, compileFlags, the toolchain) // (type, debug, sysroot, defines, compileFlags, the std module, the
// toolchain)
// collapse into a short hash so two Configurations sharing a path but // collapse into a short hash so two Configurations sharing a path but
// with different compile state can't clobber each other's outputs. // with different compile state can't clobber each other's outputs.
std::string VariantId() const { return VariantId(GetToolchain()); } std::string VariantId() const { return VariantId(GetToolchain()); }
@ -577,6 +628,25 @@ export namespace Crafter {
compileKey += "|W:"; compileKey += "|W:";
compileKey += f; compileKey += f;
} }
// So does the set every wasm unit gets before those. Unkeyed, the
// release that turned exceptions on would have kept the objects
// and PCMs compiled without them: the first unit recompiled after
// the update would import BMIs built without exceptions, which
// clang rejects, and objects nothing recompiled would let
// exceptions pass through without running their destructors.
if (target.starts_with("wasm32")) {
compileKey += "|B:";
compileKey += WasmBaselineFlags;
}
// The std module the units import. Its PCMs and objects record
// it by path, and units do not notice it moving: when the mingw
// prefix stopped being clang's sysroot as is (issue #43), the
// std module was rebuilt and the objects compiled against the
// host's glibc headers were only relinked, failing as before. A
// std module that moves directories, because anything it is
// compiled with changed, moves the build with it.
compileKey += "|S:";
compileKey += StdPcmKey(*this, toolchain);
// Sorted by ApplyStandardArgs, so flag order on the command line // Sorted by ApplyStandardArgs, so flag order on the command line
// doesn't split the cache. // doesn't split the cache.
for (const std::string& a : projectArgs) { for (const std::string& a : projectArgs) {
@ -611,9 +681,9 @@ export namespace Crafter {
std::vector<std::string> requires_; std::vector<std::string> requires_;
}; };
// Directory holding the std module PCM for this Configuration. Keyed on // Directory holding the std module PCM for this Configuration,
// target+march because a BMI is only loadable by a TU with matching target // GetCacheDir() / StdPcmKey(config, GetToolchain()): a BMI is only
// features, and suffixed with the wasm variant flags for the same reason. // loadable by a unit compiled the way it was.
fs::path StdPcmDir(const Configuration& config); fs::path StdPcmDir(const Configuration& config);
// Where Build leaves this Configuration's linked artifact: the executable // Where Build leaves this Configuration's linked artifact: the executable
@ -687,7 +757,11 @@ export namespace Crafter {
// open it). Opt-in: WASI builds destined for wasmtime/wasmer don't need // open it). Opt-in: WASI builds destined for wasmtime/wasmer don't need
// this and shouldn't carry the extra files. Call from project.cpp after // this and shouldn't carry the extra files. Call from project.cpp after
// outputName is set; index.html is generated against the current // outputName is set; index.html is generated against the current
// outputName so renaming the binary later requires another call. // outputName so renaming the binary later requires another call. In that
// runtime exit and _Exit end the program without running any more of it;
// a program that hands the thread to the browser and keeps serving calls
// parks instead, through the crafter_runtime.park import (runtime.js,
// PROGRAM_END).
void EnableWasiBrowserRuntime(Configuration& cfg); void EnableWasiBrowserRuntime(Configuration& cfg);
// Register the relaxed-SIMD codegen variant on cfg. Builds an additional // Register the relaxed-SIMD codegen variant on cfg. Builds an additional
@ -696,7 +770,8 @@ export namespace Crafter {
// it only to engines that report relaxed-SIMD support (Chrome 114+, Firefox // it only to engines that report relaxed-SIMD support (Chrome 114+, Firefox
// 120+) and falls back to the baseline outputName.wasm everywhere else // 120+) and falls back to the baseline outputName.wasm everywhere else
// (e.g. Safari, which still gates relaxed SIMD behind a flag as of mid // (e.g. Safari, which still gates relaxed SIMD behind a flag as of mid
// 2026). Idempotent. Call before EnableWasiBrowserRuntime so the emitted // 2026). Both need what WasmBaselineProbes names, so an engine without
// exception handling gets neither but an error saying so. Idempotent. Call before EnableWasiBrowserRuntime so the emitted
// variants.json manifest includes it. A no-op for non-wasm targets at build // variants.json manifest includes it. A no-op for non-wasm targets at build
// time (no wasm is produced), but harmless to declare unconditionally. // time (no wasm is produced), but harmless to declare unconditionally.
void EnableWasiRelaxedSimdVariant(Configuration& cfg); void EnableWasiRelaxedSimdVariant(Configuration& cfg);
@ -730,3 +805,16 @@ export namespace Crafter {
// for-arg-in-args loop. // for-arg-in-args loop.
ArgQuery ApplyStandardArgs(Configuration& cfg, std::span<const std::string_view> args); ArgQuery ApplyStandardArgs(Configuration& cfg, std::span<const std::string_view> args);
} }
namespace Crafter {
// WasmBaselineFlags followed by the active variant pass's
// wasmVariantFlags, each with a leading space: the wasm part of every
// compile command and of the std module's, which have to agree flag for
// flag.
std::string WasmCompileFlags(const Configuration& config);
// The flags BuildStdPcm compiles the std module with, after the base
// command and before the source, each with a leading space. StdPcmKey
// hashes the same string, so the command and the directory it writes to
// cannot disagree about what decides the BMI.
std::string StdPcmFlags(const Configuration& config);
}

View file

@ -139,3 +139,30 @@ export namespace Crafter {
// SHA for reproducible, offline-after-first-fetch builds. // SHA for reproducible, offline-after-first-fetch builds.
fs::path GitFetch(const GitSource& source); fs::path GitFetch(const GitSource& source);
} }
namespace Crafter {
// Where GitProject or LocalProject loaded a Configuration from. The
// loader keeps this beside the cache that owns the Configuration, since
// the Configuration itself cannot say: only the one they return has its
// path rebased, and the configurations a dependency's project file makes
// for itself keep a path such as "./", which resolves against the
// consumer's directory.
struct LoadedProject {
fs::path projectFile; // absolute
// What a consumer's AGENTS.md calls it: the repository's name for a
// git project, with the path below the clone's root for one that is
// not at it, and the directory's name for a local one. The same on
// every machine and for a --local checkout and its clone alike,
// which a path into the cache or a sibling checkout is not, and free
// of what a URL can carry, credentials included.
std::string name;
// Which library this is, whatever the revision: the repository and
// the project file in it for git, the project file for a local one.
// Two specs that pin one library differently are one library.
std::string identity;
};
// The LoadedProject behind a Configuration that GitProject or
// LocalProject returned; nullopt for any other: the project crafter-build
// was pointed at, and every configuration a project file makes itself.
std::optional<LoadedProject> LoadedProjectOf(const Configuration& cfg);
}

View file

@ -75,6 +75,13 @@ namespace Crafter {
// that library as a dependency, the test's GitProject loads crafter-build's // that library as a dependency, the test's GitProject loads crafter-build's
// project file in turn and the two recurse until the cycle guard throws. // project file in turn and the two recurse until the cycle guard throws.
export bool IsTopLevelProject(); export bool IsTopLevelProject();
// What the `#pragma crafter dependency` lines of `projectFile` resolved
// to the last time LoadProject loaded it in this process, in file order;
// empty when it has none or was never loaded. Unlike ProjectDependency
// this still answers once LoadProject has returned, which is when the
// dependency notes of the project crafter-build was pointed at are
// gathered (Crafter.Build:Agents).
std::vector<Configuration*> PragmaDependencies(const fs::path& projectFile);
// True while this thread loads a `#pragma crafter dependency`: the tool's // True while this thread loads a `#pragma crafter dependency`: the tool's
// project file and every sibling it loads in turn. GitProject and // project file and every sibling it loads in turn. GitProject and
// LocalProject key the project cache on it, so host tooling, which // LocalProject key the project cache on it, so host tooling, which

View file

@ -39,11 +39,20 @@ export namespace Crafter {
fs::path wasiSysroot; fs::path wasiSysroot;
// The mingw-w64 install prefix, the directory that holds // The mingw-w64 install prefix, the directory that holds
// x86_64-w64-mingw32/ and lib/gcc/x86_64-w64-mingw32/ (/usr for a // x86_64-w64-mingw32/ and lib/gcc/x86_64-w64-mingw32/ (/usr for a
// distro package), passed to clang as --sysroot for the mingw target. // distro package), absolute and lexically normal. Empty means clang's
// Empty means clang's own discovery, which follows the mingw gcc on // own discovery, which follows the mingw gcc on PATH. Set by
// PATH. Set by CRAFTER_MINGW_DIR, else the host's /usr when running // CRAFTER_MINGW_DIR, else the host's /usr when running sandboxed and
// sandboxed and it has mingw-w64 (the sandbox's own /usr never does). // it has mingw-w64 (the sandbox's own /usr never does). What the
fs::path mingwSysroot; // mingw target passes as --sysroot is MingwSysroot(), not this:
// see mingwPrefixIsSystem.
fs::path mingwPrefix;
// The prefix holds a Linux system as well as mingw-w64: the host's
// /usr in the sandbox, or a prefix whose include/ has a C library's
// features.h, as /usr has. clang searches a mingw sysroot's include/
// and lib/, so such a prefix is shown to it only through a view of
// its mingw-w64 parts; any other prefix is the sysroot itself, and
// Windows libraries installed into its include/ and lib/ are found.
bool mingwPrefixIsSystem = false;
// `clang++ -print-target-triple` and the first line of // `clang++ -print-target-triple` and the first line of
// `clang++ --version`; both empty when clang++ could not be run. // `clang++ --version`; both empty when clang++ could not be run.
std::string hostTriple; std::string hostTriple;
@ -86,6 +95,32 @@ export namespace Crafter {
const Toolchain& GetToolchain(); const Toolchain& GetToolchain();
// GetToolchain(), or a std::runtime_error carrying its error. // GetToolchain(), or a std::runtime_error carrying its error.
const Toolchain& RequireToolchain(); const Toolchain& RequireToolchain();
// Makes, in the first of `parents` that can hold one, a directory that
// shows clang only the mingw-w64 parts of `prefix`: links named
// x86_64-w64-mingw32, lib/gcc and lib64/gcc to the same paths under
// `prefix`, at <parent>/mingw-sysroot/<hash of prefix>, and returns it.
// A view already there is reused; one another process makes at the same
// moment is as good as one made here. Throws when no parent can hold
// one, with the reason for each.
fs::path MakeMingwSysroot(const fs::path& prefix, std::span<const fs::path> parents);
// The --sysroot of every compile, link and CMake configure for the mingw
// target, settled on the first call of a process: empty without a
// mingwPrefix, the prefix itself for one that holds mingw-w64 alone, and
// for a system prefix the view MakeMingwSysroot makes in crafter-build's
// cache, or in $XDG_RUNTIME_DIR/crafter.build when the cache's
// filesystem takes no symbolic links. A system prefix cannot be the
// sysroot: clang takes a mingw sysroot for a tree that holds mingw-w64
// alone and searches its include/ and lib/ as well, which for /usr are
// the Linux system's own. A Windows build then saw glibc's headers, and
// libstdc++, finding <sys/single_threaded.h>, had shared_ptr read
// __libc_single_threaded, which nothing on Windows defines (issue #43).
// Throws when the view cannot be made.
const fs::path& MingwSysroot();
// `text` with every path into the view MingwSysroot() made written as
// the same path into the prefix it shows, so that compiler messages name
// a header where the user installed it rather than in a hashed cache
// directory; `text` unchanged when no view was made.
std::string MingwViewToPrefix(std::string text);
// _LIBCPP_VERSION as the <__config> in `includeDir` (an include/c++/v1) // _LIBCPP_VERSION as the <__config> in `includeDir` (an include/c++/v1)
// defines it, XXYYZZ; 0 when the file is missing or has no such line. // defines it, XXYYZZ; 0 when the file is missing or has no such line.
std::int32_t LibcxxVersion(const fs::path& includeDir); std::int32_t LibcxxVersion(const fs::path& includeDir);

View file

@ -8,6 +8,7 @@ export import :Platform;
export import :Implementation; export import :Implementation;
export import :Shader; export import :Shader;
export import :External; export import :External;
export import :Agents;
export import :Test; export import :Test;
export import :Lint; export import :Lint;
export import :Script; export import :Script;

View file

@ -25,6 +25,86 @@
</provides> </provides>
<content_rating type="oars-1.1"/> <content_rating type="oars-1.1"/>
<releases> <releases>
<release version="6.0.0" date="2026-11-02">
<description>
<p>New</p>
<ul>
<li>The Flatpak carries lldb, with <code>lldb-server</code>, <code>lldb-dap</code> and <code>lldb-mcp</code>, from the same LLVM build as the compiler, so programs that <code>-r</code> and the test runner start can be debugged inside the sandbox they run in. The README's new Debugging section says how, including <code>flatpak enter</code> to reach a program that is already running.</li>
<li>The Flatpak carries the Khronos Vulkan validation layer, the Vulkan SDK release of the runtime's own loader. <code>VK_INSTANCE_LAYERS</code> or <code>VK_LOADER_LAYERS_ENABLE</code> turns it on from the environment, and <code>VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1</code> adds GPU-assisted validation. A layer installed on the host cannot stand in: <code>flatpak run</code> clears <code>VK_LAYER_PATH</code> and <code>VK_ADD_LAYER_PATH</code>.</li>
<li>The Flatpak carries the SPIRV-Tools programs (<code>spirv-val</code>, <code>spirv-opt</code>, <code>spirv-dis</code> and the rest), so a test that requires <code>tool:spirv-val</code> runs inside it instead of being skipped.</li>
<li>The alias in the README's install commands passes <code>--die-with-parent</code>, which ends the sandbox when its <code>flatpak run</code> process dies and, for a command typed at a bash or zsh prompt, when that shell does. Without it, killing <code>flatpak run</code> leaves a build writing into <code>bin/</code> and <code>build/</code> and a program started with <code>-r</code> running. An alias set up from an earlier README does not have it.</li>
</ul>
<p>Fixes</p>
<ul>
<li>The <code>.flatpakref</code> attached to a release carries the repository's signing key. Releases 2.0.0 to 5.0.0 attach one with a placeholder in its place, and installing from it fails with <code>GPG: Unable to export keys</code>; the one on flatpak.catcrafts.net has the key.</li>
</ul>
<p>Breaking</p>
<ul>
<li>Master has no builds of its own any more: the beta branch gets no new builds and the rolling "Latest master build" release is gone, and every change reaches users as a release on the stable branch. Beta's last build is marked end-of-life, and the next <code>flatpak update</code> of a beta installation says so: install stable with <code>flatpak install catcrafts net.catcrafts.CrafterBuild//stable</code> and remove beta with <code>flatpak uninstall net.catcrafts.CrafterBuild//beta</code>.</li>
</ul>
</description>
</release>
<release version="5.0.0" date="2026-10-31">
<description>
<p>Fixes</p>
<ul>
<li>A mingw build no longer sees the Linux system's headers and libraries. Inside the Flatpak the mingw-w64 prefix is the host's /usr, and it was clang's <code>--sysroot</code>, so clang searched /usr/include and /usr/lib for the Windows build as well: libstdc++ found glibc's <code>sys/single_threaded.h</code> and had <code>shared_ptr</code> read <code>__libc_single_threaded</code>, and every program that used one failed to link. <code>CRAFTER_MINGW_DIR=/usr</code> did the same outside the Flatpak. A prefix that also holds a Linux system (the host's /usr, or any prefix whose include/ has a C library's features.h) now reaches clang through a view of its mingw-w64 directories in the cache, and compiler messages still name its headers where they are installed. A prefix that holds mingw-w64 alone is the sysroot as before, so Windows libraries installed into its include/ and lib/ are found (#43).</li>
<li>Units that import a std module rebuilt in place, after a libc++ update or a mingw-w64 update under the same compiler, are compiled again. They used to be relinked unchanged.</li>
<li>A CMake external whose configure inputs changed, such as another compiler or <code>--sysroot</code>, is built again from an empty build directory. It used to be reconfigured in place, keeping the objects and check results of the old configure, so a mingw external built by 4.0.0 would have kept failing to link after the fix above.</li>
<li>Configurations that compile the std module differently no longer share one. Two that differed only in <code>-mtune</code> (a project built with <code>CRAFTER_BUILD_MTUNE</code> set, next to its own project.cpp) or only in sysroot took turns rebuilding it, and every module built against it, on every run.</li>
<li><code>crafter-build lint</code> fails, and says why, when a configuration's compile command cannot be assembled or its std module cannot be built. Its files used to pass every rule that needs an AST as if they were clean.</li>
</ul>
<p>Breaking</p>
<ul>
<li><code>Toolchain::mingwSysroot</code> is now <code>mingwPrefix</code>, beside <code>mingwPrefixIsSystem</code>, because clang no longer always gets the prefix itself. A project.cpp that passes the mingw sysroot to a command of its own calls <code>MingwSysroot()</code>; passing <code>mingwPrefix</code> brings the bug above back for a prefix like /usr.</li>
<li>The std module's cache directory is now keyed on every flag it is compiled with, and every build directory on the std module it imports, so the first build of each project after updating starts in fresh directories and compiles everything, as after a compiler update. <code>crafter-build clean</code> removes the old build directories; the old std module directories stay in the cache until it is cleared by hand.</li>
</ul>
</description>
</release>
<release version="4.0.0" date="2026-10-31">
<description>
<p>New</p>
<ul>
<li>A project keeps its agent instructions in <code>AGENTS.template.md</code>, and <code>crafter-build</code> and <code>crafter-build test</code> write <code>AGENTS.md</code> from it with the dependencies' notes merged in: after the template's text, or between its marker lines. <code>AGENTS.md</code> is build output, so it goes into .gitignore, and a fresh clone has none until its first build. A project with a committed <code>AGENTS.md</code> moves over with <code>git mv AGENTS.md AGENTS.template.md</code> and <code>echo /AGENTS.md &gt;&gt; .gitignore</code>. An <code>AGENTS.md</code> no build wrote, such as a committed one still beside a new template, is never replaced; every build warns until it is deleted.</li>
</ul>
<p>Fixes</p>
<ul>
<li>A build no longer changes a file the project commits. The dependency notes lived in a section of the committed <code>AGENTS.md</code>, so a dependency that edited its notes, an unpinned one that moved on, or a flag that pulled one in showed up in the project's diff as a change nobody made.</li>
</ul>
<p>Breaking</p>
<ul>
<li>Without an <code>AGENTS.template.md</code>, a build leaves <code>AGENTS.md</code> alone. A section 3.0.0 kept in it is no longer updated, and every build warns about it until the file is moved to the template or the section is deleted.</li>
</ul>
</description>
</release>
<release version="3.0.0" date="2026-10-31">
<description>
<p>New</p>
<ul>
<li>C++ exceptions work in WebAssembly builds. Every <code>wasm32-wasip1</code> unit and the std module are compiled with WebAssembly exception handling in its standardized form (exnref), against the exception-enabled libc++ the sysroot ships under <code>lib/&lt;triple&gt;/eh/</code>, and <code>-lunwind</code> and <code>-lsetjmp</code> are linked automatically, so throw and catch, libc++'s own throws, setjmp and longjmp all work. The Flatpak's WASI sysroot is built with those libraries. A page from <code>EnableWasiBrowserRuntime</code> probes for the feature and names what an older browser lacks instead of failing with a CompileError.</li>
<li>Scripts. A project.cpp can push a name and a C++ callback onto <code>cfg.scripts</code>; <code>crafter-build run &lt;name&gt; [args...]</code> runs it with everything after the name handed over untouched, a bare <code>run</code> lists the scripts, and the callback's return value is the exit status. <code>ScriptContext</code> gives it the project, the project directory, the arguments and <code>Run(command)</code>, which runs a shell command with the terminal attached. <code>Crafter::ShellQuote</code> is exported. Only the scripts of the project crafter-build was pointed at can be run, never a dependency's.</li>
<li>Dependency notes for agents. A library ships an <code>AGENTS.consumer.md</code> beside its project.cpp; <code>crafter-build</code> and <code>crafter-build test</code> keep its dependencies' notes in a marked section of a consumer's <code>AGENTS.md</code>, when the consumer has one, and never create the file. <code>cfg.dependencyNotes</code> picks the dependencies: <code>Transitive</code> (the default), <code>Direct</code> or <code>None</code>, which also removes the section.</li>
<li>In a page from <code>EnableWasiBrowserRuntime</code>, <code>exit</code> and <code>_Exit</code> end the program without running any of its code on the way out, no destructor and no <code>catch (...)</code>, with the status in <code>window.crafter_wasi.exitCode</code>. A program that hands the thread to the browser for good and goes on serving calls imports <code>crafter_runtime.park</code> instead, which never returns and leaves main's locals valid for the calls that follow.</li>
<li>The mingw target is found where it is: by the <code>x86_64-w64-mingw32-gcc</code> on PATH, through <code>CRAFTER_MINGW_DIR</code> naming another prefix, and inside the Flatpak through the host's /usr when it has mingw-w64. Its std module comes from the toolchain's own libstdc++.modules.json, and a toolchain that is not there is a build error saying where it looked, not std::terminate.</li>
</ul>
<p>Fixes</p>
<ul>
<li>A program that holds two builds of one module is refused before anything compiles, with both paths and what tells the builds apart. A sibling project that two paths loaded with different arg lists used to be built twice, every consumer took whichever build clang found first, and the program built cleanly and then read garbage or crashed (#41).</li>
<li>An external dependency shared by the configurations of one build is fetched and built once per build pass instead of once per configuration, and each clone's fetch and build hold a file lock. The tests of a project that copies its externalDependencies into each test pulled the same clone side by side and broke its FETCH_HEAD, which failed every test (#34); they now wait for one fetch.</li>
<li>Incremental builds pick up more of what changed. A shader rebuilds when a file it includes or one of its settings (stage, entry point, defines, include directories) changes, not only its own source. An executable relinks when its link command changed, such as a new link flag or a dependency's library that moved to another cache directory, not only when an object is newer.</li>
<li>Submodules are checked out at the revisions the fetched commit records, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules and not the default branch's.</li>
<li>A project that loads the same sibling as one of its <code>#pragma crafter dependency</code> tools, with the same args, got the tool's copy back: host tooling with -march=native and -fPIC, whose BMI clang then refused. Host tooling and everything it loads are now kept apart in the project cache.</li>
<li><code>crafter-build format clean</code> formats the rules matching "clean"; it used to delete bin/ and build/ and exit 0.</li>
<li><code>--</code> ends crafter-build's options, as the help has advertised: every later word is an operand of the verb. It used to be forwarded as a project arg and change the variant hash, and the words after it were still read as options.</li>
<li><code>ProjectDependency()</code> works from a static initialiser of a project file; it used to find nothing there and terminate the process.</li>
</ul>
<p>Breaking</p>
<ul>
<li>A wasm build needs a sysroot that ships the exception-enabled libc++ and libunwind (wasi-sdk 33 or later; the Flatpak's does), and stops with one line saying so without one. It refuses <code>-fno-exceptions</code>, <code>-flto</code> in compileFlags or linkFlags, and shared libraries, each with a message that says why; there is no switch back to building without exceptions. The result runs on engines with exnref: Chrome 137, Firefox 131, Safari 18.4 and wasmtime 47 or newer (the bundled one qualifies; an older wasmtime needs <code>-W exceptions=y</code>). An exception that escapes main ends the program as an uncaught WebAssembly exception without calling std::terminate, so catch it in main to report it. The flags are part of the variant hash, so the first wasm build after updating starts in a fresh directory.</li>
<li>A sibling loaded through <code>GitProject</code> or <code>LocalProject</code> without <code>.args</code> now receives the args its loading project.cpp was given, so <code>--target=</code>, <code>--march=</code>, <code>--debug</code> and the project's own flags reach every sibling down the chain; releases up to 2.2.0 forwarded nothing. Set <code>.args = {}</code> to keep the old behaviour, and give every sibling the same list: a project that two paths load with different lists now stops the build. The default is read when the spec is made, so make it inside <code>CrafterBuildProject</code>; a spec made at namespace scope forwards nothing. A <code>#pragma crafter dependency</code> still gets none of the project's args.</li>
</ul>
</description>
</release>
<release version="2.2.0" date="2026-10-16"> <release version="2.2.0" date="2026-10-16">
<description> <description>
<p>A Shader entry can carry defines, spelled like the compiler's -D (NAME is defined to 1, NAME=VALUE to VALUE), and a variant that names its output, so one source compiles into several outputs: ui.comp.glsl with variant sets becomes ui.comp.sets.spv. A configuration in which two entries share an output name is refused with both named, as are a malformed define and a variant with a path separator.</p> <p>A Shader entry can carry defines, spelled like the compiler's -D (NAME is defined to 1, NAME=VALUE to VALUE), and a variant that names its output, so one source compiles into several outputs: ui.comp.glsl with variant sets becomes ui.comp.sets.spv. A configuration in which two entries share an output name is refused with both named, as are a malformed define and a variant with a path separator.</p>

View file

@ -5,9 +5,15 @@
# Platform: a build tool needs glibc's headers and crt objects, gcc's # Platform: a build tool needs glibc's headers and crt objects, gcc's
# libgcc_s, cmake, ninja, make, git and python at run time. Everything # libgcc_s, cmake, ninja, make, git and python at run time. Everything
# compiler-shaped is ours and lives under /app: clang, lld, libclang, libc++, # compiler-shaped is ours and lives under /app: clang, lld, libclang, libc++,
# compiler-rt (native and wasm32) and the WASI sysroot, all built from one # compiler-rt (native and wasm32), lldb and the WASI sysroot, all built from
# LLVM source tree so the modules crafter-build builds and the compiler that # one LLVM source tree so the modules crafter-build builds and the compiler
# consumes them can never drift apart. # that consumes them can never drift apart.
#
# Programs that `-r` and the test runner start run in this sandbox, so the
# tools to debug them live here too: lldb (above), the Sdk's gdb, valgrind
# and strace, the Khronos Vulkan validation layers, GPU-assisted validation
# included, which neither the Sdk nor its GL extensions carry, and the
# SPIRV-Tools programs such as spirv-val.
app-id: net.catcrafts.CrafterBuild app-id: net.catcrafts.CrafterBuild
runtime: org.freedesktop.Sdk runtime: org.freedesktop.Sdk
runtime-version: '26.08' runtime-version: '26.08'
@ -25,13 +31,38 @@ finish-args:
- --socket=fallback-x11 - --socket=fallback-x11
- --socket=pulseaudio - --socket=pulseaudio
- --device=all - --device=all
# gdb and friends from the Sdk # lldb, and gdb and friends from the Sdk
- --allow=devel - --allow=devel
- --env=PATH=/app/bin:/usr/bin - --env=PATH=/app/bin:/usr/bin
cleanup: cleanup:
- /share/man - /share/man
- /share/doc - /share/doc
modules: modules:
- name: swig
# Generates lldb's Python bindings while the llvm module builds; nothing
# of it ships.
cleanup:
- '*'
sources:
- type: archive
url: https://downloads.sourceforge.net/project/swig/swig/swig-4.4.1/swig-4.4.1.tar.gz
sha256: 40162a706c56f7592d08fd52ef5511cb7ac191f3593cf07306a0a554c6281fcf
- name: libedit
# Line editing, history and completion at lldb's prompt. The Sdk has the
# ncurses it needs but not libedit itself; liblldb links the shared
# library, so that ships and the rest goes.
config-opts:
- --disable-static
cleanup:
- /include
- /lib/pkgconfig
- /lib/*.la
sources:
- type: archive
url: https://thrysoee.dk/editline/libedit-20260512-3.1.tar.gz
sha256: 432d5e7ea8b0116dd39f2eca7bc11d0eed77faa6b77ea526ace89907c23ea4a0
- name: llvm - name: llvm
buildsystem: cmake-ninja buildsystem: cmake-ninja
builddir: true builddir: true
@ -50,8 +81,20 @@ modules:
config-opts: config-opts:
- -DCMAKE_BUILD_TYPE=Release - -DCMAKE_BUILD_TYPE=Release
- -DLLVM_ENABLE_ASSERTIONS=OFF - -DLLVM_ENABLE_ASSERTIONS=OFF
- -DLLVM_ENABLE_PROJECTS=clang;lld - -DLLVM_ENABLE_PROJECTS=clang;lld;lldb
- -DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind - -DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind
# lldb with Python scripting (its bindings generated by the swig module
# above, against the Sdk's Python), libedit (the module above), curses,
# libxml2 and lzma, each set ON rather than left to the default of "if
# found", so a dependency gone missing fails the configure step instead
# of shipping an lldb without it. lldb-server, lldb-dap and lldb-mcp
# come with it.
- -DLLDB_ENABLE_PYTHON=ON
- -DLLDB_ENABLE_LIBEDIT=ON
- -DLLDB_ENABLE_CURSES=ON
- -DLLDB_ENABLE_LIBXML2=ON
- -DLLDB_ENABLE_LZMA=ON
- -DLLDB_ENABLE_LUA=OFF
- -DLLVM_TARGETS_TO_BUILD=X86;AArch64;WebAssembly - -DLLVM_TARGETS_TO_BUILD=X86;AArch64;WebAssembly
# Classic layout: libc++ in lib/ and include/c++/v1, not under a # Classic layout: libc++ in lib/ and include/c++/v1, not under a
# per-target subdirectory, the same shape as a distro install. # per-target subdirectory, the same shape as a distro install.
@ -82,6 +125,7 @@ modules:
- /include/clang - /include/clang
- /include/clang-tidy - /include/clang-tidy
- /include/lld - /include/lld
- /include/lldb
- /lib/cmake - /lib/cmake
- /lib/*.a - /lib/*.a
- /share/opt-viewer - /share/opt-viewer
@ -135,11 +179,22 @@ modules:
- -DWASI_SDK_INSTALL_TO_CLANG_RESOURCE_DIR=ON - -DWASI_SDK_INSTALL_TO_CLANG_RESOURCE_DIR=ON
- -DWASI_SDK_TARGETS=wasm32-wasip1;wasm32-wasip1-threads - -DWASI_SDK_TARGETS=wasm32-wasip1;wasm32-wasip1-threads
- -DWASI_SDK_INCLUDE_TESTS=OFF - -DWASI_SDK_INCLUDE_TESTS=OFF
# crafter-build compiles every wasm unit with C++ exceptions and links
# the libc++ and libunwind built with them, from lib/<triple>/eh. The
# default is DUAL only when the driving clang is 23 or newer and OFF
# below that; said outright, an LLVM downgrade cannot quietly ship a
# sysroot that no wasm build links against.
- -DWASI_SDK_EXCEPTIONS=DUAL
- -DCMAKE_INSTALL_PREFIX=/app - -DCMAKE_INSTALL_PREFIX=/app
post-install: post-install:
# wasi-sdk installs into <prefix>/share/wasi-sysroot; crafter-build # wasi-sdk installs into <prefix>/share/wasi-sysroot; crafter-build
# looks there (Crafter.Build:Toolchain), so nothing to move. # looks there (Crafter.Build:Toolchain), so nothing to move.
- test -f /app/share/wasi-sysroot/share/libc++/v1/std.cppm - test -f /app/share/wasi-sysroot/share/libc++/v1/std.cppm
# The wasm tests skip when these are missing, as on a distro sysroot,
# so the Flatpak checks for them itself: without them its CI would
# pass while every wasm build failed.
- test -f /app/share/wasi-sysroot/lib/wasm32-wasip1/eh/libunwind.a
- test -f /app/share/wasi-sysroot/lib/wasm32-wasip1-threads/eh/libunwind.a
sources: sources:
- type: git - type: git
url: https://github.com/WebAssembly/wasi-sdk.git url: https://github.com/WebAssembly/wasi-sdk.git
@ -208,12 +263,117 @@ modules:
url: https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.8/llvm-project-22.1.8.src.tar.xz url: https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.8/llvm-project-22.1.8.src.tar.xz
sha256: 922f1817a0df7b1489272d18134ee0087a8b068828f87ac63b9861b1a9965888 sha256: 922f1817a0df7b1489272d18134ee0087a8b068828f87ac63b9861b1a9965888
# The Khronos validation layers and the four modules they are built from,
# after llvm, so that bumping them reuses the cached LLVM build. All five
# are pinned to the Vulkan SDK release of the Sdk's own Vulkan loader and
# headers (1.4.357), at the dependency revisions that release's
# scripts/known_good.json names. When the runtime moves to a newer loader,
# move the layers to the SDK release of that loader and take the other
# four revisions from its known_good.json again. The four are built into
# /app at those revisions rather than taken from the Sdk, whose versions
# follow the runtime, and of them only SPIRV-Tools' programs ship.
- name: vulkan-headers
buildsystem: cmake-ninja
builddir: true
cleanup:
- '*'
sources:
- type: archive
url: https://github.com/KhronosGroup/Vulkan-Headers/archive/refs/tags/v1.4.357.tar.gz
sha256: 7dc0dbcf1d49dd3d7da3761c251c6097dfbaac475321a4a8a99269d3d5abecdc
- name: vulkan-utility-libraries
buildsystem: cmake-ninja
builddir: true
config-opts:
- -DCMAKE_BUILD_TYPE=Release
# CMake searches its install prefix after /usr, where the Sdk has
# Vulkan headers of its own.
- -DCMAKE_PREFIX_PATH=/app
cleanup:
- '*'
sources:
- type: archive
url: https://github.com/KhronosGroup/Vulkan-Utility-Libraries/archive/refs/tags/v1.4.357.tar.gz
sha256: e8688013cdc67dad26f15dcece91fa6b80ad595d551fceb56397fc4795efd00a
- name: spirv-headers
buildsystem: cmake-ninja
builddir: true
cleanup:
- '*'
sources:
- type: archive
url: https://github.com/KhronosGroup/SPIRV-Headers/archive/29981f65241605e08b0ede4cfeb999fe3b723c6a.tar.gz
sha256: 232899f1ad4104fb5bc377b94596c7621575eee62ad9a9e8f929b63a7dd8a7ad
- name: spirv-tools
# The optimizer GPU-assisted validation instruments shaders with, linked
# into the layer statically, and SPIRV-Tools' programs (spirv-val,
# spirv-opt, spirv-dis and the rest), which ship: a program or test that
# checks its SPIR-V with spirv-val, as Crafter.Graphics' do, finds it in
# the sandbox. The libraries and headers stay build-only.
buildsystem: cmake-ninja
builddir: true
build-options:
no-debuginfo: true
strip: true
config-opts:
- -DCMAKE_BUILD_TYPE=Release
# The installed SPIRV-Headers: its include/ holds the grammar files
# SPIRV-Tools generates its tables from, as the source tree does.
- -DSPIRV-Headers_SOURCE_DIR=/app
- -DSPIRV_SKIP_TESTS=ON
- -DSPIRV_WERROR=OFF
cleanup:
- /include/spirv-tools
- /lib/libSPIRV-Tools*.a
# Always built besides the static libraries, and linked by nothing
# here: the programs and the layer take SPIRV-Tools statically.
- /lib/libSPIRV-Tools-shared.so
- /lib/cmake/SPIRV-Tools*
- /lib/pkgconfig/SPIRV-Tools*
post-install:
- spirv-val --version
sources:
- type: archive
url: https://github.com/KhronosGroup/SPIRV-Tools/archive/b707790a898e44038547df54580022fc1cf89c3d.tar.gz
sha256: 05d8af89737bde57571c48dbd36714c9f520a69623e14de72c3be6b600e277d6
- name: vulkan-validation-layers
# VK_LAYER_KHRONOS_validation. The loader finds its manifest under
# /app/share/vulkan/explicit_layer.d through XDG_DATA_DIRS, and the
# manifest names the library without a path, which the sandbox's loader
# finds in /app/lib. GPU-AV is a setting of this layer, not a layer of
# its own (VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1, or the layer settings).
buildsystem: cmake-ninja
builddir: true
build-options:
no-debuginfo: true
strip: true
config-opts:
- -DCMAKE_BUILD_TYPE=Release
- -DCMAKE_PREFIX_PATH=/app
# ON by default: it clones every dependency with update_deps.py, which
# the build sandbox has no network for, instead of using the four
# modules above.
- -DUPDATE_DEPS=OFF
- -DBUILD_TESTS=OFF
- -DBUILD_WERROR=OFF
post-install:
- test -f /app/share/vulkan/explicit_layer.d/VkLayer_khronos_validation.json
- test -f /app/lib/libVkLayer_khronos_validation.so
sources:
- type: archive
url: https://github.com/KhronosGroup/Vulkan-ValidationLayers/archive/refs/tags/vulkan-sdk-1.4.357.0.tar.gz
sha256: 73180b11992a3554e97ebc18e6bf2b45ff9790ded9d87fc526c7bab865d1303f
- name: crafter-build - name: crafter-build
buildsystem: simple buildsystem: simple
build-options: build-options:
# The self-host build fetches Crafter.Math, Crafter.Asset, # The self-host build fetches Crafter.Math, Crafter.Asset,
# Crafter.LintRules and glslang from git; they are unpinned on purpose # Crafter.LintRules and glslang from git; they are unpinned on purpose
# (sibling repos tracking their default branch, see AGENTS.md). # (sibling repos tracking their default branch, see AGENTS.template.md).
build-args: build-args:
- --share=network - --share=network
# On this module, not at the top of the manifest: top-level build # On this module, not at the top of the manifest: top-level build

View file

@ -16,7 +16,8 @@ namespace {
struct FlatpakPaths { struct FlatpakPaths {
fs::path manifest; fs::path manifest;
// flatpak-builder's downloads, ccache and per-module cache. The // flatpak-builder's downloads, ccache and per-module cache. The
// LLVM build lives here: about 2 GB, and 25 minutes to rebuild. // LLVM build lives here: about 3 GB in all after a first run,
// which takes about 30 minutes with 56 CPU threads.
fs::path state; fs::path state;
// The OSTree repo the build is exported to; `install` reads it. // The OSTree repo the build is exported to; `install` reads it.
fs::path repo; fs::path repo;
@ -156,14 +157,14 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
if (const fs::path& llvmInclude = GetToolchain().llvmIncludeDir; !llvmInclude.empty()) { if (const fs::path& llvmInclude = GetToolchain().llvmIncludeDir; !llvmInclude.empty()) {
CrafterBuildLib->compileFlags.push_back(std::format("-I{}", llvmInclude.string())); CrafterBuildLib->compileFlags.push_back(std::format("-I{}", llvmInclude.string()));
} }
// The version the binary reports: CI sets it from the tag, a checkout // The version the binary reports: CI sets it, a checkout describes
// describes itself, anything else is a development build. A define on // itself, anything else is a development build. A define on the
// the library, so each version is its own library variant. // library, so each version is its own library variant.
std::string version; std::string version;
if (const char* env = std::getenv("CRAFTER_BUILD_VERSION"); env && *env) { if (const char* env = std::getenv("CRAFTER_BUILD_VERSION"); env && *env) {
version = env; version = env;
} else { } else {
auto described = RunCommandWithTimeout("git describe --tags --always --dirty --exclude=latest 2>/dev/null", std::chrono::seconds(10)); auto described = RunCommandWithTimeout("git describe --tags --always --dirty 2>/dev/null", std::chrono::seconds(10));
version = described.output; version = described.output;
while (!version.empty() && (version.back() == '\n' || version.back() == '\r')) version.pop_back(); while (!version.empty() && (version.back() == '\n' || version.back() == '\r')) version.pop_back();
if (described.exitCode != 0 || version.empty()) version = "0.0.0-dev"; if (described.exitCode != 0 || version.empty()) version = "0.0.0-dev";
@ -172,7 +173,7 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
// it reaches clang, which must still see a string literal. // it reaches clang, which must still see a string literal.
CrafterBuildLib->defines.push_back({"CRAFTER_BUILD_VERSION", std::format("\\\"{}\\\"", version)}); CrafterBuildLib->defines.push_back({"CRAFTER_BUILD_VERSION", std::format("\\\"{}\\\"", version)});
{ {
std::array<fs::path, 13> interfaces = { std::array<fs::path, 14> interfaces = {
"interfaces/Crafter.Build", "interfaces/Crafter.Build",
"interfaces/Crafter.Build-Toolchain", "interfaces/Crafter.Build-Toolchain",
"interfaces/Crafter.Build-Shader", "interfaces/Crafter.Build-Shader",
@ -180,6 +181,7 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
"interfaces/Crafter.Build-Interface", "interfaces/Crafter.Build-Interface",
"interfaces/Crafter.Build-Implementation", "interfaces/Crafter.Build-Implementation",
"interfaces/Crafter.Build-External", "interfaces/Crafter.Build-External",
"interfaces/Crafter.Build-Agents",
"interfaces/Crafter.Build-Clang", "interfaces/Crafter.Build-Clang",
"interfaces/Crafter.Build-Test", "interfaces/Crafter.Build-Test",
"interfaces/Crafter.Build-Lint", "interfaces/Crafter.Build-Lint",
@ -187,13 +189,14 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
"interfaces/Crafter.Build-Progress", "interfaces/Crafter.Build-Progress",
"interfaces/Crafter.Build-Asset", "interfaces/Crafter.Build-Asset",
}; };
std::array<fs::path, 12> implementations = { std::array<fs::path, 13> implementations = {
"implementations/Crafter.Build-Toolchain", "implementations/Crafter.Build-Toolchain",
"implementations/Crafter.Build-Shader", "implementations/Crafter.Build-Shader",
"implementations/Crafter.Build-Platform", "implementations/Crafter.Build-Platform",
"implementations/Crafter.Build-Interface", "implementations/Crafter.Build-Interface",
"implementations/Crafter.Build-Implementation", "implementations/Crafter.Build-Implementation",
"implementations/Crafter.Build-External", "implementations/Crafter.Build-External",
"implementations/Crafter.Build-Agents",
"implementations/Crafter.Build-Clang", "implementations/Crafter.Build-Clang",
"implementations/Crafter.Build-Test", "implementations/Crafter.Build-Test",
"implementations/Crafter.Build-Lint", "implementations/Crafter.Build-Lint",
@ -245,15 +248,18 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
cfg.AddTest("IncrementalInterfaceChange").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("IncrementalInterfaceChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("TransitiveInterfaceChange").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("TransitiveInterfaceChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("IncrementalHeaderChange").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("IncrementalHeaderChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("StdPcmRebuild").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("CompilerWarning").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("CompilerWarning").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("CleanProject").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("CleanProject").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ShaderCompile").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ShaderCompile").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ShaderVariantBuild").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ShaderVariantBuild").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ShaderIncludeChange").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ShaderIncludeChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("MingwMissing").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("MingwMissing").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("MingwSysroot").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("GitSubmodules").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("GitSubmodules").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("LinkFlagsChange").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("LinkFlagsChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ExternalArtifactTime").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ExternalArtifactTime").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ExternalReconfigure").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ConcurrentExternalFetch").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ConcurrentExternalFetch").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("SharedExternalFetch").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("SharedExternalFetch").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("StandardArgs").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("StandardArgs").Dependencies({ CrafterBuildLib.get() });
@ -264,8 +270,12 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
cfg.AddTest("SysrootCompileFlags").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("SysrootCompileFlags").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("VersionFlag").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("VersionFlag").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasiBrowserRuntime").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("WasiBrowserRuntime").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasiBrowserRuntimeExit").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasmVariants").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("WasmVariants").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasmHelloRuns").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("WasmHelloRuns").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasmExceptions").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasiSysrootWithoutExceptions").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("WasmBuildRefusals").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("RunSingleTestExit").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("RunSingleTestExit").Dependencies({ CrafterBuildLib.get() });
// LoadProject dlopens the synthesized project.so, which references // LoadProject dlopens the synthesized project.so, which references
// Crafter:: symbols (HostTarget, Configuration ctors) that have to be // Crafter:: symbols (HostTarget, Configuration ctors) that have to be
@ -288,6 +298,8 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl"); .LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
cfg.AddTest("Script").Dependencies({ CrafterBuildLib.get() }) cfg.AddTest("Script").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl"); .LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
cfg.AddTest("DependencyNotes").Dependencies({ CrafterBuildLib.get() })
.LinkFlag("-Wl,--export-dynamic").LinkFlag("-ldl");
cfg.AddTest("ConcurrentDependencyReset").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("ConcurrentDependencyReset").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("Lint").Dependencies({ CrafterBuildLib.get() }); cfg.AddTest("Lint").Dependencies({ CrafterBuildLib.get() });
} }

View file

@ -0,0 +1,4 @@
## app
Notes from app for its own consumers. They must never appear in app's own
AGENTS.md.

View file

@ -0,0 +1,4 @@
# app
The consumer's own instructions. AGENTS.md has to carry them unchanged, below
its first line and above the dependency notes.

View file

@ -0,0 +1,17 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
using namespace Crafter;
// A second project file in app/: a dependency whose AGENTS.consumer.md, by
// where it sits, is app's own.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "helper";
cfg.outputName = "helper";
cfg.type = ConfigurationType::LibraryStatic;
return cfg;
}

View file

@ -0,0 +1,27 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
#pragma crafter dependency "../tool"
using namespace Crafter;
// The consumer whose AGENTS.md the test watches. It names lib itself, reaches
// base only through lib, and uses tool as host tooling. Its own
// AGENTS.consumer.md is for its consumers and must never land in its own
// AGENTS.md, not even through helper, a dependency whose project file sits
// beside this one. testonly is only a test's dependency.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "app";
cfg.outputName = "app";
cfg.type = ConfigurationType::Executable;
// Against the working directory, as with `cd app && crafter-build`.
cfg.dependencies = {
LocalProject({ .projectFile = "../lib/project.cpp" }),
LocalProject({ .projectFile = "helper.cpp" }),
};
cfg.AddTest("t").Dependencies({ LocalProject({ .projectFile = "../testonly/project.cpp" }) });
return cfg;
}

View file

@ -0,0 +1,7 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
// app's one test, there to have a dependency of its own.
int main() {
return 0;
}

View file

@ -0,0 +1,6 @@
## base
base counts from zero.

View file

@ -0,0 +1,17 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
using namespace Crafter;
// lib's dependency, so app's only through lib: in its notes under
// Transitive, gone under Direct.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "base";
cfg.outputName = "base";
cfg.type = ConfigurationType::LibraryStatic;
return cfg;
}

View file

@ -0,0 +1,3 @@
## lib
Call lib through its module, never through its headers.

View file

@ -0,0 +1,21 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
#pragma crafter dependency "../libtool"
using namespace Crafter;
// What app depends on. It builds against base, which app only reaches
// through it. libtool is lib's own host tooling: it says nothing about using
// lib, so its notes must not reach app.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "lib";
cfg.outputName = "lib";
cfg.type = ConfigurationType::LibraryStatic;
// Loaded from lib/, so this resolves against it.
cfg.dependencies = { LocalProject({ .projectFile = "../base/project.cpp" }) };
return cfg;
}

View file

@ -0,0 +1,3 @@
## libtool
How lib is built. This must never reach app.

View file

@ -0,0 +1,16 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
using namespace Crafter;
// lib's host tooling. Like tool, a library with no sources.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "libtool";
cfg.outputName = "libtool";
cfg.type = ConfigurationType::LibraryStatic;
return cfg;
}

View file

@ -0,0 +1,3 @@
## testonly
Only a test uses this. It must never reach app.

View file

@ -0,0 +1,17 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
using namespace Crafter;
// Only app's test depends on it. A plain build never looks at the tests, so
// its notes must not reach app.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "testonly";
cfg.outputName = "testonly";
cfg.type = ConfigurationType::LibraryStatic;
return cfg;
}

View file

@ -0,0 +1,3 @@
## tool
Run tool before committing.

View file

@ -0,0 +1,18 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
using namespace Crafter;
// app's host tooling, named by its `#pragma crafter dependency`. Built for
// the host before app's project.cpp compiles, so it is a library with no
// sources: an empty archive is all the pre-pass needs.
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view>) {
Configuration cfg;
cfg.path = "./";
cfg.name = "tool";
cfg.outputName = "tool";
cfg.type = ConfigurationType::LibraryStatic;
return cfg;
}

View file

@ -0,0 +1,357 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// Dependency notes: a consumer with an AGENTS.template.md beside its
// project.cpp has every build and test run write AGENTS.md from it, with the
// AGENTS.consumer.md each dependency ships merged into a marked section.
//
// The template is the user's own file, committed and edited by hand, and
// AGENTS.md is build output that stays out of version control: notes kept in
// a committed AGENTS.md changed it under every diff. So what this guards
// against is the generated file leaking into the user's: the template
// written, an AGENTS.md someone wrote replaced (one committed before the
// template was, or one beside no template at all), one written although it
// is read-only, or replaced so that a hard link to it goes stale; and inside
// AGENTS.md, the template's text changed or the section misplaced, a write
// when nothing changed (an editor reloading), notes from a test's dependency
// or a dependency's own host tooling leaking into the consumer's, and `lint`,
// `run` or `test --list`, which promise not to touch the project, writing it
// anyway.
namespace {
std::int32_t Failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++Failures;
}
}
std::string Read(const fs::path& file) {
std::ifstream in(file, std::ios::binary);
return std::string((std::istreambuf_iterator<char>(in)), std::istreambuf_iterator<char>());
}
void Write(const fs::path& file, std::string_view content) {
std::ofstream(file, std::ios::binary | std::ios::trunc) << content;
}
std::size_t Count(std::string_view text, std::string_view needle) {
std::size_t count = 0;
for (std::size_t pos = text.find(needle); pos != std::string_view::npos; pos = text.find(needle, pos + needle.size())) ++count;
return count;
}
// MergeDependencyNotes with a throw counted as a failure rather than
// ending the run, so the cases after it are still reported.
std::string Merge(std::string_view doc, std::span<const DependencyNote> notes) {
try {
return MergeDependencyNotes(doc, notes);
} catch (const std::exception& e) {
std::println(std::cerr, "FAIL: MergeDependencyNotes threw: {}", e.what());
++Failures;
return std::format("<threw: {}>", e.what());
}
}
// Expect MergeDependencyNotes to throw; the message, or "" when it didn't.
std::string MergeFails(std::string_view doc, std::span<const DependencyNote> notes) {
try {
(void)MergeDependencyNotes(doc, notes);
} catch (const std::exception& e) {
return e.what();
}
return "";
}
std::int32_t RunWith(std::vector<std::string> args) {
std::vector<char*> argv;
for (std::string& arg : args) argv.push_back(arg.data());
return Run(static_cast<std::int32_t>(argv.size()), argv.data());
}
// LoadProject compiles a project file against the installed module
// sources; without them the end-to-end half cannot run at all.
bool ModuleSourcesInstalled() {
try {
return fs::exists(GetCrafterBuildHome() / "Crafter.Build.cppm");
} catch (const std::exception&) {
return false;
}
}
constexpr std::string_view BeginMarker = "<!-- crafter-build: dependency notes begin -->";
constexpr std::string_view EndMarker = "<!-- crafter-build: dependency notes end -->";
constexpr std::string_view GeneratedLine = "<!-- Written by crafter-build from AGENTS.template.md, and rewritten on every build. Edit that file, not this one. -->";
}
int main() {
// The merge on its own: what goes where in a document, with no project.
const std::string original = "# app\n\nThe consumer's own text.\n";
const std::vector<DependencyNote> two = {
{ .name = "one", .text = "## one\n\nFirst." },
{ .name = "two", .text = "## two\n\nSecond." },
};
const std::string merged = Merge(original, two);
Check(merged.starts_with(std::format("{}\n{}\n", original, BeginMarker)), "the section is appended after one blank line, and the text before it is untouched");
Check(merged.ends_with(std::format("{}\n", EndMarker)), "the section ends with its end marker");
const std::size_t first = merged.find("<!-- from one -->\n## one\n\nFirst.\n");
const std::size_t second = merged.find("<!-- from two -->\n## two\n\nSecond.\n");
Check(first != std::string::npos && second != std::string::npos && first < second, "each note follows a line naming its dependency, in the order given");
Check(Merge(merged, two) == merged, "merging the same notes again changes nothing");
Check(Merge(merged, {}) == original, "removing the section gives the file its own bytes back");
Check(Merge(original, {}) == original, "a file with no section and no notes is left alone");
// A section the user moved is rewritten where it stands, and removing it
// keeps the paragraphs around it apart.
const std::string section = merged.substr(original.size() + 1);
{
const std::string moved = std::format("# app\n\n{}\n## After\n\nThe user's text after it.\n", section);
const std::vector<DependencyNote> changed = { { .name = "one", .text = "## one\n\nChanged." } };
const std::string rewritten = Merge(moved, changed);
Check(rewritten.starts_with(std::format("# app\n\n{}\n", BeginMarker)), "the text before a moved section is kept");
Check(rewritten.ends_with("\n## After\n\nThe user's text after it.\n"), "the text after a moved section is kept");
Check(rewritten.contains("Changed.") && !rewritten.contains("First.") && !rewritten.contains("Second."), "a rewritten section holds exactly the new notes");
const std::string between = std::format("Para one.\n\n{}Para two.\n", section);
Check(Merge(between, {}) == "Para one.\n\nPara two.\n", "removing a section from between two paragraphs keeps them apart");
// An unclosed block after the section does not stop it being rewritten
// in place, since the section is found before it.
const std::string open = std::format("{}```\nnever closed\n", merged);
Check(MergeFails(open, changed).empty() && Merge(open, changed).contains("Changed."), "a section before an unclosed code block is still rewritten");
}
// A CRLF file gets a CRLF section and keeps it.
{
const std::string crlf = "# app\r\n\r\nText.\r\n";
const std::string withNotes = Merge(crlf, two);
Check(withNotes.starts_with(std::format("{}\r\n{}\r\n", crlf, BeginMarker)) && Count(withNotes, "\n") == Count(withNotes, "\r\n"), "a CRLF file gets a section with CRLF line ends");
Check(Merge(withNotes, two) == withNotes && Merge(withNotes, {}) == crlf, "a CRLF section is found again and removed cleanly");
}
// Markers quoted in a code block are not the section's, including a block
// that quotes a fenced block of its own; and only real fences count as
// code blocks.
{
const std::string quoted = std::format("# app\n\n````markdown\n{}\n```\nnested\n```\n{}\n````\n", BeginMarker, EndMarker);
const std::string withNotes = Merge(quoted, two);
Check(withNotes.starts_with(std::format("{}\n{}\n", quoted, BeginMarker)), "markers quoted in a code block are left alone, and a section is appended after them");
Check(Merge(withNotes, two) == withNotes, "the real section is found past the quoted markers");
// ```make``` is inline code: read as a fence, it would open a block
// that hides everything after it.
const std::string inlineCode = "# app\n\n```make``` builds everything.\n";
Check(MergeFails(inlineCode, two).empty(), "a backtick run with a backtick after it does not open a code block");
const std::string once = Merge(inlineCode, two);
Check(Merge(once, two) == once && Count(once, BeginMarker) == 1, "a section after inline code is found again");
// A fence inside a list item, closed at the item's indentation. Read
// without the list, the indented closer would open a block instead,
// and the quoted markers after it would be taken for the section.
const std::string listed = std::format("- ```bash\n x\n ```\n\n```\n{}\nThe user's quoted text.\n{}\n```\n", BeginMarker, EndMarker);
const std::string afterList = Merge(listed, two);
Check(afterList.starts_with(std::format("{}\n{}\n", listed, BeginMarker)), "a fence in a list item closes at the item's indentation");
Check(Merge(afterList, two) == afterList, "the section after a list item's fence is found again");
}
// With notes to write, guessing which part of someone's file is generated
// would overwrite their text, and missing a section would add another on
// every build.
Check(MergeFails(std::format("# app\n{}\ntext\n", BeginMarker), two).contains("line 2"), "a begin marker with no end marker is refused, naming its line");
Check(MergeFails(std::format("# app\n\n{}\n", EndMarker), two).contains("line 3"), "an end marker with no begin marker is refused");
Check(MergeFails(std::format("{}\n{}\n", EndMarker, BeginMarker), two).contains("line 1"), "an end marker before the begin marker is refused");
Check(MergeFails(std::format("{0}\n{1}\n{0}\n{1}\n", BeginMarker, EndMarker), two).contains("line 3"), "a second section is refused");
Check(MergeFails("# app\n\n```\nnever closed\n", two).contains("line 3"), "appending to a document that ends inside a code block is refused");
// With none, leaving the file alone loses nothing, so it is not an error.
const std::string broken = std::format("# app\n{}\ntext\n", BeginMarker);
Check(MergeFails(broken, {}).empty() && Merge(broken, {}) == broken, "with no notes, unbalanced markers leave the file alone without an error");
Check(Merge("# app\n\n```\nnever closed\n", {}) == "# app\n\n```\nnever closed\n", "with no notes, an unclosed code block is no error");
const std::vector<DependencyNote> markerNote = { { .name = "marker-note", .text = std::format("## bad\n{}", EndMarker) } };
Check(MergeFails(original, markerNote).contains("marker-note"), "a note with a marker line of its own is refused");
const std::vector<DependencyNote> fenceNote = { { .name = "fence-note", .text = "## bad\n```\nnever closed" } };
Check(MergeFails(original, fenceNote).contains("fence-note"), "a note that leaves a code block open is refused");
if (!ModuleSourcesInstalled()) {
std::println(std::cerr, "SKIP: no module sources to compile the fixture's project files against");
return Failures > 0 ? 1 : 77;
}
// Through LoadProject and the CLI. Staged, because each case rewrites
// AGENTS.md and every load drops a build/ tree beside its project file.
const fs::path home = fs::current_path();
const fs::path fixture = home / "tests" / "DependencyNotes" / "fixture";
const fs::path scratch = fs::temp_directory_path() / "crafter-build-dependency-notes";
std::error_code ec;
fs::remove_all(scratch, ec);
fs::create_directories(scratch);
fs::copy(fixture, scratch, fs::copy_options::recursive);
const fs::path app = scratch / "app";
const fs::path agents = app / "AGENTS.md";
const fs::path templateFile = app / "AGENTS.template.md";
const fs::path projectFile = app / "project.cpp";
const fs::path libNotes = scratch / "lib" / "AGENTS.consumer.md";
// Checked in under another name, so that this repository's tree holds no
// template a build could ever pick up.
const std::string templateText = Read(app / "AGENTS.template.md.in");
Write(templateFile, templateText);
// LocalProject resolves against the working directory, and app's paths
// are written for `cd app && crafter-build`.
fs::current_path(app);
std::array<std::string_view, 0> noArgs = {};
Configuration cfg;
try {
cfg = LoadProject(projectFile, noArgs);
} catch (const std::exception& e) {
std::println(std::cerr, "FAIL: LoadProject threw: {}", e.what());
return 1;
}
const std::string toolNote = "<!-- from tool -->\n## tool\n\nRun tool before committing.\n";
const std::string libNote = "<!-- from lib -->\n## lib\n\nCall lib through its module, never through its headers.\n";
// base's file opens and ends with blank lines, which the section drops.
const std::string baseNote = "<!-- from base -->\n## base\n\nbase counts from zero.\n";
// How every AGENTS.md written from templateText starts.
const std::string generated = std::format("{}\n{}", GeneratedLine, templateText);
Check(!fs::exists(agents) && UpdateAgentsMd(cfg, projectFile), "the first update creates AGENTS.md");
// Every case after this one reads or rewrites that file.
if (!fs::exists(agents)) return 1;
std::string text = Read(agents);
Check(text.starts_with(std::format("{}\n{}\n", generated, BeginMarker)), "AGENTS.md is the line saying a build wrote it, the template's text untouched, then the section");
Check(Read(templateFile) == templateText, "the template is never written");
const std::size_t tool = text.find(toolNote);
const std::size_t lib = text.find(libNote);
const std::size_t base = text.find(baseNote);
Check(tool != std::string::npos && lib != std::string::npos && base != std::string::npos, std::format("tool, lib and base each contribute their notes (got:\n{})", text));
Check(tool < lib && lib < base, "host tooling first, then what app names, then what lib builds against");
Check(!text.contains("Notes from app"), "app's own AGENTS.consumer.md stays out, even beside a dependency's project file");
Check(!text.contains("libtool"), "lib's host tooling stays out of its consumers' notes");
Check(!text.contains("testonly"), "a test's dependency stays out");
const fs::file_time_type written = fs::last_write_time(agents);
Check(!UpdateAgentsMd(cfg, projectFile), "an update with nothing changed writes nothing");
Check(Read(agents) == text && fs::last_write_time(agents) == written, "the bytes and the mtime are left alone when nothing changed");
cfg.dependencyNotes = DependencyNotes::Direct;
Check(UpdateAgentsMd(cfg, projectFile), "switching to Direct rewrites AGENTS.md");
text = Read(agents);
Check(text.contains(toolNote) && text.contains(libNote) && !text.contains("## base"), "Direct keeps what app names and drops what only lib does");
cfg.dependencyNotes = DependencyNotes::Transitive;
Write(libNotes, "## lib\n\nlib was rewritten.\n");
Check(UpdateAgentsMd(cfg, projectFile), "a dependency's edited notes are picked up");
text = Read(agents);
Check(text.contains("lib was rewritten.") && !text.contains("never through its headers") && Count(text, BeginMarker) == 1, "edited notes replace the old ones, in the one section");
// AGENTS.md follows the template: an edit made there shows up here, and
// the template's marker lines say where the notes go.
Write(templateFile, std::format("{}\nA paragraph added to the template.\n", templateText));
Check(UpdateAgentsMd(cfg, projectFile), "an edited template is picked up");
Check(Read(agents).starts_with(std::format("{}\nA paragraph added to the template.\n\n{}\n", generated, BeginMarker)), "the template's new text goes in ahead of the section");
Write(templateFile, std::format("# app\n\n{}\n{}\n\n## After\n\nText after the notes.\n", BeginMarker, EndMarker));
Check(UpdateAgentsMd(cfg, projectFile), "a template that places the section is picked up");
text = Read(agents);
Check(text.starts_with(std::format("{}\n# app\n\n{}\n", GeneratedLine, BeginMarker)) && text.contains(toolNote) && text.ends_with(std::format("{}\n\n## After\n\nText after the notes.\n", EndMarker)), "the section goes where the template's markers are");
Write(templateFile, templateText);
Check(UpdateAgentsMd(cfg, projectFile) && Read(agents).starts_with(generated), "AGENTS.md follows the template back");
// A CLAUDE.md hard-linked to AGENTS.md has to keep reading the same file.
fs::create_hard_link(agents, app / "CLAUDE.md");
Write(libNotes, "## lib\n\nlib, through a link.\n");
Check(UpdateAgentsMd(cfg, projectFile), "a hard-linked AGENTS.md is updated");
Check(Read(app / "CLAUDE.md") == Read(agents) && Read(agents).contains("lib, through a link.") && fs::hard_link_count(agents) == 2, "the update is visible through every link to the file");
fs::remove(app / "CLAUDE.md");
// A read-only AGENTS.md is a lock someone set on purpose. Root writes
// through the permission bits, so the check needs an ordinary user.
fs::permissions(agents, fs::perms::owner_read | fs::perms::group_read | fs::perms::others_read);
if (!std::ofstream(agents, std::ios::app).is_open()) {
const std::string locked = Read(agents);
Write(libNotes, "## lib\n\nlib, locked out.\n");
Check(!UpdateAgentsMd(cfg, projectFile) && Read(agents) == locked, "a read-only AGENTS.md is left alone");
}
fs::permissions(agents, fs::perms::owner_write, fs::perm_options::add);
// A build's own AGENTS.md is output, an edit made in it included; any
// other text in its place is someone's, and only an empty file is not.
(void)UpdateAgentsMd(cfg, projectFile);
const std::string settled = Read(agents);
Write(agents, std::format("{}\nAn edit made in AGENTS.md itself.\n", GeneratedLine));
Check(UpdateAgentsMd(cfg, projectFile) && Read(agents) == settled, "an edit made in AGENTS.md is replaced by the next build");
const std::string handWritten = "# app\n\nAn AGENTS.md committed before the template was.\n";
Write(agents, handWritten);
Check(!UpdateAgentsMd(cfg, projectFile) && Read(agents) == handWritten, "an AGENTS.md no build wrote is never replaced");
Write(agents, "");
Check(UpdateAgentsMd(cfg, projectFile) && Read(agents) == settled, "an empty AGENTS.md, all a write cut short leaves of a new one, is replaced");
cfg.dependencyNotes = DependencyNotes::None;
Check(UpdateAgentsMd(cfg, projectFile), "None rewrites AGENTS.md");
Check(Read(agents) == generated, "under None, AGENTS.md holds the template's text alone");
cfg.dependencyNotes = DependencyNotes::Transitive;
// Linked to the template, AGENTS.md would have every build write the
// file that is committed.
fs::remove(agents);
fs::create_symlink("AGENTS.template.md", agents);
Check(!UpdateAgentsMd(cfg, projectFile) && Read(templateFile) == templateText, "an AGENTS.md that links to the template is not written");
fs::remove(agents);
// A template the notes cannot be placed in leaves no AGENTS.md behind,
// not even an empty one.
Write(templateFile, std::format("# app\n\n{}\n", BeginMarker));
Check(!UpdateAgentsMd(cfg, projectFile) && !fs::exists(agents), "a template with a begin marker and no end marker writes no AGENTS.md");
// A CRLF template makes a CRLF AGENTS.md, the line a build adds included.
Write(templateFile, "# app\r\n\r\nText.\r\n");
Check(UpdateAgentsMd(cfg, projectFile), "a CRLF template is written");
text = Read(agents);
Check(text.starts_with(std::format("{}\r\n# app\r\n", GeneratedLine)) && Count(text, "\n") == Count(text, "\r\n"), "AGENTS.md from a CRLF template ends every line in CRLF");
Write(templateFile, templateText);
fs::remove(agents);
// A layout that keeps the file elsewhere and links it, which a write that
// replaced the link with a copy would quietly undo.
fs::create_directories(app / "docs");
Write(app / "docs" / "agents.md", "");
fs::create_symlink("docs/agents.md", agents);
Check(UpdateAgentsMd(cfg, projectFile), "an AGENTS.md that is a symlink is written");
Check(fs::is_symlink(agents) && Read(app / "docs" / "agents.md").contains(toolNote), "the write goes through the link, which stays a link");
fs::remove(agents);
// Without a template, AGENTS.md is someone's own: never created, and
// never written, not even to take out a section an earlier crafter-build
// kept in it, which only gets a warning.
fs::remove(templateFile);
Check(!UpdateAgentsMd(cfg, projectFile) && !fs::exists(agents), "a project without a template gets no AGENTS.md");
const std::vector<DependencyNote> oldNotes = { { .name = "lib", .text = "## lib\n\nNotes a committed AGENTS.md still holds." } };
const std::string committed = Merge(templateText, oldNotes);
Write(agents, committed);
Check(!UpdateAgentsMd(cfg, projectFile) && Read(agents) == committed, "without a template, an AGENTS.md holding a section is left alone");
fs::remove(agents);
Write(templateFile, templateText);
// The CLI: only a build or a test run writes it.
(void)RunWith({"crafter-build", "-q", "lint"});
Check(!fs::exists(agents), "lint writes no AGENTS.md");
(void)RunWith({"crafter-build", "-q", "run"});
Check(!fs::exists(agents), "run writes no AGENTS.md");
(void)RunWith({"crafter-build", "-q", "test", "--list"});
Check(!fs::exists(agents), "test --list writes no AGENTS.md");
(void)RunWith({"crafter-build", "-q", "test"});
Check(Read(agents).contains(toolNote), "a test run writes AGENTS.md");
// app has no sources, so this build fails at the link, after the update.
fs::remove(agents);
(void)RunWith({"crafter-build", "-q"});
Check(Read(agents).contains(toolNote), "a build writes AGENTS.md before compiling, so one that fails still does");
// The builds above leave a std PCM per configuration in a tmpfs; a run
// that failed keeps them for a look.
fs::current_path(home);
if (Failures == 0) fs::remove_all(scratch, ec);
return Failures > 0 ? 1 : 0;
}

View file

@ -0,0 +1,97 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// A CMake dependency configured with other inputs than this build's (another
// compiler or --sysroot, the flags crafter-build injects, which it records in
// <build>/.crafter-options) has to be built again from nothing. It was
// reconfigured in place, and CMake does not recompile what a changed
// --sysroot compiled (the Makefiles generator writes it into build.make,
// which make does not compare) and keeps the check results of the old
// configure. A mingw dependency built by a release that gave clang the
// host's /usr as the sysroot (issue #43) kept its objects reading
// __libc_single_threaded after the update.
//
// The dependency is a local repository with a one-file C library. A file
// planted in its build directory stands for what the old configure left,
// and the recorded inputs are rewritten the way another crafter-build would
// have left them. Everything is local; XDG_CACHE_HOME keeps the clone out of
// the real cache.
extern "C" int setenv(const char* name, const char* value, int overwrite);
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
void Git(const fs::path& dir, std::string_view args) {
const std::string command = std::format("git -C {} {}", ShellQuote(dir.string()), args);
auto result = RunCommandWithTimeout(command, std::chrono::seconds(60));
if (result.exitCode != 0) throw std::runtime_error(std::format("{} failed: {}", command, result.output));
}
void Write(const fs::path& file, std::string_view text) {
std::ofstream(file, std::ios::binary | std::ios::trunc) << text;
}
}
int main() {
const fs::path scratch = fs::temp_directory_path() / "crafter-build-external-reconfigure";
fs::remove_all(scratch);
fs::create_directories(scratch);
setenv("XDG_CACHE_HOME", (scratch / "cache").c_str(), 1);
setenv("GIT_CONFIG_NOSYSTEM", "1", 1);
setenv("GIT_CONFIG_GLOBAL", "/dev/null", 1);
for (const char* name : { "GIT_AUTHOR_NAME", "GIT_COMMITTER_NAME" }) setenv(name, "crafter-build test", 1);
for (const char* name : { "GIT_AUTHOR_EMAIL", "GIT_COMMITTER_EMAIL" }) setenv(name, "test@example.invalid", 1);
const fs::path repo = scratch / "reconf";
fs::create_directories(repo);
Write(repo / "lib.c", "int Reconfigured(void) { return 1; }\n");
Write(repo / "CMakeLists.txt",
"cmake_minimum_required(VERSION 3.16)\n"
"project(reconf C)\n"
"add_library(reconf STATIC lib.c)\n");
Git(repo, "init -q -b main");
Git(repo, "add .");
Git(repo, "commit -q -m reconf");
ExternalDependency dep;
dep.name = "reconf";
dep.source.url = std::format("file://{}", repo.string());
dep.builder = ExternalBuilder::CMake;
dep.libs = { "reconf" };
std::string error = BuildExternal(dep, HostTarget());
Check(error.empty(), std::format("the dependency builds: {}", error));
const fs::path build = ExternalBuildDir(dep, HostTarget());
const fs::path leftover = build / "left-by-the-old-configure";
Write(leftover, "");
error = BuildExternal(dep, HostTarget());
Check(error.empty(), std::format("and builds again: {}", error));
Check(fs::exists(leftover), "with the same inputs its build directory is kept");
Write(build / ".crafter-options", " -DCMAKE_SYSROOT=/usr\n");
error = BuildExternal(dep, HostTarget());
Check(error.empty(), std::format("configured with other inputs, it builds: {}", error));
Check(!fs::exists(leftover), "from an empty build directory");
Check(fs::exists(build / "libreconf.a"), "and the library is built again");
std::error_code ec;
fs::remove_all(scratch, ec);
if (failures > 0) {
std::println(std::cerr, "{} assertions failed", failures);
return 1;
}
return 0;
}

View file

@ -6,6 +6,9 @@ import Crafter.Build;
namespace fs = std::filesystem; namespace fs = std::filesystem;
using namespace Crafter; using namespace Crafter;
extern "C" int setenv(const char* name, const char* value, int overwrite);
extern "C" int unsetenv(const char* name);
namespace { namespace {
std::int32_t Failures = 0; std::int32_t Failures = 0;
@ -705,6 +708,31 @@ int main() {
// two assertions above cover it. // two assertions above cover it.
} }
// A configuration whose std module cannot be built is in the build graph
// all the same: its files cannot be parsed, which has to fail the run.
// Its command was cleared, which read as "not a translation unit", and
// every semantic rule passed the files. A sysroot without libc++ is one
// way there. The std module's directory is made before its build fails,
// so the cache is redirected around this run.
{
Scratch s("ast-no-std-module");
s.Write("f", "int Value = 1;\n");
Configuration cfg = s.Config({"f"});
cfg.sysroot = (s.dir / "empty-sysroot").string();
fs::create_directories(cfg.sysroot);
bool ran = false;
cfg.AddAstLintRule("needs-ast", [&ran](LintContext&) { ran = true; });
const char* realCache = std::getenv("XDG_CACHE_HOME");
const std::string saved = realCache ? realCache : "";
setenv("XDG_CACHE_HOME", (s.dir / "cache").c_str(), 1);
LintSummary summary = RunLint(cfg, Mode(LintMode::Report));
if (saved.empty()) unsetenv("XDG_CACHE_HOME");
else setenv("XDG_CACHE_HOME", saved.c_str(), 1);
Check(!ran, "ast: rule is skipped when the std module cannot be built");
Check(summary.errors > 0, "ast: a std module that cannot be built fails the run");
Check(!summary.Clean(), "ast: a std module that cannot be built is not Clean");
}
// --no-ast skips those rules deliberately and exits normally. // --no-ast skips those rules deliberately and exits normally.
{ {
Scratch s("ast-optout"); Scratch s("ast-optout");

View file

@ -0,0 +1,13 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
// probe.cpp's checks for the C sources, which are compiled by a command of
// their own.
#if __has_include(<sys/single_threaded.h>)
#error the include/ of the prefix is on the C include path: a Windows build sees the headers of the Linux system
#endif
#if !__has_include(<_mingw.h>)
#error the mingw-w64 headers are not on the C include path
#endif
void CProbe(void) {}

View file

@ -0,0 +1,6 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
// Stops in a header of the stand-in mingw-w64, so that the error clang
// reports is located in one.
#include <crafter_probe_error.h>

View file

@ -0,0 +1,16 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
// Built for x86_64-w64-mingw32 against the prefix tests/MingwSysroot stages:
// a Linux system's /usr with a mingw-w64 cross toolchain in it.
#if __has_include(<sys/single_threaded.h>)
#error the include/ of the prefix is on the include path: a Windows build sees the headers of the Linux system
#endif
#if !__has_include(<_mingw.h>)
#error the mingw-w64 headers are not on the include path
#endif
#if !__has_include(<memory>)
#error the libstdc++ headers are not on the include path: clang found no gcc version under lib/gcc of the prefix
#endif
void Probe() {}

151
tests/MingwSysroot/main.cpp Normal file
View file

@ -0,0 +1,151 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// A mingw-w64 prefix that is a whole Linux system's /usr must not hand that
// system's headers to a Windows build. Inside the Flatpak the prefix is the
// host's /usr, and clang got it as the --sysroot. clang's mingw driver takes
// a sysroot for a tree that holds mingw-w64 alone and searches its include/
// after mingw-w64's own headers, so every header mingw-w64 lacks came from
// glibc: libstdc++ found <sys/single_threaded.h> and had shared_ptr read
// __libc_single_threaded, which nothing on Windows defines, and every
// program that used one failed to link (issue #43). CRAFTER_MINGW_DIR=/usr
// did the same outside the Flatpak.
//
// The prefix staged here has the shape of such a /usr and needs no real
// mingw-w64: glibc's features.h and <sys/single_threaded.h> in include/,
// gcc's per-target directory, which clang derives libstdc++'s header
// directory from, and a stand-in mingw-w64 tree whose libstdc++ has a std
// module with nothing in it. The fixture is a static library, so nothing is
// linked, and its C++ and C sources stop the compile when they see the
// prefix's include/ or miss mingw-w64's or libstdc++'s headers.
//
// clang reaches such a prefix through a view of it (MakeMingwSysroot), which
// has to be made where symbolic links can be, and a compile error in one of
// its headers has to name the header where it is installed, not in the
// view.
extern "C" int setenv(const char* name, const char* value, int overwrite);
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
void Write(const fs::path& file, std::string_view text) {
fs::create_directories(file.parent_path());
std::ofstream(file, std::ios::binary | std::ios::trunc) << text;
}
BuildResult BuildLibrary(const fs::path& project, std::string_view name, const fs::path& cFile) {
Configuration lib;
lib.path = project;
lib.name = std::string(name);
lib.outputName = std::string(name);
lib.target = "x86_64-w64-mingw32";
// Spelled out: the default, native, names the host's CPU, which is
// no x86 CPU on an aarch64 host.
lib.march = "x86-64";
lib.mtune = "x86-64";
lib.type = ConfigurationType::LibraryStatic;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { fs::path(name) };
lib.GetInterfacesAndImplementations(ifaces, impls);
if (!cFile.empty()) lib.cFiles = { cFile };
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
return Build(lib, depResults, depMutex);
}
}
int main() {
const fs::path scratch = fs::temp_directory_path() / "crafter-build-mingw-sysroot";
std::error_code ec;
fs::remove_all(scratch, ec);
fs::create_directories(scratch);
const fs::path project = scratch / "project";
fs::copy(fs::current_path() / "tests" / "MingwSysroot" / "fixture", project, fs::copy_options::recursive);
const fs::path prefix = scratch / "usr";
Write(prefix / "include" / "features.h", "#define __GLIBC__ 2\n");
Write(prefix / "include" / "sys" / "single_threaded.h", "extern char __libc_single_threaded;\n");
fs::create_directories(prefix / "lib" / "gcc" / "x86_64-w64-mingw32" / "15.1.0");
const fs::path mingw = prefix / "x86_64-w64-mingw32";
Write(mingw / "include" / "_mingw.h", "");
Write(mingw / "include" / "crafter_probe_error.h", "#error stopped in a mingw-w64 header\n");
Write(mingw / "include" / "c++" / "15.1.0" / "memory", "");
// Through std::format because the build's import scanner reads string
// literals as code: one holding "module std;" made this file a unit of a
// module std, and the test did not build.
Write(mingw / "include" / "c++" / "15.1.0" / "bits" / "std.cc", std::format("export module {};\n", "std"));
Write(mingw / "lib" / "libstdc++.modules.json", R"({"version": 1, "modules": [{"logical-name": "std", "source-path": "../include/c++/15.1.0/bits/std.cc", "is-std-library": true}]})");
{
// The view is made in the first place that can hold one. A cache on
// a filesystem that takes no symbolic links (vfat, exFAT, some CIFS
// and FUSE mounts) holds everything else crafter-build caches, so
// its failure must not be every mingw build's: this one can make
// nothing under mingw-sysroot, which is a file.
const fs::path blocked = scratch / "blocked";
Write(blocked / "mingw-sysroot", "");
const fs::path runtime = scratch / "runtime";
const std::array<fs::path, 2> parents = { blocked, runtime };
fs::path view;
try {
view = MakeMingwSysroot(prefix, parents);
} catch (const std::exception& e) {
Check(false, std::format("a view is made in the second place when the first can hold none: {}", e.what()));
}
if (!view.empty()) {
Check(view.string().starts_with(runtime.string()), std::format("the view is in the place that could hold it: {}", view.string()));
Check(fs::equivalent(view / "x86_64-w64-mingw32", mingw, ec), "the view shows the prefix's mingw-w64 tree");
Check(fs::equivalent(view / "lib" / "gcc", prefix / "lib" / "gcc", ec), "and gcc's directories");
Check(!fs::exists(view / "include") && !fs::exists(view / "lib" / "libstdc++.modules.json"), "and nothing else of the prefix's include/ and lib/");
fs::path again;
try {
again = MakeMingwSysroot(prefix, parents);
} catch (const std::exception& e) {
Check(false, std::format("a view already there is reused: {}", e.what()));
}
Check(again == view, "a view already there is reused");
}
const std::array<fs::path, 1> nowhere = { blocked };
std::string error;
try {
MakeMingwSysroot(prefix, nowhere);
} catch (const std::exception& e) {
error = e.what();
}
Check(error.contains(prefix.string()) && error.contains((blocked / "mingw-sysroot").string()), std::format("with no place for a view, the error names the prefix and why each place failed: {}", error));
}
// Before anything resolves the toolchain, which reads both once. The
// view and the std module cache go to the scratch directory too.
setenv("CRAFTER_MINGW_DIR", prefix.c_str(), 1);
setenv("XDG_CACHE_HOME", (scratch / "cache").c_str(), 1);
const BuildResult built = BuildLibrary(project, "probe", project / "cprobe");
Check(built.result.empty(), std::format("a Windows build sees mingw-w64's and libstdc++'s headers and not the prefix's own: {}", built.result));
Check(MingwSysroot() != prefix && MingwSysroot().string().starts_with((scratch / "cache").string()), std::format("a prefix holding a Linux system reaches clang through a view in the cache: {}", MingwSysroot().string()));
const BuildResult failed = BuildLibrary(project, "failing", {});
const std::string header = (mingw / "include" / "crafter_probe_error.h").string();
Check(failed.result.contains("stopped in a mingw-w64 header"), std::format("an error in a mingw-w64 header fails the build: {}", failed.result));
Check(failed.result.contains(header) && !failed.result.contains(MingwSysroot().string()), std::format("and names the header in the prefix, not in the view: {}", failed.result));
fs::remove_all(scratch, ec);
if (failures > 0) {
std::println(std::cerr, "{} assertions failed", failures);
return 1;
}
return 0;
}

View file

@ -0,0 +1,8 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
int main() {
std::println("rebuilt");
}

View file

@ -0,0 +1,85 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// A std module rebuilt in place has to recompile the units that import it.
// The std PCM is rebuilt in its directory whenever its source changes under
// the same compiler and flags: a libc++ update, or mingw-w64's under the
// same prefix. A .cpp unit's depfile listed only its source and headers,
// because the driver records imported BMIs for --precompile alone, so such
// a unit kept its object and the next build only relinked it.
//
// The project is one main.cpp that imports std. Its object is dated before
// std.pcm and after its source, as a std module rebuilt after the object
// leaves them, and the next build has to compile it again. XDG_CACHE_HOME keeps the std PCM out of the real
// cache, whose other users the change of date would rebuild.
extern "C" int setenv(const char* name, const char* value, int overwrite);
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
BuildResult BuildOnce(Configuration& cfg) {
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
return Build(cfg, depResults, depMutex);
}
}
int main() {
const fs::path scratch = fs::temp_directory_path() / "crafter-build-std-pcm-rebuild";
std::error_code ec;
fs::remove_all(scratch, ec);
fs::create_directories(scratch);
const fs::path project = scratch / "project";
fs::copy(fs::current_path() / "tests" / "StdPcmRebuild" / "fixture", project, fs::copy_options::recursive);
setenv("XDG_CACHE_HOME", (scratch / "cache").c_str(), 1);
Configuration cfg;
cfg.path = project;
cfg.name = "rebuilt";
cfg.outputName = "rebuilt";
cfg.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
BuildResult built = BuildOnce(cfg);
Check(built.result.empty(), std::format("the project builds: {}", built.result));
const fs::path object = cfg.BuildDir() / "main_impl.o";
const fs::path stdPcm = StdPcmDir(cfg) / "std.pcm";
Check(fs::exists(object) && fs::exists(stdPcm), std::format("{} and {} exist", object.string(), stdPcm.string()));
if (!fs::exists(object) || !fs::exists(stdPcm)) return 1;
const fs::file_time_type first = fs::last_write_time(object);
built = BuildOnce(cfg);
Check(built.result.empty(), std::format("an idle build succeeds: {}", built.result));
Check(fs::last_write_time(object) == first, "and compiles nothing again");
// The source before the object and the object before std.pcm, so that
// nothing but the std module makes the object stale.
const fs::file_time_type older = fs::last_write_time(stdPcm) - std::chrono::seconds(10);
fs::last_write_time(project / "main.cpp", older - std::chrono::seconds(10));
fs::last_write_time(object, older);
built = BuildOnce(cfg);
Check(built.result.empty(), std::format("the build after the std module succeeds: {}", built.result));
Check(fs::last_write_time(object) > older, "a unit importing a std module newer than its object is compiled again");
fs::remove_all(scratch, ec);
if (failures > 0) {
std::println(std::cerr, "{} assertions failed", failures);
return 1;
}
return 0;
}

View file

@ -19,6 +19,9 @@ using namespace Crafter;
// sysroot's copy. It must not do so where there is no sysroot, for wasm (which // sysroot's copy. It must not do so where there is no sysroot, for wasm (which
// has its own sysroot handling) or for mingw (libstdc++, no libc++ at all). // has its own sysroot handling) or for mingw (libstdc++, no libc++ at all).
// The command is checked as text: no compiler runs. // The command is checked as text: no compiler runs.
extern "C" int setenv(const char* name, const char* value, int overwrite);
namespace { namespace {
std::int32_t Failures = 0; std::int32_t Failures = 0;
@ -44,6 +47,15 @@ namespace {
} }
int main() { int main() {
// A compile command names the std module's directory in the cache, and
// the mingw one can make the view of a system mingw prefix there
// (MingwSysroot), as with CRAFTER_MINGW_DIR=/usr or inside the Flatpak
// on a host with mingw-w64. Set before anything resolves the toolchain.
const fs::path scratch = fs::temp_directory_path() / "crafter-build-sysroot-compile-flags-cache";
std::error_code ec;
fs::remove_all(scratch, ec);
setenv("XDG_CACHE_HOME", scratch.c_str(), 1);
const std::string sysroot = "/opt/crafter-build-test-sysroot"; const std::string sysroot = "/opt/crafter-build-test-sysroot";
const std::string forced = std::format("-nostdinc++ -isystem {}/usr/include/c++/v1", sysroot); const std::string forced = std::format("-nostdinc++ -isystem {}/usr/include/c++/v1", sysroot);
@ -57,6 +69,7 @@ int main() {
Check(!CommandFor("wasm32-wasip1", sysroot).contains(forced), "wasm: left to its own sysroot handling"); Check(!CommandFor("wasm32-wasip1", sysroot).contains(forced), "wasm: left to its own sysroot handling");
Check(!CommandFor("x86_64-w64-mingw32", sysroot).contains(forced), "mingw: libstdc++ target, libc++ headers are not forced"); Check(!CommandFor("x86_64-w64-mingw32", sysroot).contains(forced), "mingw: libstdc++ target, libc++ headers are not forced");
fs::remove_all(scratch, ec);
if (Failures > 0) { if (Failures > 0) {
std::println(std::cerr, "{} assertions failed", Failures); std::println(std::cerr, "{} assertions failed", Failures);
return 1; return 1;

View file

@ -61,7 +61,7 @@ int main() {
Check(tc.libcxxVersion == LibcxxVersion("/usr/include/c++/v1"), "the system libc++ headers state the release of the system module source"); Check(tc.libcxxVersion == LibcxxVersion("/usr/include/c++/v1"), "the system libc++ headers state the release of the system module source");
Check(tc.stdModulesDir.empty(), "no crafter-build prefix, no shipped module sources"); Check(tc.stdModulesDir.empty(), "no crafter-build prefix, no shipped module sources");
Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "the system wasi sysroot is the default"); Check(tc.wasiSysroot == "/usr/share/wasi-sysroot", "the system wasi sysroot is the default");
Check(tc.mingwSysroot.empty(), "outside a sandbox, clang finds mingw-w64 itself: no --sysroot"); Check(tc.mingwPrefix.empty(), "outside a sandbox, clang finds mingw-w64 itself: no prefix");
Check(tc.LibcxxCompileFlags().empty() && tc.LibcxxLinkFlags().empty(), "the PATH fallback adds no libc++ flags"); Check(tc.LibcxxCompileFlags().empty() && tc.LibcxxLinkFlags().empty(), "the PATH fallback adds no libc++ flags");
} }
{ {
@ -144,7 +144,34 @@ int main() {
in.env["CRAFTER_MINGW_DIR"] = "/opt/mingw"; in.env["CRAFTER_MINGW_DIR"] = "/opt/mingw";
Toolchain tc = ResolveToolchain(in); Toolchain tc = ResolveToolchain(in);
Check(tc.wasiSysroot == "/opt/wasi", "CRAFTER_BUILD_WASI_SYSROOT overrides the wasi sysroot"); Check(tc.wasiSysroot == "/opt/wasi", "CRAFTER_BUILD_WASI_SYSROOT overrides the wasi sysroot");
Check(tc.mingwSysroot == "/opt/mingw", "CRAFTER_MINGW_DIR names the mingw prefix passed as --sysroot"); Check(tc.mingwPrefix == "/opt/mingw", "CRAFTER_MINGW_DIR names the mingw prefix");
in.env["CRAFTER_MINGW_DIR"] = "mingw";
Check(ResolveToolchain(in).mingwPrefix == fs::absolute("mingw"), "a relative CRAFTER_MINGW_DIR is resolved against the working directory once, when the toolchain is");
// The prefix names directories, so one directory has one spelling:
// otherwise a trailing slash rebuilds every mingw build for nothing.
for (const char* spelling : {"/opt/mingw/", "/opt//mingw", "/opt/./mingw", "/opt/x/../mingw"}) {
in.env["CRAFTER_MINGW_DIR"] = spelling;
Check(ResolveToolchain(in).mingwPrefix == "/opt/mingw", std::format("{} is spelled /opt/mingw (got '{}')", spelling, ResolveToolchain(in).mingwPrefix.string()));
}
}
{
// A prefix that holds a Linux system's C library, as /usr does, is
// shown to clang through a view; one that holds mingw-w64 alone is
// the sysroot itself, Windows libraries in its include/ and lib/
// included.
const fs::path staged = fs::temp_directory_path() / "crafter-build-toolchain-mingw-prefixes";
fs::remove_all(staged);
Touch(staged / "system" / "include" / "features.h");
Touch(staged / "system" / "x86_64-w64-mingw32" / "include" / "_mingw.h");
Touch(staged / "dedicated" / "include" / "SDL2" / "SDL.h");
Touch(staged / "dedicated" / "x86_64-w64-mingw32" / "include" / "_mingw.h");
ToolchainInputs in;
in.env["CRAFTER_MINGW_DIR"] = (staged / "system").string();
Check(ResolveToolchain(in).mingwPrefixIsSystem, "a prefix with a C library's features.h holds a Linux system");
in.env["CRAFTER_MINGW_DIR"] = (staged / "dedicated").string();
Check(!ResolveToolchain(in).mingwPrefixIsSystem, "a prefix of mingw-w64 and Windows libraries does not");
Check(!ResolveToolchain({}).mingwPrefixIsSystem, "no prefix, nothing to show");
fs::remove_all(staged);
} }
{ {
// Sandboxed, the host's /usr stands in for the missing mingw-w64, // Sandboxed, the host's /usr stands in for the missing mingw-w64,
@ -153,11 +180,12 @@ int main() {
fs::remove_all(host); fs::remove_all(host);
ToolchainInputs in; ToolchainInputs in;
in.hostRoot = host; in.hostRoot = host;
Check(ResolveToolchain(in).mingwSysroot.empty(), "a host without mingw-w64 gives no --sysroot"); Check(ResolveToolchain(in).mingwPrefix.empty(), "a host without mingw-w64 gives no mingw prefix");
fs::create_directories(host / "usr" / "x86_64-w64-mingw32"); fs::create_directories(host / "usr" / "x86_64-w64-mingw32");
Check(ResolveToolchain(in).mingwSysroot == host / "usr", "a host with mingw-w64 gives its /usr as the mingw prefix"); Check(ResolveToolchain(in).mingwPrefix == host / "usr", "a host with mingw-w64 gives its /usr as the mingw prefix");
Check(ResolveToolchain(in).mingwPrefixIsSystem, "the host's /usr is the host's system, with or without its C library's headers");
in.env["CRAFTER_MINGW_DIR"] = "/opt/mingw"; in.env["CRAFTER_MINGW_DIR"] = "/opt/mingw";
Check(ResolveToolchain(in).mingwSysroot == "/opt/mingw", "CRAFTER_MINGW_DIR wins over the host's"); Check(ResolveToolchain(in).mingwPrefix == "/opt/mingw", "CRAFTER_MINGW_DIR wins over the host's");
fs::remove_all(host); fs::remove_all(host);
} }
{ {

View file

@ -160,6 +160,65 @@ int main() {
Check(cfg.VariantId() == cfg.VariantId(GetToolchain()), "VariantId() is the variant under the running toolchain"); Check(cfg.VariantId() == cfg.VariantId(GetToolchain()), "VariantId() is the variant under the running toolchain");
Check(cfg.BuildDir().filename() == cfg.VariantId(GetToolchain()), "BuildDir is named for the running toolchain's variant"); Check(cfg.BuildDir().filename() == cfg.VariantId(GetToolchain()), "BuildDir is named for the running toolchain's variant");
} }
{
// The mingw-w64 prefix decides what a mingw build compiles against.
// Before issue #43 it was clang's sysroot as is, which put the host's
// headers on the include path, and with the prefix left out of the
// id, the release that fixed that reused the objects built against
// them: the next build only relinked, and failed to link as before.
Configuration mingw = MakeBase();
mingw.target = "x86_64-w64-mingw32";
Toolchain discovered;
discovered.clangxx = "/app/bin/clang++";
Toolchain hostUsr = discovered;
hostUsr.mingwPrefix = "/run/host/usr";
Toolchain optMingw = discovered;
optMingw.mingwPrefix = "/opt/mingw";
Check(mingw.VariantId(discovered) != mingw.VariantId(hostUsr), "a mingw prefix perturbs a mingw configuration's VariantId");
Check(mingw.VariantId(hostUsr) != mingw.VariantId(optMingw), "two mingw prefixes are two variants");
Configuration host = MakeBase();
Check(host.VariantId(discovered) == host.VariantId(hostUsr), "the mingw prefix leaves other targets' variants alone");
// The std module's directory is keyed the same way, and so is a
// prefix given whole against one reached through a view.
Check(StdPcmKey(mingw, discovered) != StdPcmKey(mingw, hostUsr), "a mingw prefix moves the mingw std module");
Check(StdPcmKey(mingw, hostUsr) != StdPcmKey(mingw, optMingw), "two mingw prefixes are two std modules");
Toolchain hostUsrWhole = hostUsr;
hostUsrWhole.mingwPrefixIsSystem = !hostUsr.mingwPrefixIsSystem;
Check(StdPcmKey(mingw, hostUsr) != StdPcmKey(mingw, hostUsrWhole), "a prefix given whole and one shown through a view are two std modules");
Check(mingw.VariantId(hostUsr) != mingw.VariantId(hostUsrWhole), "and two variants, through the std module");
Check(StdPcmKey(host, discovered) == StdPcmKey(host, hostUsr), "the mingw prefix leaves other targets' std modules alone");
}
{
// Everything the std module is compiled with keys its directory, not
// only target and march. -mtune was left out, so a project built
// with -mtune=generic shared one std PCM with project.cpp's
// -mtune=native, and the two rebuilt it in turn on every run, under
// every module built against it; two sysroots did the same.
Toolchain tc;
tc.clangxx = "/app/bin/clang++";
tc.clangVersion = "clang version 23.1.2";
Configuration native = MakeBase();
Configuration generic = MakeBase();
generic.mtune = "generic";
Check(StdPcmKey(native, tc) != StdPcmKey(generic, tc), "-mtune keys the std module");
Configuration sysrootA = MakeBase();
sysrootA.sysroot = "/opt/sysroot-a";
Configuration sysrootB = MakeBase();
sysrootB.sysroot = "/opt/sysroot-b";
Check(StdPcmKey(sysrootA, tc) != StdPcmKey(sysrootB, tc) && StdPcmKey(sysrootA, tc) != StdPcmKey(native, tc), "the sysroot keys the std module");
Configuration wasm = MakeBase();
wasm.target = "wasm32-wasip1";
wasm.sysroot = "/opt/wasi-sysroot";
Configuration relaxed = MakeBase();
relaxed.target = wasm.target;
relaxed.sysroot = wasm.sysroot;
relaxed.wasmVariantFlags = {"-mrelaxed-simd"};
Check(StdPcmKey(wasm, tc) != StdPcmKey(relaxed, tc), "a wasm variant pass's flags key the std module");
Check(StdPcmKey(native, tc) == StdPcmKey(MakeBase(), tc), "one configuration, one std module");
Check(StdPcmKey(native, tc).starts_with(std::format("{}-{}-{}-", native.target, native.march, tc.Key())), std::format("the directory still says target, march and toolchain (got '{}')", StdPcmKey(native, tc)));
Check(native.VariantId(tc) != generic.VariantId(tc) && sysrootA.VariantId(tc) != sysrootB.VariantId(tc), "a build moves with its std module");
}
if (Failures > 0) { if (Failures > 0) {
std::println(std::cerr, "{} assertions failed", Failures); std::println(std::cerr, "{} assertions failed", Failures);

View file

@ -87,7 +87,10 @@ int main() {
} }
// With no declared variants, variants.json is a single baseline entry // With no declared variants, variants.json is a single baseline entry
// pointing at outputName.wasm with no probes. // pointing at outputName.wasm, with only the probes for what every wasm
// build needs, SIMD128 and exception handling, so that runtime.js can
// turn an engine without them away with a message rather than a
// CompileError.
{ {
std::ifstream vin(*variants); std::ifstream vin(*variants);
std::string vjson{std::istreambuf_iterator<char>(vin), std::istreambuf_iterator<char>()}; std::string vjson{std::istreambuf_iterator<char>(vin), std::istreambuf_iterator<char>()};
@ -95,6 +98,10 @@ int main() {
std::println(std::cerr, "variants.json missing baseline url: {}", vjson); std::println(std::cerr, "variants.json missing baseline url: {}", vjson);
return 1; return 1;
} }
if (vjson.find("\"probes\":[\"simd\",\"exception-handling\"]") == std::string::npos) {
std::println(std::cerr, "variants.json baseline does not probe for SIMD and exception handling: {}", vjson);
return 1;
}
if (vjson.find("relaxed-simd") != std::string::npos) { if (vjson.find("relaxed-simd") != std::string::npos) {
std::println(std::cerr, "variants.json unexpectedly mentions relaxed-simd: {}", vjson); std::println(std::cerr, "variants.json unexpectedly mentions relaxed-simd: {}", vjson);
return 1; return 1;

View file

@ -0,0 +1,167 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
// Runs wasi-runtime/runtime.js (argv[2], copied to a .mjs so that node loads
// it as a module) once per scenario against the module assembled below, and
// prints one line per scenario for the test to check. Between _start and the
// call that ends it, the module has the frames C++ built with
// -fwasm-exceptions puts there: a destructor (catch_all_ref, then
// throw_ref), a catch (...) (catch_all) and a noexcept function (catch_all,
// then std::terminate). Each reports through env.trace when it runs, and on
// the way out of an exit or a park none of them may.
import { pathToFileURL } from "node:url";
const uleb = (n) => {
const out = [];
do {
const byte = n & 0x7f;
n >>>= 7;
out.push(n ? byte | 0x80 : byte);
} while (n);
return out;
};
const sleb = (n) => {
const out = [];
for (;;) {
const byte = n & 0x7f;
n >>= 7;
const done = (n === 0 && !(byte & 0x40)) || (n === -1 && (byte & 0x40));
out.push(done ? byte : byte | 0x80);
if (done) return out;
}
};
const vec = (items) => [...uleb(items.length), ...items.flat()];
const name = (text) => vec([...text].map((c) => [c.charCodeAt(0)]));
const section = (id, items) => {
const bytes = vec(items);
return [id, ...uleb(bytes.length), ...bytes];
};
const UNREACHABLE = 0x00, BLOCK = 0x02, IF = 0x04, ELSE = 0x05, THROW = 0x08, THROW_REF = 0x0a, END = 0x0b;
const RETURN = 0x0f, CALL = 0x10, TRY_TABLE = 0x1f, LOCAL_GET = 0x20, LOCAL_SET = 0x21, GLOBAL_GET = 0x23;
const GLOBAL_SET = 0x24, I32_CONST = 0x41, I32_EQ = 0x46;
const I32 = 0x7f, EXNREF = 0x69, NO_RESULT = 0x40, FUNC_TYPE = 0x60, CATCH_ALL = 0x02, CATCH_ALL_REF = 0x03;
const code = (locals, instructions) => {
const bytes = [...vec(locals), ...instructions, END];
return [...uleb(bytes.length), ...bytes];
};
// Types, imported functions, then the module's own.
const TAKES_I32 = 0, NOTHING = 1, GIVES_I32 = 2;
const PROC_EXIT = 0, PARK = 1, SCENARIO = 2, TRACE = 3;
const END_WITH = 4, NOEXCEPT = 5, CATCHES_ALL = 6, CLEANS_UP = 7, START = 8, FRAME = 9, QUIT = 10;
// What env.trace reports.
const DESTRUCTOR = 1, CATCH = 2, TERMINATE = 3, AFTER_PARK = 4;
// What env.scenario selects in _start.
const EXIT = 1, PARK_MAIN = 2, THROW_UNCAUGHT = 4, CRASH = 5;
const PARK_CODE = -1;
const ifScenario = (scenario, instructions) => [LOCAL_GET, 0, I32_CONST, ...sleb(scenario), I32_EQ, IF, NO_RESULT, ...instructions, END];
const module = new Uint8Array([
0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00,
...section(1, [
[FUNC_TYPE, ...vec([[I32]]), ...vec([])],
[FUNC_TYPE, ...vec([]), ...vec([])],
[FUNC_TYPE, ...vec([]), ...vec([[I32]])],
]),
...section(2, [
[...name("wasi_snapshot_preview1"), ...name("proc_exit"), 0x00, TAKES_I32],
[...name("crafter_runtime"), ...name("park"), 0x00, NOTHING],
[...name("env"), ...name("scenario"), 0x00, GIVES_I32],
[...name("env"), ...name("trace"), 0x00, TAKES_I32],
]),
...section(3, [[TAKES_I32], [TAKES_I32], [TAKES_I32], [TAKES_I32], [NOTHING], [GIVES_I32], [TAKES_I32]]),
...section(5, [[0x00, 0x01]]),
// One tag, for the exception nothing catches.
...section(13, [[0x00, TAKES_I32]]),
...section(6, [[I32, 0x01, I32_CONST, 0x00, END]]),
...section(7, [
[...name("memory"), 0x02, 0],
[...name("_start"), 0x00, START],
[...name("frame"), 0x00, FRAME],
[...name("quit"), 0x00, QUIT],
]),
...section(10, [
// END_WITH(code): park for PARK_CODE, else proc_exit(code).
code([], [LOCAL_GET, 0, I32_CONST, ...sleb(PARK_CODE), I32_EQ, IF, NO_RESULT, CALL, PARK, ELSE, LOCAL_GET, 0, CALL, PROC_EXIT, END]),
// NOEXCEPT(code): anything thrown through it is std::terminate.
code([], [
BLOCK, NO_RESULT, TRY_TABLE, NO_RESULT, ...vec([[CATCH_ALL, 0]]), LOCAL_GET, 0, CALL, END_WITH, END, RETURN, END,
I32_CONST, TERMINATE, CALL, TRACE, UNREACHABLE,
]),
// CATCHES_ALL(code): try { NOEXCEPT(code); } catch (...) {}
code([], [
BLOCK, NO_RESULT, TRY_TABLE, NO_RESULT, ...vec([[CATCH_ALL, 0]]), LOCAL_GET, 0, CALL, NOEXCEPT, END, RETURN, END,
I32_CONST, CATCH, CALL, TRACE,
]),
// CLEANS_UP(code): a frame with a destructor around CATCHES_ALL(code).
code([[...uleb(1), EXNREF]], [
BLOCK, EXNREF, TRY_TABLE, NO_RESULT, ...vec([[CATCH_ALL_REF, 0]]), LOCAL_GET, 0, CALL, CATCHES_ALL, END, RETURN, END,
LOCAL_SET, 1, I32_CONST, DESTRUCTOR, CALL, TRACE, LOCAL_GET, 1, THROW_REF,
]),
// _start: what env.scenario says; anything else returns from main.
code([[...uleb(1), I32]], [
CALL, SCENARIO, LOCAL_SET, 0,
...ifScenario(EXIT, [I32_CONST, 3, CALL, CLEANS_UP]),
...ifScenario(PARK_MAIN, [I32_CONST, 42, GLOBAL_SET, 0, I32_CONST, ...sleb(PARK_CODE), CALL, CLEANS_UP, I32_CONST, AFTER_PARK, CALL, TRACE]),
...ifScenario(THROW_UNCAUGHT, [I32_CONST, 9, THROW, 0]),
...ifScenario(CRASH, [UNREACHABLE]),
]),
// frame(): the state _start left, as a later frame callback reads it.
code([], [GLOBAL_GET, 0]),
// quit(code): an exit from a later call, through the same frames.
code([], [LOCAL_GET, 0, CALL, CLEANS_UP]),
]),
]);
// The standardized encoding (try_table) is what the module and runtime.js
// both depend on; an older node has only the legacy one.
const exnrefProbe = new Uint8Array([0, 97, 115, 109, 1, 0, 0, 0, 1, 4, 1, 96, 0, 0, 3, 2, 1, 0, 10, 8, 1, 6, 0, 31, 64, 0, 11, 11]);
if (!WebAssembly.validate(exnrefProbe)) {
console.log("SKIP: this node has no WebAssembly exception handling");
process.exit(0);
}
if (!WebAssembly.validate(module)) {
console.log("FAIL: the fixture module does not validate");
process.exit(1);
}
// What runtime.js expects of a page: no variants.json or files.json, so it
// loads CRAFTER_WASM_URL.
globalThis.window = globalThis;
globalThis.addEventListener = () => {};
window.CRAFTER_WASM_URL = "fixture.wasm";
globalThis.fetch = async (url) => {
if (String(url).split("?")[0] === "fixture.wasm") return new Response(module, { headers: { "Content-Type": "application/wasm" } });
return new Response(null, { status: 404 });
};
const runtime = pathToFileURL(process.argv[2]).href;
const run = async (label, scenario, after = () => "") => {
const trace = [];
window.crafter_webbuild_env = { scenario: () => scenario, trace: (what) => { trace.push(what); } };
delete window.crafter_wasi;
let outcome = "ended";
try {
// A query makes it a module of its own, run afresh.
await import(`${runtime}?scenario=${scenario}`);
} catch (e) {
outcome = `rejected with ${e?.constructor?.name}`;
}
const wasi = window.crafter_wasi;
console.log(`${label}: ${outcome}, exitCode=${wasi?.exitCode} parked=${wasi?.parked} trace=[${trace}]${after(wasi, trace)}`);
};
await run("exit", EXIT);
await run("park", PARK_MAIN, (wasi, trace) => {
const frame = wasi.instance.exports.frame();
let quit = "returned";
try {
wasi.instance.exports.quit(7);
} catch (e) {
quit = `threw ${e.constructor.name}`;
}
return `; frame()=${frame}; quit(7) ${quit}, exitCode=${wasi.exitCode} trace=[${trace}]`;
});
await run("return", 3);
await run("uncaught exception", THROW_UNCAUGHT);
await run("crash", CRASH);

View file

@ -0,0 +1,75 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// The browser runtime has to end a call into the program without running any
// more of it: when the program exits (exit, _Exit, a nonzero return from
// main), and when it parks to hand the thread to the browser for good, which
// Crafter::Window::StartSync does on DOM. runtime.js used to throw a
// JavaScript error out of proc_exit, and a program built with
// -fwasm-exceptions catches that. An exit then ran the destructors of every
// frame it left, a catch (...) swallowed it and carried on, and a noexcept
// function turned it into std::terminate, while the page went on calling
// into a program whose state those destructors had torn down.
//
// fixture/harness.mjs runs the real runtime.js under node against a module
// with the frames that catch between _start and the call that ends it: a
// destructor, a catch (...) and a noexcept function, each reporting when it
// runs. It checks that an exit runs none of them and records its status;
// that a park runs none of them either, nor the code after it, and leaves the
// program callable with its state intact; that an exit from a later call
// ends the program too; that returning from main ends it with status 0; and
// that a crash or an exception nothing caught still surfaces as an error.
// Skips without node, or with a node whose engine lacks exception handling.
// The Flatpak has no node, so this one runs in the native loop.
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
}
int main() {
if (std::system("which node > /dev/null 2>&1") != 0) {
std::println(std::cerr, "SKIP: node not on PATH");
return 77;
}
const fs::path scratch = fs::temp_directory_path() / "crafter-build-wasi-browser-runtime-exit";
fs::remove_all(scratch);
fs::create_directories(scratch);
// node runs a .js file as a CommonJS script, which cannot await at its
// top level the way runtime.js does.
const fs::path runtime = scratch / "runtime.mjs";
fs::copy_file(fs::current_path() / "wasi-runtime" / "runtime.js", runtime);
const fs::path harness = fs::current_path() / "tests" / "WasiBrowserRuntimeExit" / "fixture" / "harness.mjs";
const auto run = RunCommandWithTimeout(std::format("node {} {}", ShellQuote(harness.string()), ShellQuote(runtime.string())), std::chrono::seconds(60));
fs::remove_all(scratch);
if (run.output.contains("SKIP:")) {
std::println(std::cerr, "{}", run.output);
return 77;
}
Check(run.exitCode == 0, std::format("the harness ran to the end (exit {})", run.exitCode));
Check(run.output.contains("exit: ended, exitCode=3 parked=false trace=[]"), "an exit runs no destructor, catch (...) or terminate, and its status is recorded");
Check(run.output.contains("park: ended, exitCode=null parked=true trace=[]; frame()=42"), "a park runs none of them nor the code after it, and the program stays callable with its state");
Check(run.output.contains("quit(7) threw RuntimeError, exitCode=7 trace=[]"), "an exit from a later call ends the program the same way");
Check(run.output.contains("return: ended, exitCode=0 parked=false trace=[]"), "returning from main ends the program with status 0");
Check(run.output.contains("uncaught exception: rejected with Exception, exitCode=null"), "an exception nothing caught is not taken for an exit");
Check(run.output.contains("crash: rejected with RuntimeError, exitCode=null"), "a crash is not taken for an exit");
if (failures > 0) {
std::println(std::cerr, "{} assertions failed; harness output:\n{}", failures, run.output);
return 1;
}
return 0;
}

View file

@ -0,0 +1,6 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
int main() {
return 0;
}

View file

@ -0,0 +1,110 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// A WASI sysroot without exception support has to stop a wasm build with one
// line that says so. wasm builds compile with -fwasm-exceptions and link the
// libc++ and libunwind that wasi-sdk 33 and later ship under lib/<triple>/eh;
// a sysroot that lacks them (an older wasi-sdk, one built without its
// exceptions option) otherwise compiles and then fails the first link with
// "unable to find library -lunwind", which does not say why it was asked
// for. A sysroot that is not there at all, or has nothing for the target,
// gets a message of its own rather than that one. The checks run before
// anything is made, so a refused build leaves no bin/ behind, and they need
// no working wasm toolchain.
//
// A scratch sysroot is taken through these states: missing; without
// lib/wasm32-wasip1; without libunwind, for an executable and for a static
// library, which links nothing and passes; with libunwind under eh/; with
// libunwind in the triple's own directory, which the driver searches after
// eh/. The cases that pass the check fail later on the sysroot's empty std
// module source, which is not what is under test. The cache is redirected
// into the scratch directory as well: the std PCM cache is not keyed on the
// sysroot, and a PCM built from the fake source must not land where the real
// wasm tests read theirs.
extern "C" int setenv(const char* name, const char* value, int overwrite);
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
BuildResult BuildAgainst(const fs::path& project, const fs::path& sysroot, ConfigurationType type = ConfigurationType::Executable) {
Configuration app;
app.path = project;
app.name = "hello";
app.outputName = "hello";
app.target = "wasm32-wasip1";
app.sysroot = sysroot.string();
app.type = type;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
app.GetInterfacesAndImplementations(ifaces, impls);
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
return Build(app, depResults, depMutex);
}
}
int main() {
// Build() reports a missing compiler before it gets to the sysroot.
if (const Toolchain& tc = GetToolchain(); !tc.error.empty()) {
std::println(std::cerr, "SKIP: {}", tc.error);
return 77;
}
const fs::path scratch = fs::temp_directory_path() / "crafter-build-wasi-sysroot-without-exceptions";
fs::remove_all(scratch);
fs::create_directories(scratch);
setenv("XDG_CACHE_HOME", (scratch / "cache").c_str(), 1);
const fs::path project = scratch / "project";
fs::copy(fs::current_path() / "tests" / "WasiSysrootWithoutExceptions" / "fixture", project, fs::copy_options::recursive);
const fs::path sysroot = scratch / "sysroot";
const fs::path libDir = sysroot / "lib" / "wasm32-wasip1";
{
const BuildResult r = BuildAgainst(project, sysroot);
Check(r.result.contains(std::format("no WASI sysroot at {}", sysroot.string())), std::format("a missing sysroot is named as missing: {}", r.result));
}
fs::create_directories(sysroot / "share" / "libc++" / "v1");
std::ofstream(sysroot / "share" / "libc++" / "v1" / "std.cppm");
{
const BuildResult r = BuildAgainst(project, sysroot);
Check(r.result.contains(std::format("has nothing for wasm32-wasip1 (no {})", libDir.string())), std::format("a sysroot without the target's directory says so: {}", r.result));
}
fs::create_directories(libDir / "eh");
constexpr std::string_view missing = "has no C++ exception support";
{
const BuildResult r = BuildAgainst(project, sysroot);
Check(r.result.contains(missing), std::format("without libunwind the build stops on the sysroot: {}", r.result));
Check(r.result.contains((libDir / "eh" / "libunwind.a").string()), std::format("the message names the library it looked for: {}", r.result));
Check(!fs::exists(project / "bin") && !fs::exists(project / "build"), "a refused build leaves no bin/ or build/ behind");
}
Check(!BuildAgainst(project, sysroot, ConfigurationType::LibraryStatic).result.contains(missing), "a static library links nothing and passes without libunwind");
std::ofstream(libDir / "eh" / "libunwind.a");
Check(!BuildAgainst(project, sysroot).result.contains(missing), "libunwind under eh/ passes the check");
fs::remove(libDir / "eh" / "libunwind.a");
std::ofstream(libDir / "libunwind.a");
Check(!BuildAgainst(project, sysroot).result.contains(missing), "libunwind in the triple's directory passes the check");
fs::remove_all(scratch);
if (failures > 0) {
std::println(std::cerr, "{} assertions failed", failures);
return 1;
}
return 0;
}

View file

@ -0,0 +1,6 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
int main() {
return 0;
}

View file

@ -0,0 +1,85 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// What a wasm32 build refuses before it starts, now that every wasm unit has
// C++ exceptions, each with a message naming the cause:
// - -flto in compileFlags or linkFlags. LTO code generation drops the
// exception handling, and the build used to succeed with every throw
// escaping its catch (WebAssembly/wasi-sdk#629).
// - -fno-exceptions. The std module is built with exceptions, and every
// `import std;` failed on a BMI configuration mismatch.
// - A shared library. The exception-enabled libc++ is static archives
// only, and the link failed on a relocation that wants -fPIC.
// None of them leaves bin/ or build/ behind, and none needs a working wasm
// toolchain. The sysroot given does not exist, so a refusal that did not
// fire would show up as the missing sysroot instead.
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
struct Refusal {
std::string_view what;
std::vector<std::string> compileFlags;
std::vector<std::string> linkFlags;
ConfigurationType type;
std::string_view expected;
};
}
int main() {
// Build() reports a missing compiler before it gets to any of this.
if (const Toolchain& tc = GetToolchain(); !tc.error.empty()) {
std::println(std::cerr, "SKIP: {}", tc.error);
return 77;
}
const fs::path scratch = fs::temp_directory_path() / "crafter-build-wasm-build-refusals";
fs::remove_all(scratch);
fs::create_directories(scratch);
const fs::path project = scratch / "project";
fs::copy(fs::current_path() / "tests" / "WasmBuildRefusals" / "fixture", project, fs::copy_options::recursive);
const std::array refusals = {
Refusal{ "-flto in compileFlags", { "-flto=thin" }, {}, ConfigurationType::Executable, "-flto=thin is not supported on wasm32: LTO code generation drops C++ exception handling" },
Refusal{ "-flto in linkFlags", {}, { "-flto" }, ConfigurationType::Executable, "-flto is not supported on wasm32" },
Refusal{ "-fno-exceptions", { "-fno-exceptions" }, {}, ConfigurationType::Executable, "-fno-exceptions is not supported on wasm32" },
Refusal{ "a shared library", {}, {}, ConfigurationType::LibraryDynamic, "wasm32 shared libraries are not supported" },
};
for (const Refusal& refusal : refusals) {
Configuration app;
app.path = project;
app.name = "hello";
app.outputName = "hello";
app.target = "wasm32-wasip1";
app.sysroot = (scratch / "no-sysroot").string();
app.type = refusal.type;
app.compileFlags = refusal.compileFlags;
app.linkFlags = refusal.linkFlags;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
app.GetInterfacesAndImplementations(ifaces, impls);
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
const BuildResult r = Build(app, depResults, depMutex);
Check(r.result.contains(refusal.expected), std::format("{} is refused with its cause: {}", refusal.what, r.result));
Check(!fs::exists(project / "bin") && !fs::exists(project / "build"), std::format("refusing {} leaves no bin/ or build/ behind", refusal.what));
}
fs::remove_all(scratch);
if (failures > 0) {
std::println(std::cerr, "{} assertions failed", failures);
return 1;
}
return 0;
}

View file

@ -0,0 +1,24 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
#include <setjmp.h>
#include <stdint.h>
static jmp_buf target;
static void JumpBack(int32_t value) {
longjmp(target, value);
}
// setjmp's value may only be compared or switched on, never stored.
int32_t JumpRoundTrip(void) {
switch (setjmp(target)) {
case 0:
JumpBack(7);
return 0;
case 7:
return 7;
default:
return -1;
}
}

View file

@ -0,0 +1,46 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
// jump.c, compiled as C.
extern "C" std::int32_t JumpRoundTrip();
namespace {
std::int32_t unwound = 0;
struct Guard {
~Guard() { ++unwound; }
};
void Throw(std::int32_t depth) {
Guard guard;
if (depth == 0) throw std::runtime_error("thrown in wasm");
Throw(depth - 1);
}
}
int main(int argc, char** argv) {
// With an argument, nothing catches: the run has to fail.
if (argc > 1) Throw(0);
try {
Throw(3);
} catch (const std::runtime_error& e) {
std::println("caught: {} after {} destructors", e.what(), unwound);
}
try {
const std::vector<std::int32_t> empty;
std::println("unreachable: {}", empty.at(1));
} catch (const std::out_of_range&) {
std::println("caught: out_of_range from libc++");
}
// vector::at throws from a header, compiled into this unit; stoi throws
// inside libc++.a, which therefore has to be the exception-enabled one.
try {
std::println("unreachable: {}", std::stoi("not a number"));
} catch (const std::invalid_argument&) {
std::println("caught: invalid_argument from inside libc++");
}
std::println("longjmp came back with {}", JumpRoundTrip());
return 0;
}

View file

@ -0,0 +1,84 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// C++ exceptions in a wasm build, run under wasmtime: the fixture throws
// through frames whose destructors have to run, catches the exception,
// catches one from a libc++ header (vector::at) and one thrown inside
// libc++.a itself (stoi), and with an argument lets one escape main, which
// has to end the run as an uncaught exception rather than exit 0, trap or
// hang. Its C source uses setjmp, which compiles and links next to the C++
// objects only when C gets the same wasm flags, and links at all only
// because the build adds libsetjmp: the configuration asks for nothing.
//
// wasm builds used to be compiled with -fno-exceptions, so a throw did not
// compile and libc++'s own throws aborted. Each part of turning them on fails
// in its own way, and this is the test that sees it: a std module or a unit
// built without exceptions (clang rejects the BMI), a link without libunwind
// (undefined __cpp_exception), the libc++ built without exceptions (stoi
// aborts), or LLVM's default legacy encoding, which wasmtime refuses to load.
// Skips like WasmHelloRuns when there is no exception-enabled WASI sysroot or
// no wasmtime; inside the Flatpak both are bundled.
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
}
int main() {
const Toolchain& tc = GetToolchain();
const fs::path libDir = tc.wasiSysroot / "lib" / "wasm32-wasip1";
if (!fs::exists(libDir / "eh" / "libunwind.a") && !fs::exists(libDir / "libunwind.a")) {
std::println(std::cerr, "SKIP: no exception-enabled WASI sysroot at {}", tc.wasiSysroot.string());
return 77;
}
if (std::system("which wasmtime > /dev/null 2>&1") != 0) {
std::println(std::cerr, "SKIP: wasmtime not on PATH");
return 77;
}
Configuration cfg;
cfg.path = fs::current_path() / "tests" / "WasmExceptions" / "fixture";
cfg.name = "wasm-exceptions";
cfg.outputName = "wasm-exceptions";
cfg.target = "wasm32-wasip1";
cfg.type = ConfigurationType::Executable;
std::array<fs::path, 0> ifaces = {};
std::array<fs::path, 1> impls = { "main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
cfg.cFiles = { cfg.path / "jump" };
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
const BuildResult r = Build(cfg, depResults, depMutex);
if (!r.result.empty()) {
std::println(std::cerr, "build failed: {}", r.result);
return 1;
}
const fs::path wasm = ExpectedOutput(cfg);
const auto caught = RunCommandWithTimeout(std::format("wasmtime {}", wasm.string()), std::chrono::seconds(60));
Check(caught.exitCode == 0, std::format("the run that catches everything exits 0 (exit {}): {}", caught.exitCode, caught.output));
Check(caught.output.contains("caught: thrown in wasm after 4 destructors"), std::format("the exception is caught after unwinding all four frames: {}", caught.output));
Check(caught.output.contains("caught: out_of_range from libc++"), std::format("an exception from a libc++ header is catchable: {}", caught.output));
Check(caught.output.contains("caught: invalid_argument from inside libc++"), std::format("an exception from inside libc++.a is catchable: {}", caught.output));
Check(caught.output.contains("longjmp came back with 7"), std::format("setjmp in a C source works next to the C++ objects: {}", caught.output));
const auto escaped = RunCommandWithTimeout(std::format("wasmtime {} uncaught", wasm.string()), std::chrono::seconds(60));
Check(escaped.exitCode != 0 && !escaped.timedOut && !escaped.crashed && escaped.output.contains("thrown Wasm exception"), std::format("an exception escaping main ends the run as an uncaught exception (exit {}): {}", escaped.exitCode, escaped.output));
if (failures > 0) {
std::println(std::cerr, "{} assertions failed", failures);
return 1;
}
return 0;
}

View file

@ -11,12 +11,15 @@ using namespace Crafter;
// wasmtime. The other wasm tests only inspect generated files; this is the // wasmtime. The other wasm tests only inspect generated files; this is the
// one that proves the sysroot, the std module for wasm and wasm-ld all work // one that proves the sysroot, the std module for wasm and wasm-ld all work
// together. It skips when the sysroot or wasmtime is missing, so a // together. It skips when the sysroot or wasmtime is missing, so a
// development box without them is not a failure. Inside the Flatpak both are // development box without them is not a failure; a sysroot without the
// bundled, and there the skip would be the failure to look for. // exception-enabled libraries counts as missing, since wasm builds link them
// (WasiSysrootWithoutExceptions covers that message). Inside the Flatpak both
// are bundled, and there the skip would be the failure to look for.
int main() { int main() {
const Toolchain& tc = GetToolchain(); const Toolchain& tc = GetToolchain();
if (!fs::exists(tc.wasiSysroot / "include")) { const fs::path libDir = tc.wasiSysroot / "lib" / "wasm32-wasip1";
std::println(std::cerr, "SKIP: no WASI sysroot at {}", tc.wasiSysroot.string()); if (!fs::exists(libDir / "eh" / "libunwind.a") && !fs::exists(libDir / "libunwind.a")) {
std::println(std::cerr, "SKIP: no exception-enabled WASI sysroot at {}", tc.wasiSysroot.string());
return 77; return 77;
} }
if (std::system("which wasmtime > /dev/null 2>&1") != 0) { if (std::system("which wasmtime > /dev/null 2>&1") != 0) {

View file

@ -97,6 +97,11 @@ int main() {
Check(j.find("\"label\":\"relaxed-simd\"") != std::string::npos, "manifest has relaxed-simd label"); Check(j.find("\"label\":\"relaxed-simd\"") != std::string::npos, "manifest has relaxed-simd label");
Check(j.find("\"wvhelper.relaxed-simd.wasm\"") != std::string::npos, "manifest has labelled url"); Check(j.find("\"wvhelper.relaxed-simd.wasm\"") != std::string::npos, "manifest has labelled url");
Check(j.find("\"relaxed-simd\"") != std::string::npos, "manifest has relaxed-simd probe"); Check(j.find("\"relaxed-simd\"") != std::string::npos, "manifest has relaxed-simd probe");
// Every file is built with SIMD128 and exceptions, so every entry
// needs them besides its own features; the baseline needs
// nothing else.
Check(j.find("\"probes\":[\"relaxed-simd\",\"simd\",\"exception-handling\"]") != std::string::npos, "relaxed-simd entry also probes for the baseline's features");
Check(j.find("\"url\":\"wvhelper.wasm\",\"probes\":[\"simd\",\"exception-handling\"]") != std::string::npos, "baseline entry probes for its own features only");
// baseline fallback entry. // baseline fallback entry.
Check(j.find("\"wvhelper.wasm\"") != std::string::npos, "manifest has baseline url"); Check(j.find("\"wvhelper.wasm\"") != std::string::npos, "manifest has baseline url");
// The relaxed-simd variant must precede the baseline so the runtime // The relaxed-simd variant must precede the baseline so the runtime

View file

@ -176,11 +176,53 @@ function markDirty(path) {
scheduleFlush(); scheduleFlush();
} }
// ─── leaving the program ───────────────────────────────────────────────
// exit and _Exit (proc_exit) end the program. park hands the thread to the
// browser for good while the program goes on serving the calls bridges make
// into it (Crafter::Window::StartSync on DOM): the import
// crafter_runtime.park, `void park(void)`, which never returns. Either way
// the current call into the program has to end without running any more of
// its code: not the destructors of the frames it leaves, not a catch (...),
// not the std::terminate of a noexcept function. Built with
// -fwasm-exceptions, every one of those catches a JavaScript exception
// thrown through it, so the way out cannot be a throw from JS. A trap is
// the one thing wasm code cannot catch, and this module raises it in wasm
// itself, bound directly as the program's import; a trap raised from JS
// would depend on each engine keeping it uncatchable when it passes back
// into wasm. It reports first, through host.ended, which has returned by
// the time it traps:
// (import "host" "ended" (func $ended (param $how i32) (param $code i32)))
// (func (export "exit") (param $code i32)
// i32.const 1 local.get $code call $ended unreachable)
// (func (export "park")
// i32.const 2 i32.const 0 call $ended unreachable)
// The program's own state is left as it is, its shadow stack included, so
// the locals of a parked main stay valid for the calls that follow.
const PROGRAM_END = new Uint8Array([
0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00,
0x01, 0x0d, 0x03, 0x60, 0x02, 0x7f, 0x7f, 0x00, 0x60, 0x01, 0x7f, 0x00, 0x60, 0x00, 0x00,
0x02, 0x0e, 0x01, 0x04, 0x68, 0x6f, 0x73, 0x74, 0x05, 0x65, 0x6e, 0x64, 0x65, 0x64, 0x00, 0x00,
0x03, 0x03, 0x02, 0x01, 0x02,
0x07, 0x0f, 0x02, 0x04, 0x65, 0x78, 0x69, 0x74, 0x00, 0x01, 0x04, 0x70, 0x61, 0x72, 0x6b, 0x00, 0x02,
0x0a, 0x15, 0x02,
0x09, 0x00, 0x41, 0x01, 0x20, 0x00, 0x10, 0x00, 0x00, 0x0b,
0x09, 0x00, 0x41, 0x02, 0x41, 0x00, 0x10, 0x00, 0x00, 0x0b,
]);
const ENDED_BY_EXIT = 1;
const ENDED_BY_PARK = 2;
class Wasi { class Wasi {
#encodedStdin; #encodedStdin;
#envEncodedStrings; #envEncodedStrings;
#argEncodedStrings; #argEncodedStrings;
instance; instance;
// The program's exit status once it has ended, through exit or _Exit or
// by returning from main; null while it runs or is parked. A bridge that
// calls into the program on its own, such as a frame loop, stops once
// this is set: nothing of an ended program may run.
exitCode = null;
// True once the program has parked (see PROGRAM_END).
parked = false;
#fdTable; #fdTable;
#nextFd; #nextFd;
#decoder; #decoder;
@ -194,6 +236,27 @@ class Wasi {
this.#nextFd = 100; // stay clear of stdio + preopens this.#nextFd = 100; // stay clear of stdio + preopens
this.#decoder = new TextDecoder(); this.#decoder = new TextDecoder();
this.bind(); this.bind();
const end = new WebAssembly.Instance(new WebAssembly.Module(PROGRAM_END), {
host: { ended: (how, code) => this.ended(how, code) },
}).exports;
// Own properties, so instantiation takes these wasm functions as the
// program's imports rather than a JS method.
this.proc_exit = end.exit;
this.park = end.park;
}
// Called by PROGRAM_END just before its trap ends the call into the
// program. An exit after a park ends the program all the same.
ended(how, code) {
if (how === ENDED_BY_PARK) {
this.parked = true;
} else if (how === ENDED_BY_EXIT) {
console.log(`[wasi] proc_exit(${code})`);
this.exitCode = code;
// The last chance for anything written this session to reach
// OPFS before the page is torn down.
opfsFlushAll();
}
} }
bind() { bind() {
@ -805,22 +868,6 @@ class Wasi {
sizePointerBuffer[0] = envByteLength; sizePointerBuffer[0] = envByteLength;
return 0; return 0;
} }
proc_exit(code) {
// Throw a sentinel so the wasm stack unwinds back to runtime.js
// WITHOUT executing the `unreachable` instruction wasi-libc emits
// after __wasi_proc_exit (which is declared noreturn). Trapping
// there would mark the wasm call as crashed and may interrupt any
// browser-side render loop that called _Exit on purpose (e.g.
// Crafter::Window::StartSync on DOM, which hands the loop to rAF
// and then exits main without running static destructors).
console.log(`[wasi] proc_exit(${code})`);
// Trigger a final flush so anything written this session has the
// best chance of reaching OPFS before the page is torn down.
opfsFlushAll();
const e = new Error(`wasi proc_exit(${code})`);
e.crafterWasiExit = code;
throw e;
}
} }
// ─── wasm feature-detect probes ──────────────────────────────────────── // ─── wasm feature-detect probes ────────────────────────────────────────
@ -829,21 +876,28 @@ class Wasi {
// GoogleChromeLabs/wasm-feature-detect, inlined so we ship no dependency. A // GoogleChromeLabs/wasm-feature-detect, inlined so we ship no dependency. A
// variant in variants.json lists the probe names that must all pass for that // variant in variants.json lists the probe names that must all pass for that
// variant to be served; an unknown name is treated as unsupported (we fall // variant to be served; an unknown name is treated as unsupported (we fall
// through to a less demanding variant, ultimately the baseline). Keep the keys // through to a less demanding variant, ultimately the baseline, and to an
// in sync with the `probes` a WasmVariant declares in Crafter.Build. // error naming what is missing when not even the baseline fits). Keep the
// keys in sync with the `probes` a WasmVariant declares in Crafter.Build, and
// with WasmBaselineProbes, which every entry carries.
const WASM_PROBES = { const WASM_PROBES = {
// relaxed SIMD: i8x16.relaxed_swizzle (Chrome 114+, Firefox 120+; Safari // relaxed SIMD: i8x16.relaxed_swizzle (Chrome 114+, Firefox 120+; Safari
// still flag-gated as of mid-2026). // still flag-gated as of mid-2026).
"relaxed-simd": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,5,1,96,0,1,123,3,2,1,0,10,15,1,13,0,65,1,253,15,65,2,253,15,253,128,2,11])), "relaxed-simd": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,5,1,96,0,1,123,3,2,1,0,10,15,1,13,0,65,1,253,15,65,2,253,15,253,128,2,11])),
// fixed-width SIMD128 (v128.const + i8x16.popcnt) — the current baseline, // fixed-width SIMD128 (v128.const + i8x16.popcnt), which every build
// here for completeness / future baseline shifts. // uses (-msimd128). V8 and SpiderMonkey switch it off on x86 CPUs without
// SSE4.1, so it is a baseline probe even though every engine with
// exception handling has it.
"simd": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,5,1,96,0,1,123,3,2,1,0,10,10,1,8,0,65,0,253,15,253,98,11])), "simd": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,5,1,96,0,1,123,3,2,1,0,10,10,1,8,0,65,0,253,15,253,98,11])),
// tail calls (return_call). // tail calls (return_call).
"tail-call": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,10,6,1,4,0,18,0,11])), "tail-call": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,10,6,1,4,0,18,0,11])),
// bulk memory ops (memory.init). // bulk memory ops (memory.init).
"bulk-memory": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,5,3,1,0,1,10,14,1,12,0,65,0,65,0,65,0,252,10,0,0,11])), "bulk-memory": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,5,3,1,0,1,10,14,1,12,0,65,0,65,0,65,0,252,10,0,0,11])),
// legacy exception handling (try/catch). // exception handling in its standardized form (try_table, exnref; Chrome
"exception-handling": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,10,8,1,6,0,6,64,25,11,11])), // 137, Firefox 131, Safari 18.4), which every crafter-build wasm uses for
// C++ exceptions. Probing the legacy try/catch encoding instead says
// nothing about this one: V8 accepts it with exnref switched off.
"exception-handling": () => WebAssembly.validate(new Uint8Array([0,97,115,109,1,0,0,0,1,4,1,96,0,0,3,2,1,0,10,8,1,6,0,31,64,0,11,11])),
// threads/atomics — also requires SharedArrayBuffer (cross-origin isolated). // threads/atomics — also requires SharedArrayBuffer (cross-origin isolated).
"threads": () => { "threads": () => {
try { try {
@ -866,6 +920,9 @@ function variantSupported(probes) {
// Pick the wasm URL: prefer the variants.json manifest (first entry whose // Pick the wasm URL: prefer the variants.json manifest (first entry whose
// probes all pass), else fall back to the single baked window.CRAFTER_WASM_URL. // probes all pass), else fall back to the single baked window.CRAFTER_WASM_URL.
// When the manifest has no entry this engine can run, not even the baseline,
// say which features it lacks: loading the baseline anyway would only fail
// with a CompileError about an unknown opcode.
async function pickWasmUrl() { async function pickWasmUrl() {
let variants = null; let variants = null;
try { try {
@ -881,6 +938,9 @@ async function pickWasmUrl() {
return v.url; return v.url;
} }
} }
const baseline = variants[variants.length - 1];
const missing = (baseline.probes || []).filter((name) => !variantSupported([name]));
throw new Error(`[wasi] ${baseline.url} needs WebAssembly ${missing.join(", ")}, which this browser does not support; a current Chrome, Firefox or Safari does`);
} }
return window.CRAFTER_WASM_URL; return window.CRAFTER_WASM_URL;
} }
@ -970,6 +1030,7 @@ if (window.crafter_webbuild_env_ready) {
const { instance } = await WebAssembly.instantiateStreaming(fetch(wasmUrl), { const { instance } = await WebAssembly.instantiateStreaming(fetch(wasmUrl), {
wasi_snapshot_preview1: wasi, wasi_snapshot_preview1: wasi,
env: window.crafter_webbuild_env, env: window.crafter_webbuild_env,
crafter_runtime: { park: wasi.park },
}); });
wasi.instance = instance; wasi.instance = instance;
// Alias under both names — `crafter_wasi` is the canonical, but the older // Alias under both names — `crafter_wasi` is the canonical, but the older
@ -980,12 +1041,12 @@ window.crafter_webbuild_wasi = wasi;
try { try {
instance.exports._start(); instance.exports._start();
// main returned 0; any other status reaches proc_exit.
wasi.exitCode = 0;
opfsFlushAll();
} catch (e) { } catch (e) {
if (e && typeof e.crafterWasiExit === "number") { // proc_exit and park end _start with PROGRAM_END's trap after saying
// Clean exit from std::_Exit / __wasi_proc_exit. The wasm instance // which they were. Any other error, a crash or a C++ exception that
// remains valid; any rAF callback registered before exit keeps // nothing caught, is the program failing.
// running. (See Crafter::Window::StartSync DOM impl.) if (wasi.exitCode === null && !wasi.parked) throw e;
} else {
throw e;
}
} }