- Shaders record the files glslang opened (a depfile) and their settings
(source path, stage, entry point, defines, include directories) in the
build directory, and rebuild when an included file or a setting changes,
not only the shader's own source.
- An executable relinks when its link command changed, not only when an
object is newer: a link flag in project.cpp, or a dependency's library
that moved (an ExternalDependency built into a new cache directory). The
command is recorded beside the objects; its flags come from ordered sets,
so the order the dependency threads finish in can't change it.
- An external's newest artifact counts its libDirs and versioned .so names.
- Submodules are checked out at the revisions the fetched commit records,
from the URLs its .gitmodules gives, after every clone, switch, checkout
and pull, so a pinned commit builds against its own submodules, and one
moved by hand is put back; with no .gitmodules, git isn't asked.
- Each external clone's fetch and build hold a file lock: parallel builds
pulled into one clone at once ("cannot lock ref").
- mingw:
- -stdlib=libstdc++ always;
- the std module's source is found through libstdc++.modules.json in the
mingw toolchain's own library directories (the generic lib/ holds the
host's Linux one), asking the compiler once a run;
- a toolchain that isn't there is a build error saying where it looked,
not std::terminate;
- CRAFTER_MINGW_DIR names the prefix, the directory holding
x86_64-w64-mingw32/ and lib/gcc/;
- inside the Flatpak the host's /usr mingw is used.
- RunScript's exit-status check reads as the one range it is.
- Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules,
LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and
ToolchainResolve extended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RunScript caught std::exception only. Anything else a script threw went
on across the project.so boundary, and crafter-build ended in
std::terminate instead of reporting the script's failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local Flatpak build was one long flatpak-builder line in AGENTS.md,
with its state wherever the caller happened to stand, and installing
the result was not written down anywhere. It is not a rare chore: the
development box compiles with the distro's clang and the Flatpak with
its bundled LLVM, a major version apart at the moment, so anything
toolchain-sensitive has to be tried in the Flatpak too.
Both are now scripts in project.cpp, the first users of cfg.scripts.
`run flatpak` builds the manifest as it is, from the working tree, and
ends with `crafter-build --version` from inside the result. Its state
is build/flatpak/ (state/, repo/, build-dir/). That is a fixed name and
not a BuildDir() on purpose: the Flatpak is compiled by the toolchain in
the sandbox, so it does not depend on the host variant, and a
variant-keyed directory would strand the LLVM cache on every host
compiler update. The manifest already skips build/ when it copies the
sources. flatpak-builder keys its cache on the manifest's path, so the
manifest is never rewritten into a copy the way CI stamps the version
in, and a local build reports 0.0.0-dev. The only argument accepted is
--jobs=<N>: anything that ends the run without a build would leave the
smoke test checking the previous one.
`run install` installs the last export as branch `local` from a user
remote of its own and makes it current, leaving the published branch
installed. It does not build first: flatpak-builder never caches a
`type: dir` module, so every build reruns build.sh and the suite.
remote-add --if-not-exists is followed by remote-modify --url, because
the remote may exist and point at another checkout.
Both refuse to run from the Flatpak build of crafter-build, which has
no flatpak to drive.
The smoke test goes through `flatpak build` and not `<builder> --run`,
which is what CI uses. flatpak-builder 1.4.9 run as the
org.flatpak.Builder app exits 1 when the command it ran succeeded and 0
when it failed (`true` gives 1, `exit 3` gives 0), so the script
reported a good build as a failure.
Measured on the 56-core box with LLVM and the WASI sysroot cached: 260
seconds, the crafter-build module only, 29 tests passing in the sandbox.
A second `run install` after a rebuild moved the installed commit.
`clean` now also drops that state, about 2 GB and a cold LLVM build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
RunScript on hand-built configurations: the named script alone runs,
with the project, the project directory and the arguments exactly as
given; its return value comes back whole from 0 to 255 and as 1 outside
that; a script that throws exits 1. A project without scripts, a
duplicate, an untypeable name, a missing callback and an unknown name
all fail without running anything. ctx.Run returns the command's exit
code and 128+signal for a killed one, and ShellQuote carries quotes,
spaces and expansions through sh intact.
Then the same through Run() with a fixture project, because the parsing
is where this goes wrong. `run record a --help -v clean --debug
--version run` must hand all seven words to the script, leave the one
project arg that stood before `run` alone, and not delete the staged
bin/ and build/. `run clean` and `run help` look for scripts of those
names. Moving the two capture branches below the verbs in the parse
loop fails this test.
The command-line half needs the installed module sources and is skipped
without them, but only when the in-process half reported no failure.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project.cpp could describe how a project is built, tested and linted,
but not the chores around it: packaging, installing, publishing. Those
stayed shell lines in a README or in someone's history.
Configuration gains `scripts`, a plain list like `defines`. A Script is
a name and a C++ callback, as a lint rule is; there is no script
language and no config file. The callback gets a ScriptContext (the
project, the project file's directory, the arguments) and returns the
exit status. ctx.Run runs a shell command with the terminal attached
and returns its status. It is only stored when project.cpp loads, which
happens for every verb and for every project that depends on this one.
`crafter-build run <name> [args...]` runs one, a bare `run` lists them.
Only the scripts of the project crafter-build was pointed at are
runnable, never a dependency's.
Everything after the script's name is handed to the script untouched.
The two branches that do this sit ahead of every other branch of the
parse loop, --version and --help included. Below the verbs,
`run package clean` would delete build/, and any other word would land
in the project's args and so in the variant hash, moving the build to
another directory. Options before the name keep their meaning, so
project args go there.
The driver is its own partition, :Script. It refuses a project with no
scripts, a name that is empty or starts with '-' (it could never be
typed), a name declared twice (running either would be a guess) and an
unknown name, each with exit 1 and without running anything. A script
that throws exits 1. A return value outside 0..255 becomes 1: an exit
status is eight bits, and a raw wait status such as 256 would otherwise
wrap to success.
Two helpers in :Platform:
- RunCommandAttached runs a command with inherited stdio. Every helper
so far captured through popen, which is right for a compile step and
useless for a command that runs for half an hour or asks a question.
It reads the wait status through DecodeWaitStatus, which
RunCommandChecked and RunCommandWithTimeout now share.
- ShellQuote is exported. It existed twice, as ShellQuote in :External
and ShellQuoteSh in :Test, with the same behaviour; both are gone. A
script that puts a path into a command needs it too.
Not included: a string shorthand for one-command scripts (a one-line
lambda covers it) and a ctx.Build() helper (Build() is already
exported).
A new partition and new exported symbols: run ./build.sh.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Drives Run() with `format clean` on a staged project whose project file
cannot compile. Both trees must survive, and the run must fail: getting
as far as loading the project is what shows it went on to format rather
than cleaning and exiting 0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every verb is a bare word that is only read as a verb while no
conflicting verb is active; after one, the same word is a glob. The
guard on `clean` checked the test and lint verbs and then itself, where
the format verb was meant. `crafter-build format clean`, which asks for
the rules matching "clean", therefore removed both trees and exited 0
without formatting anything.
The guard now names format, like the three beside it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CI takes the newest <release> as the notes of a tag and refuses a tag
that does not match it. 2.2.0 covers what changed since 2.1.0: Shader
entries with per-entry defines and a named output variant, sysroot
compiles against the sysroot's own libc++ headers, a project.so stamp
that hashes every part of the host build, and the vendored json.hpp
removed.
The variant API is what catcrafts.net's deploy needs: Crafter.Graphics
master has used it since 2026-10-06, and a stable Flatpak that lacks it
cannot build the site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GetCompileCommand is checked as text, without a compiler: the forced
include directory is present for linux-gnu and linux-musl targets with
a sysroot, and absent without a sysroot, for wasm32 and for mingw.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
clang's driver ranks the host toolchain's own libc++ headers
(<install>/../include/c++/v1) above --sysroot. BuildStdPcm has forced
the sysroot's copy for the std module for that reason, but ordinary
compiles did not. A unit that only does `import std;` never noticed. A
unit that includes a standard header textually got the host's libc++,
configured for the host's libc: cross-building Crafter.Graphics for
aarch64 musl, Crafter.Asset's vendored gdeflate died on <chrono> with
"unknown rune table for this platform". Against a glibc sysroot the
same unit would compile against the wrong libc++ release and say
nothing.
Every non-wasm, non-mingw compile with a sysroot now gets
`-nostdinc++ -isystem <sysroot>/usr/include/c++/v1`. wasm has its own
sysroot handling and mingw is a libstdc++ target.
-nostdinc++ leaves -stdlib=libc++ unused on each compile, which clang
reports once per unit, so the command also carries
-Wno-unused-command-line-argument. The cost is that the driver's
unused-argument diagnostics are silent on sysroot builds, including for
a user's own misplaced flag (a -l or -Wl, option in compileFlags).
Not addressed: external cmake dependencies are still configured
without the target or the sysroot, and objects already built in an
existing sysroot build directory are not invalidated by the new flags.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ShaderCompile registers one source twice and checks that the variant
gets its own output name, takes the other #ifdef branch and receives
the value of a NAME=VALUE define. Staleness is pinned per output: while
only the plain output exists the variant must still be due, or Build()
would never compile a variant added to a built project. Malformed
defines and a variant with a path separator must be refused by name.
ShaderVariantBuild drives the same through Build(), which is where the
entries compile in parallel into one directory: a variant added to an
already built configuration is compiled into its own file, and two
entries with one output name fail the build without touching the
existing output. Its fixture tests the macro with `#if`, which pins
that a bare define is 1 and not empty.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Crafter.Graphics has to ship each of its compute shaders in two builds
of one source: one that indexes VK_EXT_descriptor_heap and one that
reads the same slots from a descriptor set, for drivers without the
extension. A Shader was a path, an entry point and a stage, and its
output name was derived from the path alone, so one source could only
be compiled once.
Shader gains `defines`, spelled like the compiler's -D ("NAME" is
defined to 1, "NAME=VALUE" to VALUE) and handed to glslang as a
preamble, and `variant`, which names the output: "ui.comp.glsl" with
variant "sets" becomes "ui.comp.sets.spv". OutputName() is the one
place that name is formed; Check, Compile and the forwarding of a
dependency's shaders into the consumer's bin dir all use it.
Two fields make one mistake easy: defines set, variant forgotten. Both
entries then compiled in parallel into the same file, the build
reported success with whichever finished last, and Check() called both
up to date from then on. The same has always been true of a.vert next
to a.frag. Build() now refuses a configuration in which two entries
share an output name and says which.
A define without a name, or one spanning lines, and a variant with a
path separator are rejected by name. Left to glslang the first two
surface as "-1:<line>" with no file, and the last as a failed open
after the whole compile.
Not addressed: Check() still compares only the source's mtime, so a
change to the defines, or to an included file, does not recompile
(#35).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
HostCacheSourceStamp already edits a module source with a backdated
mtime and expects the PCM cache to be rebuilt. Its own header names the
real danger, a project.so compiled against declarations that no longer
match the process it is loaded into, but it never looked at the
project.so. It now records the file's mtime before the edit and expects
a newer one after it, with project.cpp untouched, so only the stamp can
force the rebuild.
Fails without the previous commit: the stamp was the version string
alone and the project.so was reused.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The stamp that decides whether a compiled project file can be reused
was meant to cover the crafter-build version, the toolchain key, the
module-source stamp and the compile command. Only the version went in.
The four parts were joined by a format string with "\0" between the
fields, and a format string is read as a C string: it ended at the
first NUL, so std::format saw "{}" and dropped the other arguments
without a diagnostic.
A project.so is loaded into the crafter-build process and fills in
Configuration objects by their layout. With the stamp blind to the
declarations, one built against other module sources was loaded as
current whenever the version string was unchanged, which is every
working-tree build (`git describe --dirty` does not follow the tree's
contents) and every local Flatpak build. Growing a class that project
files construct then corrupts the heap: a project.so built while Shader
was 56 bytes, loaded after it had become 104, aborted with "free():
invalid size" as soon as it declared a shader. CI builds were spared
only because each carries its own version.
The parts are now appended to a std::string with NUL separators, the
way the other stamps in this file are built. Every existing project.so
carries a stamp of the version alone, so each is recompiled once.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CI takes the newest <release> as the notes of a tag and refuses a tag
that does not match it. 2.1.0 covers what changed since 2.0.0: the
bundled LLVM 23.1.2, build outputs kept per compiler, and the std
module source chosen by the sysroot's libc++ release.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README still gave the output path as bin/<name>-<target>-<march>/,
which has not been its shape since the variant hash was introduced. It
now shows the real one and says what the hash stands for, including
that a compiler update starts a new directory and `clean` removes the
old. AGENTS.md lists the toolchain among VariantId()'s inputs and notes
that a glob over variant directories matches two after an update.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two toolchains that differ only in the clang release, as the Flatpak's
do across an update, must give one Configuration two variants; so must
the same release installed under another prefix. The no-argument form
is checked to be the variant under the running toolchain, and BuildDir
to be named for it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A project built with one clang kept its outputs when crafter-build
started using another. The output directory is named by VariantId(),
which hashed every input that changes what is compiled except the
compiler itself, and staleness is decided by mtimes, which a compiler
update leaves alone. A PCM can only be read by the clang release that
wrote it, so after the bundled LLVM moved from 22 to 23 the first build
found nothing stale, and the first edit failed with "module file
Greeter.pcm uses an older format that is no longer supported" until the
project was cleaned by hand. A project with a `#pragma crafter
dependency` did not get that far: its project file imports the
dependency's PCM, so it failed to load, and for a git dependency that
PCM lives in the cache, where `crafter-build clean` does not reach. The
same happens natively when the distro updates clang.
The toolchain's key now goes into the hash, as the host PCM cache was
already keyed on it. An update starts an empty variant directory and
rebuilds into it; the previous one stays until `clean`, as after any
other change of variant, and two crafter-build versions can work in one
checkout without disturbing each other. VariantId() gains an overload
taking the toolchain so the dependence can be tested in-process, the
way ResolveToolchain and SysrootStdModule take theirs.
External cmake builds are not covered: their directory is fixed per
clone and keyed on the compiler's path, not its release. Objects of two
clang releases link together on the host, so nothing fails there, but
they are not rebuilt either.
Verified by replaying the update: three projects built with the 2.0.0
Flatpak, then built, edited and run with this one without cleaning,
then built with 2.0.0 again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The previous commit made the LLVM tree inside the wasi-sdk checkout a
git repository with `git init`, so that wasi-sdk's patch step would
apply. That was half the fix: wasi-sdk also runs `git rev-parse HEAD`
in that tree to write the sysroot's VERSION file, which on an empty
repository raises a traceback during configure and leaves VERSION
without its llvm lines. The build went on regardless, so the suite
passed and the published beta carries the truncated file.
Fetching the tree as a git source pinned to the release tag gives
wasi-sdk what it has upstream, a submodule checkout at a commit, and
removes the shell step instead of stacking a second workaround on it.
The llvm module keeps the release tarball: it is the same commit, and a
tarball is the cheaper download for a tree that is only compiled.
flatpak-builder keeps a mirror of the clone in its state directory, so
the fetch costs the cold run about 300 MB once.
Verified locally: no traceback, VERSION lists llvm 85ac56026243 and
llvm-version 23.1.2, the eight patch steps apply, the suite passes in
the sandbox and a wasm32-wasip1 program runs under the bundled wasmtime.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README said a cross build precompiles the sysroot's own std.cppm,
which was never the whole story for sysroots that do not ship one. It
now states the order (the sysroot's, the toolchain's for the same major
release, the sources crafter-build carries for other releases), what
the failure looks like when nothing matches and how to resolve it, and
lists the new directory in the install layout.
AGENTS.md records the rule for the next LLVM bump: the release the
bundled toolchain leaves behind stays available as a module-source
module, because stable distros remain on it long after.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With LLVM 23 bundled, a sysroot still on libc++ 22 without its own
module source (Alpine v3.24) is the mirror image of the case that
started this: the toolchain's source is a release ahead of the
sysroot's headers. The new libcxx-22-modules module installs libc++
22.1.8's module source under share/crafter-build/libc++-modules/22,
where SysrootStdModule looks for the release a sysroot is on.
It only configures libc++'s runtimes build and installs the cxx-modules
component: std.cppm as configure generates it and the .inc files from
the source tree, nothing compiled, about 600 KB installed. The
component is staged and copied because it also writes
lib/libc++.modules.json, which belongs to the bundled libc++.
Verified in the built Flatpak: `crafter-build --version` lists release
22, and a cross build against an Alpine v3.24 aarch64 sysroot (libc++
22.1.3) precompiles that source with clang 23, links and runs under
qemu.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LLVM 23 has been the stable series since September, and the distros
crafter-build cross-builds for have moved to it: Alpine edge ships
libc++ 23.1.2, which is what surfaced the std module mismatch fixed in
the previous commits. wasi-sdk 34 already builds against a newer LLVM
than 22 upstream, so the WASI sysroot loses nothing by following.
The bump exposed a patch step that never ran. wasi-sdk patches its LLVM
tree with `git apply` in src/llvm-project, which upstream is a
submodule. Here that directory is a tarball unpacked inside the
wasi-sdk checkout, and git, finding the outer repository, treats every
patched path as outside the current directory, prints "Skipped patch"
and exits 0. LLVM 22 built without the three patches; libc++ 23 does
not compile for WASI without the one that keeps the z/OS locale shim
out of other platforms' builds (strtod_l redeclared with an abi_tag).
`git init` on the unpacked tree makes it a repository of its own, so
wasi-sdk's step applies whatever patch list the pinned tag carries, as
it does upstream, and fails loudly when a patch no longer fits.
Built locally: the whole manifest, the suite inside the sandbox (26
passed, WasmHelloRuns under the bundled wasmtime included), and aarch64
musl cross builds against Alpine edge and v3.24 sysroots that run under
qemu. The native development loop is unaffected and stays on whatever
clang the machine has.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SysrootStdModule runs over staged layouts that need no compiler: a
sysroot with its own module source, one on a point release of the
toolchain's libc++, one on the next major release with a shipped
source, one nothing was shipped for, one without libc++ at all, and the
two cases where a release cannot be read. The last block drives Build()
against a sysroot claiming libc++ 99 and expects the one-line
explanation back instead of compiler output, which is the failure the
fix replaces.
ToolchainResolve additionally checks that the release is read from the
libc++ the module source comes from (the prefix's, an explicit libc++
install's, the system's) and where the shipped sources are looked up.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A cross build against a sysroot whose libc++ differs from the
toolchain's failed while precompiling the std module, with a page of
"no member named 'runtime_format' in namespace 'std'". The module source
is a list of using-declarations for what one libc++ release declares, so
it only compiles against that release's headers. BuildStdPcm wants the
sysroot's own <sysroot>/usr/share/libc++/v1/std.cppm, but Alpine
packages the headers without it, and the fallback was the toolchain's
copy whatever its release. That held while the compiler came from the
same distro as the sysroot; with the Flatpak's libc++ 22.1.8 and Alpine
edge on 23.1.2 it no longer does.
The release is now read from _LIBCPP_VERSION in the sysroot's and the
toolchain's <__config>, and SysrootStdModule picks in this order: the
sysroot's own source, the toolchain's when both are the same major
release (a release branch only takes fixes), the copy crafter-build
ships for that release under share/crafter-build/libc++-modules/<major>.
When none fits the build stops with one line naming both releases and
the directory the matching source belongs in, before clang is run. If
either release cannot be read the toolchain's source is tried as
before, since there is nothing to compare.
Matching sources live in crafter-build's own share directory rather
than the LLVM prefix: they are data this tool ships for sysroots, not
part of any one compiler install, and "sources ship" already describes
that directory. `--version` prints the toolchain's libc++ release and
the releases sources are shipped for, the first thing to look at when
such a build fails.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
flatpak-builder printed "Building module llvm" on every CI run instead
of "Cache hit". The build environment, CRAFTER_BUILD_VERSION included,
sat in the manifest's top-level build options, and those are part of
every module's cache key; the version changes with each commit, so no
module ever hit. Runs still took twelve minutes rather than eighty only
because ccache made the LLVM rebuild cheap.
The variables only matter to crafter-build's own module, so they move
there, and CI rewrites the version on that module. LLVM, the WASI
sysroot and wasmtime are now keyed on nothing that changes between
commits.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every non-tag build called itself 0.0.0-dev+<hash>, a placeholder that
said nothing about which release it came after and put a `+` in the
bundle's download URL. A master build now uses `git describe`, as a
local checkout already does: 2.0.0-3-g21746b1 is three commits past
2.0.0. The placeholder remains only for history that has no tag yet.
Three more things the tag path needed before its first run. The
forgejo-release action takes the notes text in `release-notes` and a
path in `release-notes-file`; the workflow passed a path to the former,
so the release body would have been the file name. The tag filter is
spelled with globs so it reads the same as a workflow pattern or a plain
glob. Python, used to extract the notes from the metainfo, was only in
the container as somebody else's dependency and is now listed. The
2.0.0 entry carries the date it is tagged on.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The publish step created the OSTree repo only when the directory was
missing. On the runner host the web root exists before the first
publish, as an empty directory, so the init was skipped and
build-commit-from failed with `opendir(objects)`. The check now looks
for the repo's config file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The privileged runner that already existed is registered to another
repository, so its labels cannot serve this one. CI runs on a separate
privileged runner registered to Crafter.Build only, with four volumes
rather than three now that the signing key is mounted from the host.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The runner is our own machine, so the repo signing key does not need to
pass through the forge as a secret. It is generated once under
/etc/crafter-flatpak on the host and bind-mounted read-only into the job
container, where each run imports it into a throwaway keyring. One less
secret to manage, and the private key never leaves the box that uses it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README leads with the Flatpak install, keeps the native loop as the
way to build from source, and drops the tarball, musl and makepkg text
that no longer describes anything CI produces. The install layout shows
the /app tree and the per-app cache directory.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One job replaces the tarball pipeline. On a pull request it builds the
Flatpak with flatpak-builder, which runs the bootstrap, the self-host
build and the test suite inside the sandbox, and smoke-tests the result
with `crafter-build --version` and the hello-world example. On a push to
master it also bundles the build, promotes it into the live repo as the
`beta` branch and refreshes the rolling `latest` prerelease. On a tag
X.Y.Z it promotes as `stable`, refuses to proceed unless the metainfo's
newest release is that version, and publishes a release whose notes are
that entry's description.
The job runs on the `flatpak` runner label: the privileged forgejo-runner
on the Hetzner box, with the Sdk install, the flatpak-builder state and
the repo's web root as volumes. The state volume is what keeps a normal
run short: flatpak-builder rebuilds the LLVM and WASI modules only when
their sources change. The live repo is canonical and never built into;
builds are promoted with build-commit-from, then build-update-repo signs
and prunes it and generates static deltas.
The musl job, the per-march tarballs, the toolchain packages the runner
used to install and the PKGBUILD go: the Flatpak is the one artifact.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
wasi-sdk's sysroot build hands its compiler-rt sub-build
CMAKE_C_COMPILER_TARGET but nothing for the ASM language, so the two
wasm/*.S stubs in the builtins (the setjmp/longjmp and C++ exception
tags) were assembled for x86-64. wasm-ld warned on every wasm link that
the archive members were neither Wasm objects nor bitcode, and anything
that reaches those tags, which crafter-build's -wasm-enable-sjlj does,
would fail to link. wasi-sdk's own toolchain never sees it because its
clang defaults to wasm32-wasip1; the sysroot-only mode against a host
clang does. A one-line patch adds CMAKE_ASM_COMPILER_TARGET to that
sub-build; it belongs upstream.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three findings from building the manifest locally.
LLVM's runtimes build installs libc++ under lib/<triple>/ by default.
clang finds it there, but the rpath crafter-build adds for a toolchain
outside /usr points at <prefix>/lib, and so does every distro layout the
resolver was modelled on; the bootstrap failed to start with
libc++.so.1 missing. LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF puts the
runtimes in lib/ and include/c++/v1.
flatpak-builder's default CFLAGS carry -g. For all of LLVM that is a
30 GB build tree and a multi-gigabyte .Debug extension; the module now
builds without debug info and strips, keeping the hardening flags. The
no-debuginfo key also moves under build-options, where flatpak-builder
actually reads it.
The same default CFLAGS and LDFLAGS (-fcf-protection, -z relro) reach
wasi-sdk's nested wasm32 builds through the environment, and clang
rejects them for that target. The wasi-sysroot module clears them.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
packaging/flatpak/ holds the manifest for net.catcrafts.CrafterBuild and
its side files. The runtime is the Freedesktop Sdk, not the Platform: a
build tool needs glibc's headers and crt objects, gcc's libgcc_s, cmake,
ninja, git and python at run time. Everything compiler-shaped is ours
under /app, built from one LLVM 22.1.8 source tree: clang, lld, libclang,
libc++ with its std module, compiler-rt for the host and for wasm32, the
WASI sysroot (wasi-sdk 34 in sysroot-only mode, driven by the clang just
built) and wasmtime for running the result. One tree means the modules
crafter-build builds and the compiler that consumes them cannot drift
apart, and the toolchain resolver finds all of it beside the executable
with no configuration.
The crafter-build module runs the bootstrap, the self-host build and the
test suite inside the sandbox, so a Flatpak that installs has passed the
suite with the toolchain it ships. CRAFTER_BUILD_MARCH is x86-64-v2 for
the shipped binary; user projects keep -march=native.
WasmHelloRuns compiles a fixture for wasm32-wasip1 and runs it under
wasmtime. The existing wasm tests only inspect generated files, so this
is the one that proves the sysroot, the std module for wasm and wasm-ld
work together; it skips where the sysroot or wasmtime is missing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
std.pcm was rebuilt when it was older than std.cppm, and project.so when
it was older than the crafter-build executable. Both inputs are installed
files, and an OSTree deployment (the Flatpak) gives every installed file
mtime 0: std.cppm and the executable always looked older than the cache,
so a clang bump would never rebuild std.pcm and a crafter-build update
would never recompile project.so. clang crashes on a PCM from another
compiler instead of reporting it, so the failure would have been a
backtrace on the user's first build after an update.
Each artifact now has a one-line stamp beside it. std.pcm's covers the
precompile command and the module source's bytes, which also catches a
libc++ update under the same clang. project.so's covers the crafter-build
version, the toolchain key, the module source stamp and the compile
command; project.cpp's and the pre-dependency artifacts' mtimes remain
inputs, since those live next to it with real timestamps. The module
source stamp stands in for the executable: project.so resolves Crafter::
symbols from the running binary, so only declarations matter, and an
implementation-only rebuild of crafter-build no longer recompiles it.
The std.pcm stamp folds into the module cache stamp, so a rebuilt
std.pcm rebuilds the PCMs that import it.
The cache directory carries the toolchain key
(<target>-<march>-<clang version>-<hash>), so a compiler bump starts
cold instead of handing the new clang PCMs it will crash on. LoadProject
and Build() now derive the directory from the same StdPcmDir. Old
<target>-<march> directories are left behind; `clean` never touched the
cache and still does not.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Inside the Flatpak the bundled toolchain compiled every bootstrap object,
then glslang's configure failed its test link with `unable to find
library -lc++`. clang searches its per-target runtime directory
(lib/<triple>) and the system library paths for libc++, not
<prefix>/lib, so with the classic layout a toolchain outside /usr needs
the search path spelled out, just as its loader needs the rpath.
LibcxxLinkFlags and build.sh now emit both, the same pair an explicit
CRAFTER_BUILD_LIBCXX_DIR already got.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every clang++, clang and llvm-ar the tool spawned was a bare name on
PATH, the std module source was /usr/share/libc++/v1/std.cppm, the WASI
sysroot /usr/share/wasi-sysroot and the mingw headers
/usr/x86_64-w64-mingw32, spelled out at eighteen call sites. A
toolchain installed anywhere else, which is what the Flatpak ships
under /app, had no way in.
Crafter.Build:Toolchain resolves all of it once per process:
CRAFTER_BUILD_LLVM_DIR, else a clang++ installed beside crafter-build
(<prefix>/bin, found through the same walk GetCrafterBuildHome does),
else PATH. What sits beside the compiler is used when present: its
libclang.so, its clang-c headers, its std module source, its WASI
sysroot. An explicit prefix without a clang++ is an error rather than a
fallback, for the same reason CRAFTER_BUILD_LIBCLANG was already
exclusive. CRAFTER_BUILD_LIBCXX_DIR points at a libc++ install other
than the compiler's own, and CRAFTER_MINGW_DIR now does what its help
line always claimed: it is passed as --sysroot for the mingw target.
Binaries linked by a toolchain outside /usr get an rpath to its lib
dir, since its libc++.so.1 is not on the loader's default path.
HostTarget() is the resolved clang's -print-target-triple. A clang built
without a vendor reports x86_64-unknown-linux-gnu where Arch's reports
x86_64-pc-linux-gnu, so target equality that decides whether a test runs
locally and which targets a bare `test` sweeps goes through SameTarget,
which ignores the vendor field. The strings themselves are never
rewritten: clang rejects a PCM whose triple differs from the importing
unit's. The examples and README stop hardcoding the host triple; the
default is already the host.
`crafter-build --version` prints the version baked in at build time
(CRAFTER_BUILD_VERSION, else `git describe`, else 0.0.0-dev), the
compiler and the host triple, and is answered before any argument is
forwarded to project.cpp. The version is a define on the library, so
each version is its own library variant and relinks the executable and
tests once per commit.
build.sh gains a shebang and `set -eu`, honours CRAFTER_BUILD_LLVM_DIR
and CRAFTER_BUILD_VERSION, skips the glslang clone when it is already
there, and writes a fresh archive instead of appending to one that may
hold members from an older layout.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
README and AGENTS.md stop documenting build.cmd, LIBCXX_DIR, the
DLL-plus-launcher design rule and the cmd-over-SSH gotcha. The runner
section also drops Ssh, SshWin and Wsl, which were removed from the
code earlier but still had README entries, and the stale pointer to a
deleted cross-arch test. ci.yaml loses the commented-out Windows build,
staging and upload steps and the comments that counted them.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
crafter-build is moving to a single Flatpak distribution, and there is
no Windows host in that picture: no build.cmd, no MSVC or mingw host
build, no DLL plus launcher. Delete every host-Windows path instead of
leaving it to rot unverified: the cmd.exe command runner and quoting,
the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants
with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port
probe, the LoadLibrary libclang loader, the console-width and isatty
shims, and the preprocessor guards around the Linux bodies.
The x86_64-w64-mingw32 cross-compile target stays. It is target logic
(output names, -femulated-tls, the libstdc++ link extras, copying a
dependency DLL next to a mingw exe, wine as the runner) and did not
depend on any of the removed code. The msvc target goes with the host
that was the only thing able to use it.
Three target-path warts fixed on the way: ExpectedOutput named a mingw
static library <name>.lib while the archive step wrote lib<name>.a, so
those libraries relinked on every run; the import lib is no longer
copied next to a mingw exe, since the dependency's own bin dir is
already on the link line; and llvm-ar now writes every archive, not
only the LTO ones, so Debug and Release archives are made the same way.
CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary.
Nothing downstream used them, so they are removed rather than kept as
an empty macro nobody can verify.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Crafter.LintRules declares a test that links crafter-build, fetched
with GitProject. crafter-build's own project.cpp loads Crafter.LintRules
as a `#pragma crafter dependency`, so the chain was: crafter-build loads
LintRules, LintRules' test wiring loads crafter-build's project.cpp from
the external cache, which loads LintRules again, and the cycle guard
threw. Every command that loads project.cpp in this repository has
failed since that test was added, including the last CI run on master.
A project file had no way to tell whether it was the one the user
pointed crafter-build at or a dependency being loaded for somebody
else, so it could not skip wiring that only its own build needs.
IsTopLevelProject() answers that from the loading-frame stack
ProjectDependency() already reads: one frame is the top-level project
file, a second is only ever pushed for a dependency. Outside any project
file it is true as well, which is what a test harness constructing
Configurations expects.
The ProjectDependency test records the answer from both the app and the
greeter project files and checks each.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Brings in `#pragma crafter dependency`, the switch to Crafter.LintRules
for the house style, and the contributor guide in AGENTS.md.
The branch predates the musl host work on master, and both rewrote the
command that compiles project.so, so the merge reconciles them by hand:
- LoadProject compiles project.so for hostConfig.target (HostTarget(),
from master) with the pre-dependency module path and archives from
the branch. -fuse-ld=lld appears once, with both reasons in the
comment.
- ForceHostTooling added -fPIC only for the literal x86_64-pc-linux-gnu
triple. hostConfig now comes from HostTarget(), so on Alpine the host
is x86_64-alpine-linux-musl and the dependency archive would not have
been position-independent. It now matches any *-linux-gnu or
*-linux-musl host, as project.cpp and the cmake flags do since the
musl change, and its comment no longer argues against HostTarget().
- The ProjectDependency test compared the dependency's target against
the glibc triple, which the musl CI job would fail; it compares
against HostTarget().
- README keeps master's paragraph on the bootstrap triple. AGENTS.md
describes the test block as `if (linuxTarget)`, which is what
project.cpp has now.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
