docs: debugging inside the Flatpak

The README gets a Debugging section: the debugging tools the sandbox
carries, that a program to debug needs cfg.debug (which --debug sets
where the project calls ApplyStandardArgs), how to start the tools with
--command, why a debugger in one flatpak run cannot see a program that
another one started (each has process IDs of its own), and flatpak
enter as the way into a running program's sandbox, with the ptrace
opt-in that takes where ptrace_scope is 1. It warns off lldb's
gdb-remote from a second sandbox: when a process in lldb's own sandbox
has the remote program's process ID, lldb takes that process's
executable for the program's. And it shows which variables turn the
layer and GPU-AV on from the environment, and that VK_LAYER_PATH and
VK_ADD_LAYER_PATH do not reach the sandbox.

AGENTS.template.md gets the module layout and the rule that the layer
follows the Sdk's Vulkan loader. Its CI section says that a change to a
module's sources or build-commands builds it again and every module
after it, so that a new version of swig or libedit, which come before
LLVM, now rebuilds it too, and loses the cold-run time it gave, written
on 2026-10-03, before any of these modules were in the manifest.

AGENTS.template.md

@@ -160,7 +160,7 @@ These were decided deliberately. Do not undo them without the maintainer's say-s
 
 ## CI and publishing
 
-`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, so the LLVM and WASI modules are rebuilt only when their sources change; a cold run takes about 80 minutes on 8 cores, a warm one a fraction of that), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it.
+`.forgejo/workflows/ci.yaml` runs on the `crafter-flatpak` runner label: a dedicated privileged forgejo-runner on the Hetzner box (`forgejo-runner-flatpak.service`, config `/etc/forgejo-runner/config-flatpak.yaml`), registered to this repository only and the only runner with that label. It is privileged because bubblewrap inside the job container needs it, and separate from the other runners so that privilege is not shared with any other repository. Its config allows four volumes, which the workflow mounts: `crafter-flatpak-system` (the Sdk install at `/var/lib/flatpak`), `crafter-flatpak-builder` (flatpak-builder state and ccache, which later runs reuse: flatpak-builder builds a module again when its sources or its `build-commands` change, and every module after it with it, so a new version of `swig` or `libedit`, which come before LLVM, also rebuilds the LLVM and WASI modules), `/var/www/flatpak`, the web root Caddy serves as `https://flatpak.catcrafts.net/`, and `/etc/crafter-flatpak`, which holds the repo signing key (`signing-key.asc`, no passphrase, and `key-id`), generated once on the host and mounted read-only into the job; nothing is stored in the forge. The live repo at `/var/www/flatpak/repo` is canonical; builds are promoted into it with `flatpak build-commit-from`, never built into it.
 
 Keep the label unique to that runner. Plain `flatpak` also belongs to the organisation-wide runner (`forgejo-runner-org-flatpak.service`), which serves every Catcrafts repository and on purpose allows no volumes. Whenever this job lands there, flatpak-builder stops with `The state dir (/root/.cache/flatpak-builder) is not on the same filesystem as the target dir`; the top of the job log shows which runner took it and lists every volume it ignored.
 
@@ -197,6 +197,7 @@ Local builds are two scripts in `project.cpp`. Run them with the native binary:
 - The bootstrap binary is compiled without `CRAFTER_BUILD_HAS_ASSET`, so `cfg.assets` only works in the self-hosted binary. Changing asset extensions means updating two maps in `Crafter.Build-Clang.cpp`: `assetOutput` and the transitive copy loop.
 - A warning is printed only when its unit actually recompiles. A build with nothing to do is silent, which is not evidence that a warning is gone.
 - A std module source only compiles against the headers of its own libc++ release, and a sysroot does not always ship one (Alpine). The Flatpak therefore carries the sources of the release next to the bundled one under `share/crafter-build/libc++-modules/<major>/`, one `libcxx-<major>-modules` manifest module each. When the bundled LLVM is bumped, keep the release it leaves behind as such a module: sysroots on the stable distro stay on it for months. The native loop has no such directory; stage one by hand under `share/crafter-build/` to test against a foreign sysroot.
+- The Flatpak carries the debugging tools for what it builds, because those programs run in its sandbox: lldb is a project of the `llvm` module, with `LLDB_ENABLE_PYTHON`, `LIBEDIT`, `CURSES`, `LIBXML2` and `LZMA` set ON so that a missing dependency fails the configure step (SWIG through Python, which needs it), and the `swig` and `libedit` modules come before it. The Khronos validation layer is the `vulkan-validation-layers` module plus four build-only modules before it, all after the LLVM and WASI modules, so that bumping them reuses the cached LLVM and WASI builds. The layer follows the Sdk's Vulkan loader: when `runtime-version` moves to a runtime with a newer loader, move the layer to the Vulkan SDK release of that loader and take the other four revisions from that release's `scripts/known_good.json`.
 - The wasm compile flags live in one place, `WasmBaselineFlags` in `Crafter.Build-Clang.cppm`: the std module and every unit have to agree on them, and `VariantId()` and the std PCM directory hash them, so a change rebuilds instead of mixing old and new BMIs. They enable C++ exceptions, so the link takes the exception-enabled libc++ and libunwind from the sysroot's `lib/<triple>/eh/` (wasi-sdk 33 and later). The Flatpak's sysroot has them and a distro's may not; where they are missing the wasm tests skip, so run those through the Flatpak. A browser page checks the engine with the `WasmBaselineProbes` named in `variants.json`.
 - `WasiBrowserRuntimeExit` runs `wasi-runtime/runtime.js` under node and skips where there is none. The Flatpak has no node, so CI never runs it: after touching `runtime.js`, run the suite in the native loop with node on `PATH`.
 - `VariantId()` names the output directory, so CI globs `bin/crafter.build-exe-<target>-<march>-*/` rather than predicting the hash. The hash includes the compiler, so after a toolchain update the previous variant's directories stay in `bin/` and `build/` until `clean`, and such a glob matches both.

README.md

@@ -4,7 +4,7 @@ A C++26 modules build system. Project descriptions are written in C++ with no JS
 
 ## Install
 
-crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler:
+crafter-build ships as a Flatpak that carries its own clang, lld, libc++ and WASI sysroot, so every machine builds with the same compiler, and lldb and the Vulkan validation layers to debug the result (see [Debugging](#debugging)):
 
 ```bash
 flatpak remote-add --if-not-exists catcrafts https://flatpak.catcrafts.net/catcrafts.flatpakrepo
@@ -364,6 +364,32 @@ Things to know:
 - Inside the Flatpak, commands run in its sandbox: the host's files are visible, but the tools on `PATH` are the ones the Flatpak ships (`git`, `cmake`, `python3`, `tar`, ...), not everything installed on the host.
 - A return value outside 0 to 255 is not an exit status and becomes 1. So does a script that throws, with the exception's message printed.
 
+## Debugging
+
+Programs that `-r` and the test runner start run inside the Flatpak's sandbox, and so do the tools to debug them: lldb, with `lldb-server`, `lldb-dap` and `lldb-mcp`, from the same LLVM build as the compiler; `gdb`, `gdbserver`, `valgrind` and `strace` from the Freedesktop Sdk; and the Khronos Vulkan validation layer. A default build is optimised (`-O3`, and ThinLTO except for wasm) and carries no debug information: set `cfg.debug = true` for a program you want to debug, which compiles its C++ units with `-g` and links it without LTO; its C and CUDA sources (`cfg.cFiles`, `cfg.cuda`) stay at `-O3` without debug information. A `project.cpp` that calls `ApplyStandardArgs` sets it from the `--debug` argument, as in `crafter-build --debug -r`. Start the tools with `--command`:
+
+```bash
+flatpak run --die-with-parent --command=lldb net.catcrafts.CrafterBuild ./bin/myapp-<variant>/myapp
+flatpak run --die-with-parent --command=valgrind net.catcrafts.CrafterBuild ./bin/myapp-<variant>/myapp
+```
+
+Every `flatpak run` is a sandbox of its own, with process IDs of its own, so a debugger in one cannot see a program that another one, such as `crafter-build -r`, started. Either start the program under the debugger, or run the debugger inside that program's sandbox with `flatpak enter`, giving the instance `flatpak ps` lists for net.catcrafts.CrafterBuild and the debugger's full path:
+
+```bash
+flatpak ps --columns=instance,application
+flatpak enter <instance> /app/bin/lldb -n myapp
+```
+
+Where `/proc/sys/kernel/yama/ptrace_scope` is 1, a debugger in the sandbox, where processes hold no capabilities, can attach to a program that is not its descendant only when the program allowed it with `prctl(PR_SET_PTRACER, …)`: with the debugger's process ID, or with `PR_SET_PTRACER_ANY` for any process. Otherwise the attach fails. A program started under the debugger is its descendant. Prefer `flatpak enter` to `lldb-server gdbserver` in one sandbox and lldb's `gdb-remote` from another: when a process in lldb's own sandbox has the remote program's process ID, lldb takes that process's executable for the program's: symbols come out wrong, and a breakpoint in the program's source finds no location.
+
+The validation layer is enabled the usual ways, by the program itself or from the environment: `flatpak run` passes `VK_INSTANCE_LAYERS`, `VK_LOADER_LAYERS_ENABLE` and the layer's `VK_KHRONOS_VALIDATION_*` settings into the sandbox. `VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1` adds GPU-assisted validation (GPU-AV):
+
+```bash
+VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_KHRONOS_VALIDATION_GPUAV_ENABLE=1 crafter-build -r
+```
+
+The layer is the Vulkan SDK release of the runtime's own Vulkan loader. `VK_LAYER_PATH` and `VK_ADD_LAYER_PATH` are cleared on the way into the sandbox, so a layer installed on the host is out of reach.
+
 ## Examples
 
 [examples/](examples/) contains progressively larger self-contained projects: