Crafter.Asset/AGENTS.md

# AGENTS.md

Guidance for coding agents and contributors working on Crafter.Asset itself. There is no README: [AGENTS.consumer.md](AGENTS.consumer.md) describes the library as its users see it, is written for the agents of the projects that depend on it, and crafter-build copies it into their AGENTS.md when they build. When this file and the code disagree, the code wins; the interfaces in `interfaces/` are the reference.

## Project overview

Crafter.Asset is the asset library of the Crafter engine: a C++ modules library (`import Crafter.Asset;`) that decodes source assets (images through stb_image, Wavefront OBJ, binary glTF), compresses vertex and pixel streams with GDeflate into files the GPU decompresses through `VK_EXT_memory_decompression`, and reads those back, with a CPU fallback. It depends on Crafter.Math for its vector and matrix types and is built with [crafter-build](https://forgejo.catcrafts.net/Catcrafts/Crafter.Build). Its consumers are Crafter.Graphics, the renderer, which uploads the compressed streams, and crafter-build itself, which links the library to compress a project's `cfg.assets` at build time. Both pull this repository's `master` unpinned.

| Path | What lives there |
|---|---|
| `interfaces/Crafter.Asset.cppm` | The umbrella module: `export import`s the four partitions. |
| `interfaces/Crafter.Asset-<Partition>.cppm` | The partitions. `:Texture` and `:Mesh` are templates and inline functions only and have no implementation unit. |
| `implementations/Crafter.Asset-Compression.cpp`, `-GLB.cpp` | Implementation units (`module Crafter.Asset;`) of the two partitions with out-of-line code. Compression `#include`s the vendored GDeflate wrappers; GLB is the only unit that sees nlohmann/json. |
| `implementations/main.cpp` | `crafter-asset`, a CLI that converts a `.png` or `.obj` into the raw `.tex`/`.mesh` formats, with a `--test-compression` round trip. Compiled only with `--exe`. |
| `project.cpp` | The build description: Crafter.Math as a dependency, libdeflate's C sources, the partitions, the `--exe` switch and the test. |
| `tests/GLB/main.cpp` | The one test. Builds GLBs in memory and loads them. |
| `lib/` | Vendored: `gdeflate/` (Microsoft's GDeflate reference, Apache-2.0, carrying NVIDIA's libdeflate 1.8 fork, MIT), `json.hpp` (nlohmann/json 3.12.0, MIT), `stb_image.h` 2.30 and `stb_image_resize2.h` 2.18 (public domain). Not edited, with the one exception under "Design rules". |
| `bin/`, `build/` | Generated by crafter-build, one `Crafter.Asset-<target>-<march>-<mtune>-<hash>/` per variant. Gitignored. |

Partitions and what they own:

- `:Compression` is the GDeflate layer. `CompressStreams` turns a list of byte spans into one `CompressedBlob`: the streams concatenated, plus a `RegionMeta` table whose layout is `VkDecompressMemoryRegionEXT` without the device addresses. Every stream is its own GDeflate tile stream, so the GPU decompresses regions independently. `DecompressCPU` is the fallback, `WriteBlob`/`ReadBlob` serialise a blob behind a format's own header, and `Fatal` prints and aborts.
- `:Texture` is `TextureAsset<T>`: decoding anything stb_image reads (`LoadPNG`, `LoadPNGFromMemory`, `SizeFromMemory`) to 8-bit or floating-point RGBA, opacity classification, `Resize`, the raw `Save`/`Load` and the compressed `SaveCompressed`/`LoadCompressedTexture` (`.ctex`). The stb implementations are compiled here, in the interface's global module fragment.
- `:Mesh` is `MeshAsset<T>`: OBJ parsing (`LoadOBJ`, the per-material `LoadOBJSplit`, `LoadMTL`), the vertex types, tangent generation, raw `Save`/`Load` and compressed `SaveCompressed`/`LoadCompressedMesh` (`.cmesh`). `MeshAsset<void>` is positions and indices only.
- `:GLB` is `GlbAsset::Load`, a glTF 2.0 binary loader: nodes, meshes with morph targets, skins, materials, textures, images, cameras and `KHR_lights_punctual`, with the JSON and BIN chunks and the `extras` kept verbatim for what it doesn't interpret.

## Dev environment

The build tool is crafter-build, as the Flatpak `net.catcrafts.CrafterBuild` or a native build of the sibling checkout; `crafter-build --version` prints the compiler it resolved. The Flatpak compiles with its bundled LLVM and a native build with the distro's clang, which can be a major version apart. Both work from this directory.

Loading `project.cpp` clones Crafter.Math from `forgejo.catcrafts.net` into `~/.cache/crafter.build/external/` and, since it is unpinned, runs `git pull` on it on every invocation, so a build needs the network. `--local` takes the sibling checkout `../Crafter.Math` instead. The flag stays in the args handed to Crafter.Math's `project.cpp`, and a consumer built with `--local` hands it to this one, which is how a `--local` build gets one copy of each module in the program instead of two.

```bash
crafter-build                              # bin/Crafter.Asset-<variant>/libCrafter.Asset.a and the PCMs
crafter-build --local                      # the same, against ../Crafter.Math
crafter-build test                         # build, then run the GLB test
crafter-build test --list                  # enumerate without running
crafter-build test --debug                 # debug build of the library and the test
crafter-build --exe                        # bin/Crafter.Asset-<variant>/crafter-asset; declares no test
crafter-build --target=wasm32-wasip1       # the wasm library: no x86 cpu_features, no tests
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine   # Windows cross-build, test under wine
crafter-build clean                        # delete bin/ and build/
```

Project flags (`--local`, `--exe`, `--debug`) can go before or after `test`. Flags `ApplyStandardArgs` doesn't interpret are part of the variant hash, so each combination has its own `bin/` and `build/` directory, and the directories of earlier variants and earlier compilers stay until `clean`; glob for a directory rather than predicting its hash.

There are no lint rules in `project.cpp`, so `crafter-build lint` and `format` have nothing to do here; style is kept by hand (see "Code style"). This repository has no CI of its own. crafter-build's CI builds it as a dependency on every crafter-build PR, which is where a break on `master` is noticed first. Before you call a change done: `crafter-build test` passes on the host, and `crafter-build --target=wasm32-wasip1` still compiles when you touched Compression or Texture, which have wasm-only branches.

Dependency notes: a crafter-build newer than 2.2.0 keeps a section of this file holding the `AGENTS.consumer.md` of every dependency. Crafter.Math ships none at the time of writing, so there is no section; when it starts to, the next build appends one at the end of this file and says so. Commit it. Don't add the markers by hand: a section without notes is removed again on the next build.

## Testing instructions

Tests are declared in `project.cpp`, and only for a native library build:

```cpp
if (!isWasm && cfg.type == ConfigurationType::LibraryStatic) {
    cfg.AddTest("GLB").Dependencies({ &cfg });
}
```

The source is `tests/<Name>/main.cpp`, an ordinary `int main()` that links `libCrafter.Asset.a` and inherits the target, march and debug setting. Exit 0 passes, nonzero fails, 77 skips. Each test runs in its own process with a 60 s default timeout (`test --timeout=<seconds>`); the output of a failed or timed-out test is in `build/test-logs/<name>.log`.

Conventions, from `tests/GLB/main.cpp`:

- Open with a comment that states what is covered and which failures it guards against.
- `Check(ok, what)` prints `PASS` or `FAIL` with the description and counts failures; `main` returns `EXIT_FAILURE` when any failed. `CheckError(loaded, text, what)` checks that a load failed with `text` in its message, `CheckLoads` the opposite, printing the error. Don't stop at the first failure.
- Fixtures are built in memory: `BuildGlb(json, bin)` lays out the container, `LoadJson` loads it, and `Replace(json, {{from, to}})` derives a broken file from a good one and fails the test when `from` isn't in the JSON, so a case can't silently stop testing what it meant to. Only the path-based cases touch the disk, under `fs::temp_directory_path()` through `TempFile`, named per run so two runs don't collide.
- A timing check compares against a smaller instance of the same work (the 100,000-node chain against one a quarter as deep), never against the clock, which flaked in debug builds.
- `CRAFTER_ASSET_TEST_GLB=<path to a skinned .glb>` adds a check on a real file (every joint's world matrix times its inverse-bind matrix is the identity at rest); without it that case prints `SKIP` and the test still passes.

What isn't covered: Compression, Texture and Mesh have no test under `tests/`. `crafter-asset --test-compression` round-trips GDeflate at the tile boundaries (1, 65535, 65536, 65537 bytes and 16 MiB) but isn't part of `crafter-build test`, and the consumers' tests (Crafter.Graphics's `MeshDecompressStagingRelease` and its Decompression example) are the only round-trip coverage of the file formats. A change to those partitions adds a `tests/<Name>/main.cpp` and its `AddTest` line, in the GLB test's shape; a bug fix ships with a test that fails before it.

## Code style

Nothing enforces it here, so match the surrounding code:

- The first two lines of every first-party source are `//SPDX-License-Identifier: LGPL-3.0-only` and `//SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®`, no space after the slashes, then a blank line. Vendored files keep their own headers.
- `import std;` everywhere; standard headers are never `#include`d. C headers and the vendored headers go in the global module fragment, after `module;`.
- Four-space indentation, no tabs. `PascalCase` for types and functions, `camelCase` for variables and members, `k`-prefixed constants in anonymous namespaces (`kMagic`, `kDecodeFloor`); the file-format constants sit in `TextureAssetFormat` and `MeshAssetFormat`.
- Fixed-width types. `int` only where a C API (stb) takes it, converted at the boundary. A size read from a file is compared in a width it can't wrap in before it is narrowed; `ViewBytes` and `ReadAccessor` are the pattern.
- `std::expected<T, std::string>` for anything that can fail on user input. The JSON helpers (`Integer`, `Index`, `Number`, `Array`) return a fallback rather than throw, because a `-fno-exceptions` build aborts on a throw.
- `std::println` and `std::format` in new code. `main.cpp`'s iostream output is older and not worth churning.
- Comments explain why: the failure mode, the constraint, the rejected alternative. The GLB unit and the recent commit bodies are the voice. Keep them, and write them for new non-obvious code.
- K&R braces; `if (x) return ...;` on one line when it fits.

## Design rules

These were decided deliberately. Don't undo them without the maintainer's say-so.

1. **Two error regimes, on purpose.** User content, anything a program loads at runtime that it didn't build itself (a GLB, an image through `LoadPNGFromMemory`), comes back as the error of a `std::expected`, and the program decides. The pipeline's own output (`.ctex` and `.cmesh` through `LoadCompressed*`, `DecompressCPU`) goes through `Compression::Fatal`, which prints and aborts: a bad magic, version or size there means the build's data doesn't match the binary, and nothing sensible can be done at that point. `Fatal` rather than `throw` keeps the module buildable under `-fno-exceptions`, which the wasm targets are. `LoadPNG` is the one exception: it throws `std::runtime_error` so crafter-build can report which file failed, and is `Fatal` where there are no exceptions. A new loader goes on the side its input belongs to; never `Fatal` on user content.
2. **The compressed formats are a contract with Crafter.Graphics.** `.ctex` is the magic `CGDT`, version 1, a header and one region, the pixels. `.cmesh` is `CGDM`, version 1, a header and three regions in the order vertices, indices, vertex data; always three, even from `MeshAsset<void>`, where the data region is empty. `RegionMeta` is three `std::uint64_t`, the `VkDecompressMemoryRegionEXT` fields minus the two addresses, and each stream is a GDeflate 1.0 tile stream at the maximum level, which the GPU decodes as is. A layout change bumps `version` in the `*AssetFormat` namespace, changes the writer and the reader together, and lands with the Crafter.Graphics change that reads it. The raw `Save`/`Load` formats (`.tex`, `.mesh`) have no magic and no version; don't grow them without adding one.
3. **Non-template code lives in an implementation unit.** `:Texture` and `:Mesh` are templates and inline functions and behave like headers; the Compression and GLB entry points are out of line in `implementations/` so they land in `libCrafter.Asset.a`, which is why the default `ConfigurationType` is `LibraryStatic`. A new non-template function goes in an implementation unit.
4. **Third-party code stays out of the interfaces.** nlohmann/json is included only in the GLB implementation unit, and the interface hands `extras` on as JSON text, so the parser is nobody else's compile-time cost. stb is the one exception, included with its implementation in the `:Texture` interface's global module fragment, because `LoadPNG` is a template. Don't add another.
5. **Every size from a file is checked before it is used.** The GLB loader compares offsets, counts and strides in 64 bits, as the file has them, before narrowing, validates every index (`CheckReferences`), finds cycles in the node graph in linear time, writes `extras` without recursion, and charges every allocation of vertices, indices and pixels to a decode budget (`decodeLimit`, by default 1 GiB plus 64 bytes per byte of file), so a small file can't allocate without bound. Images are refused from their header past 65535 a side or 2^28 pixels, which is where stb's 32-bit arithmetic wraps. Anything new that reads bytes keeps that discipline, and gets a broken-file case in the test.
6. **`lib/` is vendored.** It isn't restyled or refactored. A fix to it, such as the mingw `_rotr` macro collision in libdeflate (`021ced6`), is a minimal patch to the vendored copy whose commit message says so, so that the next update of the vendored version knows to reapply it.
7. **Targets branch at runtime in `project.cpp` and on `__wasm__` in the sources.** wasm skips libdeflate's x86 `cpu_features` (inline assembly), has no `_Float16` and no exceptions, and runs no tests. There is no host-platform split: the host is Linux and mingw is a cross-compile target.
8. **The libdeflate include path is absolute**, `fs::absolute("lib/gdeflate/libdeflate")`: as a dependency this project is compiled from the consumer's directory, and a relative `-I` falls through to a system libdeflate of a different ABI. Keep it absolute, and do the same for any include path added.

## Commit and PR guidelines

- Branch from `master`; PRs go to `master` on `forgejo.catcrafts.net/Catcrafts/Crafter.Asset`. Agent branches are named `claude/<topic>`. Pushes go over SSH: the push URL is `forgejo@forgejo.catcrafts.net:Catcrafts/Crafter.Asset.git`, the fetch URL HTTPS.
- Subject: `type(scope): imperative summary` with `feat`, `fix`, `refactor`, `docs` or `test`, the partition as the scope (`glb`, `texture`, `compression`, ...), as the recent commits do; the older one-word subjects aren't the model. The body says what changed, grouped, and why. `git log -2` shows the shape.
- A change to an interface lands on every consumer's next build, since Crafter.Graphics and crafter-build pull `master` unpinned, and crafter-build links this library into itself. Build a consumer against the change before merging: Crafter.Graphics with `--local` from its checkout, or crafter-build's self-host build. What crafter-build calls: `TextureAsset<Vector<std::uint8_t, 4, 4>>::LoadPNG<std::uint8_t>`, `Resize`, `SaveCompressed`; `MeshAsset<VertexNormalTangentUVPacked>::LoadOBJ`, `LoadOBJSplit`, `SaveCompressed`; `LoadMTL`.
- Update [AGENTS.consumer.md](AGENTS.consumer.md) in the same change when the API, a file format, a limit or an error behaviour changes. It is what the consumers' agents read, and it reaches their AGENTS.md on their next build.
- Never commit `bin/` or `build/` (gitignored), and don't edit `lib/` outside design rule 6.

## Security considerations

- The loaders parse untrusted bytes, and the GLB loader and `LoadPNGFromMemory` are the ones that meet user content. Their defences are design rule 5; the "broken files" section of the GLB test is their regression suite. A new field read from a file gets a broken-file case.
- stb_image is the image attack surface. It is pinned at 2.30 in `lib/`; the size limits in `TextureAsset` exist because stb's own arithmetic overflows past them, and an encoded image is bounded to `INT_MAX` bytes because that is what stb takes.
- `Compression::Fatal` aborts the process. Right for build output that doesn't match the binary, wrong for anything a user can hand the program (design rule 1).
- `project.cpp` is compiled and run by crafter-build, and clones and builds Crafter.Math from its default branch on every invocation. Pin a `commit` in the `GitProject` spec if that ever matters.
- The test writes only under `fs::temp_directory_path()`.

## Gotchas

- Adding a partition is three edits: the interface file, the `ifaces` array in `project.cpp`, and the umbrella `Crafter.Asset.cppm`; an implementation unit goes into both `impls` arrays, the `--exe` branch and the library branch.
- `STB_IMAGE_IMPLEMENTATION` and `STB_IMAGE_RESIZE_IMPLEMENTATION` are defined once, in `Crafter.Asset-Texture.cppm`. Defining them anywhere else in this tree, or in a consumer, duplicates the symbols at link time. `CrafterAssetClearStbiFailure`, beside them, is a plain global function because stb's failure reason is a static only that fragment can reset; it isn't API.
- The GDeflate C++ wrappers (`GDeflateCompress.cpp`, `GDeflateDecompress.cpp`) are `#include`d into the Compression implementation unit because `cfg.cFiles` compiles only `.c` files. Don't list them in `project.cpp`.
- `LoadOBJ` deduplicates vertices with a linear scan and is quadratic, and takes only triangulated faces with `v/vt/vn` on every vertex. `LoadOBJSplit` hashes, fan-triangulates and is what crafter-build's `BuildOBJBundle` uses for Sponza-sized inputs. Fix performance there, not by speeding up `LoadOBJ`.
- `MeshAsset<void>::Load` reads a two-count header and can't read a `MeshAsset<T>::Save` file, which has three. The compressed formats share one layout, which is why the data region is always present.
- `TextureAsset::Resize` is constrained to 4-byte pixels (RGBA8); the `_Float16` and `float` textures have no resize.
- The GLB loader decodes every image at load, eagerly, and refuses images with a `uri` (external file or data URI): a GLB is expected to carry its images in the BIN chunk. Only PNG and JPEG are decoded, by their signatures; another format stays an empty image, and a texture pointing at one is an error.
- There is no wasm test because wasm runs no tests here; a wasm-only branch is checked only by building `--target=wasm32-wasip1`.