Compare commits
3 changed files with 1 additions and 189 deletions
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -1,3 +1,2 @@
|
||||||
build/
|
build/
|
||||||
bin/
|
bin/
|
||||||
AGENTS.md
|
|
||||||
|
|
@ -1,63 +0,0 @@
|
||||||
## Crafter.Asset
|
|
||||||
|
|
||||||
The asset library of the Crafter engine. It decodes source assets (images through stb_image, Wavefront OBJ, binary glTF), GDeflate-compresses vertex and pixel streams into `.ctex` and `.cmesh` files that the GPU decompresses through `VK_EXT_memory_decompression`, and reads those files back, with a CPU fallback. `import Crafter.Asset;` gets all of it; the vector and matrix types are Crafter.Math's, so `import Crafter.Math;` as well. Everything is in `namespace Crafter`, the compression layer in `Crafter::Compression`. Source and issues: https://forgejo.catcrafts.net/Catcrafts/Crafter.Asset (LGPL-3.0-only; the vendored GDeflate is Apache-2.0, libdeflate and nlohmann/json MIT, stb public domain). These notes are its `AGENTS.consumer.md`: a fix to them goes there, not into the copy crafter-build writes.
|
|
||||||
|
|
||||||
### Textures
|
|
||||||
|
|
||||||
`TextureAsset<T>` is `sizeX`, `sizeY` (`std::uint16_t`), `opaque` and `std::vector<T> pixels`, row-major from the top-left, with `T` a pixel vector of Crafter.Math: `Vector<std::uint8_t, 4, 4>` (RGBA8, 4 bytes), `Vector<_Float16, 4, 4>` or `Vector<float, 4, 4>`.
|
|
||||||
|
|
||||||
- `TextureAsset<T>::LoadPNG<TT>(path)` decodes any format stb_image reads (PNG, JPEG, TGA, BMP, GIF, PSD, HDR, PNM) to RGBA: 8-bit for `TT = std::uint8_t`, 0..1 for `_Float16`, `float` or `double`; `TT` is `T`'s component type. It throws `std::runtime_error` naming the file and stb's reason when the file doesn't decode (and aborts where there are no exceptions). `LoadPNGFromMemory<TT>(bytes)` returns `std::expected<TextureAsset<T>, std::string>` instead, for an image inside a larger asset; `SizeFromMemory(bytes)` reads only the header, as `std::expected<std::pair<std::uint32_t, std::uint32_t>, std::string>`. All three refuse images over 65535 a side, over 2^28 pixels (16384×16384), or over `INT_MAX` encoded bytes.
|
|
||||||
- `opaque` is classified at decode: `OpaqueType::FullyOpaque` (every alpha is full), `SemiOpaque` (only full or zero: a cutout, no blending needed) or `Transparent`.
|
|
||||||
- `Resize(w, h)` is a bilinear resize in place, RGBA8 only; Crafter.Graphics uses it to give every layer of a texture array the same size.
|
|
||||||
- `Save(path)` and `Load(path)` are the raw `.tex` format: `u16 sizeX, u16 sizeY, u8 opaque, pixels`, no magic, no version. `LoadInfo(path)` reads that header as a `TextureAssetInfo`; `Load(path, T* pixels, sizeX, sizeY)` reads the pixels into memory you own, such as a mapped staging buffer.
|
|
||||||
- `SaveCompressed(path)` writes `.ctex`. `LoadCompressedTexture(path)` returns a `CompressedTextureAsset { sizeX, sizeY, opaque, pixelStride, blob }` without decompressing: one region, `sizeX * sizeY * pixelStride` bytes when decompressed.
|
|
||||||
|
|
||||||
### Meshes
|
|
||||||
|
|
||||||
`MeshAsset<T>` is `vertexes` (`Vector<float, 3, 3>`: 12 bytes each, tightly packed), `indexes` (`std::uint32_t`, a triangle list) and `datas`, one `T` per vertex. `T` is normally `VertexNormalTangentUVPacked`: `normal`, `tangent` (`Vector<float, 3, 4>`) and `uv` (`Vector<float, 2, 4>`), each padded to 16 bytes, 48 in all. `MeshAsset<void>` has positions and indices only.
|
|
||||||
|
|
||||||
- `MeshAsset<T>::LoadOBJ(path)` reads an OBJ into one mesh. Faces must be triangles with `v/vt/vn` on every vertex. Identical vertices are merged, with a linear scan, so it is quadratic: for anything large use `LoadOBJSplit(path)`, which returns one `std::pair<std::string, MeshAsset<T>>` per `usemtl` group in encounter order (faces before any `usemtl` go under `""`), merges through a hash, fan-triangulates polygons and skips degenerate UV triangles when computing tangents. Both accumulate per-vertex tangents from the UVs. `LoadMTL(path)` gives `std::unordered_map<std::string, MtlMaterial>`, each `MtlMaterial { mapKd }` the diffuse texture path relative to the `.mtl`'s directory, or empty; nothing else of a material is read.
|
|
||||||
- `Save(path)` and `Load(path)` are the raw `.mesh` format: `u32 vertexCount, u32 indexCount, u32 dataCount, vertexes, indexes, datas`, no magic, no version. `MeshAsset<void>`'s raw form has no `dataCount`, so the two can't read each other's files.
|
|
||||||
- `SaveCompressed(path)` writes `.cmesh`. `LoadCompressedMesh(path)` returns a `CompressedMeshAsset { vertexCount, indexCount, dataCount, dataStride, blob }`: always three regions, in order vertices (`vertexCount * 12` bytes), indices (`indexCount * 4`) and vertex data (`dataCount * dataStride`; for `MeshAsset<void>` the count and both sizes are 0). `dataStride` is `sizeof(T)` at save time; check it against the `T` you are about to read the bytes as.
|
|
||||||
|
|
||||||
### Compressed blobs and the GPU path
|
|
||||||
|
|
||||||
`Compression::CompressStreams(spans)` compresses each span as an independent GDeflate stream (64 KiB tiles, maximum level) and returns a `CompressedBlob { bytes, regions }`. `regions[i]` is a `RegionMeta { srcOffset, compressedSize, decompressedSize }`, three `std::uint64_t`: the fields of `VkDecompressMemoryRegionEXT` minus the two device addresses. Upload `bytes` to a buffer, add its address to each `srcOffset`, point each region's destination where its output goes, and the GPU decompresses them with `VK_EXT_memory_decompression` (method GDeflate 1.0). An empty input becomes a region with both sizes 0; skip it. `blob.TotalDecompressedSize()` sums the regions.
|
|
||||||
|
|
||||||
`DecompressCPU(blob, outputs)` is the fallback and the reference, single-threaded: `outputs.size()` must equal `regions.size()` and each `outputs[i].size()` its region's `decompressedSize`, exactly, or it aborts.
|
|
||||||
|
|
||||||
`WriteBlob` and `ReadBlob` serialise a blob behind a format's own header: `u32 regionCount`, then `regionCount` `RegionMeta`s, then `u64 payloadSize` and the payload. Every integer in every format is written in native byte order, as laid out in memory; all supported targets are little-endian.
|
|
||||||
|
|
||||||
### GLB
|
|
||||||
|
|
||||||
`GlbAsset::Load(path, decodeLimit = std::nullopt)` and `Load(std::span<const std::byte> glb, decodeLimit)` return `std::expected<GlbAsset, std::string>`. Every error starts with `GLB: ` and says what is wrong, with the path appended by the file overload. A broken file is an error, never a crash or an abort; that is the contract. What it reads:
|
|
||||||
|
|
||||||
- `nodes`, each with `parent` (-1 at a root) and `children`, the local transform as `translation`, `rotation` (quaternion, xyzw), `scale`, or, when `hasMatrix`, as `matrix`, and `mesh`, `skin`, `camera`, `light` indices (-1 when absent), per-node morph `weights` and `extras`. `sceneRoots` are the default scene's roots (the file's `scene`, else scene 0, else every parentless node).
|
|
||||||
- `meshes[].primitives[]`, triangle lists only: `positions`, `normals` (`Vector<float, 3, 3>`), `tangents` (`Vector<float, 4, 4>`, the bitangent's handedness in `w`; computed from the UV set the material's normal map samples when the file has none and the primitive has normals), `uv0`, `uv1` (`Vector<float, 2, 2>`), `joints` (`std::array<std::uint16_t, 4>`), `weights` (`std::array<float, 4>`, summing to 1; the four heaviest when the file gives more), `indices` (`std::uint32_t`, generated as 0..n-1 for an unindexed primitive), `targets` (morph targets: position, normal and tangent deltas) and `material` (-1 for none). A stream the primitive doesn't have is empty; one it has holds one entry per vertex. Strips and fans arrive as lists; points and lines arrive as primitives with no vertices, so indices into `primitives` still match the file. `meshes[].weights` are the morph weights, 0 where the file has none, and `meshes[].extras` is where Blender and three.js put `targetNames`.
|
|
||||||
- `skins` with `joints`, `inverseBindMatrices` (identity when absent) and `skeleton`. `materials` with the PBR factors, texture refs as `GlbTextureRef { texture, texCoord (0 or 1), scale }` (`scale` is `normalTexture`'s scale or `occlusionTexture`'s strength), `alphaMode`, `alphaCutoff`, `doubleSided`, and `KHR_materials_emissive_strength`, `KHR_materials_transmission`, `KHR_materials_ior`. `textures` (`image`, `sampler`), `samplers` (GL enums as in the file; filters 0 when unset, wrap 10497 by default). `images` as `TextureAsset<Vector<std::uint8_t, 4, 4>>`, every one decoded at load from the BIN chunk when its bytes are PNG or JPEG; another format (WebP, KTX2) stays empty with `sizeX == 0`, a texture pointing at one is an error, and an image with a `uri` (external file or data URI) is an error. `cameras` look down their node's -Z with +Y up (`yfov` in radians; `aspectRatio` 0 means the viewport's; `zfar` 0 means infinite). `lights` of `KHR_lights_punctual` sit at their node's origin and shine down -Z (`intensity` in candela, lux for directional; `range` 0 means unlimited; cone angles in radians).
|
|
||||||
- `json` and `bin` are the two chunks verbatim, and `extras` (of the scene, each node, mesh, material and light) are JSON text: animations, other extensions, Blender's custom properties and anything else the loader doesn't interpret are there for you to parse with a JSON library of your own.
|
|
||||||
|
|
||||||
Matrices: `GlbNode::matrix` and `GlbSkin::inverseBindMatrices` are `MatrixRowMajor<float, 4, 4, 1>` in Crafter.Math's row-vector form (translation in row 3), which is glTF's column-major array copied into the rows unchanged. A vertex transforms as `v * M`; a TRS node's local matrix is `MatrixRowMajor<float, 4, 4, 1>::TRS(translation, rotation, scale)`; a node's world matrix is `local * parentWorld`; and at rest `world[joint] * inverseBind[joint]` is the identity for every joint of a skin.
|
|
||||||
|
|
||||||
Refusals and limits: a file whose `extensionsRequired` names anything but `KHR_lights_punctual`, `KHR_materials_emissive_strength`, `KHR_materials_ior` or `KHR_materials_transmission` is an error, as glTF asks; a `.gltf` with external `.bin` buffers isn't a GLB and isn't read. What is decoded (vertices, indices, pixels) is held to `decodeLimit` bytes, by default 1 GiB plus 64 bytes per byte of file, and a file past it is an error rather than an allocation, so a few KB of JSON can't point a thousand primitives at one buffer and exhaust memory. Running out of memory short of the limit throws `std::bad_alloc`, as anywhere. Every image is decoded at load, so a GLB with large textures costs their RGBA8 size up front.
|
|
||||||
|
|
||||||
There is no compressed form of a `GlbAsset` and no `.glb` step in crafter-build's asset pipeline: load one at runtime, or convert in a build script, copying primitives into a `MeshAsset` and calling `SaveCompressed`; `images[i]` is a `TextureAsset` already, so `images[i].SaveCompressed(path)` works as is.
|
|
||||||
|
|
||||||
### Build-time pipeline, in crafter-build
|
|
||||||
|
|
||||||
crafter-build links this library and compresses a project's `cfg.assets` while building it, when Crafter.Asset is reachable through `cfg.dependencies`:
|
|
||||||
|
|
||||||
- A file entry `.png`, `.tga`, `.jpg`, `.jpeg` or `.bmp` (case-insensitive) becomes `<name>.ctex` in the bin directory, decoded as RGBA8; `.obj` becomes `<name>.cmesh` through `MeshAsset<VertexNormalTangentUVPacked>::LoadOBJ`. Any other extension fails the build.
|
|
||||||
- A directory entry is recursed and mirrored under `<dirname>/` in the bin directory: those extensions compressed, every other file copied unchanged, so a `mod.json` beside them keeps referring to `cannon/base.cmesh`.
|
|
||||||
- A file is skipped when its output is newer than its source. Outputs are forwarded into a consuming executable's bin directory, like shaders and `cfg.files`.
|
|
||||||
- `BuildOBJBundle(objPath, mtlPath, outDir, albedoSize = 1024)`, from `import Crafter.Build;` in a `project.cpp`, splits an OBJ by material into `mesh_<i>.cmesh`, resizes each distinct diffuse map to `albedoSize` square as `tex_<i>.ctex`, and writes `scene.txt` (albedo count, mesh count, then one albedo index per mesh, -1 for none). It returns at once when `scene.txt` exists; forward the directory's files through `cfg.files`. Crafter.Graphics's Sponza example is the pattern.
|
|
||||||
- `crafter-asset`, the CLI this repository builds with `crafter-build --exe`, writes the raw `.tex` and `.mesh` formats, not the compressed ones: `crafter-asset in.png [out.tex] [--format u8|f16]`, `crafter-asset in.obj [out.mesh]`.
|
|
||||||
|
|
||||||
### Errors
|
|
||||||
|
|
||||||
| Call | On bad input |
|
|
||||||
|---|---|
|
|
||||||
| `GlbAsset::Load`, `LoadPNGFromMemory`, `SizeFromMemory` | `std::unexpected` with a message. User content: you decide. |
|
|
||||||
| `LoadPNG` | throws `std::runtime_error` (crafter-build reports it per file); aborts on wasm. |
|
|
||||||
| `LoadCompressedTexture`, `LoadCompressedMesh`, `DecompressCPU`, `CompressStreams` | `Compression::Fatal`: prints `Crafter.Asset fatal: ...` to stderr and `std::abort()`s. These read the build's own output, and a mismatch means the data and the binary disagree. Never route user content through them. |
|
|
||||||
| raw `Load`, `LoadInfo`, `LoadOBJ`, `LoadOBJSplit`, `LoadMTL` | no checks: a missing file reads as zeros or empty, a malformed OBJ can throw from `std::stoi`. |
|
|
||||||
|
|
@ -1,124 +0,0 @@
|
||||||
# AGENTS.md
|
|
||||||
|
|
||||||
Guidance for coding agents and contributors working on Crafter.Asset itself. There is no README: [AGENTS.consumer.md](AGENTS.consumer.md) describes the library as its users see it, is written for the agents of the projects that depend on it, and crafter-build copies it into their AGENTS.md when they build. When this file and the code disagree, the code wins; the interfaces in `interfaces/` are the reference.
|
|
||||||
|
|
||||||
## Project overview
|
|
||||||
|
|
||||||
Crafter.Asset is the asset library of the Crafter engine: a C++ modules library (`import Crafter.Asset;`) that decodes source assets (images through stb_image, Wavefront OBJ, binary glTF), compresses vertex and pixel streams with GDeflate into files the GPU decompresses through `VK_EXT_memory_decompression`, and reads those back, with a CPU fallback. It depends on Crafter.Math for its vector and matrix types and is built with [crafter-build](https://forgejo.catcrafts.net/Catcrafts/Crafter.Build). Its consumers are Crafter.Graphics, the renderer, which uploads the compressed streams, and crafter-build itself, which links the library to compress a project's `cfg.assets` at build time. Both pull this repository's `master` unpinned.
|
|
||||||
|
|
||||||
| Path | What lives there |
|
|
||||||
|---|---|
|
|
||||||
| `interfaces/Crafter.Asset.cppm` | The umbrella module: `export import`s the four partitions. |
|
|
||||||
| `interfaces/Crafter.Asset-<Partition>.cppm` | The partitions. `:Texture` and `:Mesh` are templates and inline functions only and have no implementation unit. |
|
|
||||||
| `implementations/Crafter.Asset-Compression.cpp`, `-GLB.cpp` | Implementation units (`module Crafter.Asset;`) of the two partitions with out-of-line code. Compression `#include`s the vendored GDeflate wrappers; GLB is the only unit that sees nlohmann/json. |
|
|
||||||
| `implementations/main.cpp` | `crafter-asset`, a CLI that converts a `.png` or `.obj` into the raw `.tex`/`.mesh` formats, with a `--test-compression` round trip. Compiled only with `--exe`. |
|
|
||||||
| `project.cpp` | The build description: Crafter.Math as a dependency, libdeflate's C sources, the partitions, the `--exe` switch and the test. |
|
|
||||||
| `tests/GLB/main.cpp` | The one test. Builds GLBs in memory and loads them. |
|
|
||||||
| `lib/` | Vendored: `gdeflate/` (Microsoft's GDeflate reference, Apache-2.0, carrying NVIDIA's libdeflate 1.8 fork, MIT), `json.hpp` (nlohmann/json 3.12.0, MIT), `stb_image.h` 2.30 and `stb_image_resize2.h` 2.18 (public domain). Not edited, with the one exception under "Design rules". |
|
|
||||||
| `bin/`, `build/` | Generated by crafter-build, one `Crafter.Asset-<target>-<march>-<mtune>-<hash>/` per variant. Gitignored. |
|
|
||||||
|
|
||||||
Partitions and what they own:
|
|
||||||
|
|
||||||
- `:Compression` is the GDeflate layer. `CompressStreams` turns a list of byte spans into one `CompressedBlob`: the streams concatenated, plus a `RegionMeta` table whose layout is `VkDecompressMemoryRegionEXT` without the device addresses. Every stream is its own GDeflate tile stream, so the GPU decompresses regions independently. `DecompressCPU` is the fallback, `WriteBlob`/`ReadBlob` serialise a blob behind a format's own header, and `Fatal` prints and aborts.
|
|
||||||
- `:Texture` is `TextureAsset<T>`: decoding anything stb_image reads (`LoadPNG`, `LoadPNGFromMemory`, `SizeFromMemory`) to 8-bit or floating-point RGBA, opacity classification, `Resize`, the raw `Save`/`Load` and the compressed `SaveCompressed`/`LoadCompressedTexture` (`.ctex`). The stb implementations are compiled here, in the interface's global module fragment.
|
|
||||||
- `:Mesh` is `MeshAsset<T>`: OBJ parsing (`LoadOBJ`, the per-material `LoadOBJSplit`, `LoadMTL`), the vertex types, tangent generation, raw `Save`/`Load` and compressed `SaveCompressed`/`LoadCompressedMesh` (`.cmesh`). `MeshAsset<void>` is positions and indices only.
|
|
||||||
- `:GLB` is `GlbAsset::Load`, a glTF 2.0 binary loader: nodes, meshes with morph targets, skins, materials, textures, images, cameras and `KHR_lights_punctual`, with the JSON and BIN chunks and the `extras` kept verbatim for what it doesn't interpret.
|
|
||||||
|
|
||||||
## Dev environment
|
|
||||||
|
|
||||||
The build tool is crafter-build, as the Flatpak `net.catcrafts.CrafterBuild` or a native build of the sibling checkout; `crafter-build --version` prints the compiler it resolved. The Flatpak compiles with its bundled LLVM and a native build with the distro's clang, which can be a major version apart. Both work from this directory.
|
|
||||||
|
|
||||||
Loading `project.cpp` clones Crafter.Math from `forgejo.catcrafts.net` into `~/.cache/crafter.build/external/` and, since it is unpinned, runs `git pull` on it on every invocation, so a build needs the network. `--local` takes the sibling checkout `../Crafter.Math` instead. The flag stays in the args handed to Crafter.Math's `project.cpp`, and a consumer built with `--local` hands it to this one, which is how a `--local` build gets one copy of each module in the program instead of two.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
crafter-build # bin/Crafter.Asset-<variant>/libCrafter.Asset.a and the PCMs
|
|
||||||
crafter-build --local # the same, against ../Crafter.Math
|
|
||||||
crafter-build test # build, then run the GLB test
|
|
||||||
crafter-build test --list # enumerate without running
|
|
||||||
crafter-build test --debug # debug build of the library and the test
|
|
||||||
crafter-build --exe # bin/Crafter.Asset-<variant>/crafter-asset; declares no test
|
|
||||||
crafter-build --target=wasm32-wasip1 # the wasm library: no x86 cpu_features, no tests
|
|
||||||
crafter-build test --target=x86_64-w64-mingw32 --runner=cmd:wine # Windows cross-build, test under wine
|
|
||||||
crafter-build clean # delete bin/ and build/
|
|
||||||
```
|
|
||||||
|
|
||||||
Project flags (`--local`, `--exe`, `--debug`) can go before or after `test`. Flags `ApplyStandardArgs` doesn't interpret are part of the variant hash, so each combination has its own `bin/` and `build/` directory, and the directories of earlier variants and earlier compilers stay until `clean`; glob for a directory rather than predicting its hash.
|
|
||||||
|
|
||||||
There are no lint rules in `project.cpp`, so `crafter-build lint` and `format` have nothing to do here; style is kept by hand (see "Code style"). This repository has no CI of its own. crafter-build's CI builds it as a dependency on every crafter-build PR, which is where a break on `master` is noticed first. Before you call a change done: `crafter-build test` passes on the host, and `crafter-build --target=wasm32-wasip1` still compiles when you touched Compression or Texture, which have wasm-only branches.
|
|
||||||
|
|
||||||
Dependency notes: a crafter-build newer than 2.2.0 keeps a section of this file holding the `AGENTS.consumer.md` of every dependency. Crafter.Math ships none at the time of writing, so there is no section; when it starts to, the next build appends one at the end of this file and says so. Commit it. Don't add the markers by hand: a section without notes is removed again on the next build.
|
|
||||||
|
|
||||||
## Testing instructions
|
|
||||||
|
|
||||||
Tests are declared in `project.cpp`, and only for a native library build:
|
|
||||||
|
|
||||||
```cpp
|
|
||||||
if (!isWasm && cfg.type == ConfigurationType::LibraryStatic) {
|
|
||||||
cfg.AddTest("GLB").Dependencies({ &cfg });
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The source is `tests/<Name>/main.cpp`, an ordinary `int main()` that links `libCrafter.Asset.a` and inherits the target, march and debug setting. Exit 0 passes, nonzero fails, 77 skips. Each test runs in its own process with a 60 s default timeout (`test --timeout=<seconds>`); the output of a failed or timed-out test is in `build/test-logs/<name>.log`.
|
|
||||||
|
|
||||||
Conventions, from `tests/GLB/main.cpp`:
|
|
||||||
|
|
||||||
- Open with a comment that states what is covered and which failures it guards against.
|
|
||||||
- `Check(ok, what)` prints `PASS` or `FAIL` with the description and counts failures; `main` returns `EXIT_FAILURE` when any failed. `CheckError(loaded, text, what)` checks that a load failed with `text` in its message, `CheckLoads` the opposite, printing the error. Don't stop at the first failure.
|
|
||||||
- Fixtures are built in memory: `BuildGlb(json, bin)` lays out the container, `LoadJson` loads it, and `Replace(json, {{from, to}})` derives a broken file from a good one and fails the test when `from` isn't in the JSON, so a case can't silently stop testing what it meant to. Only the path-based cases touch the disk, under `fs::temp_directory_path()` through `TempFile`, named per run so two runs don't collide.
|
|
||||||
- A timing check compares against a smaller instance of the same work (the 100,000-node chain against one a quarter as deep), never against the clock, which flaked in debug builds.
|
|
||||||
- `CRAFTER_ASSET_TEST_GLB=<path to a skinned .glb>` adds a check on a real file (every joint's world matrix times its inverse-bind matrix is the identity at rest); without it that case prints `SKIP` and the test still passes.
|
|
||||||
|
|
||||||
What isn't covered: Compression, Texture and Mesh have no test under `tests/`. `crafter-asset --test-compression` round-trips GDeflate at the tile boundaries (1, 65535, 65536, 65537 bytes and 16 MiB) but isn't part of `crafter-build test`, and the consumers' tests (Crafter.Graphics's `MeshDecompressStagingRelease` and its Decompression example) are the only round-trip coverage of the file formats. A change to those partitions adds a `tests/<Name>/main.cpp` and its `AddTest` line, in the GLB test's shape; a bug fix ships with a test that fails before it.
|
|
||||||
|
|
||||||
## Code style
|
|
||||||
|
|
||||||
Nothing enforces it here, so match the surrounding code:
|
|
||||||
|
|
||||||
- The first two lines of every first-party source are `//SPDX-License-Identifier: LGPL-3.0-only` and `//SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®`, no space after the slashes, then a blank line. Vendored files keep their own headers.
|
|
||||||
- `import std;` everywhere; standard headers are never `#include`d. C headers and the vendored headers go in the global module fragment, after `module;`.
|
|
||||||
- Four-space indentation, no tabs. `PascalCase` for types and functions, `camelCase` for variables and members, `k`-prefixed constants in anonymous namespaces (`kMagic`, `kDecodeFloor`); the file-format constants sit in `TextureAssetFormat` and `MeshAssetFormat`.
|
|
||||||
- Fixed-width types. `int` only where a C API (stb) takes it, converted at the boundary. A size read from a file is compared in a width it can't wrap in before it is narrowed; `ViewBytes` and `ReadAccessor` are the pattern.
|
|
||||||
- `std::expected<T, std::string>` for anything that can fail on user input. The JSON helpers (`Integer`, `Index`, `Number`, `Array`) return a fallback rather than throw, because a `-fno-exceptions` build aborts on a throw.
|
|
||||||
- `std::println` and `std::format` in new code. `main.cpp`'s iostream output is older and not worth churning.
|
|
||||||
- Comments explain why: the failure mode, the constraint, the rejected alternative. The GLB unit and the recent commit bodies are the voice. Keep them, and write them for new non-obvious code.
|
|
||||||
- K&R braces; `if (x) return ...;` on one line when it fits.
|
|
||||||
|
|
||||||
## Design rules
|
|
||||||
|
|
||||||
These were decided deliberately. Don't undo them without the maintainer's say-so.
|
|
||||||
|
|
||||||
1. **Two error regimes, on purpose.** User content, anything a program loads at runtime that it didn't build itself (a GLB, an image through `LoadPNGFromMemory`), comes back as the error of a `std::expected`, and the program decides. The pipeline's own output (`.ctex` and `.cmesh` through `LoadCompressed*`, `DecompressCPU`) goes through `Compression::Fatal`, which prints and aborts: a bad magic, version or size there means the build's data doesn't match the binary, and nothing sensible can be done at that point. `Fatal` rather than `throw` keeps the module buildable under `-fno-exceptions`, which the wasm targets are. `LoadPNG` is the one exception: it throws `std::runtime_error` so crafter-build can report which file failed, and is `Fatal` where there are no exceptions. A new loader goes on the side its input belongs to; never `Fatal` on user content.
|
|
||||||
2. **The compressed formats are a contract with Crafter.Graphics.** `.ctex` is the magic `CGDT`, version 1, a header and one region, the pixels. `.cmesh` is `CGDM`, version 1, a header and three regions in the order vertices, indices, vertex data; always three, even from `MeshAsset<void>`, where the data region is empty. `RegionMeta` is three `std::uint64_t`, the `VkDecompressMemoryRegionEXT` fields minus the two addresses, and each stream is a GDeflate 1.0 tile stream at the maximum level, which the GPU decodes as is. A layout change bumps `version` in the `*AssetFormat` namespace, changes the writer and the reader together, and lands with the Crafter.Graphics change that reads it. The raw `Save`/`Load` formats (`.tex`, `.mesh`) have no magic and no version; don't grow them without adding one.
|
|
||||||
3. **Non-template code lives in an implementation unit.** `:Texture` and `:Mesh` are templates and inline functions and behave like headers; the Compression and GLB entry points are out of line in `implementations/` so they land in `libCrafter.Asset.a`, which is why the default `ConfigurationType` is `LibraryStatic`. A new non-template function goes in an implementation unit.
|
|
||||||
4. **Third-party code stays out of the interfaces.** nlohmann/json is included only in the GLB implementation unit, and the interface hands `extras` on as JSON text, so the parser is nobody else's compile-time cost. stb is the one exception, included with its implementation in the `:Texture` interface's global module fragment, because `LoadPNG` is a template. Don't add another.
|
|
||||||
5. **Every size from a file is checked before it is used.** The GLB loader compares offsets, counts and strides in 64 bits, as the file has them, before narrowing, validates every index (`CheckReferences`), finds cycles in the node graph in linear time, writes `extras` without recursion, and charges every allocation of vertices, indices and pixels to a decode budget (`decodeLimit`, by default 1 GiB plus 64 bytes per byte of file), so a small file can't allocate without bound. Images are refused from their header past 65535 a side or 2^28 pixels, which is where stb's 32-bit arithmetic wraps. Anything new that reads bytes keeps that discipline, and gets a broken-file case in the test.
|
|
||||||
6. **`lib/` is vendored.** It isn't restyled or refactored. A fix to it, such as the mingw `_rotr` macro collision in libdeflate (`021ced6`), is a minimal patch to the vendored copy whose commit message says so, so that the next update of the vendored version knows to reapply it.
|
|
||||||
7. **Targets branch at runtime in `project.cpp` and on `__wasm__` in the sources.** wasm skips libdeflate's x86 `cpu_features` (inline assembly), has no `_Float16` and no exceptions, and runs no tests. There is no host-platform split: the host is Linux and mingw is a cross-compile target.
|
|
||||||
8. **The libdeflate include path is absolute**, `fs::absolute("lib/gdeflate/libdeflate")`: as a dependency this project is compiled from the consumer's directory, and a relative `-I` falls through to a system libdeflate of a different ABI. Keep it absolute, and do the same for any include path added.
|
|
||||||
|
|
||||||
## Commit and PR guidelines
|
|
||||||
|
|
||||||
- Branch from `master`; PRs go to `master` on `forgejo.catcrafts.net/Catcrafts/Crafter.Asset`. Agent branches are named `claude/<topic>`. Pushes go over SSH: the push URL is `forgejo@forgejo.catcrafts.net:Catcrafts/Crafter.Asset.git`, the fetch URL HTTPS.
|
|
||||||
- Subject: `type(scope): imperative summary` with `feat`, `fix`, `refactor`, `docs` or `test`, the partition as the scope (`glb`, `texture`, `compression`, ...), as the recent commits do; the older one-word subjects aren't the model. The body says what changed, grouped, and why. `git log -2` shows the shape.
|
|
||||||
- A change to an interface lands on every consumer's next build, since Crafter.Graphics and crafter-build pull `master` unpinned, and crafter-build links this library into itself. Build a consumer against the change before merging: Crafter.Graphics with `--local` from its checkout, or crafter-build's self-host build. What crafter-build calls: `TextureAsset<Vector<std::uint8_t, 4, 4>>::LoadPNG<std::uint8_t>`, `Resize`, `SaveCompressed`; `MeshAsset<VertexNormalTangentUVPacked>::LoadOBJ`, `LoadOBJSplit`, `SaveCompressed`; `LoadMTL`.
|
|
||||||
- Update [AGENTS.consumer.md](AGENTS.consumer.md) in the same change when the API, a file format, a limit or an error behaviour changes. It is what the consumers' agents read, and it reaches their AGENTS.md on their next build.
|
|
||||||
- Never commit `bin/` or `build/` (gitignored), and don't edit `lib/` outside design rule 6.
|
|
||||||
|
|
||||||
## Security considerations
|
|
||||||
|
|
||||||
- The loaders parse untrusted bytes, and the GLB loader and `LoadPNGFromMemory` are the ones that meet user content. Their defences are design rule 5; the "broken files" section of the GLB test is their regression suite. A new field read from a file gets a broken-file case.
|
|
||||||
- stb_image is the image attack surface. It is pinned at 2.30 in `lib/`; the size limits in `TextureAsset` exist because stb's own arithmetic overflows past them, and an encoded image is bounded to `INT_MAX` bytes because that is what stb takes.
|
|
||||||
- `Compression::Fatal` aborts the process. Right for build output that doesn't match the binary, wrong for anything a user can hand the program (design rule 1).
|
|
||||||
- `project.cpp` is compiled and run by crafter-build, and clones and builds Crafter.Math from its default branch on every invocation. Pin a `commit` in the `GitProject` spec if that ever matters.
|
|
||||||
- The test writes only under `fs::temp_directory_path()`.
|
|
||||||
|
|
||||||
## Gotchas
|
|
||||||
|
|
||||||
- Adding a partition is three edits: the interface file, the `ifaces` array in `project.cpp`, and the umbrella `Crafter.Asset.cppm`; an implementation unit goes into both `impls` arrays, the `--exe` branch and the library branch.
|
|
||||||
- `STB_IMAGE_IMPLEMENTATION` and `STB_IMAGE_RESIZE_IMPLEMENTATION` are defined once, in `Crafter.Asset-Texture.cppm`. Defining them anywhere else in this tree, or in a consumer, duplicates the symbols at link time. `CrafterAssetClearStbiFailure`, beside them, is a plain global function because stb's failure reason is a static only that fragment can reset; it isn't API.
|
|
||||||
- The GDeflate C++ wrappers (`GDeflateCompress.cpp`, `GDeflateDecompress.cpp`) are `#include`d into the Compression implementation unit because `cfg.cFiles` compiles only `.c` files. Don't list them in `project.cpp`.
|
|
||||||
- `LoadOBJ` deduplicates vertices with a linear scan and is quadratic, and takes only triangulated faces with `v/vt/vn` on every vertex. `LoadOBJSplit` hashes, fan-triangulates and is what crafter-build's `BuildOBJBundle` uses for Sponza-sized inputs. Fix performance there, not by speeding up `LoadOBJ`.
|
|
||||||
- `MeshAsset<void>::Load` reads a two-count header and can't read a `MeshAsset<T>::Save` file, which has three. The compressed formats share one layout, which is why the data region is always present.
|
|
||||||
- `TextureAsset::Resize` is constrained to 4-byte pixels (RGBA8); the `_Float16` and `float` textures have no resize.
|
|
||||||
- The GLB loader decodes every image at load, eagerly, and refuses images with a `uri` (external file or data URI): a GLB is expected to carry its images in the BIN chunk. Only PNG and JPEG are decoded, by their signatures; another format stays an empty image, and a texture pointing at one is an error.
|
|
||||||
- There is no wasm test because wasm runs no tests here; a wasm-only branch is checked only by building `--target=wasm32-wasip1`.
|
|
||||||
Loading…
Reference in a new issue