## Crafter.Graphics The Crafter engine's graphics library: a C++ module, `import Crafter.Graphics;`, that opens a window, drives a Vulkan 1.3+ device through `VK_EXT_descriptor_heap` (bindless: no descriptor-set layouts, push data through `vkCmdPushDataEXT`), traces rays through `VK_KHR_ray_tracing_pipeline`, draws UI from compute shaders, and reads keyboards, mice, touch, gamepads and the clipboard. A `wasm32-*` build of the same surface runs in the browser on WebGPU with a software ray tracer. Everything is in `namespace Crafter`. The vectors and matrices are Crafter.Math's (`import Crafter.Math;`) and the compressed asset formats Crafter.Asset's (`import Crafter.Asset;`); both ship notes of their own. Window callbacks are `Crafter::Event` from Crafter.Event (`import Crafter.Event;`): `EventListener sub(&window.onMouseMove, [&] { ... });` subscribes, and the listener's destructor unsubscribes. Source, examples and issues: https://forgejo.catcrafts.net/Catcrafts/Crafter.Graphics (LGPL-3.0-only; the examples under `examples/` are MIT and meant to be copied). These notes are its `AGENTS.consumer.md`: a fix to them goes there, not into the copy crafter-build writes. ### Building against it - The target picks the backend, in the library and in your code. Any `wasm32-*` target builds the DOM + WebGPU library; everything else builds Vulkan, with Wayland on Linux (there is no X11 backend) or Win32 on `x86_64-w64-mingw32`. The library defines `CRAFTER_GRAPHICS_WINDOW_WAYLAND`, `_WIN32` or `_DOM` for its own units only, so a wasm consumer sets `cfg.target = "wasm32-wasip1"`, `cfg.defines.push_back({"CRAFTER_GRAPHICS_WINDOW_DOM", ""})`, `cfg.compileFlags.push_back("-msimd128")` (the dependency PCMs were compiled with it) and `EnableWasiBrowserRuntime(cfg)`, as `examples/HelloUI/project.cpp` does. Portable code `#ifdef CRAFTER_GRAPHICS_WINDOW_DOM`s at the few sites where the backends differ; the library has no shared signatures that are no-ops on one side. - The library's link flags (the Vulkan loader, `wayland-client`, `xkbcommon`, `udev`, or the Win32 libraries) come with the dependency. Its shaders (`ui-*.comp.spv` and `ui-*.comp.sets.spv`) and, for wasm, `dom-env.js` and `dom-webgpu.js` are copied into your executable's bin directory, as are your `cfg.shaders` outputs (`.spv` from `.glsl`) and `cfg.files`. Paths are resolved against the working directory at runtime, so run from the bin directory, as `crafter-build -r` does. - Your own shaders: `cfg.shaders.emplace_back(fs::path("raygen.glsl"), std::string("main"), ShaderType::RayGen)`, with `Compute`, `RayGen`, `Miss`, `ClosestHit`, `AnyHit`, `Intersect` and `Callable`. A compute shader for the UI heap is registered twice, see "Custom compute shaders". WGSL for the DOM build ships through `cfg.files` and is loaded at runtime. - Project args the library understands, passed through your `args`: `--local` (sibling checkouts of Crafter.Event, Crafter.Math and Crafter.Asset instead of clones), `--timing` (meant to add frame timing counters; the build does not compile with it at the moment, issue #178), `--no-webgpu` (wasm: a DOM-only page without the WebGPU bridge, which otherwise requests a GPU adapter on every load and replaces the page body with an error where WebGPU is missing). - Needs at runtime: Vulkan 1.3 (1.4 is used where the GPU has it), a presentable surface, and the features the library's own shaders use. `VK_EXT_descriptor_heap` and ray tracing are optional groups, see "Device and features". Mesa's llvmpipe runs the compute/UI path, and recent Mesa advertises the heap. ### A program, and what the loop does ```cpp Device::Initialize({.vulkan10 = {.shaderInt16 = VK_TRUE}}); // the features your shaders use; on the DOM build Initialize() takes none Window window(1280, 720, "Title"); // `static` on the DOM build VkCommandBuffer init = window.StartInit(); // a one-time setup command buffer DescriptorHeapVulkan heap; heap.Initialize(/*images*/ 16, /*buffers*/ 16, /*samplers*/ 4); window.descriptorHeap = &heap; // bound at the start of every frame // create buffers, images, meshes, pipelines and passes, recording into `init` ... window.passes.push_back(&pass); // RenderPass*: a UIRenderer, an RTPass, your own window.FinishInit(); // submits `init` and waits for it window.Render(); // one frame window.StartUpdate(); // frames on the compositor's clock from here window.StartSync(); // the event loop; never returns ``` - Include `vulkan/vulkan.h` yourself, before the imports (the examples wrap it in `#ifndef CRAFTER_GRAPHICS_WINDOW_DOM`): the library includes it only in its own global module fragments, and neither Vulkan's types nor its macros (`VK_TRUE`, the `VK_*` flags) cross an `import`. On the DOM build `Device::Initialize()` takes no arguments and `DeviceFeatures` does not exist, which is why the dual-target examples keep a separate DOM `main`. - `StartSync` runs the platform loop until `open` is false (`onClose` fires first). Each pass of the loop dispatches platform events into the `Window` events, runs `Gamepad::Tick()` and `onBeforeUpdate`, and runs `Update()` for a window that is due. On Wayland a window on the compositor's clock updates from its frame callback during that event dispatch, before the pass's `onBeforeUpdate`, and the loop wakes for every event, so `onBeforeUpdate` fires several times per frame; on Win32 and in the browser `onBeforeUpdate` precedes the same pass's `Update`. `updateInterval`, set before `StartUpdate`, makes a window update every so often whether visible or not; pair it with `SetPresentMode(PresentMode::Mailbox)` so a present never waits for the display. `AddPollFd(fd, callback)` (Wayland) wakes the loop for a socket or an eventfd. - `Update()` computes `mouseDelta` and calls `Render()`, which waits for the fence of its frame slot (three frames in flight, `Window::numFrames`; the slot is `window.currentFrame`), acquires a swapchain image (`window.currentImage`; the swapchain has as many images as its present mode needs, not necessarily three), writes it into this window's swapchain slots of the heap, reclaims deferred deletions, begins `drawCmdBuffers[currentFrame]`, binds the heap (or the descriptor set), fires `onUpdate(FrameTime)`, records every pass in `passes` in order with a barrier on the swapchain image between them, transitions the image to present, submits and presents. Per-frame GPU work of your own (a TLAS rebuild, a physics dispatch) is recorded from `onUpdate` into `window.drawCmdBuffers[window.currentFrame]`; the heaps are bound by then and the passes come after. A `Resize` or `SetPresentMode` from a listener of the frame (an input action ticked in `onUpdate`, say) takes effect after its present. `onResize` fires after a swapchain rebuild (a new size or present mode); the swapchain slots follow by themselves, and anything else that holds the old images or is sized after the window is made again there. - A `RenderPass` is `Record(cmd, frameIdx, window)` plus `SwapchainStage()`, which must return the stage that touches the swapchain image (`COMPUTE_SHADER`, `RAY_TRACING_SHADER` or `TRANSFER`): the frame's barriers take their stage and access masks from it. `frameIdx` is the frame slot, `window.currentFrame`, not the image: the frame's image is `window.images[window.currentImage]`. The image stays in `VK_IMAGE_LAYOUT_GENERAL` through the whole pass list, with `STORAGE | TRANSFER_SRC` usage and `TRANSFER_DST` where the surface allows it (`window.swapchainUsage`). Nothing clears it: draw a background quad, or clear from a transfer pass where `TRANSFER_DST` is there. - `window.width` and `height` are physical pixels, as is `currentMousePos`. `CaptureNextFrame(path)` saves the next frame as a PNG. `SetTitle`, `SetAppId`, `SetCursorImage`, `SetDefaultCursor`; `SetTextInput(active, x, y, w, h, purpose)` tells the compositor's input method a text field has focus, which opens an on-screen keyboard on a phone. - Several windows can exist (on Wayland `Device::windows` lists them; the other backends keep no list), and may share a heap, each writing only its own swapchain slots, but they share one frame clock: one rendering window is the assumption behind the deletion queue, the staging rings and the font atlas's evictions. Without a `Window`, nothing advances `Device::frameCounter`: a program that draws text without one steps it itself after each frame's work, or its full atlas never frees a cell and new glyphs stay blank. ### Device and features - `Device::Initialize(required = {}, optional = {})` creates the instance, weighs every GPU and creates the device. `DeviceFeatures` holds the five core feature structs, `vulkan10` to `vulkan14`; only their `VkBool32`s are read, so `{.vulkan12 = {.shaderFloat16 = VK_TRUE}}` is a complete argument. The library enables what its own shaders use and nothing more: a `uint16_t` spec constant needs `.vulkan10.shaderInt16`, fp16 arithmetic `.vulkan12.shaderFloat16`, 16-bit storage `.vulkan11.storageBuffer16BitAccess`, scalar layout `.vulkan12.scalarBlockLayout`. A GPU lacking a required feature is skipped; when none is left the call throws naming each GPU's gaps, and may be called again asking for less. Optional features are enabled where the GPU has them; `Device::enabledFeatures` says which, `supportedFeatures` what the GPU has, `apiVersion` the version in use (`vulkan14` features need a 1.4 GPU; `maintenance5` is always on). `CRAFTER_GRAPHICS_DEVICE=` picks GPU `n`. - `VulkanShader` checks every module's SPIR-V capabilities against what was enabled and throws naming the file and the field to pass (`vulkan10.shaderInt16`), or saying the GPU lacks it. Most drivers accept such a module silently, so this is the error you get here instead of a wrong frame on another machine. - Optional groups, decided once at `Initialize`: `Device::descriptorHeapSupported` (`VK_EXT_descriptor_heap`), `rayTracingSupported` (needs the heap too), `memoryDecompressionSupported` (`VK_EXT_memory_decompression`: `.cmesh` and `.ctex` decompress on the GPU; otherwise the library decodes them on the CPU), `deviceFaultSupported`, `swapchainMaintenance1Supported` (`VK_KHR_swapchain_maintenance1`, with the instance's `surface_maintenance1`, `surfaceMaintenance1Supported`: a swapchain names its present mode and gets that mode's own image count; otherwise the stricter of the mode's and the surface's). The RT entry points throw through `Device::RequireRayTracing` without ray tracing; the compute/UI path runs everywhere. - `Device::device`, `queue`, `commandPool`, `physDevice`, `instance`, `memoryProperties`, `deviceProperties`, `rayTracingProperties` and `descriptorHeapProperties` are public statics for raw Vulkan. `Device::GetMemoryType(typeBits, required, preferred = 0)` and `PreferDirectDeviceWrite(bytes)` are the allocation helpers. - `Device::pipelineCache` is shared by every pipeline and written to `Device::pipelineCachePath` (`pipeline_cache.bin` in the working directory) at exit. Set the path before `Initialize`, and gitignore the file. ### Lifetimes Frames are pipelined three deep and nothing waits for the GPU at steady state, so: - Every library object that owns a Vulkan handle (`VulkanBuffer`, `Mesh`, `TlasWithBuffer`, `VulkanShader`, `ComputeShader`, `PipelineRTVulkan`, and `ImageVulkan`'s staging buffer) is move-only and releases the handle through `Device::EnqueueDeletion`; the render loop frees it `Device::framesInFlight` frames later. Destroying one mid-frame is safe, and so is destroying it from another thread. `VulkanBuffer::Clear()` frees at once and is only for a GPU known to be idle; `DeferredClear()` is the queued form. A program with no `Window` reclaims nothing on its own: call `Device::DrainDeletions()` after `vkQueueWaitIdle(Device::queue)`. - Static owners (a `static Mesh`, a `static VulkanBuffer`) are fine: their destructors run after the driver has unloaded, and the queue allows for that by never being destroyed itself. `RenderingElement3D::tlases` is leaked for the same reason. - `Window` is neither movable nor copyable, and its destructor waits for the device, so a window can be closed and another opened. - `ImageVulkan` has no destructor: call `Destroy()` once the GPU is done with it. `FontAtlas` waits for the queue and destroys its own image. - Heap slots (`ImageSlot`, `BufferSlot`, `SamplerSlot`) are RAII handles that return the slot on destruction, and a returned slot is handed out again at once. Free a slot only when no frame in flight reads it (after a `vkQueueWaitIdle`, or at a scene switch), and before its heap goes. In descriptor-set mode a slot write destroys the previous view immediately: same rule. - `RenderingElement3D::elements` holds raw pointers: `Add`/`Remove` elements whose storage does not move (reserve the vector, or make them `static`). `UIRenderer::ShapeText` and `FontAtlas` cache by `Font*`: call `ui.InvalidateFont(font)`, which clears both, before destroying or replacing a `Font` (the atlas keeps the `Font*` of a glyph waiting for a cell until the next frame). ### The bindless heap `DescriptorHeapVulkan::Initialize(images, buffers, samplers)` sizes one resource heap and one sampler heap per frame in flight (`resourceHeap[Window::numFrames]`, `samplerHeap[...]`, mapped `VulkanBuffer`s) and the slot allocators. `AllocateImageSlots(n)`, `AllocateBufferSlots(n)` and `AllocateSamplerSlots(n)` return `{firstElement, count}` ranges (contiguous; freed runs are reused), `Free*Slots` returns them, and running out throws naming the pool and the counts. `Initialize` throws too when its buffer slots would need shader-visible indices past 16 bits. - **Buffer slot indices are relative.** `AllocateBufferSlots` indexes the heap's buffer region, which starts after the image region, while a GLSL `descriptor_heap` SSBO array indexes buffer descriptors from heap byte 0. The shader-visible index is `heap.bufferStartElement + raw`. The RAII `BufferSlot` converts to exactly that (its `operator std::uint16_t()`), `UIRenderer::RegisterBuffer` returns one, and raw code passes `heap.bufferStartElement` as a spec constant: the RT examples declare `layout(constant_id = 0) const uint16_t bufferStart = 0us;` and index `topLevelAS[bufferStart]`, which is why they ask for `shaderInt16`. Image slots index the heap as they are, in image-descriptor units; samplers live in their own heap. A wrong buffer index reads zeros, which looks like every field of every item being zero. In descriptor-set mode `bufferStartElement` is 0, so code that goes through the conversion is right in both. - Writing a descriptor by hand: `Device::vkWriteResourceDescriptorsEXT` into `heap.resourceHeap[f].value + heap.ImageByteOffset(imageRange.firstElement)` or `heap.BufferByteOffset(bufferRange.firstElement)`, `Device::descriptorHeapProperties.imageDescriptorSize` or `bufferDescriptorSize` bytes long, for each frame `f`, then `heap.resourceHeap[f].FlushDevice()`. `BufferByteOffset` takes the raw index `AllocateBufferSlots` returned and adds the region start itself; handing it a `BufferSlot`, which converts to the absolute index, adds the start twice and puts the descriptor in the wrong place. `ImageByteOffset` takes an image slot as it is. Per-frame resources (`tlases[f]`) are written into frame `f`'s heap; a shared resource into every frame's. The swapchain image is neither, since frames and images pair up differently from frame to frame: list its image slot once with `heap.AddSwapchainSlot(slot, window)`, and the window writes the image each frame draws into there, a storage image in `GENERAL`, in both binding models and across resizes (a slot listed during a frame is written for that frame). `examples/VulkanTriangle/main.cpp` is the pattern. The `UIRenderer::Register*` helpers do this for you in both binding models. - Binding the heap outside the frame loop: `Window::Render` binds `window.descriptorHeap` in every frame it records. A command buffer it does not record that dispatches against the heap (`window.StartInit()`'s, say) calls `heap.Bind(cmd, frame)`, which binds that frame's heaps with their reserved ranges, or in descriptor-set mode its descriptor set. Don't build the `VkBindHeapInfoEXT` by hand: where the reserved range sits is the heap's to decide. - **Descriptor-set fallback.** On a driver without `VK_EXT_descriptor_heap` (Mesa Turnip on Adreno) `Device::descriptorHeapSupported` is false and the same slots live in one bindless descriptor set per frame; slot handles, item structs, `UIDispatchHeader` and the `UIRenderer` API are unchanged, so application code does not branch. Ray tracing is unavailable there. `CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP` forces this path on a driver that has the heap, to test it. Raw `vkWriteResourceDescriptorsEXT` code is heap-only; a consumer that must run on such a driver stays on the `UIRenderer::Register*` helpers. ### Buffers, images, samplers - `VulkanBuffer`: `Create(usage, memoryFlags, count, preferredFlags = 0)`; `Resize(...)` reuses the allocation when the request fits and the usage matches, otherwise queues the old one and creates; `size` (bytes), `buffer`, `address` (non-zero only with `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT`), `memoryPropertyFlagsChosen`. With `Mapped`, `value` is the `T*` of a persistent map: after writing, call `FlushDevice()` (whole buffer), `FlushDevice(offset, bytes)` (a range, rounded to `nonCoherentAtomSize`) or `FlushDevice(cmd, dstAccess, dstStage)`, which flushes and records the host-write barrier; all three are no-ops on coherent memory. `FlushHost()` is the invalidate for readback. A UI item buffer is `GraphicsBuffer` with `STORAGE_BUFFER | SHADER_DEVICE_ADDRESS` usage and `HOST_VISIBLE` memory with `DEVICE_LOCAL` as the preferred fourth argument (the examples pass both as required, which throws on a GPU without a host-visible device-local type), flushed each frame before the dispatch that reads it. - Static geometry: `VulkanBuffer::UploadDeviceLocal(usage, src, count, cmd, dstAccess, dstStage)` places the buffer in device memory, written directly on ReBAR and UMA and staged through a per-frame staging ring on a small BAR, and records the barrier to `dstStage`. `UploadDeviceLocalRange(src, offset, count, cmd, dstAccess, dstStage)` re-uploads the dirty window `[offset, offset + count)`; `src` points at the first dirty element (`vertices.data() + offset`), not at the array's start. A re-upload into a buffer a frame may still read is ordered after those reads on the GPU. - `ImageVulkan` (`Image2D` is the portable alias): `Create(width, height, mipLevels, cmd, format, usage, layout, consumerStage = RAY_TRACING_SHADER, streamed = false)` makes the image and a mapped staging buffer `image.buffer`; fill `image.buffer.value` and `Update(cmd, layout)` copies it in, builds the mip chain by blits and transitions to `layout` for `consumerStage`. `Update(const CompressedTextureAsset&, cmd, layout)` decompresses a `.ctex` on the GPU, or the CPU. A static image releases its staging after the first upload; `streamed = true` keeps it (the font atlas, through `UpdateRegion`). Pass `VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT` as `consumerStage` for an image a compute shader samples; the RT default is corrected to compute on a device without ray tracing. `image`, `imageView` and `descriptor` are public for raw descriptor writes; `Destroy()` releases. - `SamplerVulkan` makes one linear, repeat, 16x anisotropic sampler at construction (`textureSampler`). The UI has `UIRenderer::RegisterSampler(const VkSamplerCreateInfo&)` and `RegisterLinearClampSampler()`. ### UI Three tiers, on purpose: Tier 1 is any compute shader you write (`ComputeShader`, `UIRenderer::Dispatch`); Tier 2 the four standard kernels, their item structs and the `UIRenderer` helpers; Tier 3 stateless functions that append items to your buffers. The library has no widget tree, layout engine, theme, hit-testing or focus management: state lives in your PODs, hit-testing is `Rect::Contains` against the rects you drew, and a Tier 3 component that does not fit is copied out of `implementations/Crafter.Graphics-UIComponents.cpp` and edited, not subclassed. Setup: `FontAtlas atlas; atlas.Initialize(init);` then `UIRenderer ui; ui.fontAtlas = &atlas; ui.Initialize(window, heap, init); window.passes.push_back(&ui);`. `Initialize` loads the five kernels by file name from the working directory (`ui-quads.comp.spv` and so on; the `.sets.spv` builds must sit beside them) and registers the swapchain images and the atlas. Then, per item buffer, `auto slot = ui.RegisterBuffer(buf);` gives a `BufferSlot` ready for a shader. Per frame, in an `EventListener` on `ui.onBuild` (fired from the pass's `Record` with `{cmd, frameIdx}`): 1. Rebuild the item arrays in your mapped buffers, in draw order: later items overdraw earlier ones. Lay out with `Rect::FromWindow(window)`, `SubRect(size, Anchor)` and `Inset(...)`. Keep one set of item buffers (and slots) per frame in flight, `Window::numFrames` of them, and fill the one at `frameIdx`: a frame still on the GPU reads its items as it runs, and a later frame's items written into the same buffer can name atlas cells that frame drew other glyphs in (`examples/HelloUI`). 2. `buf.FlushDevice(cmd, VK_ACCESS_SHADER_READ_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT)` for each buffer with items. 3. Dispatch: `ui.DispatchQuads(cmd, slot, count, clipRectPx = {0, 0, 1e9, 1e9})`, `DispatchCircles`, `DispatchImages`, `DispatchText`, or `ui.DispatchFused(cmd, {quadsSlot, qc}, {circlesSlot, cc}, {imagesSlot, ic}, {glyphsSlot, gc})`, which composites the four categories in that fixed order (quads, circles, images, text) with one load and one store; an empty category costs nothing. The two forms mix; a custom `ui.Dispatch` between two `DispatchFused` calls is the flush point. - The item structs are `std430` PODs mirrored byte for byte in `shaders/ui-shared.glsl`, and frozen: `QuadItem` (64 bytes: `x, y, w, h`, `r, g, b, a`, corner radii `cTL, cTR, cBR, cBL`, `outline, oR, oG, oB`), `CircleItem` (48: `cx, cy, radius, _p0`, colour, outline), `ImageItem` (64: rect, `u0, v0, u1, v1`, tint, `texSlot, sampSlot, _p1, _p2`), `GlyphItem` (48: rect, atlas uv, colour). Colours are straight alpha, blended source-over; coordinates are window pixels. - Cost: every kernel dispatches one workgroup per 8x8 screen tile and each pixel walks every item (behind a per-tile cull), so a dispatch is `O(W*H*N)`, fine up to a few hundred items at 1080p. Splitting one buffer into several dispatches does not help. Thousands of items want a different shader (tile binning), which is Tier 1. - Text: `Font font("font.ttf")` (TrueType through stb_truetype); `font.GetLineWidth(text, px)`, `LineHeight(px)`, `AscentPx(px)`, `NearestCursorByte(text, px, x)`. The atlas is a 1024 by 1024 SDF atlas at a 32 px base size, filled lazily per codepoint and re-uploaded by dirty rectangle. Glyphs scale from it with the edge ramped over one screen pixel at any size and aspect, on both backends; far above 32 px corners round off and fine detail goes, and below about 4 px the ramp narrows and thin strokes bolden. `ui.ShapeText(font, px, x, baselineY, utf8, color, out, capacity, &advance, TextAlign::Left)` writes `GlyphItem`s: single line, no kerning, no wrapping, a shaped-run cache behind it. Once the atlas is full, a glyph that finds no free room draws blank for one frame but keeps its advance, and gets a cell when the next frame begins, taken from the glyphs drawn least recently, never one drawn in the last `Device::framesInFlight + 1` frames. So a `GlyphItem` holds for the frame it was shaped for: shape text every frame (the Tier 3 components do, and a cached run costs a copy) rather than drawing a glyph buffer filled in an earlier frame. `atlas.EnsureAndGet(font, codepoint)` returns `const Glyph&` and never fails; its UVs hold only when `Resident()`, and a `Lookup` pointer only until the atlas's next `EnsureAndGet`, `Touch` or `Update`. A Tier 1 kernel that samples `FontAtlasImageSlot()` through `ui.Dispatch` sees the frame's new glyphs: every dispatch uploads the atlas first. - Tier 3: fill a `UIBuffer` (`quads`, `quadCount`, `quadCap`; `glyphs`, `glyphCount`, `glyphCap`; `images`, `imageCount`, `imageCap`; `atlas`; `renderer`; a null pointer makes the components that need it no-ops) and call `DrawButton(buf, rect, label, hovered, pressed, font, px, ButtonColors)`, `DrawCheckbox(buf, rect, checked, hovered, CheckboxColors)`, `DrawSlider(buf, rect, t01, dragging, SliderColors)`, `DrawProgressBar(buf, rect, t01, ProgressColors)`, `DrawText(buf, text, x, baselineY, font, px, color, align)` (returns the advance) and `DrawImage(buf, rect, textureSlot, samplerSlot, tint, uv)`. Colours are small `*Colors` aggregates you hold; there is no `Theme`. - `InputField` is the model for a stateful component: a POD (`value`, `type`, `focused`, `cursorPos`), free functions you call from your own listeners (`InputField_OnText(field, utf8)` from `onTextInput`, `InputField_OnKey(field, code)` from `onRawKeyDown`, `InputField_HitTestCursor(field, rect, clickX, font, px, colors)`), `InputField_TryParse(field)`, and `DrawInputField(buf, field, rect, font, px, colors, caretVisible)`; the blink is yours. - Images in the UI: `auto tex = ui.RegisterImage(image, VK_FORMAT_R8G8B8A8_UNORM);` (an `ImageSlot`; the layout defaults to `SHADER_READ_ONLY_OPTIMAL`), `auto samp = ui.RegisterLinearClampSampler();`, then both go into `ImageItem::texSlot` and `sampSlot`. `ui.OutImageSlot()`, `FontAtlasImageSlot()` and `FontAtlasSamplerSlot()` expose the renderer's own slots. ### Custom compute shaders (Tier 1) A GLSL compute shader that dispatches beside the standard kernels follows `shaders/ui-shared.glsl`, which the build puts on the include path: ```glsl #version 460 #extension GL_GOOGLE_include_directive : enable #include "ui-shared.glsl" struct MyItem { vec4 a; }; // mirrored in C++ with a static_assert on its size layout(UI_HEAP_BUFFERS, std430) readonly buffer MyBuf { MyItem items[]; } myHeap[]; layout(push_constant) uniform PC { UIDispatchHeader hdr; /* your fields after it */ } pc; layout(local_size_x = 8, local_size_y = 8, local_size_z = 1) in; void main() { ivec2 px; if (!uiResolveScreenPixel(pc.hdr, px)) return; // read myHeap[pc.hdr.itemBuffer].items[i]; imageLoad/imageStore uiImages[pc.hdr.outImage] } ``` - Declare bindless resources with `UI_HEAP_IMAGES`, `UI_HEAP_TEXTURES`, `UI_HEAP_SAMPLERS` and `UI_HEAP_BUFFERS`, never a literal `descriptor_heap` or a set and binding, and register the source twice in `project.cpp`: once plain, once with `sets.defines = {"CRAFTER_GRAPHICS_DESCRIPTOR_SETS"}; sets.variant = "sets";`, which yields `.spv` and `.sets.spv`. `ComputeShader::Load(".spv", heap)` picks the build the device needs and throws naming a missing file. `examples/CustomShader/project.cpp` has the lines. - `UIDispatchHeader` (48 bytes; in C++ `outImage, itemBuffer, surfaceWidth, surfaceHeight, clipX, clipY, clipW, clipH, itemCount, frameIdx, flags, _pad`, and in GLSL the same bytes as `uint outImage, itemBuffer; uvec2 surfaceSize; vec4 clipRectPx; uint itemCount, frameIdx, flags, _pad`) must be the first member of the push block; `ui.FillHeader(slot, count, clipRectPx, flags)` fills it. The high bit of `flags` (`kUIFlagClip`) is the renderer's, the rest are yours. Keep the push block a multiple of 4 bytes and within 256; the descriptor-set path throws past the device's `maxPushConstantsSize`. - `ui.Dispatch(cmd, shader, &pc, sizeof(pc), ceil(W / 8), ceil(H / 8))` binds, pushes and dispatches, with a `SHADER_WRITE -> SHADER_READ | SHADER_WRITE` barrier on the swapchain image before every dispatch but the frame's first, so a custom shader sees what the dispatch before it drew. `ComputeShader::Dispatch(cmd, push, bytes, gx, gy, gz, tlasAddress = 0)` is the same without the barrier; a compute shader that ray-queries the TLAS passes `RenderingElement3D::tlases[frameIdx].address`, for the NVIDIA workaround below. - A shader that draws text from the font atlas (`ui.FontAtlasImageSlot()` and `FontAtlasSamplerSlot()`), a copy of `ui-text.comp.glsl` among them, decodes it with `ui-shared.glsl`'s helpers rather than arithmetic of its own: `uiGlyphAtlasSize(texSlot, samplerSlot)`, then `uiGlyphRamp(item.uv, 1.0 / item.rect.zw, atlasSize)` once per glyph and `uiGlyphCoverage(texSlot, samplerSlot, uv, ramp, atlasSize)` per pixel, with dynamically uniform slots. A texel holds stb_truetype's `UI_SDF_ON_EDGE + UI_SDF_PIXEL_DIST_SCALE * d` (128 + 32 d), `d` the distance from the outline in atlas pixels, positive inside, truncated to a byte; `uiSdfDistance(sample)` gives `d` back. Multiplying a sample by 32 instead of dividing by 32/255 gives a ramp a quarter of a pixel wide, and pixelated text. - Keep the one-workgroup-per-screen-tile shape for anything that composites. A workgroup per item races on overlapping pixels: Vulkan orders nothing between the workgroups of one dispatch. - A `ComputeShader` can also run outside the UI pass, dispatched from `onUpdate` into the frame's command buffer with barriers of your own. ### Ray tracing (Vulkan) Heap-only: check `Device::rayTracingSupported`, or expect `std::runtime_error` from the first RT call. `examples/VulkanTriangle/main.cpp` is the whole setup. - Shaders: `VulkanShader shader("raygen.spv", "main", VK_SHADER_STAGE_RAYGEN_BIT_KHR, &specInfo);` (`nullptr` for no specialisation); `ShaderBindingTableVulkan table; table.Init(shaders);`; `PipelineRTVulkan pipeline; pipeline.Init(cmd, raygenGroups, missGroups, hitGroups, table, maxRecursionDepth = 1);` with spans of `VkRayTracingShaderGroupCreateInfoKHR` indexing the shader array. **`maxRecursionDepth` counts the raygen as 1**: a closest-hit that traces a shadow ray needs 2, and tracing past the pipeline's depth loses the device. - Geometry: `Mesh mesh; mesh.Build(std::span> vertices, std::span indices, cmd, RTBuildOptions{});` builds a BLAS and sets `mesh.blasAddr`; `Build(const CompressedMeshAsset&, cmd)` builds from a `.cmesh`, decompressed on the GPU where supported; `BuildProcedural(aabbs, opaque, cmd)` builds from `RTAabb` boxes for a `VK_RAY_TRACING_SHADER_GROUP_TYPE_PROCEDURAL_HIT_GROUP_KHR` with an intersection shader, and has an overload taking a device address a compute pass wrote. `RTBuildOptions` is `.preference` (`FastTrace`, `FastBuild`), `.allowUpdate` and `.noDuplicateAnyHit`; with `allowUpdate`, `Refit(vertices, indices, cmd)` and its dirty-range overload update in place and keep `blasAddr`. A rebuild into storage that still fits keeps `blasAddr` too; read it again after a rebuild that grows. - Instances: a `RenderingElement3D` holds a `VkAccelerationStructureInstanceKHR instance` (`transform`, `mask`, `instanceCustomIndex`, `instanceShaderBindingTableRecordOffset`, `flags`, `accelerationStructureReference = mesh.blasAddr`; Crafter.Math's `MatrixRowMajor::Store(reinterpret_cast(instance.transform.matrix))` writes the transform), `userMetadata` (copied per instance id into `tlases[f].metadataBuffer`) and `transformOwnedByGpu`. `RenderingElement3D::Add(&e)` and `Remove(&e)` register it. `RenderingElement3D::BuildTLAS(cmd, frameIndex, preference = FastTrace)` builds or refits `tlases[frameIndex]` from every registered element: once per frame slot at setup (`0`, `1`, `2`), and per frame from `onUpdate` for `window.currentFrame` when instances move. `e.MarkHostDataDirty()` after a change to an element's host fields lets the per-frame copy skip unchanged slots. `tlases[f].address` goes into a `VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR` descriptor of frame `f` (see "The bindless heap") and changes only when the TLAS outgrows its storage. - The pass: `RTPass rtPass(&pipeline); rtPass.pushPtr = &camera; rtPass.pushBytes = sizeof(camera);` pushes your data at offset 0 of the shaders' push block before every trace (the bytes are read at record time, so updating `camera` before the frame is enough); `width` and `height` trace at a size other than the window's. `window.passes.push_back(&rtPass);`. - In GLSL, with the extensions the examples enable (`GL_EXT_ray_tracing`, `GL_EXT_descriptor_heap`, `GL_EXT_nonuniform_qualifier` for a non-constant heap index, `GL_EXT_shader_explicit_arithmetic_types_int16` for a `uint16_t` spec constant, and `GL_EXT_shader_image_load_formatted` or a `writeonly` image): `layout(descriptor_heap) uniform accelerationStructureEXT topLevelAS[];`, `layout(descriptor_heap) uniform writeonly image2D image[];`, indices from spec constants or push data. The output image is the swapchain image at an image slot listed with `AddSwapchainSlot`, in `VK_IMAGE_LAYOUT_GENERAL`. Start from `examples/VulkanTriangle/raygen.glsl`. - **NVIDIA.** On the proprietary driver (still on `615.78.08`) reading an acceleration structure through the heap loses the device. The library works around it transparently: `VulkanShader` rewrites the SPIR-V so the heap AS read becomes a TLAS-address load from the push block plus `OpConvertUToAccelerationStructureKHR`, and `RTPass` and `ComputeShader` push the address. Nothing changes in your shaders or code, except that the rewrite appends an 8-byte member after your push block, and that it turns `shaderInt64` on for itself only: a shader of yours that declares `Int64` still has to ask for `shaderInt64`, or it runs on NVIDIA alone. `CRAFTER_GRAPHICS_NO_AS_WORKAROUND` disables the rewrite, to probe a new driver. Second note: on `610.43.02`, indexing a `descriptor_heap` array with a runtime value inside a hit shader faulted (fixed in `615.78.08`). The portable pattern, which the Sponza example uses, is one descriptor at a spec-constant slot and dynamic indexing inside it: all geometry in one SSBO indexed by element offset, materials in one `texture2DArray` indexed by layer. `examples/Sponza/README.md` has the GLSL. ### Input - `Window` events, `Event` unless noted: `onMouseMove` (then `currentMousePos`, `lastMousePos`, `mouseDelta`), `onMouseEnter` and `onMouseLeave` with `mouseInside` (native only: the DOM build declares the two events but never fires them, and has no `mouseInside`), `onMouseLeftClick`, `onMouseLeftHold`, `onMouseLeftRelease` and the right-button trio (`mouseLeftHeld`, `mouseRightHeld`), `onMouseScroll` (`Event`: whole detents, reinterpret the payload as `int32`, positive is wheel down, every backend normalised to one per notch), `onRawKeyDown`, `onRawKeyHold`, `onRawKeyUp` (`Event`, raw platform codes; `heldKeys`), `onTextInput` (`Event`, UTF-8; repeats are synthesised on Wayland), `onPreedit` and `onTextInputDelete` (input methods), `onTouchDown`, `onTouchMove`, `onTouchUp` (`Event`; the first finger also drives the pointer events), `onClose`, `onResize`, `onBeforeUpdate`, `onUpdate` (`Event`: `now`, `delta`). - Keys: `Key(CrafterKeys::Space)` is `consteval` and folds to the platform's raw code (a Win32 scancode, a Linux keycode), so bindings stay portable in source and compare against `onRawKeyDown`'s `KeyCode` at runtime. Bindings serialised with `Input::BindingToString` are platform-specific. - `Input::Map`: `auto& jump = map.AddAction("Jump", Input::ActionType::Button);` (`Button`, `Axis1D`, `Vector2`), `jump.bindings = { Input::KeyBind{Key(CrafterKeys::Space)}, Input::GamepadButtonBind{id, Gamepad::Button::South} };` (also `MouseButtonBind`, `MouseScrollBind`, `MouseDeltaBind{scale}`, `GamepadAxisBind`, `GamepadStickBind{id, Gamepad::Stick::Left}`, `WASDBind{up, down, left, right}`), `map.Attach(window);`, and `map.Tick();` once per frame, after `Gamepad::Tick()`, which the loop runs before `onBeforeUpdate`. Every `Tick` recomputes the mouse delta and drains the scroll accumulator, and on Wayland `onBeforeUpdate` fires on every loop wake, so either tick from `onUpdate`, or tick in `onBeforeUpdate` and consume the delta and scroll actions in that same listener, as the Sponza example does. Read `action.pressed`, `value` or `vector2`, or subscribe to `onPerformed`, `onCanceled`, `onValueChanged`, `onVector2Changed`. `StartRebind(action, CaptureMask, callback)` captures the next input. - `Gamepad::connected`, `onConnected`, `onDisconnected`, per-device `buttons[]`, `axes[]` and events, `Rumble(dev, low, high, duration)`, `FindById(id)`; buttons are named by position (`South`, `East`, `West`, `North`). `Clipboard::SetText(text)` and `GetText()` (`std::optional`). ### WebGPU and the DOM build The same C++ surface, with these differences: - The `Window` is the page, and `Device::Initialize()` takes no arguments. Make the window `static`: `StartSync` hands control to `requestAnimationFrame` and `_Exit(0)`s without running static destructors, so nothing after it runs on any target. `StayAlive` is `StartSync` without the animation loop, for pages that only touch the DOM, and is `StartSync` natively. `Window::numFrames` is 1, so `tlases[0]` and `BuildTLAS(cmd, 0)`. `GraphicsCommandBuffer` is an opaque `std::uint32_t`. - `GraphicsBuffer::Create(count)`, `FlushDevice()` without a command buffer (`queue.writeBuffer`), `FlushDeviceRange`, `FlushDeviceSlot`, and `EnqueueReadback` with `PollReadback` for GPU to CPU. `Image2D::Create(w, h)` is `rgba8unorm`, `Update(const CompressedTextureAsset&)` decompresses on the CPU, `AllocateSlot(heap)` registers it; `Image2DArray` stacks layers with a CPU mip chain; `StorageImage2D` is a GPU-written float texture. `AllocateLinearClampSampler(heap)` and `AllocateLinearRepeatSampler(heap)`. - `DispatchImages` has a six-argument overload with one texture and one sampler for the whole dispatch (WebGPU bind groups); the four-argument overload binds the font atlas instead. `DispatchFused` issues the four kernels in order through the four-argument overloads, so quads, circles and text match the Vulkan result but an images category does not: draw images with the six-argument `DispatchImages` on this backend. - Custom compute in the UI pass: `WebGPUComputeShader::Load(wgslPath, bindings, rayQuery = false)`, where each `UICustomBinding` is `{group >= 2, binding, kind, format, pushOffset}` with `kind` one of `Buffer`, `SampledTexture`, `Sampler`, `SampledTextureArray`, `BufferReadWrite`, `StorageTexture`; the renderer reads the slot at `pushOffset` out of your push data and binds that handle. Group 0 binding 0 is the header uniform and group 1 the ping-pong images, both declared by your WGSL; with `rayQuery` true the library injects its own group 1 (the RT heap and the `rayQuery*` API) and your shader must not declare one. Standalone compute: `PlainComputeShader::Load(wgsl, pushUniformSize, bindings, rayQuery)` and `Dispatch(push, bytes, handles, gx, gy, gz)` run anywhere in the frame, `handles` parallel to `bindings`. It has no library bind groups: group 0 binding 0 is your optional push uniform (`pushUniformSize` bytes, 0 for none) and your bindings start at group 1, or at group 2 with `rayQuery`, which reserves group 1 the same way. - Ray tracing is a library-built wavefront tracer composed from your WGSL stages: `WebGPUShader(fs::path("raygen.wgsl"), entryFn, WebGPURTStage::Raygen | Miss | ClosestHit | AnyHit | Intersection | Resolve)` (wrap the path: a bare string literal is ambiguous with the constructor that takes the source itself), with the entry signatures documented in `interfaces/Crafter.Graphics-ShaderBindingTableWebGPU.cppm`; `ShaderBindingTableWebGPU::Init`, `RTShaderGroup{.type = RTShaderGroupType::General | TrianglesHitGroup | ProceduralHitGroup, ...}`, `PipelineRTWebGPU::Init(cmd, raygen, miss, hit, sbt, bindings, hdrOutputFormat)`. Raygen calls `rtEmitPrimaryRay`; closest-hit and miss run in the SHADE stage and call `rtEmitRay` (continuation and shadow rays) and `rtAccumulate` (atomic, any number of rays per pixel). `RTPass` adds `maxDepth` (bounces), `raysPerPixel` (the ray buffer budget), `handlesPtr` and `handlesCount` (your bindings' handles) and `outTexHandle` (HDR output). `Mesh::Build`, `BuildProcedural` and `RenderingElement3D` keep their API, but `Refit` and the host-span `RefitProcedural` rebuild the host BVH under a new `blasAddr`: set `instance.accelerationStructureReference` again before the next `BuildTLAS`, or portable deforming-mesh code keeps tracing the old geometry here. Only the device-buffer `RefitProcedural` keeps the handle. - `Crafter::Dom::HtmlElement` and `HtmlElementPtr` (move-only DOM handles with typed listeners), `Crafter::Router` (history, location, query parsing) and `Crafter::Dom::*Event` exist only here. `WebGPU::SetCanvasMount(elementId)` renders into an element instead of the page. - Examples with a DOM build ship `.glsl` and `.wgsl` side by side; `RTStress`, `RayQueryPick`, `HDRBloom` and `RTMultiShadow` are WebGPU-only. ### Environment variables All are "set to any value", read once at `Device::Initialize` or swapchain creation. | Variable | Effect | |---|---| | `CRAFTER_GRAPHICS_VALIDATION` | The Khronos validation layer with GPU-assisted validation, plus info messages; off by default. `Device::validationErrorCount` counts the errors. | | `CRAFTER_GRAPHICS_DEVICE=` | Use GPU number `n` as Vulkan lists them, never another; an error when it can't run the application. | | `CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP` | Take the descriptor-set path on a driver that has the heap (and lose ray tracing), as a heap-less driver would. | | `CRAFTER_GRAPHICS_NO_AS_WORKAROUND` | Turn the NVIDIA SPIR-V rewrite off, to check a new driver. | | `CRAFTER_GRAPHICS_NO_VULKAN_1_4` | Create a Vulkan 1.3 instance on a 1.4 GPU. | | `CRAFTER_PRESENT_IMMEDIATE` | Present immediately, uncapped, for benchmarks. | ### Testing a consumer `Device::Initialize` on Linux connects to Wayland right after creating the instance, so even a windowless test needs a compositor (`WAYLAND_DISPLAY`). `Window::CaptureNextFrame(png)` followed by `Render()` writes a frame to read back and compare; decode it with a private copy of `stb_image.h` (`STB_IMAGE_STATIC`), since the library compiles only the writer. Assert `Device::validationErrorCount == 0` only when `CRAFTER_GRAPHICS_VALIDATION` is set, as the count stays zero without the layer. Return 77 (crafter-build's skip) when `!Device::rayTracingSupported` and the test needs it. A test with GPU work and no `Window` calls `vkQueueWaitIdle(Device::queue)` and `Device::DrainDeletions()` before it ends. The library's own `tests/` has one of each kind to copy. ### Examples to copy from `HelloWindow` (a window), `HelloUI` (all three UI tiers, text, hit-testing), `CustomShader` (a Tier 1 shader in GLSL and WGSL, registered twice), `VulkanTriangle` (the smallest RT setup, raw descriptor writes), `Sponza` (`.cmesh` and `.ctex` through `Mesh::Build` and `Image2D`, texture arrays, a free camera on `Input::Map`), `RTVolume` (procedural geometry with intersection and any-hit shaders), `InputSystem` (actions, rebinding, gamepads), `HelloDom` (DOM and router, no GPU), `Decompression` (the asset round trip). All MIT.