Crafter.Graphics/AGENTS.consumer.md

Crafter.Graphics

The Crafter engine's graphics library: a C++ module, import Crafter.Graphics;, that opens a window, drives a Vulkan 1.3+ device through VK_EXT_descriptor_heap (bindless: no descriptor-set layouts, push data through vkCmdPushDataEXT), traces rays through VK_KHR_ray_tracing_pipeline, draws UI from compute shaders, and reads keyboards, mice, touch, gamepads and the clipboard. A wasm32-* build of the same surface runs in the browser on WebGPU with a software ray tracer. Everything is in namespace Crafter. The vectors and matrices are Crafter.Math's (import Crafter.Math;) and the compressed asset formats Crafter.Asset's (import Crafter.Asset;); both ship notes of their own. Window callbacks are Crafter::Event<T> from Crafter.Event (import Crafter.Event;): EventListener<void> sub(&window.onMouseMove, [&] { ... }); subscribes, and the listener's destructor unsubscribes. Source, examples and issues: https://forgejo.catcrafts.net/Catcrafts/Crafter.Graphics (LGPL-3.0-only; the examples under examples/ are MIT and meant to be copied). These notes are its AGENTS.consumer.md: a fix to them goes there, not into the copy crafter-build writes.

Building against it

• The library's project.cpp needs crafter-build 3.0.0 or later; older ones fail to compile it, for every target.
• The target picks the backend, in the library and in your code. Any wasm32-* target builds the DOM + WebGPU library; everything else builds Vulkan, with Wayland on Linux (there is no X11 backend) or Win32 on x86_64-w64-mingw32. The library defines CRAFTER_GRAPHICS_WINDOW_WAYLAND, _WIN32 or _DOM for its own units only, so a wasm consumer sets cfg.target = "wasm32-wasip1", cfg.defines.push_back({"CRAFTER_GRAPHICS_WINDOW_DOM", ""}), cfg.compileFlags.push_back("-msimd128") (the dependency PCMs were compiled with it) and EnableWasiBrowserRuntime(cfg), as examples/HelloUI/project.cpp does. Portable code #ifdef CRAFTER_GRAPHICS_WINDOW_DOMs at the few sites where the backends differ; the library has no shared signatures that are no-ops on one side.
• The library's link flags (the Vulkan loader, wayland-client, xkbcommon, udev, or the Win32 libraries) come with the dependency. On x86_64-w64-mingw32 that includes an import library for vulkan-1.dll, so linking for Win32 needs no Vulkan SDK. The library makes it at configure time with the llvm-dlltool beside crafter-build's llvm-ar, from the loader's own export list (Vulkan-Loader's loader/vulkan-1.def, fetched unpinned). Under the Flatpak a Win32 executable still fails to link on __libc_single_threaded (Crafter.Build #43). Until that is fixed, add a C file defining char __libc_single_threaded = 0; to the executable, as the library's Win32Link test does. Its shaders (ui-*.comp.spv and ui-*.comp.sets.spv) and, for wasm, dom-env.js and dom-webgpu.js are copied into your executable's bin directory, as are your cfg.shaders outputs (<name>.spv from <name>.glsl) and cfg.files. Paths are resolved against the working directory at runtime, so run from the bin directory, as crafter-build -r does.
• Your own shaders: cfg.shaders.emplace_back(fs::path("raygen.glsl"), std::string("main"), ShaderType::RayGen), with Compute, RayGen, Miss, ClosestHit, AnyHit, Intersect and Callable. A compute shader for the UI heap is registered twice, see "Custom compute shaders". WGSL for the DOM build ships through cfg.files and is loaded at runtime.
• Project args the library understands, passed through your args: --local (sibling checkouts of Crafter.Event, Crafter.Math and Crafter.Asset instead of clones), --timing (meant to add frame timing counters; the build does not compile with it at the moment, issue #178), --no-webgpu (wasm: a DOM-only page without the WebGPU bridge, which otherwise requests a GPU adapter on every load and replaces the page body with an error where WebGPU is missing).
• Needs at runtime: Vulkan 1.3 (1.4 is used where the GPU has it), a presentable surface, and the features the library's own shaders use. VK_EXT_descriptor_heap and ray tracing are optional groups, see "Device and features". Mesa's llvmpipe runs the compute/UI path, and recent Mesa advertises the heap.

A program, and what the loop does

Device::Initialize({.vulkan10 = {.shaderInt16 = VK_TRUE}});   // the features your shaders use; on the DOM build Initialize() takes none
Window window(1280, 720, "Title");                             // `static` on the DOM build
VkCommandBuffer init = window.StartInit();                     // a one-time setup command buffer
DescriptorHeapVulkan heap;
heap.Initialize(/*images*/ 16, /*buffers*/ 16, /*samplers*/ 4);
window.descriptorHeap = &heap;                                 // bound at the start of every frame
// create buffers, images, meshes, pipelines and passes, recording into `init` ...
window.passes.push_back(&pass);                                // RenderPass*: a UIRenderer, an RTPass, your own
window.FinishInit();                                           // submits `init` and waits for it
window.Render();                                               // one frame
window.StartUpdate();                                          // frames on the compositor's clock from here
window.StartSync();                                            // the event loop; never returns

• Include vulkan/vulkan.h yourself, before the imports (the examples wrap it in #ifndef CRAFTER_GRAPHICS_WINDOW_DOM): the library includes it only in its own global module fragments, and neither Vulkan's types nor its macros (VK_TRUE, the VK_* flags) cross an import. On the DOM build Device::Initialize() takes no arguments and DeviceFeatures does not exist, which is why the dual-target examples keep a separate DOM main.
• StartSync runs the platform loop until open is false (onClose fires first). Each pass of the loop dispatches platform events into the Window events, runs Gamepad::Tick() and onBeforeUpdate, and runs Update() for a window that is due. On Wayland a window on the compositor's clock updates from its frame callback during that event dispatch, before the pass's onBeforeUpdate, and the loop wakes for every event, so onBeforeUpdate fires several times per frame; on Win32 and in the browser onBeforeUpdate precedes the same pass's Update. updateInterval, set before StartUpdate, makes a window update every so often whether visible or not; pair it with SetPresentMode(PresentMode::Mailbox) so a present never waits for the display. AddPollFd(fd, callback) (Wayland) wakes the loop for a socket or an eventfd.
• Update() computes mouseDelta and calls Render(), which waits for the fence of its frame slot (three frames in flight, Window::numFrames; the slot is window.currentFrame), acquires a swapchain image (window.currentImage; the swapchain has as many images as its present mode needs, not necessarily three), writes it into this window's swapchain slots of the heap, reclaims deferred deletions, begins drawCmdBuffers[currentFrame], binds the heap (or the descriptor set), fires onUpdate(FrameTime), records every pass in passes in order with a barrier on the swapchain image between them, transitions the image to present, submits and presents. Per-frame GPU work of your own (a TLAS rebuild, a physics dispatch) is recorded from onUpdate into window.drawCmdBuffers[window.currentFrame]; the heaps are bound by then and the passes come after. A Resize or SetPresentMode from a listener of the frame (an input action ticked in onUpdate, say) takes effect after its present. onResize fires after a swapchain rebuild (a new size or present mode); the swapchain slots follow by themselves, and anything else that holds the old images or is sized after the window is made again there.
• A RenderPass is Record(cmd, frameIdx, window) plus SwapchainStage(), which must return the stage that touches the swapchain image (COMPUTE_SHADER, RAY_TRACING_SHADER or TRANSFER): the frame's barriers take their stage and access masks from it. frameIdx is the frame slot, window.currentFrame, not the image: the frame's image is window.images[window.currentImage]. The image stays in VK_IMAGE_LAYOUT_GENERAL through the whole pass list, with STORAGE | TRANSFER_SRC usage and TRANSFER_DST where the surface allows it (window.swapchainUsage). Nothing clears it: draw a background quad, or clear from a transfer pass where TRANSFER_DST is there.
• window.width and height are physical pixels, as is currentMousePos. CaptureNextFrame(path) saves the next frame as a PNG. SetTitle, SetAppId, SetCursorImage, SetDefaultCursor; SetTextInput(active, x, y, w, h, purpose) tells the compositor's input method a text field has focus, which opens an on-screen keyboard on a phone.
• Several windows can exist (on Wayland Device::windows lists them; the other backends keep no list), and may share a heap, each writing only its own swapchain slots, but they share one frame clock: one rendering window is the assumption behind the deletion queue, the staging rings and the font atlas's evictions. Without a Window, nothing advances Device::frameCounter: a program that draws text without one steps it itself after each frame's work, or its full atlas never frees a cell and new glyphs stay blank.

Device and features

• Device::Initialize(required = {}, optional = {}) creates the instance, weighs every GPU and creates the device. DeviceFeatures holds the five core feature structs, vulkan10 to vulkan14; only their VkBool32s are read, so {.vulkan12 = {.shaderFloat16 = VK_TRUE}} is a complete argument. The library enables what its own shaders use and nothing more: a uint16_t spec constant needs .vulkan10.shaderInt16, fp16 arithmetic .vulkan12.shaderFloat16, 16-bit storage .vulkan11.storageBuffer16BitAccess, scalar layout .vulkan12.scalarBlockLayout. A GPU lacking a required feature is skipped; when none is left the call throws naming each GPU's gaps, and may be called again asking for less. Optional features are enabled where the GPU has them; Device::enabledFeatures says which, supportedFeatures what the GPU has, apiVersion the version in use (vulkan14 features need a 1.4 GPU; maintenance5 is always on). CRAFTER_GRAPHICS_DEVICE=<n> picks GPU n.
• VulkanShader checks every module's SPIR-V capabilities against what was enabled and throws naming the file and the field to pass (vulkan10.shaderInt16), or saying the GPU lacks it. Most drivers accept such a module silently, so this is the error you get here instead of a wrong frame on another machine.
• Optional groups, decided once at Initialize: Device::descriptorHeapSupported (VK_EXT_descriptor_heap), rayTracingSupported (needs the heap too), memoryDecompressionSupported (VK_EXT_memory_decompression: .cmesh and .ctex decompress on the GPU; otherwise the library decodes them on the CPU), deviceFaultSupported, swapchainMaintenance1Supported (VK_KHR_swapchain_maintenance1, with the instance's surface_maintenance1, surfaceMaintenance1Supported: a swapchain names its present mode and gets that mode's own image count; otherwise the stricter of the mode's and the surface's). The RT entry points throw through Device::RequireRayTracing without ray tracing; the compute/UI path runs everywhere.
• Device::device, queue, commandPool, physDevice, instance, memoryProperties, deviceProperties, rayTracingProperties and descriptorHeapProperties are public statics for raw Vulkan. Device::GetMemoryType(typeBits, required, preferred = 0) and PreferDirectDeviceWrite(bytes) are the allocation helpers.
• Device::pipelineCache is shared by every pipeline and written to Device::pipelineCachePath (pipeline_cache.bin in the working directory) at exit. Set the path before Initialize, and gitignore the file.

Lifetimes

Frames are pipelined three deep and nothing waits for the GPU at steady state, so:

• Every library object that owns a Vulkan handle (VulkanBuffer, Mesh, TlasWithBuffer, VulkanShader, ComputeShader, PipelineRTVulkan, and ImageVulkan's staging buffer) is move-only and releases the handle through Device::EnqueueDeletion; the render loop frees it Device::framesInFlight frames later. Destroying one mid-frame is safe, and so is destroying it from another thread. VulkanBuffer::Clear() frees at once and is only for a GPU known to be idle; DeferredClear() is the queued form. A program with no Window reclaims nothing on its own: call Device::DrainDeletions() after vkQueueWaitIdle(Device::queue).
• Static owners (a static Mesh, a static VulkanBuffer) are fine: their destructors run after the driver has unloaded, and the queue allows for that by never being destroyed itself. RenderingElement3D::tlases is leaked for the same reason.
• Window is neither movable nor copyable, and its destructor waits for the device, so a window can be closed and another opened.
• ImageVulkan has no destructor: call Destroy() once the GPU is done with it. FontAtlas waits for the queue and destroys its own image.
• Heap slots (ImageSlot, BufferSlot, SamplerSlot) are RAII handles that return the slot on destruction, and a returned slot is handed out again at once. Free a slot only when no frame in flight reads it (after a vkQueueWaitIdle, or at a scene switch), and before its heap goes. In descriptor-set mode a slot write destroys the previous view immediately: same rule.
• RenderingElement3D::elements holds raw pointers: Add/Remove elements whose storage does not move (reserve the vector, or make them static). UIRenderer::ShapeText and FontAtlas cache by Font*: call ui.InvalidateFont(font), which clears both, before destroying or replacing a Font (the atlas keeps the Font* of a glyph waiting for a cell until the next frame).

The bindless heap

DescriptorHeapVulkan::Initialize(images, buffers, samplers) sizes one resource heap and one sampler heap per frame in flight (resourceHeap[Window::numFrames], samplerHeap[...], mapped VulkanBuffers) and the slot allocators. AllocateImageSlots(n), AllocateBufferSlots(n) and AllocateSamplerSlots(n) return {firstElement, count} ranges (contiguous; freed runs are reused), Free*Slots returns them, and running out throws naming the pool and the counts. Initialize throws too when its buffer slots would need shader-visible indices past 16 bits.

• Buffer slot indices are relative. AllocateBufferSlots indexes the heap's buffer region, which starts after the image region, while a GLSL descriptor_heap SSBO array indexes buffer descriptors from heap byte 0. The shader-visible index is heap.bufferStartElement + raw. The RAII BufferSlot converts to exactly that (its operator std::uint16_t()), UIRenderer::RegisterBuffer returns one, and raw code passes heap.bufferStartElement as a spec constant: the RT examples declare layout(constant_id = 0) const uint16_t bufferStart = 0us; and index topLevelAS[bufferStart], which is why they ask for shaderInt16. Image slots index the heap as they are, in image-descriptor units; samplers live in their own heap. A wrong buffer index reads zeros, which looks like every field of every item being zero. In descriptor-set mode bufferStartElement is 0, so code that goes through the conversion is right in both.
• Writing a descriptor by hand: Device::vkWriteResourceDescriptorsEXT into heap.resourceHeap[f].value + heap.ImageByteOffset(imageRange.firstElement) or heap.BufferByteOffset(bufferRange.firstElement), Device::descriptorHeapProperties.imageDescriptorSize or bufferDescriptorSize bytes long, for each frame f, then heap.resourceHeap[f].FlushDevice(). BufferByteOffset takes the raw index AllocateBufferSlots returned and adds the region start itself; handing it a BufferSlot, which converts to the absolute index, adds the start twice and puts the descriptor in the wrong place. ImageByteOffset takes an image slot as it is. Per-frame resources (tlases[f]) are written into frame f's heap; a shared resource into every frame's. The swapchain image is neither, since frames and images pair up differently from frame to frame: list its image slot once with heap.AddSwapchainSlot(slot, window), and the window writes the image each frame draws into there, a storage image in GENERAL, in both binding models and across resizes (a slot listed during a frame is written for that frame). examples/VulkanTriangle/main.cpp is the pattern. The UIRenderer::Register* helpers do this for you in both binding models.
• Binding the heap outside the frame loop: Window::Render binds window.descriptorHeap in every frame it records. A command buffer it does not record that dispatches against the heap (window.StartInit()'s, say) calls heap.Bind(cmd, frame), which binds that frame's heaps with their reserved ranges, or in descriptor-set mode its descriptor set. Don't build the VkBindHeapInfoEXT by hand: where the reserved range sits is the heap's to decide.
• Descriptor-set fallback. On a driver without VK_EXT_descriptor_heap (Mesa Turnip on Adreno) Device::descriptorHeapSupported is false and the same slots live in one bindless descriptor set per frame; slot handles, item structs, UIDispatchHeader and the UIRenderer API are unchanged, so application code does not branch. Ray tracing is unavailable there. CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP forces this path on a driver that has the heap, to test it. Raw vkWriteResourceDescriptorsEXT code is heap-only; a consumer that must run on such a driver stays on the UIRenderer::Register* helpers.

Buffers, images, samplers

• VulkanBuffer<T, Mapped>: Create(usage, memoryFlags, count, preferredFlags = 0); Resize(...) reuses the allocation when the request fits and the usage matches, otherwise queues the old one and creates; size (bytes), buffer, address (non-zero only with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT), memoryPropertyFlagsChosen. With Mapped, value is the T* of a persistent map: after writing, call FlushDevice() (whole buffer), FlushDevice(offset, bytes) (a range, rounded to nonCoherentAtomSize) or FlushDevice(cmd, dstAccess, dstStage), which flushes and records the host-write barrier; all three are no-ops on coherent memory. FlushHost() is the invalidate for readback. A UI item buffer is GraphicsBuffer<QuadItem, true> with STORAGE_BUFFER | SHADER_DEVICE_ADDRESS usage and HOST_VISIBLE memory with DEVICE_LOCAL as the preferred fourth argument (the examples pass both as required, which throws on a GPU without a host-visible device-local type), flushed each frame before the dispatch that reads it.
• Static geometry: VulkanBuffer<T, false>::UploadDeviceLocal(usage, src, count, cmd, dstAccess, dstStage) places the buffer in device memory, written directly on ReBAR and UMA and staged through a per-frame staging ring on a small BAR, and records the barrier to dstStage. UploadDeviceLocalRange(src, offset, count, cmd, dstAccess, dstStage) re-uploads the dirty window [offset, offset + count); src points at the first dirty element (vertices.data() + offset), not at the array's start. A re-upload into a buffer a frame may still read is ordered after those reads on the GPU.
• ImageVulkan<Pixel> (Image2D<Pixel> is the portable alias): Create(width, height, mipLevels, cmd, format, usage, layout, consumerStage = RAY_TRACING_SHADER, streamed = false) makes the image and a mapped staging buffer image.buffer; fill image.buffer.value and Update(cmd, layout) copies it in, builds the mip chain by blits and transitions to layout for consumerStage. Update(const CompressedTextureAsset&, cmd, layout) decompresses a .ctex on the GPU, or the CPU. A static image releases its staging after the first upload; streamed = true keeps it (the font atlas, through UpdateRegion). Pass VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT as consumerStage for an image a compute shader samples; the RT default is corrected to compute on a device without ray tracing. image, imageView and descriptor are public for raw descriptor writes; Destroy() releases.
• SamplerVulkan<Pixel> makes one linear, repeat, 16x anisotropic sampler at construction (textureSampler). The UI has UIRenderer::RegisterSampler(const VkSamplerCreateInfo&) and RegisterLinearClampSampler().

UI

Three tiers, on purpose: Tier 1 is any compute shader you write (ComputeShader, UIRenderer::Dispatch); Tier 2 the four standard kernels, their item structs and the UIRenderer helpers; Tier 3 stateless functions that append items to your buffers. The library has no widget tree, layout engine, theme, hit-testing or focus management: state lives in your PODs, hit-testing is Rect::Contains against the rects you drew, and a Tier 3 component that does not fit is copied out of implementations/Crafter.Graphics-UIComponents.cpp and edited, not subclassed.

Setup: FontAtlas atlas; atlas.Initialize(init); then UIRenderer ui; ui.fontAtlas = &atlas; ui.Initialize(window, heap, init); window.passes.push_back(&ui);. Initialize loads the five kernels by file name from the working directory (ui-quads.comp.spv and so on; the .sets.spv builds must sit beside them) and registers the swapchain images and the atlas. Then, per item buffer, auto slot = ui.RegisterBuffer(buf); gives a BufferSlot ready for a shader.

Per frame, in an EventListener<UIBuildArgs> on ui.onBuild (fired from the pass's Record with {cmd, frameIdx}):

1. Rebuild the item arrays in your mapped buffers, in draw order: later items overdraw earlier ones. Lay out with Rect::FromWindow(window), SubRect(size, Anchor) and Inset(...). Keep one set of item buffers (and slots) per frame in flight, Window::numFrames of them, and fill the one at frameIdx: a frame still on the GPU reads its items as it runs, and a later frame's items written into the same buffer can name atlas cells that frame drew other glyphs in (examples/HelloUI).
2. buf.FlushDevice(cmd, VK_ACCESS_SHADER_READ_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT) for each buffer with items.
3. Dispatch: ui.DispatchQuads(cmd, slot, count, clipRectPx = {0, 0, 1e9, 1e9}), DispatchCircles, DispatchImages, DispatchText, or ui.DispatchFused(cmd, {quadsSlot, qc}, {circlesSlot, cc}, {imagesSlot, ic}, {glyphsSlot, gc}), which composites the four categories in that fixed order (quads, circles, images, text) with one load and one store; an empty category costs nothing. The two forms mix; a custom ui.Dispatch between two DispatchFused calls is the flush point.

• The item structs are std430 PODs mirrored byte for byte in shaders/ui-shared.glsl, and frozen: QuadItem (64 bytes: x, y, w, h, r, g, b, a, corner radii cTL, cTR, cBR, cBL, outline, oR, oG, oB), CircleItem (48: cx, cy, radius, _p0, colour, outline), ImageItem (64: rect, u0, v0, u1, v1, tint, texSlot, sampSlot, _p1, _p2), GlyphItem (48: rect, atlas uv, colour). Colours are straight alpha, blended source-over; coordinates are window pixels.
• Cost: every kernel dispatches one workgroup per 8x8 screen tile and each pixel walks every item (behind a per-tile cull), so a dispatch is O(W*H*N), fine up to a few hundred items at 1080p. Splitting one buffer into several dispatches does not help. Thousands of items want a different shader (tile binning), which is Tier 1.
• Text: Font font("font.ttf") (TrueType through stb_truetype); font.GetLineWidth(text, px), LineHeight(px), AscentPx(px), NearestCursorByte(text, px, x). The atlas is a 1024 by 1024 SDF atlas at a 32 px base size, filled lazily per codepoint and re-uploaded by dirty rectangle. Glyphs scale from it with the edge ramped over one screen pixel at any size and aspect, on both backends; far above 32 px corners round off and fine detail goes, and below about 4 px the ramp narrows and thin strokes bolden. ui.ShapeText(font, px, x, baselineY, utf8, color, out, capacity, &advance, TextAlign::Left) writes GlyphItems: single line, no kerning, no wrapping, a shaped-run cache behind it. Once the atlas is full, a glyph that finds no free room draws blank for one frame but keeps its advance, and gets a cell when the next frame begins, taken from the glyphs drawn least recently, never one drawn in the last Device::framesInFlight + 1 frames. So a GlyphItem holds for the frame it was shaped for: shape text every frame (the Tier 3 components do, and a cached run costs a copy) rather than drawing a glyph buffer filled in an earlier frame. atlas.EnsureAndGet(font, codepoint) returns const Glyph& and never fails; its UVs hold only when Resident(), and a Lookup pointer only until the atlas's next EnsureAndGet, Touch or Update. A Tier 1 kernel that samples FontAtlasImageSlot() through ui.Dispatch sees the frame's new glyphs: every dispatch uploads the atlas first.
• Tier 3: fill a UIBuffer (quads, quadCount, quadCap; glyphs, glyphCount, glyphCap; images, imageCount, imageCap; atlas; renderer; a null pointer makes the components that need it no-ops) and call DrawButton(buf, rect, label, hovered, pressed, font, px, ButtonColors), DrawCheckbox(buf, rect, checked, hovered, CheckboxColors), DrawSlider(buf, rect, t01, dragging, SliderColors), DrawProgressBar(buf, rect, t01, ProgressColors), DrawText(buf, text, x, baselineY, font, px, color, align) (returns the advance) and DrawImage(buf, rect, textureSlot, samplerSlot, tint, uv). Colours are small *Colors aggregates you hold; there is no Theme.
• InputField is the model for a stateful component: a POD (value, type, focused, cursorPos), free functions you call from your own listeners (InputField_OnText(field, utf8) from onTextInput, InputField_OnKey(field, code) from onRawKeyDown, InputField_HitTestCursor(field, rect, clickX, font, px, colors)), InputField_TryParse<T>(field), and DrawInputField(buf, field, rect, font, px, colors, caretVisible); the blink is yours.
• Images in the UI: auto tex = ui.RegisterImage(image, VK_FORMAT_R8G8B8A8_UNORM); (an ImageSlot; the layout defaults to SHADER_READ_ONLY_OPTIMAL), auto samp = ui.RegisterLinearClampSampler();, then both go into ImageItem::texSlot and sampSlot. ui.OutImageSlot(), FontAtlasImageSlot() and FontAtlasSamplerSlot() expose the renderer's own slots.

Custom compute shaders (Tier 1)

A GLSL compute shader that dispatches beside the standard kernels follows shaders/ui-shared.glsl, which the build puts on the include path:

#version 460
#extension GL_GOOGLE_include_directive : enable
#include "ui-shared.glsl"
struct MyItem { vec4 a; };                          // mirrored in C++ with a static_assert on its size
layout(UI_HEAP_BUFFERS, std430) readonly buffer MyBuf { MyItem items[]; } myHeap[];
layout(push_constant) uniform PC { UIDispatchHeader hdr; /* your fields after it */ } pc;
layout(local_size_x = 8, local_size_y = 8, local_size_z = 1) in;
void main() {
    ivec2 px;
    if (!uiResolveScreenPixel(pc.hdr, px)) return;
    // read myHeap[pc.hdr.itemBuffer].items[i]; imageLoad/imageStore uiImages[pc.hdr.outImage]
}

• Declare bindless resources with UI_HEAP_IMAGES, UI_HEAP_TEXTURES, UI_HEAP_SAMPLERS and UI_HEAP_BUFFERS, never a literal descriptor_heap or a set and binding, and register the source twice in project.cpp: once plain, once with sets.defines = {"CRAFTER_GRAPHICS_DESCRIPTOR_SETS"}; sets.variant = "sets";, which yields <name>.spv and <name>.sets.spv. ComputeShader::Load("<name>.spv", heap) picks the build the device needs and throws naming a missing file. examples/CustomShader/project.cpp has the lines.
• UIDispatchHeader (48 bytes; in C++ outImage, itemBuffer, surfaceWidth, surfaceHeight, clipX, clipY, clipW, clipH, itemCount, frameIdx, flags, _pad, and in GLSL the same bytes as uint outImage, itemBuffer; uvec2 surfaceSize; vec4 clipRectPx; uint itemCount, frameIdx, flags, _pad) must be the first member of the push block; ui.FillHeader(slot, count, clipRectPx, flags) fills it. The high bit of flags (kUIFlagClip) is the renderer's, the rest are yours. Keep the push block a multiple of 4 bytes and within 256; the descriptor-set path throws past the device's maxPushConstantsSize.
• ui.Dispatch(cmd, shader, &pc, sizeof(pc), ceil(W / 8), ceil(H / 8)) binds, pushes and dispatches, with a SHADER_WRITE -> SHADER_READ | SHADER_WRITE barrier on the swapchain image before every dispatch but the frame's first, so a custom shader sees what the dispatch before it drew. ComputeShader::Dispatch(cmd, push, bytes, gx, gy, gz, tlasAddress = 0) is the same without the barrier; a compute shader that ray-queries the TLAS passes RenderingElement3D::tlases[frameIdx].address, for the NVIDIA workaround below.
• A shader that draws text from the font atlas (ui.FontAtlasImageSlot() and FontAtlasSamplerSlot()), a copy of ui-text.comp.glsl among them, decodes it with ui-shared.glsl's helpers rather than arithmetic of its own: uiGlyphAtlasSize(texSlot, samplerSlot), then uiGlyphRamp(item.uv, 1.0 / item.rect.zw, atlasSize) once per glyph and uiGlyphCoverage(texSlot, samplerSlot, uv, ramp, atlasSize) per pixel, with dynamically uniform slots. A texel holds stb_truetype's UI_SDF_ON_EDGE + UI_SDF_PIXEL_DIST_SCALE * d (128 + 32 d), d the distance from the outline in atlas pixels, positive inside, truncated to a byte; uiSdfDistance(sample) gives d back. Multiplying a sample by 32 instead of dividing by 32/255 gives a ramp a quarter of a pixel wide, and pixelated text.
• Keep the one-workgroup-per-screen-tile shape for anything that composites. A workgroup per item races on overlapping pixels: Vulkan orders nothing between the workgroups of one dispatch.
• A ComputeShader can also run outside the UI pass, dispatched from onUpdate into the frame's command buffer with barriers of your own.

Ray tracing (Vulkan)

Heap-only: check Device::rayTracingSupported, or expect std::runtime_error from the first RT call. examples/VulkanTriangle/main.cpp is the whole setup.

• Shaders: VulkanShader shader("raygen.spv", "main", VK_SHADER_STAGE_RAYGEN_BIT_KHR, &specInfo); (nullptr for no specialisation); ShaderBindingTableVulkan table; table.Init(shaders);; PipelineRTVulkan pipeline; pipeline.Init(cmd, raygenGroups, missGroups, hitGroups, table, maxRecursionDepth = 1); with spans of VkRayTracingShaderGroupCreateInfoKHR indexing the shader array. maxRecursionDepth counts the raygen as 1: a closest-hit that traces a shadow ray needs 2, and tracing past the pipeline's depth loses the device.
• Geometry: Mesh mesh; mesh.Build(std::span<Vector<float, 3, 3>> vertices, std::span<std::uint32_t> indices, cmd, RTBuildOptions{}); builds a BLAS and sets mesh.blasAddr; Build(const CompressedMeshAsset&, cmd) builds from a .cmesh, decompressed on the GPU where supported; BuildProcedural(aabbs, opaque, cmd) builds from RTAabb boxes for a VK_RAY_TRACING_SHADER_GROUP_TYPE_PROCEDURAL_HIT_GROUP_KHR with an intersection shader, and has an overload taking a device address a compute pass wrote. RTBuildOptions is .preference (FastTrace, FastBuild), .allowUpdate and .noDuplicateAnyHit; with allowUpdate, Refit(vertices, indices, cmd) and its dirty-range overload update in place and keep blasAddr. A rebuild into storage that still fits keeps blasAddr too; read it again after a rebuild that grows.
• Instances: a RenderingElement3D holds a VkAccelerationStructureInstanceKHR instance (transform, mask, instanceCustomIndex, instanceShaderBindingTableRecordOffset, flags, accelerationStructureReference = mesh.blasAddr; Crafter.Math's MatrixRowMajor<float, 4, 3, 1>::Store(reinterpret_cast<float*>(instance.transform.matrix)) writes the transform), userMetadata (copied per instance id into tlases[f].metadataBuffer) and transformOwnedByGpu. RenderingElement3D::Add(&e) and Remove(&e) register it. RenderingElement3D::BuildTLAS(cmd, frameIndex, preference = FastTrace) builds or refits tlases[frameIndex] from every registered element: once per frame slot at setup (0, 1, 2), and per frame from onUpdate for window.currentFrame when instances move. e.MarkHostDataDirty() after a change to an element's host fields lets the per-frame copy skip unchanged slots. tlases[f].address goes into a VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR descriptor of frame f (see "The bindless heap") and changes only when the TLAS outgrows its storage.
• The pass: RTPass rtPass(&pipeline); rtPass.pushPtr = &camera; rtPass.pushBytes = sizeof(camera); pushes your data at offset 0 of the shaders' push block before every trace (the bytes are read at record time, so updating camera before the frame is enough); width and height trace at a size other than the window's. window.passes.push_back(&rtPass);.
• In GLSL, with the extensions the examples enable (GL_EXT_ray_tracing, GL_EXT_descriptor_heap, GL_EXT_nonuniform_qualifier for a non-constant heap index, GL_EXT_shader_explicit_arithmetic_types_int16 for a uint16_t spec constant, and GL_EXT_shader_image_load_formatted or a writeonly image): layout(descriptor_heap) uniform accelerationStructureEXT topLevelAS[];, layout(descriptor_heap) uniform writeonly image2D image[];, indices from spec constants or push data. The output image is the swapchain image at an image slot listed with AddSwapchainSlot, in VK_IMAGE_LAYOUT_GENERAL. Start from examples/VulkanTriangle/raygen.glsl.
• NVIDIA. On the proprietary driver (still on 615.78.08) reading an acceleration structure through the heap loses the device. The library works around it transparently: VulkanShader rewrites the SPIR-V so the heap AS read becomes a TLAS-address load from the push block plus OpConvertUToAccelerationStructureKHR, and RTPass and ComputeShader push the address. Nothing changes in your shaders or code, except that the rewrite appends an 8-byte member after your push block, and that it turns shaderInt64 on for itself only: a shader of yours that declares Int64 still has to ask for shaderInt64, or it runs on NVIDIA alone. CRAFTER_GRAPHICS_NO_AS_WORKAROUND disables the rewrite, to probe a new driver. Second note: on 610.43.02, indexing a descriptor_heap array with a runtime value inside a hit shader faulted (fixed in 615.78.08). The portable pattern, which the Sponza example uses, is one descriptor at a spec-constant slot and dynamic indexing inside it: all geometry in one SSBO indexed by element offset, materials in one texture2DArray indexed by layer. examples/Sponza/README.md has the GLSL.

Input

• Window events, Event<void> unless noted: onMouseMove (then currentMousePos, lastMousePos, mouseDelta), onMouseEnter and onMouseLeave with mouseInside (native only: the DOM build declares the two events but never fires them, and has no mouseInside), onMouseLeftClick, onMouseLeftHold, onMouseLeftRelease and the right-button trio (mouseLeftHeld, mouseRightHeld), onMouseScroll (Event<std::uint32_t>: whole detents, reinterpret the payload as int32, positive is wheel down, every backend normalised to one per notch), onRawKeyDown, onRawKeyHold, onRawKeyUp (Event<KeyCode>, raw platform codes; heldKeys), onTextInput (Event<const std::string_view>, UTF-8; repeats are synthesised on Wayland), onPreedit and onTextInputDelete (input methods), onTouchDown, onTouchMove, onTouchUp (Event<TouchPoint>; the first finger also drives the pointer events), onClose, onResize, onBeforeUpdate, onUpdate (Event<FrameTime>: now, delta).
• Keys: Key(CrafterKeys::Space) is consteval and folds to the platform's raw code (a Win32 scancode, a Linux keycode), so bindings stay portable in source and compare against onRawKeyDown's KeyCode at runtime. Bindings serialised with Input::BindingToString are platform-specific.
• Input::Map: auto& jump = map.AddAction("Jump", Input::ActionType::Button); (Button, Axis1D, Vector2), jump.bindings = { Input::KeyBind{Key(CrafterKeys::Space)}, Input::GamepadButtonBind{id, Gamepad::Button::South} }; (also MouseButtonBind, MouseScrollBind, MouseDeltaBind{scale}, GamepadAxisBind, GamepadStickBind{id, Gamepad::Stick::Left}, WASDBind{up, down, left, right}), map.Attach(window);, and map.Tick(); once per frame, after Gamepad::Tick(), which the loop runs before onBeforeUpdate. Every Tick recomputes the mouse delta and drains the scroll accumulator, and on Wayland onBeforeUpdate fires on every loop wake, so either tick from onUpdate, or tick in onBeforeUpdate and consume the delta and scroll actions in that same listener, as the Sponza example does. Read action.pressed, value or vector2, or subscribe to onPerformed, onCanceled, onValueChanged, onVector2Changed. StartRebind(action, CaptureMask, callback) captures the next input.
• Gamepad::connected, onConnected, onDisconnected, per-device buttons[], axes[] and events, Rumble(dev, low, high, duration), FindById(id); buttons are named by position (South, East, West, North). Clipboard::SetText(text) and GetText() (std::optional<std::string>).

WebGPU and the DOM build

The same C++ surface, with these differences:

• The Window is the page, and Device::Initialize() takes no arguments. Make the window static: StartSync hands control to requestAnimationFrame and _Exit(0)s without running static destructors, so nothing after it runs on any target. StayAlive is StartSync without the animation loop, for pages that only touch the DOM, and is StartSync natively. Window::numFrames is 1, so tlases[0] and BuildTLAS(cmd, 0). GraphicsCommandBuffer is an opaque std::uint32_t.
• GraphicsBuffer<T, true>::Create(count), FlushDevice() without a command buffer (queue.writeBuffer), FlushDeviceRange, FlushDeviceSlot, and EnqueueReadback with PollReadback for GPU to CPU. Image2D<T>::Create(w, h) is rgba8unorm, Update(const CompressedTextureAsset&) decompresses on the CPU, AllocateSlot(heap) registers it; Image2DArray<T> stacks layers with a CPU mip chain; StorageImage2D is a GPU-written float texture. AllocateLinearClampSampler(heap) and AllocateLinearRepeatSampler(heap).
• DispatchImages has a six-argument overload with one texture and one sampler for the whole dispatch (WebGPU bind groups); the four-argument overload binds the font atlas instead. DispatchFused issues the four kernels in order through the four-argument overloads, so quads, circles and text match the Vulkan result but an images category does not: draw images with the six-argument DispatchImages on this backend.
• Custom compute in the UI pass: WebGPUComputeShader::Load(wgslPath, bindings, rayQuery = false), where each UICustomBinding is {group >= 2, binding, kind, format, pushOffset} with kind one of Buffer, SampledTexture, Sampler, SampledTextureArray, BufferReadWrite, StorageTexture; the renderer reads the slot at pushOffset out of your push data and binds that handle. Group 0 binding 0 is the header uniform and group 1 the ping-pong images, both declared by your WGSL; with rayQuery true the library injects its own group 1 (the RT heap and the rayQuery* API) and your shader must not declare one. Standalone compute: PlainComputeShader::Load(wgsl, pushUniformSize, bindings, rayQuery) and Dispatch(push, bytes, handles, gx, gy, gz) run anywhere in the frame, handles parallel to bindings. It has no library bind groups: group 0 binding 0 is your optional push uniform (pushUniformSize bytes, 0 for none) and your bindings start at group 1, or at group 2 with rayQuery, which reserves group 1 the same way.
• Ray tracing is a library-built wavefront tracer composed from your WGSL stages: WebGPUShader(fs::path("raygen.wgsl"), entryFn, WebGPURTStage::Raygen | Miss | ClosestHit | AnyHit | Intersection | Resolve) (wrap the path: a bare string literal is ambiguous with the constructor that takes the source itself), with the entry signatures documented in interfaces/Crafter.Graphics-ShaderBindingTableWebGPU.cppm; ShaderBindingTableWebGPU::Init, RTShaderGroup{.type = RTShaderGroupType::General | TrianglesHitGroup | ProceduralHitGroup, ...}, PipelineRTWebGPU::Init(cmd, raygen, miss, hit, sbt, bindings, hdrOutputFormat). Raygen calls rtEmitPrimaryRay; closest-hit and miss run in the SHADE stage and call rtEmitRay (continuation and shadow rays) and rtAccumulate (atomic, any number of rays per pixel). RTPass adds maxDepth (bounces), raysPerPixel (the ray buffer budget), handlesPtr and handlesCount (your bindings' handles) and outTexHandle (HDR output). Mesh::Build, BuildProcedural and RenderingElement3D keep their API, but Refit and the host-span RefitProcedural rebuild the host BVH under a new blasAddr: set instance.accelerationStructureReference again before the next BuildTLAS, or portable deforming-mesh code keeps tracing the old geometry here. Only the device-buffer RefitProcedural keeps the handle.
• Crafter::Dom::HtmlElement and HtmlElementPtr (move-only DOM handles with typed listeners), Crafter::Router (history, location, query parsing) and Crafter::Dom::*Event exist only here. WebGPU::SetCanvasMount(elementId) renders into an element instead of the page.
• Examples with a DOM build ship .glsl and .wgsl side by side; RTStress, RayQueryPick, HDRBloom and RTMultiShadow are WebGPU-only.

Environment variables

All are "set to any value", read once at Device::Initialize or swapchain creation.

Variable	Effect
CRAFTER_GRAPHICS_VALIDATION	The Khronos validation layer with GPU-assisted validation, plus info messages; off by default. Device::validationErrorCount counts the errors.
CRAFTER_GRAPHICS_DEVICE=<n>	Use GPU number n as Vulkan lists them, never another; an error when it can't run the application.
CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP	Take the descriptor-set path on a driver that has the heap (and lose ray tracing), as a heap-less driver would.
CRAFTER_GRAPHICS_NO_AS_WORKAROUND	Turn the NVIDIA SPIR-V rewrite off, to check a new driver.
CRAFTER_GRAPHICS_NO_VULKAN_1_4	Create a Vulkan 1.3 instance on a 1.4 GPU.
CRAFTER_PRESENT_IMMEDIATE	Present immediately, uncapped, for benchmarks.

Testing a consumer

Device::Initialize on Linux connects to Wayland right after creating the instance, so even a windowless test needs a compositor (WAYLAND_DISPLAY). Window::CaptureNextFrame(png) followed by Render() writes a frame to read back and compare; decode it with a private copy of stb_image.h (STB_IMAGE_STATIC), since the library compiles only the writer. Assert Device::validationErrorCount == 0 only when CRAFTER_GRAPHICS_VALIDATION is set, as the count stays zero without the layer. Return 77 (crafter-build's skip) when !Device::rayTracingSupported and the test needs it. A test with GPU work and no Window calls vkQueueWaitIdle(Device::queue) and Device::DrainDeletions() before it ends. The library's own tests/ has one of each kind to copy.

Examples to copy from

HelloWindow (a window), HelloUI (all three UI tiers, text, hit-testing), CustomShader (a Tier 1 shader in GLSL and WGSL, registered twice), VulkanTriangle (the smallest RT setup, raw descriptor writes), Sponza (.cmesh and .ctex through Mesh::Build and Image2D, texture arrays, a free camera on Input::Map), RTVolume (procedural geometry with intersection and any-hit shaders), InputSystem (actions, rebinding, gamepads), HelloDom (DOM and router, no GPU), Decompression (the asset round trip). All MIT.