# UI System for Crafter.Graphics2 ## Context The old UI system (still visible in [/home/jorijn/repos/Crafter/Crafter.Graphics/](../../repos/Crafter/Crafter.Graphics/)) was stripped out of Crafter.Graphics2 because it was painful to use. Real-world usages — [/home/jorijn/repos/3DForts/implementations/Forts3D-MainMenu.cpp](../../repos/3DForts/implementations/Forts3D-MainMenu.cpp) (250-line constructor) and [/home/jorijn/repos/3DForts/implementations/Forts3D-OptionsMenu.cpp](../../repos/3DForts/implementations/Forts3D-OptionsMenu.cpp) (510 lines, manual descriptor surgery on every page change) — show the symptoms: 7-param `Anchor2D` repeated everywhere, manual `parent.children.push_back`, manual per-frame `UpdatePosition`, 8-`EventListener`-fields-per-widget callback boilerplate, no widget abstraction at all (users compose `RenderingElement2D` + `MouseElement` + `EventListener` by hand), no batching, no glyph atlas, character-by-character glyph render every frame. The new system must be: - User-friendly and future-proof (declarative composition, automatic layout, no manual lifetime juggling). - At least as performant as the old one (single batched draw, glyph atlas, dirty-tracked layout). - Cleanly integrated into the existing 3D ray-traced pipeline. **Any change that touches the 3D path needs explicit OK from the user.** - CPU-runnable as a nice-to-have — but explicitly *not* via a custom CPU rasterizer if it can't beat llvmpipe. ## Design summary **Three-layer architecture:** 1. **Widget layer (user-facing):** declarative builder API. Widgets are value types with chained `.method()` configuration; composite containers (`VStack`, `HStack`, `Stack`, `Overlay`, `Grid`, `TabView`, `ScrollView`) take children as `&&` parameter packs and own them inside a `UIScene` arena. The only handle user code keeps is `WidgetRef` (a stable typed reference into the scene). 2. **Layout / event layer:** two-pass measure/arrange (WPF/Avalonia/Flutter convention). Hit-testing is automatic from the laid-out tree; events route via capture → tunnel → bubble. Focus is tree-cycled with Tab. 3. **Rendering layer:** widgets emit `UIItem` records into a single per-frame SSBO; one compute-shader dispatch composites everything onto the swapchain image. SDF glyph atlas means one texture serves all sizes/scales. **Key decisions (locked):** - **Compute shader, not graphics pipeline.** The swapchain has only `VK_IMAGE_USAGE_STORAGE_BIT` ([Crafter.Graphics-Window.cpp:958](../../repos/Crafter/Crafter.Graphics2/implementations/Crafter.Graphics-Window.cpp#L958)). A compute pass writes the same way the existing RT pipeline does, in the same `VK_IMAGE_LAYOUT_GENERAL`. No swapchain change, no renderpass machinery, no extra barriers. - **SDF glyph atlas, not per-size raster.** One R8 atlas (1024² growable to 4096²) built lazily from `stb_truetype`. Survives DPI / fractional-scale changes without re-bake. Aliases cleanly at any scale via shader smoothstep. - **No tile-binning in V1.** A naive front-to-back per-pixel item scan over a ≤500-item draw list comfortably hits sub-millisecond on integrated GPUs. The data layout supports adding tile-binning later without changing the public API. - **No custom CPU rasterizer.** Same Vulkan code path runs on llvmpipe via the loader. Documented as the CPU fallback. - **One descriptor heap, bound once per frame, never re-bound.** Per the [`VK_EXT_descriptor_heap` proposal](https://docs.vulkan.org/features/latest/features/proposals/VK_EXT_descriptor_heap.html), heap rebinds are expensive and applications should "stick to the same heap throughout the lifetime of the application". UIScene allocates its slots from the same `Window::descriptorHeap` the user already creates for 3D — never its own. Detail in §"Descriptor management". - **Window owns a list of `RenderPass*`, not a single pipeline pointer.** Replaces the current `Window::pipeline` / `Window::descriptorHeap` direct-bind model. Future-proofs for post-processing, debug overlays, RTT, multi-pass effects. Detail in §"Window integration". - **Layout is dirty-tracked.** `Observable::set()` marks owning widgets dirty; layout re-walks only the dirty subtree. - **Text supports per-glyph styling** via `TextRun`. Each glyph is already a separate draw-list item, so per-run styling adds zero shader complexity — just per-item properties. ## Window integration — `RenderPass` refactor The current `Window::Render` ([Crafter.Graphics-Window.cpp:730-844](../../repos/Crafter/Crafter.Graphics2/implementations/Crafter.Graphics-Window.cpp#L730-L844)) hardcodes a single ray-tracing pipeline: it binds `Window::pipeline` ([Crafter.Graphics-Window.cppm:193](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L193)) and calls `vkCmdTraceRaysKHR` directly. That assumption breaks the moment we want UI on top, post-processing, debug overlays, or any multi-pass effect. **Refactor**: replace the single-pipeline pointer with a vector of render passes. ```cpp // new module: Crafter.Graphics-RenderPass.cppm struct RenderPass { virtual void Record(VkCommandBuffer cmd, std::uint32_t frameIdx) = 0; virtual ~RenderPass() = default; }; // in Window (replaces `pipeline` and `descriptorHeap` pointer fields) std::vector passes; DescriptorHeapVulkan* descriptorHeap; // still here, but now SHARED across all passes std::optional> clearColor; // optional initial clear; default nullopt ``` `Window::Render` becomes: 1. `vkAcquireNextImageKHR`. 2. Begin command buffer. 3. Barrier `UNDEFINED → GENERAL` (existing, unchanged). 4. Bind shared descriptor heap (resource + sampler) **once**. 5. If `clearColor.has_value()`: `vkCmdClearColorImage`. 6. For each `pass : passes`: `pass->Record(cmd, currentBuffer);` — Window inserts a storage→storage memory barrier between consecutive passes that both write to the swapchain image. (V1: insert always; cheap enough. V2: pass-declared write/read sets.) 7. Barrier `GENERAL → PRESENT_SRC_KHR` (existing, unchanged). 8. End / submit / present (existing, unchanged). **3D usage** (replaces the inline `vkCmdTraceRaysKHR` in `Window::Render`): ```cpp // new helper struct shipped with the library: Crafter.Graphics-RTPass.cppm struct RTPass : RenderPass { PipelineRTVulkan* pipeline; void Record(VkCommandBuffer cmd, std::uint32_t frame) override { vkCmdBindPipeline(cmd, VK_PIPELINE_BIND_POINT_RAY_TRACING_KHR, pipeline->pipeline); Device::vkCmdTraceRaysKHR(cmd, &pipeline->raygenRegion, &pipeline->missRegion, &pipeline->hitRegion, &pipeline->callableRegion, width, height, 1); } }; // user code (replaces existing pattern in VulkanTriangle/main.cpp:195-196) RTPass rt{&pipeline}; window.passes.push_back(&rt); ``` **UI usage**: `UIScene` *is* a `RenderPass` (publicly inherits or composes). Its constructor `push_back`s itself onto `window.passes`. The compute dispatch happens inside `UIScene::Record`. **UI-only scenes** (main menu with no 3D): the user simply doesn't add an `RTPass`. UI is the only pass. Optionally set `window.clearColor = {0,0,0,1}` for a known background. No stock pipeline, no `nullptr` checks needed. This **is** an API break: `VulkanTriangle/main.cpp` line 195-196 changes from `window.pipeline = &pipeline; window.descriptorHeap = &descriptorHeap;` to `window.descriptorHeap = &descriptorHeap; RTPass rt{&pipeline}; window.passes.push_back(&rt);`. The user OK'd this in feedback ("I don't mind breaking the existing API as long as it's discussed and done for a good reason"). The reason: future-proof multi-pass support, decouples Window from RT specifically. ## Public API — concrete shape ### Static main menu (replaces ~250 lines of [Forts3D-MainMenu.cpp:24-273](../../repos/3DForts/implementations/Forts3D-MainMenu.cpp#L24-L273)) ```cpp import Crafter.Graphics; using namespace Crafter::UI; UIScene scene(window); scene.Root( Stack{}.background(Image{assetFolder/"Background.tex"}).children( Image{logoTex}.anchor(Anchor::TopCenter).size(Length::Pct(80), Length::Auto()), VStack{}.anchor(Anchor::BottomLeft).padding(8, 16).spacing(6).children( Button{"Sandbox"}.style(themes.menu).onClick([&]{ SwitchScene(Active::Game); }), Button{"Options"}.style(themes.menu).onClick([&]{ SwitchScene(Active::Options); }), Button{"Exit"} .style(themes.exit).onClick([&]{ std::_Exit(0); }) ), Text{std::format("V1.0.0-{}", BUILD_TARGET)} .anchor(Anchor::BottomRight).padding(8).size(15).color(textColor) ) ); ``` ### Dynamic options screen with tabs and bound input fields ```cpp struct OptionsScene { UIScene scene; Options temp = options; OptionsScene(Window& w) : scene(w) { scene.Root(VStack{}.children( HStack{}.height(Length::Pct(7.5)).background(panelBg).padding(8).children( Button{"Exit"}.onClick([&]{ SwitchScene(Active::Main); }), Spacer{}, Text{"OPTIONS"}.size(30), Spacer{}, Button{"Save"}.onClick([&]{ options = temp; SwitchScene(Active::Main); }) ), TabView{} .tab("Graphics", VStack{}.spacing(4).children( OptionRow{"Resolution"}.right(InputField{}.bind(temp.resX, temp.resY)), OptionRow{"Max lights"}.right(InputField{}.bind(temp.maxLights)) )) .tab("Input", BuildInputPage()) .tab("Audio", BuildAudioPage()) )); } }; ``` `InputField{}.bind(member)` does both display (via `std::format`) and parse (via `std::from_chars`); no manual `TryParse`. `TabView` swaps content automatically on tab click — no manual erase/push_back/`UpdateElements`/`CreateBuffer`/`ReorderBuffer` (compare to [Forts3D-OptionsMenu.cpp:307-509](../../repos/3DForts/implementations/Forts3D-OptionsMenu.cpp#L307-L509)). ### Frame-updated HUD overlay ```cpp Observable fps; Observable health{100}; scene.Root(Overlay{}.children( Text{}.bindFmt("{:.1f} fps", fps).anchor(Anchor::TopRight).padding(8), ProgressBar{}.bindValue(health, 0, 100).anchor(Anchor::BottomCenter).size(Length::Px(300), Length::Px(20)) )); window.onUpdate += [&](FrameTime t){ fps = 1.0 / t.delta.count(); health = currentHealth; }; ``` `Observable` mutation marks only its widget dirty; layout re-walks only that subtree; only the affected items in the SSBO are rewritten. ### Per-glyph text styling Most call sites want a single style — that's the simple shorthand: ```cpp Text{"Welcome"}.size(24).color(textColor); ``` For mixed styling — common in HUDs ("Damage: 250" with the number colored), tooltips, dialog text, or future code-editor-like surfaces — pass an explicit list of `TextRun`: ```cpp Text{}.runs( TextRun{"Health: "}.color(white), TextRun{std::format("{}", hp)}.color(hp < 25 ? red : green).bold(), TextRun{" / "}.color(grey), TextRun{"100"}.color(white) ).size(20); // base size; runs can override ``` Each run can independently set `color`, `size`, `weight`, `italic`, `underline`, `strikethrough`, and (V2) `font`. Internally each glyph already produces its own `UIItem`, so per-run styling is just per-item properties — zero shader complexity. Layout splits the runs at line-wrap points naturally. Bound text uses the same machinery: ```cpp Text{}.bind(observableRuns); // Observable> Text{}.bindFmt("{:.1f} fps", fpsObservable); // single-style shorthand for the common case ``` ### User-defined composites ```cpp struct OptionRow { std::string label; Widget rightChild; OptionRow(std::string l) : label(std::move(l)) {} OptionRow& right(Widget w) && { rightChild = std::move(w); return *this; } operator Widget() && { return HStack{}.padding(4, 8).children( Text{label}.size(18).expand(), std::move(rightChild) ); } }; ``` No inheritance, no event-listener fields, no `parent.children.push_back`. A composite is just a function returning a `Widget`. ## Layout system - **Two-pass measure/arrange.** `Measure(availableSize) → desiredSize`; `Arrange(finalRect)`. Cached per-node in `LayoutResult { Rect rectPx; Rect clipPx; }`. - **Units:** `Length::Px(float)` (logical px), `Length::Pct(float)` (% of parent on same axis), `Length::Auto()` (use measured desired), `Length::Frac(float)` (weighted fill — Flutter `Expanded`). - **DPI / fractional-scale:** `UIScene` reads `Window::scale` ([Crafter.Graphics-Window.cppm:136](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L136)) once per layout pass. `Px` values multiply by scale exactly once at logical→device conversion. `Pct` is scale-invariant by construction. - **Containers V1:** `Stack`, `HStack`, `VStack`, `Overlay`, `Grid`. No CSS-style auto-everything. ## Event / interaction model `UIScene` subscribes to existing `Window` events ([Crafter.Graphics-Window.cppm:75-95](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L75-L95)) — no new Window input API needed. - **Hit-testing:** top-down walk, gather containers whose `clipPx` contains cursor, pick topmost interactive leaf. - **Routing:** capture (drag/modal) → tunnel (`onPreviewKeyDown`) → bubble (`onKeyDown`). Handler returns `Handled` to stop. - **Focus:** `UIScene::focused` is a `WidgetRef<>`. Click grabs focus. Tab/Shift-Tab cycles focusables in tree order. `onTextInput` / `onKeyDown` start bubbling from `focused`. - **Drag/scroll:** `Capture()` / `Release()` on the event arg. `ScrollView` captures wheel and drag-with-button-down. ## Rendering pipeline ### Per-frame draw list Single mapped SSBO `VulkanBuffer itemBuf[Window::numFrames]` — matches the existing 3-frame ring pattern ([Crafter.Graphics-Window.cppm:180](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L180)). ```glsl struct UIItem { uint type; // 0=rect, 1=roundRect, 2=glyph, 3=image, 4=line, 5=clipPush, 6=clipPop uint flags; vec2 posPx; // top-left, device px vec2 sizePx; vec4 color; // primary vec4 colorB; // gradient stop / shadow vec4 uvRect; // glyph atlas uv or image source rect uint imageIdx; // bindless image slot (0 = none / use atlas) uint cornerRadiusPx; vec2 reserved; }; // 96 bytes ``` Tree walk emits items front-to-back. Clip stack is encoded inline as `clipPush`/`clipPop` items (Skia/Slate trick — keeps shader simple). ### Compute shader (skeleton) `shaders/ui.comp.glsl` → `ui.spv`, loaded via existing [Crafter.Graphics-ShaderVulkan.cppm:39](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-ShaderVulkan.cppm#L39). ```glsl layout(local_size_x = 16, local_size_y = 16) in; layout(binding = 0, rgba8) uniform image2D outImage; // swapchain layout(binding = 1) buffer ItemBuffer { UIItem items[]; }; layout(binding = 2) uniform sampler2D fontAtlas; // SDF layout(binding = 3) uniform sampler2D bindlessImages[]; void main() { ivec2 p = ivec2(gl_GlobalInvocationID.xy); vec4 dst = imageLoad(outImage, p); // composite OVER existing 3D content ClipStack stack; for (uint i = 0; i < pc.itemCount; ++i) { UIItem it = items[i]; if (it.type == TYPE_CLIP_PUSH) { stack.push(it); continue; } if (it.type == TYPE_CLIP_POP) { stack.pop(); continue; } if (!stack.contains(p)) continue; vec4 src = ShadeItem(it, vec2(p)); dst = vec4(src.rgb + dst.rgb*(1-src.a), src.a + dst.a*(1-src.a)); } imageStore(outImage, p, dst); } ``` Dispatch: `vkCmdDispatch(cmd, ceil(width/16), ceil(height/16), 1)` once per frame. Tile-binning V2 replaces the inner loop with a per-tile range without API change. ### Glyph atlas - One `ImageVulkan` 1024² (growable). Created once at `UIScene` init. - Shelf-allocator inserts glyphs lazily; `Font` already exposes `stbtt_fontinfo` ([Crafter.Graphics-Font.cppm:35](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Font.cppm#L35)). - SDF rasterized via `stbtt_GetGlyphSDF`. One atlas serves all sizes via shader-side smoothstep on screen-space derivative. - Updates batched: layout collects missing glyphs per frame, single `ImageVulkan::Update` ([Crafter.Graphics-ImageVulkan.cppm:97](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-ImageVulkan.cppm#L97)) before dispatch. ### Descriptor management — single shared heap **Hard rule**: the descriptor heap is bound exactly once per frame by `Window::Render`, and never re-bound. All passes (RT, UI, future post-processing) read from the same heap; they index into different *slot ranges* via push constants or specialization constants. This requires extending `DescriptorHeapVulkan` ([Crafter.Graphics-DescriptorHeapVulkan.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm)) with a slot-allocator API on top of its existing fixed-size pre-allocation: ```cpp struct DescriptorHeapVulkan { // existing: resourceHeap[Window::numFrames], samplerHeap[Window::numFrames], etc. // new — bump allocators with optional free-lists in V2 struct ImageSlotRange { std::uint16_t firstElement; std::uint16_t count; }; struct BufferSlotRange { std::uint16_t firstElement; std::uint16_t count; }; struct SamplerSlotRange{ std::uint16_t firstElement; std::uint16_t count; }; ImageSlotRange AllocateImageSlots(std::uint16_t count); BufferSlotRange AllocateBufferSlots(std::uint16_t count); SamplerSlotRange AllocateSamplerSlots(std::uint16_t count); // existing helpers GetBufferOffset / GetBufferOffsetElement still useful // for translating a slot-range to a host-write address. }; ``` **3D usage (migration)**: today, [VulkanTriangle/main.cpp:162-187](../../repos/Crafter/Crafter.Graphics2/examples/VulkanTriangle/main.cpp#L162-L187) hand-computes addresses with `descriptorHeap.bufferStartElement` and `descriptorHeap.bufferStartOffset`. Migrated: ```cpp auto rtImageSlots = descriptorHeap.AllocateImageSlots(3); // 3 swapchain views auto rtTlasSlots = descriptorHeap.AllocateBufferSlots(3); // 3 TLAS addrs // vkWriteResourceDescriptorsEXT writes into those specific offsets // raygen.glsl reads them via specialization constants for the slot indices ``` **UI usage**: `UIScene::Initialize(window)` calls: ```cpp auto atlasSlot = window.descriptorHeap->AllocateImageSlots(1); auto bindlessSlots = window.descriptorHeap->AllocateImageSlots(64); // Image{} widgets auto itemBufSlots = window.descriptorHeap->AllocateBufferSlots(Window::numFrames); auto samplerSlots = window.descriptorHeap->AllocateSamplerSlots(2); // linear + nearest ``` UI's compute shader uses these slot indices passed in via push constants. **No second heap, no rebind**. **Sizing the heap**: the user must size `DescriptorHeapVulkan::Initialize(images, buffers, samplers)` to fit *all* subsystems combined. The library can ship a `Window::EnsureDescriptorHeap(images, buffers, samplers)` helper that lazily creates a default-sized heap (e.g. 128 images, 32 buffers, 16 samplers) if the user hasn't explicitly created one — UI calls this first thing. **For UI-only scenes** (no 3D): user can skip creating a heap entirely; `UIScene` calls `EnsureDescriptorHeap` and gets a sensible default. ### Dispatch site (inside `UIScene::Record(cmd, frame)`) 1. `vkCmdBindPipeline(VK_PIPELINE_BIND_POINT_COMPUTE, uiPipeline)`. 2. (No heap binding — already bound by `Window::Render`.) 3. Push constants: item count, surface size, scale, slot-range starts (atlas, bindless base, item buf, samplers). 4. `vkCmdDispatch(cmd, ceil(width/16), ceil(height/16), 1)`. The `Window`-inserted storage→storage memory barrier between passes ensures the previous pass's writes are visible. ## New files (matching existing convention) Add to [interfaces/](../../repos/Crafter/Crafter.Graphics2/interfaces/): | File | Role | |---|---| | `Crafter.Graphics-RenderPass.cppm` | `RenderPass` base. Used by both UI and 3D. | | `Crafter.Graphics-RTPass.cppm` | `RTPass` helper that records a `vkCmdTraceRaysKHR` call from a `PipelineRTVulkan*`. | | `Crafter.Graphics-UI.cppm` | `:UI` partition; re-exports the sub-partitions. | | `Crafter.Graphics-UI-Length.cppm` | `Length`, `Anchor`, `Edges`, `Color`. | | `Crafter.Graphics-UI-Widget.cppm` | `Widget` (handle), `WidgetRef`, `Observable`. | | `Crafter.Graphics-UI-Widgets.cppm` | Stock widgets (see scope below) including `Text` + `TextRun`. | | `Crafter.Graphics-UI-Layout.cppm` | Measure/arrange engine. | | `Crafter.Graphics-UI-Hit.cppm` | Hit-testing + capture/tunnel/bubble router. | | `Crafter.Graphics-UI-Theme.cppm` | `Theme` struct + `themes::default_dark`. | | `Crafter.Graphics-UI-Atlas.cppm` | SDF glyph atlas atop `Font`. | | `Crafter.Graphics-UI-DrawList.cppm` | `UIItem` + tree→buffer emitter. | | `Crafter.Graphics-UI-Renderer.cppm` | Compute pipeline, per-frame item buffers, dispatch (implements `RenderPass`). | | `Crafter.Graphics-UI-Scene.cppm` | `UIScene` — the only thing user code constructs; owns its `RenderPass` instance. | Add to [implementations/](../../repos/Crafter/Crafter.Graphics2/implementations/): `Crafter.Graphics-UI-Layout.cpp`, `Crafter.Graphics-UI-Hit.cpp`, `Crafter.Graphics-UI-Atlas.cpp`, `Crafter.Graphics-UI-Renderer.cpp`, `Crafter.Graphics-UI-Scene.cpp`. (`RenderPass` and `RTPass` headers are header-only.) Add `shaders/ui.comp.glsl` — compiled to `ui.spv`, loaded via existing [Crafter.Graphics-ShaderVulkan.cppm:39](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-ShaderVulkan.cppm#L39) `ifstream` pattern. ### Updates to existing files - [project.cpp](../../repos/Crafter/Crafter.Graphics2/project.cpp): bump `ifaces` array from 17 → 30 (RenderPass + RTPass + 11 UI), `impls` from 5 → 10. - [interfaces/Crafter.Graphics.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics.cppm): add `export import :RenderPass;`, `export import :RTPass;`, `export import :UI;`. - [interfaces/Crafter.Graphics-Window.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm): replace `PipelineRTVulkan* pipeline;` field (line 193) with `std::vector passes;` and `std::optional> clearColor;`. Keep `DescriptorHeapVulkan* descriptorHeap;` (now shared across all passes). Add forward decl of `RenderPass`. - [implementations/Crafter.Graphics-Window.cpp](../../repos/Crafter/Crafter.Graphics2/implementations/Crafter.Graphics-Window.cpp) `Window::Render` lines 730-844: replace the inline RT-bind-and-trace block (780-804) with: bind heap once (782-802 stays, lifted out of the per-pass code path), optional `vkCmdClearColorImage` if `clearColor`, then `for (auto* p : passes) { p->Record(cmd, currentBuffer); insertStorageBarrier(); }`. - [interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm): add slot-allocator API (`AllocateImageSlots`, `AllocateBufferSlots`, `AllocateSamplerSlots`) plus internal bump-allocator state. Existing static helpers stay. - [examples/VulkanTriangle/main.cpp](../../repos/Crafter/Crafter.Graphics2/examples/VulkanTriangle/main.cpp): migrate to slot-allocator + `RTPass` + `window.passes.push_back`. Lines 162-196 are the affected region; replacement is shorter. ## Migration story (for projects like 3DForts and `VulkanTriangle`) | Old | New | |---|---| | Compose `RenderingElement2D` + `MouseElement` + `EventListener` per widget | Stock widgets from `:UI-Widgets`. | | 7-param `Anchor2D` everywhere | Builder methods (`anchor`, `padding`, `size`); often unneeded inside layout containers. | | Manual `parent.children.push_back` | `.children(...)` | | Per-frame `UpdateElements` / `CreateBuffer` / `ReorderBuffer` / `UpdatePosition` | Implicit; `UIScene` does it. | | Manual `DescriptorHeapVulkan::Initialize` + `WriteDescriptors` per scene with hand-computed offsets | One shared `DescriptorHeapVulkan` for the application; subsystems call `AllocateImageSlots`/`AllocateBufferSlots`/`AllocateSamplerSlots`. | | Custom `RaygenMenu.spv` shader per UI scene | Drop; UI uses one library compute shader. | | `window.pipeline = &p; window.descriptorHeap = &h;` | `window.descriptorHeap = &h; window.passes.push_back(&pass);` for each pass (RT, UI, etc.). UI-only scenes don't push an RT pass — they just rely on `window.clearColor` for background. | The `VulkanTriangle` example migration is the smallest concrete example: lines 162-196 of [main.cpp](../../repos/Crafter/Crafter.Graphics2/examples/VulkanTriangle/main.cpp#L162-L196) shrink because slot offsets come from the allocator instead of being hand-computed; `window.pipeline = ...` becomes `RTPass rt{&pipeline}; window.passes.push_back(&rt);`. Migration is mostly *deletion*. A typical 350-line constructor becomes ~30 lines of declarative tree. ## Verification plan 1. **Build:** `cd Crafter.Graphics2 && crafter-build --vulkan` produces the static lib including all new modules. 2. **GPU smoke:** `examples/UI/main.cpp` (a new example) — a window showing nested stacks, a button that toggles its label, a focused text input, and a progress bar bound to an `Observable` driven from `onUpdate`. All on real GPU. 3. **CPU smoke:** same example with `VK_ICD_FILENAMES=…/lvp_icd.x86_64.json` to force llvmpipe. Confirm visual parity, framerate ≥30 fps for a static menu. 4. **Performance regression check:** port `Forts3D-MainMenu.cpp` and `Forts3D-OptionsMenu.cpp` to the new API; A/B compare frame time on the same hardware. Target: equal or better than old. 5. **3D pipeline non-regression:** after Phase 1, `examples/VulkanTriangle/main.cpp` (now using `RTPass` + `window.passes`) renders identically to its pre-refactor behavior — same triangle, same colors, same framerate. This is the gate before any UI code is written. 6. **Mixed scene:** a third example combines `VulkanTriangle`'s ray-traced triangle with an HUD overlay (two-pass: `RTPass` + `UIScene`); confirms the `RenderPass`/inter-pass-barrier integration works end-to-end and UI composites correctly over RT output. ## Implementation order Roughly 18-20 working days (~4 weeks) for one engineer. **Phase 1 is the foundational refactor that touches the 3D path; it must land cleanly before any UI code is written, with `VulkanTriangle` working at every step.** **Phase 1 — refactor** (must keep `VulkanTriangle` running after each step): 1. Add `RenderPass` base + `RTPass` helper. ½ day. 2. Extend `DescriptorHeapVulkan` with slot-allocator API (bump allocators). ½ day. 3. Refactor `Window::Render`: replace single-pipeline bind/trace with `passes` loop, lift heap-bind out, add optional `clearColor` clear, insert inter-pass barriers. 1 day. 4. Migrate `VulkanTriangle/main.cpp` to the new API. ½ day. **Gate to Phase 2 — must run identically to today.** **Phase 2 — UI core**: 5. `UI-Length`, `UI-Widget` (values + handle + observable). 1 day. 6. `UI-Layout` measure/arrange engine. 1.5 days. 7. Stock widgets in priority order: stacks → Text (with `TextRun`) → Button → Image → InputField → ScrollView → TabView → ProgressBar. 3.5 days. 8. `UI-Theme`. ½ day. 9. `UI-Hit` (hit-test + capture/tunnel/bubble router). 1 day. **Phase 3 — UI rendering**: 10. `UI-Atlas` SDF on top of `Font`. 1.5 days. 11. `UI-DrawList`. ½ day. 12. `shaders/ui.comp.glsl`. 1 day, iterating with renderer. 13. `UI-Renderer` (`RenderPass`-implementing compute dispatcher). 2 days. 14. `UI-Scene` — wires everything to `Window` and registers itself in `window.passes`. ½ day. **Phase 4 — validation & migration**: 15. `examples/UI/main.cpp` showing static menu, dynamic input, HUD overlay over RT. 1 day. 16. Migrate `Forts3D-MainMenu` + `Forts3D-OptionsMenu` as the integration test. 1 day each. ## Decisions locked from user feedback - **Window integration:** `Window::passes` refactor (not the `onAfterTrace` event) — chosen for future-proofing despite breaking `VulkanTriangle`. Migration is part of Phase 1. - **Descriptor heap:** single shared heap, slot-allocator API on `DescriptorHeapVulkan`. **Never re-bound mid-frame.** This was the hard dealbreaker; the original "UI owns its own heap" plan is dropped. - **Widget set V1:** `Stack`, `HStack`, `VStack`, `Overlay`, `Grid`, `Text` (with `TextRun`), `Button`, `Image`, `InputField` (string / integral / float / bound enum), `ScrollView`, `TabView`, `ProgressBar`, `Spacer`. Slider/Checkbox/Dropdown deferred to V2 unless requested. - **Theming:** flat `Theme` struct with named slots, per-instance override via `.style(...)`. No cascading. - **Animation:** the existing `Animation` is not woven into the UI API; users may drive `Observable` from any source (including `Animation` if they want). No `Animated` adapter, no `.fadeIn(2s)` shortcuts in V1. - **Text V1:** single-font, LTR only, soft-wrap on space, kerning from stb. **Per-glyph styling is in V1** via `TextRun` (cheap to add, addresses the "text was a mess" feedback). ICU-grade BiDi/complex-shaping is V2+. - **SPIR-V delivery:** `ifstream` the `.spv` (matches existing `VulkanShader` pattern). - **Multi-window:** one `UIScene` per `Window`. Users wanting cross-window mirroring do it manually. - **RTT / world-space UI:** V2. - **Hot-reload:** V2 (nice to have, not blocking). - **Heap auto-creation:** `UIScene` calls `Window::EnsureDescriptorHeap(128 images, 32 buffers, 16 samplers)` if the user hasn't already attached one. Users wanting tighter control can pre-create their own heap before constructing `UIScene`.