new UI system

This commit is contained in:
Jorijn van der Graaf 2026-05-01 23:35:37 +02:00
commit 216972e73a
82 changed files with 4837 additions and 3243 deletions

435
claude-rewrite-plan.md Normal file
View file

@ -0,0 +1,435 @@
# UI System for Crafter.Graphics2
## Context
The old UI system (still visible in [/home/jorijn/repos/Crafter/Crafter.Graphics/](../../repos/Crafter/Crafter.Graphics/)) was stripped out of Crafter.Graphics2 because it was painful to use. Real-world usages — [/home/jorijn/repos/3DForts/implementations/Forts3D-MainMenu.cpp](../../repos/3DForts/implementations/Forts3D-MainMenu.cpp) (250-line constructor) and [/home/jorijn/repos/3DForts/implementations/Forts3D-OptionsMenu.cpp](../../repos/3DForts/implementations/Forts3D-OptionsMenu.cpp) (510 lines, manual descriptor surgery on every page change) — show the symptoms: 7-param `Anchor2D` repeated everywhere, manual `parent.children.push_back`, manual per-frame `UpdatePosition`, 8-`EventListener`-fields-per-widget callback boilerplate, no widget abstraction at all (users compose `RenderingElement2D` + `MouseElement` + `EventListener` by hand), no batching, no glyph atlas, character-by-character glyph render every frame.
The new system must be:
- User-friendly and future-proof (declarative composition, automatic layout, no manual lifetime juggling).
- At least as performant as the old one (single batched draw, glyph atlas, dirty-tracked layout).
- Cleanly integrated into the existing 3D ray-traced pipeline. **Any change that touches the 3D path needs explicit OK from the user.**
- CPU-runnable as a nice-to-have — but explicitly *not* via a custom CPU rasterizer if it can't beat llvmpipe.
## Design summary
**Three-layer architecture:**
1. **Widget layer (user-facing):** declarative builder API. Widgets are value types with chained `.method()` configuration; composite containers (`VStack`, `HStack`, `Stack`, `Overlay`, `Grid`, `TabView`, `ScrollView`) take children as `&&` parameter packs and own them inside a `UIScene` arena. The only handle user code keeps is `WidgetRef<T>` (a stable typed reference into the scene).
2. **Layout / event layer:** two-pass measure/arrange (WPF/Avalonia/Flutter convention). Hit-testing is automatic from the laid-out tree; events route via capture → tunnel → bubble. Focus is tree-cycled with Tab.
3. **Rendering layer:** widgets emit `UIItem` records into a single per-frame SSBO; one compute-shader dispatch composites everything onto the swapchain image. SDF glyph atlas means one texture serves all sizes/scales.
**Key decisions (locked):**
- **Compute shader, not graphics pipeline.** The swapchain has only `VK_IMAGE_USAGE_STORAGE_BIT` ([Crafter.Graphics-Window.cpp:958](../../repos/Crafter/Crafter.Graphics2/implementations/Crafter.Graphics-Window.cpp#L958)). A compute pass writes the same way the existing RT pipeline does, in the same `VK_IMAGE_LAYOUT_GENERAL`. No swapchain change, no renderpass machinery, no extra barriers.
- **SDF glyph atlas, not per-size raster.** One R8 atlas (1024² growable to 4096²) built lazily from `stb_truetype`. Survives DPI / fractional-scale changes without re-bake. Aliases cleanly at any scale via shader smoothstep.
- **No tile-binning in V1.** A naive front-to-back per-pixel item scan over a ≤500-item draw list comfortably hits sub-millisecond on integrated GPUs. The data layout supports adding tile-binning later without changing the public API.
- **No custom CPU rasterizer.** Same Vulkan code path runs on llvmpipe via the loader. Documented as the CPU fallback.
- **One descriptor heap, bound once per frame, never re-bound.** Per the [`VK_EXT_descriptor_heap` proposal](https://docs.vulkan.org/features/latest/features/proposals/VK_EXT_descriptor_heap.html), heap rebinds are expensive and applications should "stick to the same heap throughout the lifetime of the application". UIScene allocates its slots from the same `Window::descriptorHeap` the user already creates for 3D — never its own. Detail in §"Descriptor management".
- **Window owns a list of `RenderPass*`, not a single pipeline pointer.** Replaces the current `Window::pipeline` / `Window::descriptorHeap` direct-bind model. Future-proofs for post-processing, debug overlays, RTT, multi-pass effects. Detail in §"Window integration".
- **Layout is dirty-tracked.** `Observable<T>::set()` marks owning widgets dirty; layout re-walks only the dirty subtree.
- **Text supports per-glyph styling** via `TextRun`. Each glyph is already a separate draw-list item, so per-run styling adds zero shader complexity — just per-item properties.
## Window integration — `RenderPass` refactor
The current `Window::Render` ([Crafter.Graphics-Window.cpp:730-844](../../repos/Crafter/Crafter.Graphics2/implementations/Crafter.Graphics-Window.cpp#L730-L844)) hardcodes a single ray-tracing pipeline: it binds `Window::pipeline` ([Crafter.Graphics-Window.cppm:193](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L193)) and calls `vkCmdTraceRaysKHR` directly. That assumption breaks the moment we want UI on top, post-processing, debug overlays, or any multi-pass effect.
**Refactor**: replace the single-pipeline pointer with a vector of render passes.
```cpp
// new module: Crafter.Graphics-RenderPass.cppm
struct RenderPass {
virtual void Record(VkCommandBuffer cmd, std::uint32_t frameIdx) = 0;
virtual ~RenderPass() = default;
};
// in Window (replaces `pipeline` and `descriptorHeap` pointer fields)
std::vector<RenderPass*> passes;
DescriptorHeapVulkan* descriptorHeap; // still here, but now SHARED across all passes
std::optional<Vector<float, 4>> clearColor; // optional initial clear; default nullopt
```
`Window::Render` becomes:
1. `vkAcquireNextImageKHR`.
2. Begin command buffer.
3. Barrier `UNDEFINED → GENERAL` (existing, unchanged).
4. Bind shared descriptor heap (resource + sampler) **once**.
5. If `clearColor.has_value()`: `vkCmdClearColorImage`.
6. For each `pass : passes`: `pass->Record(cmd, currentBuffer);` — Window inserts a storage→storage memory barrier between consecutive passes that both write to the swapchain image. (V1: insert always; cheap enough. V2: pass-declared write/read sets.)
7. Barrier `GENERAL → PRESENT_SRC_KHR` (existing, unchanged).
8. End / submit / present (existing, unchanged).
**3D usage** (replaces the inline `vkCmdTraceRaysKHR` in `Window::Render`):
```cpp
// new helper struct shipped with the library: Crafter.Graphics-RTPass.cppm
struct RTPass : RenderPass {
PipelineRTVulkan* pipeline;
void Record(VkCommandBuffer cmd, std::uint32_t frame) override {
vkCmdBindPipeline(cmd, VK_PIPELINE_BIND_POINT_RAY_TRACING_KHR, pipeline->pipeline);
Device::vkCmdTraceRaysKHR(cmd, &pipeline->raygenRegion, &pipeline->missRegion,
&pipeline->hitRegion, &pipeline->callableRegion,
width, height, 1);
}
};
// user code (replaces existing pattern in VulkanTriangle/main.cpp:195-196)
RTPass rt{&pipeline};
window.passes.push_back(&rt);
```
**UI usage**: `UIScene` *is* a `RenderPass` (publicly inherits or composes). Its constructor `push_back`s itself onto `window.passes`. The compute dispatch happens inside `UIScene::Record`.
**UI-only scenes** (main menu with no 3D): the user simply doesn't add an `RTPass`. UI is the only pass. Optionally set `window.clearColor = {0,0,0,1}` for a known background. No stock pipeline, no `nullptr` checks needed.
This **is** an API break: `VulkanTriangle/main.cpp` line 195-196 changes from `window.pipeline = &pipeline; window.descriptorHeap = &descriptorHeap;` to `window.descriptorHeap = &descriptorHeap; RTPass rt{&pipeline}; window.passes.push_back(&rt);`. The user OK'd this in feedback ("I don't mind breaking the existing API as long as it's discussed and done for a good reason"). The reason: future-proof multi-pass support, decouples Window from RT specifically.
## Public API — concrete shape
### Static main menu (replaces ~250 lines of [Forts3D-MainMenu.cpp:24-273](../../repos/3DForts/implementations/Forts3D-MainMenu.cpp#L24-L273))
```cpp
import Crafter.Graphics;
using namespace Crafter::UI;
UIScene scene(window);
scene.Root(
Stack{}.background(Image{assetFolder/"Background.tex"}).children(
Image{logoTex}.anchor(Anchor::TopCenter).size(Length::Pct(80), Length::Auto()),
VStack{}.anchor(Anchor::BottomLeft).padding(8, 16).spacing(6).children(
Button{"Sandbox"}.style(themes.menu).onClick([&]{ SwitchScene(Active::Game); }),
Button{"Options"}.style(themes.menu).onClick([&]{ SwitchScene(Active::Options); }),
Button{"Exit"} .style(themes.exit).onClick([&]{ std::_Exit(0); })
),
Text{std::format("V1.0.0-{}", BUILD_TARGET)}
.anchor(Anchor::BottomRight).padding(8).size(15).color(textColor)
)
);
```
### Dynamic options screen with tabs and bound input fields
```cpp
struct OptionsScene {
UIScene scene;
Options temp = options;
OptionsScene(Window& w) : scene(w) {
scene.Root(VStack{}.children(
HStack{}.height(Length::Pct(7.5)).background(panelBg).padding(8).children(
Button{"Exit"}.onClick([&]{ SwitchScene(Active::Main); }),
Spacer{}, Text{"OPTIONS"}.size(30), Spacer{},
Button{"Save"}.onClick([&]{ options = temp; SwitchScene(Active::Main); })
),
TabView{}
.tab("Graphics", VStack{}.spacing(4).children(
OptionRow{"Resolution"}.right(InputField<Resolution>{}.bind(temp.resX, temp.resY)),
OptionRow{"Max lights"}.right(InputField<uint16_t>{}.bind(temp.maxLights))
))
.tab("Input", BuildInputPage())
.tab("Audio", BuildAudioPage())
));
}
};
```
`InputField<T>{}.bind(member)` does both display (via `std::format`) and parse (via `std::from_chars`); no manual `TryParse`. `TabView` swaps content automatically on tab click — no manual erase/push_back/`UpdateElements`/`CreateBuffer`/`ReorderBuffer` (compare to [Forts3D-OptionsMenu.cpp:307-509](../../repos/3DForts/implementations/Forts3D-OptionsMenu.cpp#L307-L509)).
### Frame-updated HUD overlay
```cpp
Observable<float> fps;
Observable<int> health{100};
scene.Root(Overlay{}.children(
Text{}.bindFmt("{:.1f} fps", fps).anchor(Anchor::TopRight).padding(8),
ProgressBar{}.bindValue(health, 0, 100).anchor(Anchor::BottomCenter).size(Length::Px(300), Length::Px(20))
));
window.onUpdate += [&](FrameTime t){ fps = 1.0 / t.delta.count(); health = currentHealth; };
```
`Observable<T>` mutation marks only its widget dirty; layout re-walks only that subtree; only the affected items in the SSBO are rewritten.
### Per-glyph text styling
Most call sites want a single style — that's the simple shorthand:
```cpp
Text{"Welcome"}.size(24).color(textColor);
```
For mixed styling — common in HUDs ("Damage: 250" with the number colored), tooltips, dialog text, or future code-editor-like surfaces — pass an explicit list of `TextRun`:
```cpp
Text{}.runs(
TextRun{"Health: "}.color(white),
TextRun{std::format("{}", hp)}.color(hp < 25 ? red : green).bold(),
TextRun{" / "}.color(grey),
TextRun{"100"}.color(white)
).size(20); // base size; runs can override
```
Each run can independently set `color`, `size`, `weight`, `italic`, `underline`, `strikethrough`, and (V2) `font`. Internally each glyph already produces its own `UIItem`, so per-run styling is just per-item properties — zero shader complexity. Layout splits the runs at line-wrap points naturally.
Bound text uses the same machinery:
```cpp
Text{}.bind(observableRuns); // Observable<std::vector<TextRun>>
Text{}.bindFmt("{:.1f} fps", fpsObservable); // single-style shorthand for the common case
```
### User-defined composites
```cpp
struct OptionRow {
std::string label;
Widget rightChild;
OptionRow(std::string l) : label(std::move(l)) {}
OptionRow& right(Widget w) && { rightChild = std::move(w); return *this; }
operator Widget() && {
return HStack{}.padding(4, 8).children(
Text{label}.size(18).expand(), std::move(rightChild)
);
}
};
```
No inheritance, no event-listener fields, no `parent.children.push_back`. A composite is just a function returning a `Widget`.
## Layout system
- **Two-pass measure/arrange.** `Measure(availableSize) → desiredSize`; `Arrange(finalRect)`. Cached per-node in `LayoutResult { Rect rectPx; Rect clipPx; }`.
- **Units:** `Length::Px(float)` (logical px), `Length::Pct(float)` (% of parent on same axis), `Length::Auto()` (use measured desired), `Length::Frac(float)` (weighted fill — Flutter `Expanded`).
- **DPI / fractional-scale:** `UIScene` reads `Window::scale` ([Crafter.Graphics-Window.cppm:136](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L136)) once per layout pass. `Px` values multiply by scale exactly once at logical→device conversion. `Pct` is scale-invariant by construction.
- **Containers V1:** `Stack`, `HStack`, `VStack`, `Overlay`, `Grid`. No CSS-style auto-everything.
## Event / interaction model
`UIScene` subscribes to existing `Window` events ([Crafter.Graphics-Window.cppm:75-95](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L75-L95)) — no new Window input API needed.
- **Hit-testing:** top-down walk, gather containers whose `clipPx` contains cursor, pick topmost interactive leaf.
- **Routing:** capture (drag/modal) → tunnel (`onPreviewKeyDown`) → bubble (`onKeyDown`). Handler returns `Handled` to stop.
- **Focus:** `UIScene::focused` is a `WidgetRef<>`. Click grabs focus. Tab/Shift-Tab cycles focusables in tree order. `onTextInput` / `onKeyDown` start bubbling from `focused`.
- **Drag/scroll:** `Capture()` / `Release()` on the event arg. `ScrollView` captures wheel and drag-with-button-down.
## Rendering pipeline
### Per-frame draw list
Single mapped SSBO `VulkanBuffer<UIItem, true> itemBuf[Window::numFrames]` — matches the existing 3-frame ring pattern ([Crafter.Graphics-Window.cppm:180](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm#L180)).
```glsl
struct UIItem {
uint type; // 0=rect, 1=roundRect, 2=glyph, 3=image, 4=line, 5=clipPush, 6=clipPop
uint flags;
vec2 posPx; // top-left, device px
vec2 sizePx;
vec4 color; // primary
vec4 colorB; // gradient stop / shadow
vec4 uvRect; // glyph atlas uv or image source rect
uint imageIdx; // bindless image slot (0 = none / use atlas)
uint cornerRadiusPx;
vec2 reserved;
}; // 96 bytes
```
Tree walk emits items front-to-back. Clip stack is encoded inline as `clipPush`/`clipPop` items (Skia/Slate trick — keeps shader simple).
### Compute shader (skeleton)
`shaders/ui.comp.glsl``ui.spv`, loaded via existing [Crafter.Graphics-ShaderVulkan.cppm:39](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-ShaderVulkan.cppm#L39).
```glsl
layout(local_size_x = 16, local_size_y = 16) in;
layout(binding = 0, rgba8) uniform image2D outImage; // swapchain
layout(binding = 1) buffer ItemBuffer { UIItem items[]; };
layout(binding = 2) uniform sampler2D fontAtlas; // SDF
layout(binding = 3) uniform sampler2D bindlessImages[];
void main() {
ivec2 p = ivec2(gl_GlobalInvocationID.xy);
vec4 dst = imageLoad(outImage, p); // composite OVER existing 3D content
ClipStack stack;
for (uint i = 0; i < pc.itemCount; ++i) {
UIItem it = items[i];
if (it.type == TYPE_CLIP_PUSH) { stack.push(it); continue; }
if (it.type == TYPE_CLIP_POP) { stack.pop(); continue; }
if (!stack.contains(p)) continue;
vec4 src = ShadeItem(it, vec2(p));
dst = vec4(src.rgb + dst.rgb*(1-src.a), src.a + dst.a*(1-src.a));
}
imageStore(outImage, p, dst);
}
```
Dispatch: `vkCmdDispatch(cmd, ceil(width/16), ceil(height/16), 1)` once per frame. Tile-binning V2 replaces the inner loop with a per-tile range without API change.
### Glyph atlas
- One `ImageVulkan<R8>` 1024² (growable). Created once at `UIScene` init.
- Shelf-allocator inserts glyphs lazily; `Font` already exposes `stbtt_fontinfo` ([Crafter.Graphics-Font.cppm:35](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Font.cppm#L35)).
- SDF rasterized via `stbtt_GetGlyphSDF`. One atlas serves all sizes via shader-side smoothstep on screen-space derivative.
- Updates batched: layout collects missing glyphs per frame, single `ImageVulkan::Update` ([Crafter.Graphics-ImageVulkan.cppm:97](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-ImageVulkan.cppm#L97)) before dispatch.
### Descriptor management — single shared heap
**Hard rule**: the descriptor heap is bound exactly once per frame by `Window::Render`, and never re-bound. All passes (RT, UI, future post-processing) read from the same heap; they index into different *slot ranges* via push constants or specialization constants.
This requires extending `DescriptorHeapVulkan` ([Crafter.Graphics-DescriptorHeapVulkan.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm)) with a slot-allocator API on top of its existing fixed-size pre-allocation:
```cpp
struct DescriptorHeapVulkan {
// existing: resourceHeap[Window::numFrames], samplerHeap[Window::numFrames], etc.
// new — bump allocators with optional free-lists in V2
struct ImageSlotRange { std::uint16_t firstElement; std::uint16_t count; };
struct BufferSlotRange { std::uint16_t firstElement; std::uint16_t count; };
struct SamplerSlotRange{ std::uint16_t firstElement; std::uint16_t count; };
ImageSlotRange AllocateImageSlots(std::uint16_t count);
BufferSlotRange AllocateBufferSlots(std::uint16_t count);
SamplerSlotRange AllocateSamplerSlots(std::uint16_t count);
// existing helpers GetBufferOffset / GetBufferOffsetElement still useful
// for translating a slot-range to a host-write address.
};
```
**3D usage (migration)**: today, [VulkanTriangle/main.cpp:162-187](../../repos/Crafter/Crafter.Graphics2/examples/VulkanTriangle/main.cpp#L162-L187) hand-computes addresses with `descriptorHeap.bufferStartElement` and `descriptorHeap.bufferStartOffset`. Migrated:
```cpp
auto rtImageSlots = descriptorHeap.AllocateImageSlots(3); // 3 swapchain views
auto rtTlasSlots = descriptorHeap.AllocateBufferSlots(3); // 3 TLAS addrs
// vkWriteResourceDescriptorsEXT writes into those specific offsets
// raygen.glsl reads them via specialization constants for the slot indices
```
**UI usage**: `UIScene::Initialize(window)` calls:
```cpp
auto atlasSlot = window.descriptorHeap->AllocateImageSlots(1);
auto bindlessSlots = window.descriptorHeap->AllocateImageSlots(64); // Image{} widgets
auto itemBufSlots = window.descriptorHeap->AllocateBufferSlots(Window::numFrames);
auto samplerSlots = window.descriptorHeap->AllocateSamplerSlots(2); // linear + nearest
```
UI's compute shader uses these slot indices passed in via push constants. **No second heap, no rebind**.
**Sizing the heap**: the user must size `DescriptorHeapVulkan::Initialize(images, buffers, samplers)` to fit *all* subsystems combined. The library can ship a `Window::EnsureDescriptorHeap(images, buffers, samplers)` helper that lazily creates a default-sized heap (e.g. 128 images, 32 buffers, 16 samplers) if the user hasn't explicitly created one — UI calls this first thing.
**For UI-only scenes** (no 3D): user can skip creating a heap entirely; `UIScene` calls `EnsureDescriptorHeap` and gets a sensible default.
### Dispatch site (inside `UIScene::Record(cmd, frame)`)
1. `vkCmdBindPipeline(VK_PIPELINE_BIND_POINT_COMPUTE, uiPipeline)`.
2. (No heap binding — already bound by `Window::Render`.)
3. Push constants: item count, surface size, scale, slot-range starts (atlas, bindless base, item buf, samplers).
4. `vkCmdDispatch(cmd, ceil(width/16), ceil(height/16), 1)`.
The `Window`-inserted storage→storage memory barrier between passes ensures the previous pass's writes are visible.
## New files (matching existing convention)
Add to [interfaces/](../../repos/Crafter/Crafter.Graphics2/interfaces/):
| File | Role |
|---|---|
| `Crafter.Graphics-RenderPass.cppm` | `RenderPass` base. Used by both UI and 3D. |
| `Crafter.Graphics-RTPass.cppm` | `RTPass` helper that records a `vkCmdTraceRaysKHR` call from a `PipelineRTVulkan*`. |
| `Crafter.Graphics-UI.cppm` | `:UI` partition; re-exports the sub-partitions. |
| `Crafter.Graphics-UI-Length.cppm` | `Length`, `Anchor`, `Edges`, `Color`. |
| `Crafter.Graphics-UI-Widget.cppm` | `Widget` (handle), `WidgetRef<T>`, `Observable<T>`. |
| `Crafter.Graphics-UI-Widgets.cppm` | Stock widgets (see scope below) including `Text` + `TextRun`. |
| `Crafter.Graphics-UI-Layout.cppm` | Measure/arrange engine. |
| `Crafter.Graphics-UI-Hit.cppm` | Hit-testing + capture/tunnel/bubble router. |
| `Crafter.Graphics-UI-Theme.cppm` | `Theme` struct + `themes::default_dark`. |
| `Crafter.Graphics-UI-Atlas.cppm` | SDF glyph atlas atop `Font`. |
| `Crafter.Graphics-UI-DrawList.cppm` | `UIItem` + tree→buffer emitter. |
| `Crafter.Graphics-UI-Renderer.cppm` | Compute pipeline, per-frame item buffers, dispatch (implements `RenderPass`). |
| `Crafter.Graphics-UI-Scene.cppm` | `UIScene` — the only thing user code constructs; owns its `RenderPass` instance. |
Add to [implementations/](../../repos/Crafter/Crafter.Graphics2/implementations/):
`Crafter.Graphics-UI-Layout.cpp`, `Crafter.Graphics-UI-Hit.cpp`, `Crafter.Graphics-UI-Atlas.cpp`, `Crafter.Graphics-UI-Renderer.cpp`, `Crafter.Graphics-UI-Scene.cpp`. (`RenderPass` and `RTPass` headers are header-only.)
Add `shaders/ui.comp.glsl` — compiled to `ui.spv`, loaded via existing [Crafter.Graphics-ShaderVulkan.cppm:39](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-ShaderVulkan.cppm#L39) `ifstream` pattern.
### Updates to existing files
- [project.cpp](../../repos/Crafter/Crafter.Graphics2/project.cpp): bump `ifaces` array from 17 → 30 (RenderPass + RTPass + 11 UI), `impls` from 5 → 10.
- [interfaces/Crafter.Graphics.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics.cppm): add `export import :RenderPass;`, `export import :RTPass;`, `export import :UI;`.
- [interfaces/Crafter.Graphics-Window.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-Window.cppm): replace `PipelineRTVulkan* pipeline;` field (line 193) with `std::vector<RenderPass*> passes;` and `std::optional<Vector<float, 4>> clearColor;`. Keep `DescriptorHeapVulkan* descriptorHeap;` (now shared across all passes). Add forward decl of `RenderPass`.
- [implementations/Crafter.Graphics-Window.cpp](../../repos/Crafter/Crafter.Graphics2/implementations/Crafter.Graphics-Window.cpp) `Window::Render` lines 730-844: replace the inline RT-bind-and-trace block (780-804) with: bind heap once (782-802 stays, lifted out of the per-pass code path), optional `vkCmdClearColorImage` if `clearColor`, then `for (auto* p : passes) { p->Record(cmd, currentBuffer); insertStorageBarrier(); }`.
- [interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm](../../repos/Crafter/Crafter.Graphics2/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm): add slot-allocator API (`AllocateImageSlots`, `AllocateBufferSlots`, `AllocateSamplerSlots`) plus internal bump-allocator state. Existing static helpers stay.
- [examples/VulkanTriangle/main.cpp](../../repos/Crafter/Crafter.Graphics2/examples/VulkanTriangle/main.cpp): migrate to slot-allocator + `RTPass` + `window.passes.push_back`. Lines 162-196 are the affected region; replacement is shorter.
## Migration story (for projects like 3DForts and `VulkanTriangle`)
| Old | New |
|---|---|
| Compose `RenderingElement2D` + `MouseElement` + `EventListener` per widget | Stock widgets from `:UI-Widgets`. |
| 7-param `Anchor2D` everywhere | Builder methods (`anchor`, `padding`, `size`); often unneeded inside layout containers. |
| Manual `parent.children.push_back` | `.children(...)` |
| Per-frame `UpdateElements` / `CreateBuffer` / `ReorderBuffer` / `UpdatePosition` | Implicit; `UIScene` does it. |
| Manual `DescriptorHeapVulkan::Initialize` + `WriteDescriptors` per scene with hand-computed offsets | One shared `DescriptorHeapVulkan` for the application; subsystems call `AllocateImageSlots`/`AllocateBufferSlots`/`AllocateSamplerSlots`. |
| Custom `RaygenMenu.spv` shader per UI scene | Drop; UI uses one library compute shader. |
| `window.pipeline = &p; window.descriptorHeap = &h;` | `window.descriptorHeap = &h; window.passes.push_back(&pass);` for each pass (RT, UI, etc.). UI-only scenes don't push an RT pass — they just rely on `window.clearColor` for background. |
The `VulkanTriangle` example migration is the smallest concrete example: lines 162-196 of [main.cpp](../../repos/Crafter/Crafter.Graphics2/examples/VulkanTriangle/main.cpp#L162-L196) shrink because slot offsets come from the allocator instead of being hand-computed; `window.pipeline = ...` becomes `RTPass rt{&pipeline}; window.passes.push_back(&rt);`.
Migration is mostly *deletion*. A typical 350-line constructor becomes ~30 lines of declarative tree.
## Verification plan
1. **Build:** `cd Crafter.Graphics2 && crafter-build --vulkan` produces the static lib including all new modules.
2. **GPU smoke:** `examples/UI/main.cpp` (a new example) — a window showing nested stacks, a button that toggles its label, a focused text input, and a progress bar bound to an `Observable<float>` driven from `onUpdate`. All on real GPU.
3. **CPU smoke:** same example with `VK_ICD_FILENAMES=…/lvp_icd.x86_64.json` to force llvmpipe. Confirm visual parity, framerate ≥30 fps for a static menu.
4. **Performance regression check:** port `Forts3D-MainMenu.cpp` and `Forts3D-OptionsMenu.cpp` to the new API; A/B compare frame time on the same hardware. Target: equal or better than old.
5. **3D pipeline non-regression:** after Phase 1, `examples/VulkanTriangle/main.cpp` (now using `RTPass` + `window.passes`) renders identically to its pre-refactor behavior — same triangle, same colors, same framerate. This is the gate before any UI code is written.
6. **Mixed scene:** a third example combines `VulkanTriangle`'s ray-traced triangle with an HUD overlay (two-pass: `RTPass` + `UIScene`); confirms the `RenderPass`/inter-pass-barrier integration works end-to-end and UI composites correctly over RT output.
## Implementation order
Roughly 18-20 working days (~4 weeks) for one engineer. **Phase 1 is the foundational refactor that touches the 3D path; it must land cleanly before any UI code is written, with `VulkanTriangle` working at every step.**
**Phase 1 — refactor** (must keep `VulkanTriangle` running after each step):
1. Add `RenderPass` base + `RTPass` helper. ½ day.
2. Extend `DescriptorHeapVulkan` with slot-allocator API (bump allocators). ½ day.
3. Refactor `Window::Render`: replace single-pipeline bind/trace with `passes` loop, lift heap-bind out, add optional `clearColor` clear, insert inter-pass barriers. 1 day.
4. Migrate `VulkanTriangle/main.cpp` to the new API. ½ day. **Gate to Phase 2 — must run identically to today.**
**Phase 2 — UI core**:
5. `UI-Length`, `UI-Widget` (values + handle + observable). 1 day.
6. `UI-Layout` measure/arrange engine. 1.5 days.
7. Stock widgets in priority order: stacks → Text (with `TextRun`) → Button → Image → InputField → ScrollView → TabView → ProgressBar. 3.5 days.
8. `UI-Theme`. ½ day.
9. `UI-Hit` (hit-test + capture/tunnel/bubble router). 1 day.
**Phase 3 — UI rendering**:
10. `UI-Atlas` SDF on top of `Font`. 1.5 days.
11. `UI-DrawList`. ½ day.
12. `shaders/ui.comp.glsl`. 1 day, iterating with renderer.
13. `UI-Renderer` (`RenderPass`-implementing compute dispatcher). 2 days.
14. `UI-Scene` — wires everything to `Window` and registers itself in `window.passes`. ½ day.
**Phase 4 — validation & migration**:
15. `examples/UI/main.cpp` showing static menu, dynamic input, HUD overlay over RT. 1 day.
16. Migrate `Forts3D-MainMenu` + `Forts3D-OptionsMenu` as the integration test. 1 day each.
## Decisions locked from user feedback
- **Window integration:** `Window::passes` refactor (not the `onAfterTrace` event) — chosen for future-proofing despite breaking `VulkanTriangle`. Migration is part of Phase 1.
- **Descriptor heap:** single shared heap, slot-allocator API on `DescriptorHeapVulkan`. **Never re-bound mid-frame.** This was the hard dealbreaker; the original "UI owns its own heap" plan is dropped.
- **Widget set V1:** `Stack`, `HStack`, `VStack`, `Overlay`, `Grid`, `Text` (with `TextRun`), `Button`, `Image`, `InputField` (string / integral / float / bound enum), `ScrollView`, `TabView`, `ProgressBar`, `Spacer`. Slider/Checkbox/Dropdown deferred to V2 unless requested.
- **Theming:** flat `Theme` struct with named slots, per-instance override via `.style(...)`. No cascading.
- **Animation:** the existing `Animation<T>` is not woven into the UI API; users may drive `Observable<T>` from any source (including `Animation<T>` if they want). No `Animated<T>` adapter, no `.fadeIn(2s)` shortcuts in V1.
- **Text V1:** single-font, LTR only, soft-wrap on space, kerning from stb. **Per-glyph styling is in V1** via `TextRun` (cheap to add, addresses the "text was a mess" feedback). ICU-grade BiDi/complex-shaping is V2+.
- **SPIR-V delivery:** `ifstream` the `.spv` (matches existing `VulkanShader` pattern).
- **Multi-window:** one `UIScene` per `Window`. Users wanting cross-window mirroring do it manually.
- **RTT / world-space UI:** V2.
- **Hot-reload:** V2 (nice to have, not blocking).
- **Heap auto-creation:** `UIScene` calls `Window::EnsureDescriptorHeap(128 images, 32 buffers, 16 samplers)` if the user hasn't already attached one. Users wanting tighter control can pre-create their own heap before constructing `UIScene`.