vertexBuffer/indexBuffer/aabbBuffer were HOST_VISIBLE (system RAM), so the
BLAS build, every refit, and any hit-shader geometry fetch read them over
PCIe. The usage flags (SHADER_DEVICE_ADDRESS, plus STORAGE on the index
buffer) exist precisely to expose geometry for hit-shader fetch, the dominant
RT shading pattern.
Add VulkanBuffer::UploadDeviceLocal, which places a CPU-written/GPU-read
buffer in device-local memory and picks the upload mechanism at runtime via
the #89 strategy (Device::PreferDirectDeviceWrite):
- ReBAR/UMA: allocate HOST_VISIBLE|DEVICE_LOCAL (DEVICE_LOCAL preferred),
map transiently, memcpy, flush-if-non-coherent. No staging buffer.
- no/small BAR: allocate pure DEVICE_LOCAL + TRANSFER_DST, fill a transient
HOST_VISIBLE staging buffer, vkCmdCopyBuffer, then DeferredClear the
staging buffer onto the fence-keyed deletion queue (#101/#102) so it
outlives the copy submit.
The geometry buffers become non-mapped so the destination is free to be
device-local-only. Same-size re-uploads reuse the allocation, so the device
address stays stable across an in-place AS UPDATE refit. The compressed Build
path now allocates vertex/index as pure DEVICE_LOCAL — the GPU decompressor
fills them directly, no host write.
Tests: BLASBuildOptions asserts device-local placement on the triangle,
procedural, and (budget-forced) staged paths, and exercises the staged copy +
deferred-deletion + in-place refit under GPU-assisted validation with zero
validation errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SaveFrame's readback staging buffer was allocated HOST_VISIBLE |
HOST_COHERENT, whose first matching type is write-combined (uncached) on
most discrete GPUs. The CPU then reads the entire frame back from it —
and reads from write-combined memory are pathologically slow.
Select a HOST_CACHED type instead (preferring a cached-coherent type
when one exists), falling back to any HOST_VISIBLE type if no cached type
is advertised (#59 mechanism). When the chosen type is cached-but-not-
coherent, invalidate the mapped range before the CPU read — the read-path
counterpart of FlushDevice (#60) — gated on the chosen type's actual
coherency, not the requested flags. Whole-range invalidate sidesteps
nonCoherentAtomSize.
Deliberately not routed through the #89 upload-strategy helper, which is
upload-only and steers toward write-combined DEVICE_LOCAL|HOST_VISIBLE —
the worst type for readback.
Verified by driving SaveFrame from the VulkanTriangle example: the
barycentric RGB triangle reads back with correct pixels and channel
order on an RTX 4090.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The shader binding table is written once at pipeline creation and read by
the GPU on every vkCmdTraceRaysKHR thereafter, yet it was allocated plain
HOST_VISIBLE (system RAM) — so every trace dispatch read raygen/miss/hit
records over PCIe.
Route the SBT allocation through Device::PreferDirectDeviceWrite (#89):
when a DEVICE_LOCAL|HOST_VISIBLE type exists, prefer DEVICE_LOCAL on top
of the required HOST_VISIBLE so the buffer lands in mappable device
memory (GPU reads hit local VRAM; the one-time memcpy goes write-combined
over PCIe). When no combined type exists, the preferred hint is dropped
and GetMemoryType (#59) falls back to plain HOST_VISIBLE — current
behaviour. The SBT is tiny so a small BAR window's budget is never the
constraint.
The buffer stays mapped and the existing FlushDevice gates on the chosen
memory type's coherency flags (#60), so a direct-write type lacking
HOST_COHERENT is still flushed correctly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MeshDecompressStagingRelease drives the real VK_EXT_memory_decompression /
GDeflate path on a headless device and asserts the staging is handed to the
deferred-deletion queue (not pinned), the build is validation-clean with
byte-correct decompressed data, and the entry retires only after framesInFlight
frames. Skips the GPU-path assertions when the extension is absent (CPU fallback
never allocates the staging).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The compressed Mesh::Build / ImageVulkan::Update paths kept their host-visible
`compressedStaging` (holding the GDeflate streams) alive for the whole life of
the mesh/image, pinning host-visible memory long after the single GPU decompress
that reads it has retired.
Release it via VulkanBuffer::DeferredClear() right after recording the decompress.
The recorded vkCmdDecompressMemoryEXT still references compressedStaging.address,
so it must outlive the submit — exactly what the fence-keyed deletion queue
(#101/#102) guarantees: it retires the allocation only after framesInFlight frames
have elapsed, by which point the submit's fence has cleared. Between Builds/Updates
the handle is null; the next call's Resize re-creates it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The TLAS instance buffer is rebuilt/refit every frame and is co-written by
both the CPU (host-authored fields) and the GPU (the compute shader writes
the transform field in place for transformOwnedByGpu elements). Allocated
HOST_VISIBLE | HOST_COHERENT (system RAM), both GPU accesses crossed PCIe:
the compute write of the transform, and the AS build's read of the instances.
Upgrade the request to HOST_VISIBLE with a best-effort DEVICE_LOCAL preference
(BAR/VRAM), keeping the single shared, persistently-mapped buffer — no staging,
no copy, no extra barrier. A whole-buffer staging copy was rejected: it would
clobber the GPU-written transforms in this in-place co-write design. Now the
compute write and AS build stay in local VRAM; the CPU's write-only,
sequential, never-read-back field writes are the ideal write-combined BAR load.
Correctness:
- DEVICE_LOCAL is a preference only (depends on #59): GetMemoryType falls back
to plain HOST_VISIBLE (the previous behaviour) on GPUs without resizable BAR.
- HOST_COHERENT is dropped from the required set — the combined type may not be
coherent. The existing FlushDevice(cmd, ...) already establishes host->build
visibility and gates its flush-skip on the *chosen* type's flags (#60), so a
non-coherent BAR type still flushes correctly. The barrier is never removed.
metadataBuffer gets the same upgrade separately (#75).
Extends the TLASHighWaterMark hardware test to assert the chosen memory type
matches the HOST_VISIBLE | prefer-DEVICE_LOCAL request and lands on a
DEVICE_LOCAL type where one is reachable; the existing zero-validation-errors
check covers the dropped HOST_COHERENT path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A static (!allowUpdate) mesh can never refit, so its per-mesh DEVICE_LOCAL
scratch buffer is dead weight the moment the build's GPU work completes —
yet it was kept alive for the mesh's whole lifetime. In many-mesh scenes
(where static meshes dominate) this is real, accumulating VRAM waste.
Release it via scratchBuffer.DeferredClear() right after recording a fresh
build for a static mesh. The old "free-after-build is a UAF, the GPU is
still building" hazard is gone: the fence-keyed deferred-deletion queue
(#101/#102) retires the allocation only once the build's frame has passed
its fence. A later Refit on a static mesh takes the rebuild fallback, whose
Resize sees the nulled handle and re-Creates the scratch. Refit-capable
(allowUpdate) meshes keep their scratch as before — an in-place UPDATE
reuses it each frame.
Extends the BLASBuildOptions hardware test to assert the scratch is retained
for allowUpdate meshes and released for static ones (including across a
static-mesh refit rebuild), with the validation layer confirming zero errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A hardware acceleration-structure UPDATE cannot change topology, so the
index buffer is immutable on the Refit UPDATE path. Re-uploading it was a
wasted memcpy + FlushDevice + barrier of unchanged data every refit —
genuinely per-frame for deforming meshes.
Skip the index memcpy and its FlushDevice on the UPDATE branch; the
RecordBLASBuild call still reads the stable, unchanged indexBuffer.address.
The `indicies` span is now read only for its count. Document the API
nuance: a bitwise-different but topologically-equivalent index array is
ignored on refit, consistent with "a refit may only move vertex positions."
The full-rebuild fallback path still re-uploads both buffers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Since #40 dropped the per-frame vkQueueWaitIdle, frames are pipelined:
a resource the CPU is done with may still be read by the GPU for up to
numFrames-1 more frames. VulkanBuffer::Resize's destroy-and-recreate
path was therefore a live use-after-free (#63), no longer masked by the
wait-idle.
Device gains a monotonic, frame-counter-keyed deletion queue:
EnqueueDeletion tags {buffer, memory} with the current frameCounter;
ReclaimDeletions (called per frame after the fence wait) frees entries
once framesInFlight frames have elapsed; DrainDeletions frees everything
after a wait-idle. VulkanBuffer::DeferredClear hands handles to the queue
and nulls the handle, and Resize uses it instead of immediate Clear().
Window::Render sets framesInFlight at init, reclaims after the per-image
fence wait, bumps Device::frameCounter once per frame, and drains on the
resize / OUT_OF_DATE wait-idle paths. The destructor keeps immediate
Clear() (callers destroying mid-flight remain responsible, unchanged).
Adds the DeferredDeletion test: drives the retire timing on a real
headless device with real buffers, stepping Device::frameCounter to pin
the exact reclaim frame, asserting validation stays silent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mip-generation upload path issued N+1 layout-transition barriers per
chain. The final blit's destination level is never read again, so its
dedicated TRANSFER_DST -> TRANSFER_SRC barrier was redundant: the level was
transitioned to SRC only to keep the final SRC -> consumer transition uniform.
Skip that last per-level barrier and fold the final level into the closing
transition, which now batches two VkImageMemoryBarrier entries
([0, mipLevels-1) from TRANSFER_SRC, the last level from TRANSFER_DST) into a
single vkCmdPipelineBarrier. One fewer barrier call per chain; the N-1
interleaved per-level barriers stay one-at-a-time since each is read by the
next blit. Applied to both the host-upload and GPU-decompress Update paths.
Adds the MipChainBarrierBatch regression test driving the pure barrier
builder (no GPU device needed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WriteBufferDescriptor / WriteSampledImageDescriptor / RegisterSampler /
WriteSwapchainDescriptors each FlushDevice()'d the entire multi-KB per-frame
descriptor heap after writing one few-byte descriptor. Add a ranged
VulkanBuffer::FlushDevice(offset, bytes) that flushes only the touched bytes,
rounding the range outward to nonCoherentAtomSize as vkFlushMappedMemoryRanges
requires (the WHOLE_SIZE path sidestepped that). The coherent-memory gate from
issue #60 is preserved — coherent memory still skips the flush entirely.
The descriptor writers now self-flush their written range, so the redundant
whole-heap flushes in UIRenderer::Initialize and the resize callback are gone.
Rounding lives in AlignMappedFlushRange (pure math) and is unit-tested without
a device in the new VulkanBufferRangedFlush test; the change was also verified
end-to-end by running HelloUI on a real GPU with validation layers (font-atlas
glyphs, sampler and storage-image descriptors all flush correctly, no VUIDs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BuildTLAS reallocated the host-visible instanceBuffer and metadataBuffer
on every topology change. They only ever need to hold at least
primitiveCount entries (the AS build reads exactly primitiveCount via
tlasRangeInfo, and the copy loop writes [0, primitiveCount)), so a
shrink — or any growth that still fits the previous capacity — can reuse
the existing allocation. Gate the two Resize calls on a high-water-mark
check, removing two of the four reallocations on a count change. The AS
storage + scratch rebuild below is unchanged: it is tied to the AS
itself and dominates this path regardless.
Adds tests/TLASHighWaterMark driving the real hardware AS-build path: it
asserts a shrink (and within-capacity growth) reuses the buffers while a
growth past the high-water reallocates, that builtInstanceCount still
tracks the live count, and that feeding an oversized instance buffer to
the build produces zero Vulkan validation errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PipelineCacheValidation drives Device::PipelineCacheDataCompatible
directly (no GPU): synthetic device identities + 32-byte cache headers
assert that a matching header is accepted while foreign vendor/device,
a bumped UUID (driver update), unknown version, undersized headerSize,
and short/empty blobs are all rejected. Mirrors the GPU-free style of
MemoryTypeFallback / UploadStrategy. Also gitignore the serialized
pipeline_cache.bin a real-device test run drops in the working dir.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
vkCreateComputePipelines and vkCreateRayTracingPipelinesKHR were passed
VK_NULL_HANDLE, so the driver couldn't reuse compiled shader binaries
across the pipelines built at startup (4 UI shaders + user/RT pipelines)
or across runs.
Add one process-wide Device::pipelineCache. LoadPipelineCache (called
from Initialize) seeds it from pipeline_cache.bin when the file's
VkPipelineCacheHeaderVersionOne header matches this device's
vendorID / deviceID / pipelineCacheUUID; a stale or foreign blob is
discarded so the driver never rejects it. SavePipelineCache is registered
with std::atexit, serialising the cache at process shutdown without any
explicit teardown hook. Both create sites now pass the cache; a null
handle stays valid, so no create-site null check is needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resize previously always destroyed + reallocated. It now reuses the
existing allocation in place when the new request still fits the created
capacity and the immutable-at-create properties match: usage flags are
fixed at create, and the chosen memory type must still satisfy the
required property flags. preferredPropertyFlags is a best-effort perf
hint and is intentionally not part of the guard. On reuse the buffer
handle, device address and mapped pointer are preserved; only `size`
shrinks to the new logical extent.
Tracks capacity and the created usage flags on VulkanBufferBase, set at
Create and carried through the move constructor.
Adds VulkanBufferResizeReuse, a device-free regression test that drives
the reuse guard directly and asserts the in-place path issues no Vulkan
call.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Device::PreferDirectDeviceWrite(size) so the memory-placement cluster
(#65/#72/#73/#75) no longer re-derives where a CPU-written, GPU-read buffer
should live. It picks the strategy at runtime from Device::memoryProperties:
- No DEVICE_LOCAL|HOST_VISIBLE type (no resizable BAR) -> false (must stage).
- ReBAR/UMA (BAR heap >= 90% of largest DEVICE_LOCAL heap) -> true: map and
write directly; a staging copy would be pure overhead.
- Small BAR window (BAR heap << VRAM) -> true only for buffers within a
per-window budget (1/8 of the window); large buffers stage so they don't
exhaust the window (#58).
Complements GetMemoryType (#59): #59 picks the memory *type*, this picks the
*strategy*. Upload-only — readback (#74) wants HOST_CACHED and stays separate.
A VK_EXT_memory_budget-driven remaining-space check is noted as a follow-up.
Tested by tests/UploadStrategy (pure CPU over synthetic ReBAR / UMA /
small-window / no-BAR layouts, no GPU device needed), mirroring
MemoryTypeFallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FlushDevice/FlushHost called vkFlushMappedMemoryRanges /
vkInvalidateMappedMemoryRanges unconditionally. Record the chosen
memory type's propertyFlags at Create time (via the index GetMemoryType
returns, not the requested flags) and early-return when HOST_COHERENT.
The cmd overload still always emits its pipeline barrier — only the
redundant host-side flush/invalidate is gated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a fifth UI compute path — DispatchFused — alongside the four
per-element Dispatch* calls. It composites quads → circles → images →
text (canonical back-to-front order) in ONE dispatch that loads the
destination image once and stores once, eliminating the per-pass
load+store and the inter-pass VkMemoryBarriers a run of consecutive
Dispatch* calls would pay. The win materialises at >= 2 non-empty
categories; an absent category is a zero-trip, push-constant-uniform
loop (~free).
The new uber-kernel (shaders/ui-fused.comp.glsl) reuses each category's
exact cooperative shared-memory tile-cull + per-pixel accumulate, so a
fused category is pixel-identical to its standalone pass. Categories run
sequentially per thread and share one set of shared-memory scratch, so
the VGPR/shared high-water mark stays at the per-category level, not the
sum.
Additive and non-breaking: the per-element Dispatch* calls and the
frozen 48-byte UIDispatchHeader are untouched. DispatchFused gets its own
128-byte push-constant layout (UIFusedHeader: four uvec4 headers + four
per-category clip vec4s). To interleave a custom ui.Dispatch() between
standard passes, bracket it with two DispatchFused calls — the dispatch
boundary is the explicit, app-declared flush point.
WebGPU (Vulkan-second): DispatchFused falls back to the per-element
Dispatch* calls in canonical order — one texture/sampler per dispatch
there can't feed the bindless image phase — so the result matches; only
the load/store fusion is Vulkan-only.
HelloUI now composites its background quads, mouse circle, and label
text in a single DispatchFused. Verified rendering identical on a live
Vulkan/Wayland session.
Resolves#47
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Font::GetLineWidth called stbtt_GetCodepointHMetrics for every glyph on
every invocation. InputField_HitTestCursor rescans the whole field once
per character, making caret hit-testing O(n²) in HMetrics lookups.
Memoise advances per Font via Font::AdvanceUnits: ASCII in a flat array,
the rest in a map. Advances are cached in unscaled font *units* and
rescaled per call — Glyph::advance in the FontAtlas is baked at kBaseSize
and is wrong at any other size, so it cannot be reused. The per-glyph
(int) truncation in GetLineWidth is preserved exactly.
Adds the FontAdvanceCache CPU-only regression test covering size
independence of the cached unit advance, the per-glyph-truncated rescale
pipeline, cache-hit determinism, width scaling with size, and the
non-ASCII map path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
uiResolveScreenPixel ran four float compares against hdr.clipRectPx for
every pixel of every UI dispatch, even though the clip rect is the
full-surface default {0,0,1e9,1e9} in the overwhelmingly common case.
Reserve the high bit of the header flags word (UI_FLAG_CLIP) and have
FillHeader set it only when the clip rect is narrower than the surface
(extracted into the unit-testable UIRenderer::ClipFlags). The shader
gates the compares on the bit; the branch is push-constant-uniform so it
never diverges. When the rect covers the surface the skipped compares
could never reject an in-surface pixel, so output is pixel-identical.
Mirrored on the WebGPU/WGSL path (dom-webgpu.js) for parity. Adds the
UIClipFlag CPU test covering the gate decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
InputField_HitTestCursor mapped a click x to a cursor byte offset by
calling Font::GetLineWidth on every prefix value[0..i] for i=1..size.
Each call re-walks from byte 0, so the hit test was O(n^2) glyph-metric
lookups (each an uncached stbtt cmap binary search). It also iterated
raw byte boundaries, so on multibyte UTF-8 input it could return an
offset splitting a codepoint — an invalid position for the byte-based
cursorPos.
Add Font::NearestCursorByte: one left-to-right cumulative-advance pass
(O(n) metrics, scale computed once) over codepoint boundaries, returning
the byte offset of the boundary nearest the target. Cumulative advance is
monotonic, so the scan stops past the closest boundary. The hit test now
delegates to it.
Adds tests/InputFieldHitTest, a pure-CPU regression test (no Vulkan
device/window): a px sweep across ASCII and multibyte strings compared
against an independent brute-force nearest-boundary oracle, plus the
left/right/empty/end-of-text edge cases and a codepoint-boundary invariant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
UIRenderer::Dispatch inserted a queue-wide VkMemoryBarrier
(SHADER_WRITE -> SHADER_READ|WRITE) before every dispatch after the
first, flushing/invalidating all shader caches even though the only
hazard is the read-modify-write of the single swapchain storage image
that every UI pass shares.
Narrow it to a VkImageMemoryBarrier scoped to that image's subresource
(COMPUTE -> COMPUTE, GENERAL -> GENERAL, no layout transition). Same
correctness — later passes still observe earlier passes' writes — but
only this image's caches are flushed; unrelated resources are no longer
invalidated.
This is a narrower cache flush only: the execution dependency stays
compute->compute queue-wide, so passes still do not overlap. Eliminating
the barriers entirely requires pass fusion (#47) and is intentionally
out of scope here.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GetMemoryType was a bare first-superset match that threw whenever no
memory type satisfied the requested property flags. Callers combining a
mandatory flag with a perf-only one (HOST_VISIBLE | DEVICE_LOCAL for the
descriptor heaps) would then fail allocation on any device without a
host-visible device-local heap (no resizable BAR).
It now takes a `preferred` mask distinct from `required`: a type
satisfying both is chosen first, falling back to a required-only match
when the preference is unavailable, and throwing only when even
`required` cannot be met. VulkanBuffer::Create/Resize gain an optional
trailing `preferredPropertyFlags`, and the descriptor heaps now treat
DEVICE_LOCAL as a preference on top of mandatory HOST_VISIBLE.
This does not touch any flush/barrier behaviour — coherency is not
assumed anywhere here.
Adds tests/MemoryTypeFallback, a pure-CPU test that installs synthetic
memory layouts and exercises selection, preference, fallback, and the
unsatisfiable-required throw.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
