perf(rt): allocate TLAS instance buffer in BAR/VRAM, not system RAM (#65)

The TLAS instance buffer is rebuilt/refit every frame and is co-written by
both the CPU (host-authored fields) and the GPU (the compute shader writes
the transform field in place for transformOwnedByGpu elements). Allocated
HOST_VISIBLE | HOST_COHERENT (system RAM), both GPU accesses crossed PCIe:
the compute write of the transform, and the AS build's read of the instances.

Upgrade the request to HOST_VISIBLE with a best-effort DEVICE_LOCAL preference
(BAR/VRAM), keeping the single shared, persistently-mapped buffer — no staging,
no copy, no extra barrier. A whole-buffer staging copy was rejected: it would
clobber the GPU-written transforms in this in-place co-write design. Now the
compute write and AS build stay in local VRAM; the CPU's write-only,
sequential, never-read-back field writes are the ideal write-combined BAR load.

Correctness:
- DEVICE_LOCAL is a preference only (depends on #59): GetMemoryType falls back
  to plain HOST_VISIBLE (the previous behaviour) on GPUs without resizable BAR.
- HOST_COHERENT is dropped from the required set — the combined type may not be
  coherent. The existing FlushDevice(cmd, ...) already establishes host->build
  visibility and gates its flush-skip on the *chosen* type's flags (#60), so a
  non-coherent BAR type still flushes correctly. The barrier is never removed.

metadataBuffer gets the same upgrade separately (#75).

Extends the TLASHighWaterMark hardware test to assert the chosen memory type
matches the HOST_VISIBLE | prefer-DEVICE_LOCAL request and lands on a
DEVICE_LOCAL type where one is reachable; the existing zero-validation-errors
check covers the dropped HOST_COHERENT path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
catbot 2026-06-17 17:37:39 +00:00
commit d8bc8a4e45
2 changed files with 69 additions and 1 deletions

View file

@ -106,9 +106,24 @@ void RenderingElement3D::BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTB
// STORAGE_BUFFER_BIT is required because the application's compute
// shaders bind these buffers as storage SSBOs (e.g. to write
// per-instance transforms directly into the TLAS instance data).
//
// instanceBuffer prefers HOST_VISIBLE | DEVICE_LOCAL (BAR/VRAM) so the
// two per-frame GPU accesses stay on-chip instead of crossing PCIe: the
// compute shader writes the transform field in place, and the AS build
// reads the whole instance. The CPU still writes the host-authored
// fields below — those are write-only, sequential, never-read-back
// writes, the ideal write-combined BAR workload. DEVICE_LOCAL is a
// best-effort preference: GetMemoryType falls back to plain
// HOST_VISIBLE (the previous behaviour) when no combined type exists
// (no resizable BAR). HOST_COHERENT is intentionally dropped from the
// required set — the combined type may not be coherent, and the
// FlushDevice(cmd, ...) below already establishes host->build
// visibility, gating its flush-skip on the *chosen* type's flags. The
// single shared buffer is preserved: a whole-buffer copy would clobber
// the GPU-written transforms, so staging is deliberately avoided.
if (tlas.instanceBuffer.buffer == VK_NULL_HANDLE
|| primitiveCount > tlas.instanceBuffer.size / sizeof(VkAccelerationStructureInstanceKHR)) {
tlas.instanceBuffer.Resize(VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT | VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_BUILD_INPUT_READ_ONLY_BIT_KHR | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, primitiveCount);
tlas.instanceBuffer.Resize(VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT | VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_BUILD_INPUT_READ_ONLY_BIT_KHR | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, primitiveCount, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
tlas.metadataBuffer.Resize(VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, primitiveCount);
}
}