Catcrafts/Crafter.Graphics
Watch
2
0
Fork
You've already forked Crafter.Graphics
 0
Code Issues 25 Pull requests Projects Releases Packages Wiki Activity Actions

fix(rt): rebuild TLASes in place, after earlier traces, and queue their handles like a Mesh's

Browse source 
BuildTLAS
- A rebuild for a new instance count or build preference used to create a
  new acceleration structure and destroy the old one at once, though the
  command buffer being recorded could still use it (a scene and then a
  model built in one setup). The acceleration structure now spans its whole
  storage buffer, and a rebuild that fits builds into it again, so the
  handle and tlases[i].address stay the same and nothing is destroyed. A
  TLAS that outgrows its storage gets new storage and a new handle; the old
  handle goes to the deletion queue ahead of the storage it lives in. It is
  queued after the scratch is resized, and its address goes with it, so a
  Resize or create that throws leaves no address of a queued acceleration
  structure behind.
- The barrier before the build also orders earlier traces and compute ray
  queries of the TLAS, which a refit or a rebuild in place would otherwise
  overwrite while they read it (BuildTlasWriteBarrier). Its source access
  mask holds no read bit: one does nothing, but makes synchronization
  validation 1.4.363 report the second of two builds in one command buffer
  as a write-after-write hazard.
- TlasReadBarrier is now TlasBarrier, shared by both barriers.
- Documented: call BuildTLAS once every earlier submission that used
  tlases[index] has completed, and when the address stays the same.

TlasWithBuffer
- Moves hand the acceleration structure over, and destroying or assigning
  over one queues it ahead of its storage. The implicit members dropped the
  handle over freed storage, or left two owners and a moved-from TLAS that
  refit through buffers it no longer had.

Tests
- TLASLifetime: destroying, moving, move-assigning and resetting a
  TlasWithBuffer.
- TLASHighWaterMark skips without ray tracing. It asserts the acceleration
  structure is rebuilt in place on a shrink or a growth within capacity,
  that the old handle is queued before its storage on a growth past it, and
  that builds in one command buffer keep the handle while they fit, queue
  it once when one outgrows the storage, and free it only on its retire
  frame.
- TLASBuildBarrier checks the barrier before the build too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Jorijn van der Graaf 2026-10-31 02:32:48 +01:00
commit 397ff2db51
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
6 changed files with 516 additions and 74 deletions
Download patch file Download diff file Expand all files Collapse all files

116
implementations/Crafter.Graphics-RenderingElement3D.cpp
View file

@ -9,6 +9,40 @@ import std;
using namespace Crafter;
TlasWithBuffer::TlasWithBuffer(TlasWithBuffer&& other)
: address(std::exchange(other.address, 0))
, buffer(std::move(other.buffer))
, accelerationStructure(std::exchange(other.accelerationStructure, VK_NULL_HANDLE))
, instanceBuffer(std::move(other.instanceBuffer))
, scratchBuffer(std::move(other.scratchBuffer))
, metadataBuffer(std::move(other.metadataBuffer))
, builtInstanceCount(other.builtInstanceCount)
, builtFlags(other.builtFlags)
, uploadedVersion(std::move(other.uploadedVersion)) {
}
TlasWithBuffer& TlasWithBuffer::operator=(TlasWithBuffer&& other) {
if (this == &other) return *this;
// Ahead of buffer, which it lives in and which queues its own as it
// takes other's over.
Device::EnqueueDeletion(accelerationStructure);
address = std::exchange(other.address, 0);
buffer = std::move(other.buffer);
accelerationStructure = std::exchange(other.accelerationStructure, VK_NULL_HANDLE);
instanceBuffer = std::move(other.instanceBuffer);
scratchBuffer = std::move(other.scratchBuffer);
metadataBuffer = std::move(other.metadataBuffer);
builtInstanceCount = other.builtInstanceCount;
builtFlags = other.builtFlags;
uploadedVersion = std::move(other.uploadedVersion);
return *this;
}
TlasWithBuffer::~TlasWithBuffer() {
// The body runs before the members' destructors, so the handle is queued
// ahead of buffer, which it lives in.
Device::EnqueueDeletion(accelerationStructure);
}
std::vector<RenderingElement3D*> RenderingElement3D::elements;
@ -50,8 +84,8 @@ void RenderingElement3D::BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTB
// Refit (UPDATE) is allowed when the count matches the count this AS
// was last built for and the build flags are unchanged. A change forces
// a full rebuild because the AS storage and instance buffer were sized
// for the old count (and a refit must inherit the original build flags).
// a full rebuild, since a refit must keep the primitive count and the
// build flags of the build it updates.
// Refit is dramatically cheaper at scale (millions of instances) — it
// walks the existing BVH and updates AABBs rather than reconstructing
// topology.
@ -60,17 +94,8 @@ void RenderingElement3D::BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTB
|| primitiveCount != tlas.builtInstanceCount
|| buildFlags != tlas.builtFlags;
{
VkMemoryBarrier asBarrier {
.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER,
.srcAccessMask = VK_ACCESS_ACCELERATION_STRUCTURE_WRITE_BIT_KHR,
.dstAccessMask = VK_ACCESS_ACCELERATION_STRUCTURE_READ_BIT_KHR | VK_ACCESS_ACCELERATION_STRUCTURE_WRITE_BIT_KHR
};
vkCmdPipelineBarrier(cmd,
VK_PIPELINE_STAGE_ACCELERATION_STRUCTURE_BUILD_BIT_KHR,
VK_PIPELINE_STAGE_ACCELERATION_STRUCTURE_BUILD_BIT_KHR,
0, 1, &asBarrier, 0, nullptr, 0, nullptr);
}
const TlasBarrier writeBarrier = BuildTlasWriteBarrier();
vkCmdPipelineBarrier(cmd, writeBarrier.srcStage, writeBarrier.dstStage, 0, 1, &writeBarrier.memory, 0, nullptr, 0, nullptr);
if (topologyChanged) {
// Grow the host-visible inputs on a high-water mark rather than
@ -81,8 +106,9 @@ void RenderingElement3D::BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTB
// over from a larger earlier frame is reused as-is. This drops the
// two host-buffer reallocations on a count decrease (and on an
// increase that still fits the previous high-water capacity),
// leaving only the AS storage + scratch rebuild below, which is tied
// to the AS itself and dominates this path regardless.
// leaving only the AS build below, which dominates this path
// regardless; the AS storage and scratch follow a high-water mark
// of their own.
//
// instanceBuffer and metadataBuffer grow in lockstep (always resized
// together to the same entry count), so one capacity check covers
@ -241,7 +267,7 @@ void RenderingElement3D::BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTB
};
if (topologyChanged) {
// Query sizes for the fresh build, allocate AS storage + scratch.
// Query sizes for the fresh build, and make room for it.
VkAccelerationStructureBuildSizesInfoKHR tlasBuildSizes {
.sType = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_BUILD_SIZES_INFO_KHR
};
@ -258,36 +284,50 @@ void RenderingElement3D::BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTB
// An empty TLAS may need no scratch at all, but the buffer can't be
// zero-size.
tlas.scratchBuffer.Resize(VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, std::max<VkDeviceSize>(tlasBuildSizes.buildScratchSize, 1));
tlas.buffer.Resize(VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_STORAGE_BIT_KHR | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT | VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_BUILD_INPUT_READ_ONLY_BIT_KHR, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, tlasBuildSizes.accelerationStructureSize);
// Destroy the previous AS handle before creating a new one — the
// pre-refit path leaked here on every frame.
if (tlas.accelerationStructure != VK_NULL_HANDLE) {
Device::vkDestroyAccelerationStructureKHR(Device::device, tlas.accelerationStructure, nullptr);
// The acceleration structure spans all of its storage, so a build
// that fits the storage builds into it again: a BUILD may write any
// acceleration structure created at least as large as it needs. The
// handle and address then stay the same across count and flag
// changes, and nothing is queued. Only a TLAS that outgrows its
// storage gets new storage and a new acceleration structure.
if (tlas.accelerationStructure == VK_NULL_HANDLE
|| tlasBuildSizes.accelerationStructureSize > tlas.buffer.capacity) {
// The old one may still be used earlier in this command buffer,
// by an earlier build of this index or a trace of it, so it's
// queued rather than destroyed; ahead of the storage it lives in,
// which the Resize below queues. Nothing in flight uses it (see
// BuildTLAS's precondition). The address goes with it, so a
// Resize or create that throws doesn't leave the address of a
// queued acceleration structure behind.
Device::EnqueueDeletion(tlas.accelerationStructure);
tlas.accelerationStructure = VK_NULL_HANDLE;
tlas.address = 0;
tlas.buffer.Resize(VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_STORAGE_BIT_KHR | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT | VK_BUFFER_USAGE_ACCELERATION_STRUCTURE_BUILD_INPUT_READ_ONLY_BIT_KHR, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, tlasBuildSizes.accelerationStructureSize);
VkAccelerationStructureCreateInfoKHR tlasCreateInfo {
.sType = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_CREATE_INFO_KHR,
.buffer = tlas.buffer.buffer,
.offset = 0,
.size = tlas.buffer.capacity,
.type = VK_ACCELERATION_STRUCTURE_TYPE_TOP_LEVEL_KHR,
};
Device::CheckVkResult(Device::vkCreateAccelerationStructureKHR(Device::device, &tlasCreateInfo, nullptr, &tlas.accelerationStructure));
VkAccelerationStructureDeviceAddressInfoKHR addrInfo {
.sType = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_DEVICE_ADDRESS_INFO_KHR,
.accelerationStructure = tlas.accelerationStructure
};
tlas.address = Device::vkGetAccelerationStructureDeviceAddressKHR(Device::device, &addrInfo);
}
VkAccelerationStructureCreateInfoKHR tlasCreateInfo {
.sType = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_CREATE_INFO_KHR,
.buffer = tlas.buffer.buffer,
.offset = 0,
.size = tlasBuildSizes.accelerationStructureSize,
.type = VK_ACCELERATION_STRUCTURE_TYPE_TOP_LEVEL_KHR,
};
Device::CheckVkResult(Device::vkCreateAccelerationStructureKHR(Device::device, &tlasCreateInfo, nullptr, &tlas.accelerationStructure));
VkAccelerationStructureDeviceAddressInfoKHR addrInfo {
.sType = VK_STRUCTURE_TYPE_ACCELERATION_STRUCTURE_DEVICE_ADDRESS_INFO_KHR,
.accelerationStructure = tlas.accelerationStructure
};
tlas.address = Device::vkGetAccelerationStructureDeviceAddressKHR(Device::device, &addrInfo);
tlas.builtInstanceCount = primitiveCount;
tlas.builtFlags = buildFlags;
}
// For UPDATE mode, src == dst (in-place refit). For BUILD, src is
// VK_NULL_HANDLE and dst is the freshly-created handle.
// VK_NULL_HANDLE and dst is the current handle, kept or fresh.
tlasBuildGeometryInfo.scratchData.deviceAddress = tlas.scratchBuffer.address;
tlasBuildGeometryInfo.dstAccelerationStructure = tlas.accelerationStructure;
tlasBuildGeometryInfo.srcAccelerationStructure =
@ -302,6 +342,6 @@ void RenderingElement3D::BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTB
VkAccelerationStructureBuildRangeInfoKHR* tlasRangeInfoPP = &tlasRangeInfo;
Device::vkCmdBuildAccelerationStructuresKHR(cmd, 1, &tlasBuildGeometryInfo, &tlasRangeInfoPP);
const TlasReadBarrier readBarrier = BuildTlasReadBarrier();
const TlasBarrier readBarrier = BuildTlasReadBarrier();
vkCmdPipelineBarrier(cmd, readBarrier.srcStage, readBarrier.dstStage, 0, 1, &readBarrier.memory, 0, nullptr, 0, nullptr);
}