Crafter.Graphics/interfaces/Crafter.Graphics-RenderingElement3D.cppm
catbot b5b8c04237 perf(tlas): dirty-track the per-frame TLAS instance+metadata upload (#118)
BuildTLAS rebuilt the host instance+metadata buffers with an O(n) copy of
every 64 B VkAccelerationStructureInstanceKHR + metadata entry every frame,
unconditionally, then flushed the whole high-water capacity (VK_WHOLE_SIZE)
on both buffers. At the millions-of-instances target that copy dominates the
CPU frame, and the whole-buffer flush costs on non-coherent BAR/VRAM.

Add a generation counter so only changed host-authored fields are copied, and
feed the same dirty span into the ranged FlushDevice(offset, bytes) overload:

- RenderingElement3D::hostDataVersion + MarkHostDataDirty() (bumps a global
  monotonic counter). TlasWithBuffer::uploadedVersion records, per frame, the
  version last copied into each slot. A slot is copied only when its element
  advanced past the recorded version; version 0 ("untracked") reads dirty every
  frame, so callers that don't opt in keep the prior copy-every-frame behaviour.
  Globally-unique versions make this correct under relocation (Remove's
  swap-and-pop, and remove+add that nets the same count on the refit path)
  without tracking element identity. The reset on every topology change covers
  buffer reallocation and the reshuffled element->slot mapping.
- The dirty [first, last] envelope drives both the copy and the flush: a new
  VulkanBuffer::FlushDevice(cmd, access, stage, offset, bytes) overload flushes
  + barriers just that span for instanceBuffer, and the ranged
  FlushDevice(offset, bytes) for metadataBuffer. When nothing is dirty both are
  skipped — the skipped HOST->build barrier only ever ordered host writes, never
  the application's compute-written GPU-owned transform (that compute->build
  ordering is the caller's, and is unchanged).

Constraint honoured: transformOwnedByGpu transforms are still never host-copied.
The API field/method are mirrored on the WebGPU class for source portability
(the WebGPU build re-uploads its small mirror wholesale and ignores the version).

New test TLASInstanceDirtyTracking drives the real RT device and reads back the
host-mapped buffers to assert: tracked elements upload once then skip until
re-marked, untracked elements always upload, and relocation on the refit path
re-uploads exactly the moved slots — with zero validation-layer errors over the
ranged flush.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 19:58:45 +00:00

269 lines
15 KiB
C++
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/*
Crafter®.Graphics
Copyright (C) 2026 Catcrafts®
catcrafts.net
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License version 3.0 as published by the Free Software Foundation;
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
module;
#ifndef CRAFTER_GRAPHICS_WINDOW_DOM
#include "vulkan/vulkan.h"
#endif // !CRAFTER_GRAPHICS_WINDOW_DOM
export module Crafter.Graphics:RenderingElement3D;
import :RT;
#ifndef CRAFTER_GRAPHICS_WINDOW_DOM
import std;
import :Mesh;
import :VulkanBuffer;
import Crafter.Math;
import :Window;
export namespace Crafter {
struct TlasWithBuffer {
VkDeviceAddress address = 0;
VulkanBuffer<char, false> buffer;
VkAccelerationStructureKHR accelerationStructure = VK_NULL_HANDLE;
VulkanBuffer<VkAccelerationStructureInstanceKHR, true> instanceBuffer;
VulkanBuffer<char, false> scratchBuffer;
// Parallel to instanceBuffer, indexed by TLAS instance ID. Filled
// from each element's userMetadata during BuildTLAS. Consumers
// (e.g. ray-query collision) bind this in the descriptor heap and
// look up via rayQueryGetIntersectionInstanceIdEXT to recover
// application-side per-instance data without touching the
// Vulkan-mandated instanceCustomIndex (which renderers may already
// use for their own encoding).
VulkanBuffer<std::uint32_t, true> metadataBuffer;
// Last instance count this TLAS was built (not refit) for. When
// elements.size() matches this, BuildTLAS does an in-place refit
// (UPDATE mode) which is dramatically cheaper than a full rebuild
// — refit walks the existing BVH and updates AABBs, while rebuild
// reconstructs the topology from scratch. A change in count forces
// a fresh rebuild because the AS is sized for that primitive count.
std::uint32_t builtInstanceCount = 0;
// Build flags the current AS was created with — ALLOW_UPDATE (always
// set, since BuildTLAS refits in place) plus the chosen
// FAST_TRACE/FAST_BUILD preference bit. A refit (UPDATE mode) must
// use flags identical to the originating build, so a change in the
// requested preference forces a full rebuild rather than a refit.
VkBuildAccelerationStructureFlagsKHR builtFlags = 0;
// Per-slot record of the RenderingElement3D::hostDataVersion that
// BuildTLAS last copied into this frame's instanceBuffer/metadataBuffer
// — parallel to them, sized to the live instance count. The per-frame
// copy re-uploads (and flushes) only the slots whose element advanced
// past the recorded version; an element left at version 0 ("untracked")
// reads dirty every frame, so callers that never opt in keep the
// pre-#118 copy-every-frame behaviour. Reset to all-zero on every
// topology change, since a rebuild reshuffles which element occupies
// each slot (and may have reallocated the buffers). See
// RenderingElement3D::hostDataVersion.
std::vector<std::uint64_t> uploadedVersion;
};
class RenderingElement3D {
public:
RTInstance instance;
// Position in `elements`, maintained by Add/Remove for O(1) swap-and-pop.
// Sentinel value = not currently registered.
std::uint32_t indexInElements = std::numeric_limits<std::uint32_t>::max();
// Application-defined per-instance tag, copied verbatim into
// tlases[*].metadataBuffer at this element's TLAS instance ID
// every BuildTLAS. Crafter doesn't interpret it.
std::uint32_t userMetadata = 0;
// When true, BuildTLAS skips copying instance.transform into the
// TLAS instance buffer — the application's compute shader writes
// the transform field directly into instanceBuffer at this
// element's TLAS instance ID. Other instance fields (mask,
// customIndex, SBT offset, BLAS reference) are still copied from
// the CPU instance struct.
//
// Used to take per-frame transform updates off the CPU for bodies
// whose transforms derive from GPU-side state (physics nodes that
// already live on the GPU).
bool transformOwnedByGpu = false;
// Monotonic version of this element's host-authored TLAS data — the
// instance fields the CPU writes (everything except a GPU-owned
// transform; see transformOwnedByGpu) plus userMetadata. BuildTLAS
// records, per frame, the version it last copied into each buffer slot
// (TlasWithBuffer::uploadedVersion) and re-copies a slot only when its
// element has advanced past the recorded version — so a TLAS dominated
// by instances whose host fields are set once and then left alone (the
// millions-of-GPU-driven-bodies target) pays no per-frame host copy or
// flush after the first upload.
//
// 0 is the "untracked" sentinel: an element left at 0 is copied every
// frame exactly as before this optimization, so code that mutates
// instance/userMetadata without opting in stays correct. Opt in by
// calling MarkHostDataDirty after every host-data change — the standard
// dirty-flag contract (mark on change; the upload clears it until the
// next change). The GPU-owned transform is exempt: BuildTLAS never
// host-copies it, so changing it needs no mark.
std::uint64_t hostDataVersion = 0;
// Stamp this element's host data with a fresh global version so the next
// BuildTLAS of each frame re-uploads (and flushes) its slot. Call after
// changing any host-authored instance field or userMetadata. See
// hostDataVersion.
void MarkHostDataDirty() { hostDataVersion = ++hostDataVersionCounter; }
// Source of globally-unique, monotonically-increasing host-data
// versions. Global rather than per-element so a version value names a
// unique (element, edit) pair: a per-slot recorded version then equals
// the slot's current occupant only when that exact element's exact edit
// was the last thing written there. That uniqueness is what makes the
// relocation cases — swap-and-pop in Remove, or a remove+add that nets
// the same instance count and so takes the refit path — fall out
// correctly without tracking element identity: a relocated element's
// version never collides with the (different) element's version recorded
// for that slot. 64-bit, so it does not wrap in practice.
inline static std::uint64_t hostDataVersionCounter = 0;
static std::vector<RenderingElement3D*> elements;
inline static TlasWithBuffer tlases[Window::numFrames];
// Build (or in-place refit) the TLAS for frame `index`. `preference`
// maps onto the FAST_TRACE/FAST_BUILD bits exactly as Mesh::Build
// does for a BLAS: FastTrace (the default) spends more build time for
// faster traversal — right for a TLAS rebuilt at most once per frame
// and traced many times; FastBuild trades traversal speed for cheaper
// builds, for scenes whose instance set churns hard every frame.
// ALLOW_UPDATE is always set (refit is intrinsic to this path), so it
// is not exposed as an option. Changing `preference` between frames
// forces a full rebuild on the next call, since a refit must inherit
// the original build's flags.
static void BuildTLAS(VkCommandBuffer cmd, std::uint32_t index, RTBuildPreference preference = RTBuildPreference::FastTrace);
// Register / unregister with `elements`. Use these instead of touching
// the vector directly: linear find+erase is O(n) and pathological at
// the body counts physics targets (millions of braces).
static void Add(RenderingElement3D* e);
static void Remove(RenderingElement3D* e);
};
}
#endif // !CRAFTER_GRAPHICS_WINDOW_DOM
#ifdef CRAFTER_GRAPHICS_WINDOW_DOM
import std;
import :Mesh;
import :WebGPU;
import :WebGPUBuffer;
import :Window;
export namespace Crafter {
// Per-frame TLAS storage. WebGPU has no real swapchain frame count
// (Window::numFrames = 1 on DOM), so this is effectively a singleton —
// the array form is kept for API symmetry with the Vulkan side so user
// code that indexes `tlases[frameIdx]` ports unchanged.
struct TlasWithBuffer {
// Host-visible instance buffer holding RTInstance entries — same
// layout as Vulkan's VkAccelerationStructureInstanceKHR, so user
// code touching .instance.mask / .flags / .transform.matrix is
// identical across backends. Also bound as a storage SSBO so
// application compute shaders (e.g. physics-tlas-transform.comp.wgsl)
// can write the .transform field directly when
// RenderingElement3D::transformOwnedByGpu is set.
WebGPUBuffer<RTInstance, true> instanceBuffer;
// Per-instance application metadata; parallel to instanceBuffer,
// identical semantics to the Vulkan-side counterpart.
WebGPUBuffer<std::uint32_t, true> metadataBuffer;
// GPU-built TLAS data: one TLASEntry per instance, written each
// BuildTLAS by a compute pass on the JS bridge. Read by traceRay /
// rayQuery as `@group(1) @binding(0) tlas: array<TLASEntry>`.
// TLASEntry layout: 96 bytes — aabbMin (12) + maskHGoffset (4) +
// aabbMax (12) + blasHandle (4) + invTransform 3x4 mat (48) +
// customIndex (4) + _pad (12). Defined in the WGSL traversal
// library; never directly read by C++.
WebGPUBuffer<char, false> buffer;
// GPU LBVH support — see additional/dom-webgpu.js's TLAS-build
// pipeline.
//
// entryOrder: per-frame permutation array of u32, indexing into
// `buffer` (the TLASEntry[] array). Populated by the radix-sort
// pass to spatially-coherent Morton order, then consumed by the
// BVH construction + traversal passes. In Stage 1 (this
// baseline) it's the identity permutation written by
// tlasBuildMain alongside the entries.
WebGPUBuffer<char, false> entryOrder;
// mortonCodes: per-instance 32-bit Morton codes computed from the
// world-AABB centroid, used as the radix-sort key. Written by
// tlasBuildMain.
WebGPUBuffer<char, false> mortonCodes;
// bvhNodes: 2N_PADDED - 1 sweep-tree BVH nodes built per frame
// by the LBVH-build compute pass. Each node 32 bytes (aabbMin +
// pad, aabbMax + pad). N_PADDED = 65536 (hardcoded in WGSL).
// Internal nodes [0, N_PADDED-1); leaves [N_PADDED-1, 2*N_PADDED-1).
// Node i's children are 2i+1, 2i+2 (implicit perfect binary
// tree). Cap: 65536 instances per scene.
WebGPUBuffer<char, false> bvhNodes;
// tlasBins: dead, kept allocated as a 64-byte placeholder so the
// existing wgpuBuildTLAS C++ signature doesn't need a churn.
// The pre-LBVH 64-bin partition was replaced by the full BVH.
WebGPUBuffer<char, false> tlasBins;
// Sort ping-pong buffers for the radix sort. Each pass reads
// from one and writes to the other, swapping role. Layout per
// element: 1 u32 packed key = (morton16 << 16) | tlasIndex16.
// Sized for N_PADDED.
WebGPUBuffer<char, false> sortTempA;
WebGPUBuffer<char, false> sortTempB;
std::uint32_t builtInstanceCount = 0;
};
class RenderingElement3D {
public:
RTInstance instance{};
std::uint32_t indexInElements = std::numeric_limits<std::uint32_t>::max();
std::uint32_t userMetadata = 0;
// Application compute shader writes the transform field of this
// element's instanceBuffer slot directly — BuildTLAS preserves it.
bool transformOwnedByGpu = false;
// API-symmetric with the Vulkan side so portable code that opts its
// instances into host-data dirty tracking compiles unchanged. The
// WebGPU BuildTLAS re-uploads the whole CPU mirror every build (the
// counts this path targets are small), so the version is not consulted
// here — it exists purely for cross-backend source compatibility.
std::uint64_t hostDataVersion = 0;
void MarkHostDataDirty() { hostDataVersion = ++hostDataVersionCounter; }
inline static std::uint64_t hostDataVersionCounter = 0;
static std::vector<RenderingElement3D*> elements;
inline static TlasWithBuffer tlases[Window::numFrames];
// Repopulate the TLAS for frame `index`. WebGPU path always does
// a fresh build (no refit) — the GPU build pass is cheap at the
// ~10100 instance counts the design targets; LBVH-for-TLAS is a
// future optimization for larger scenes.
//
// BuildTLAS is now split into Upload + Build so a physics
// compute pass (e.g. physics-tlas-transform) can run between the
// CPU mirror upload and the GPU LBVH build. The compute pass
// writes the per-instance transform bytes that BuildTLAS leaves
// intact for elements flagged transformOwnedByGpu, and those
// writes have to land before the LBVH reads them. The combined
// BuildTLAS is kept as a convenience for callers that don't
// interleave a compute pass (e.g. the ctor-time first build).
static void BuildTLASUpload(WebGPUCommandEncoderRef cmd, std::uint32_t index);
static void BuildTLASBuild(WebGPUCommandEncoderRef cmd, std::uint32_t index);
// `preference` mirrors the Vulkan signature for portability but has
// no effect here: the software LBVH-for-TLAS path has no hardware
// FAST_TRACE/FAST_BUILD knob (same as Mesh::Build on WebGPU). The
// parameter exists purely so portable code compiles unchanged.
static void BuildTLAS(WebGPUCommandEncoderRef cmd, std::uint32_t index, RTBuildPreference preference = RTBuildPreference::FastTrace);
static void Add(RenderingElement3D* e);
static void Remove(RenderingElement3D* e);
};
}
#endif // CRAFTER_GRAPHICS_WINDOW_DOM