feat(webgpu-rt): atomic pixel accumulator + raysPerPixel — >1 ray per pixel per bounce (#30) #31
6 changed files with 89 additions and 19 deletions
feat(webgpu-rt): atomic pixel accumulator + raysPerPixel for >1 ray/pixel/bounce (#30)
wfAccum becomes array<atomic<u32>> (4 slots/pixel, f32 bit patterns — same 16 B/pixel footprint) and rtAccumulate CASes each channel, so N rays for one pixel may resolve in the same SHADE pass without the read-modify-write racing. GENERATE clears with atomicStore (bitcast(0.0) == 0u); RESOLVE atomicLoads + bitcasts the vec4 it hands runResolve. Capacity half: RTPass::raysPerPixel (default 1) scales the wavefront ray/hit/payload buffers to raysPerPixel·W·H rays per bounce so the per-light emits actually fit instead of being dropped by rtEmitRay's capacity guard. The accumulator stays per-pixel. No flag-gating: uncontended the CAS succeeds first try — RTStress SHADE stays at its documented ~1.0 ms and master-vs-branch renders are byte-identical, so single-ray consumers pay nothing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit
27d7e84cb3
|
|
@ -101,7 +101,9 @@ shaded through an intersection shader with an any-hit cut-out.
|
||||||
bounce loop (`dispatchWorkgroupsIndirect`). TRACE carries zero user
|
bounce loop (`dispatchWorkgroupsIndirect`). TRACE carries zero user
|
||||||
code (traversal + intersection only); user raygen calls
|
code (traversal + intersection only); user raygen calls
|
||||||
`rtEmitPrimaryRay`, and closesthit / miss run in SHADE where they
|
`rtEmitPrimaryRay`, and closesthit / miss run in SHADE where they
|
||||||
`rtEmitRay` continuation/shadow rays and `rtAccumulate` radiance. An
|
`rtEmitRay` continuation/shadow rays and `rtAccumulate` radiance
|
||||||
|
(atomic — any number of rays per pixel per bounce, e.g. one shadow
|
||||||
|
ray per light). An
|
||||||
optional Resolve shader tonemaps the linear accumulator. See
|
optional Resolve shader tonemaps the linear accumulator. See
|
||||||
[WAVEFRONT-DESIGN.md](WAVEFRONT-DESIGN.md).
|
[WAVEFRONT-DESIGN.md](WAVEFRONT-DESIGN.md).
|
||||||
- **ComputeShader / WebGPUComputeShader** — Tier 1 wrapper used by the
|
- **ComputeShader / WebGPUComputeShader** — Tier 1 wrapper used by the
|
||||||
|
|
|
||||||
|
|
@ -21,12 +21,17 @@ compute pass, dispatch sizes driven by `dispatchWorkgroupsIndirect`.
|
||||||
- **RESOLVE** (1 thread/pixel, 8×8): reads accum slot, runs user `resolve_main`
|
- **RESOLVE** (1 thread/pixel, 8×8): reads accum slot, runs user `resolve_main`
|
||||||
if present else passthrough; writes outImage.
|
if present else passthrough; writes outImage.
|
||||||
|
|
||||||
## Buffers (rtState, sized to 2*W*H rays)
|
## Buffers (rtState; per-bounce ray capacity = `RTPass::raysPerPixel`·W·H)
|
||||||
- `wfRaysA`,`wfRaysB`: array<WfRay>, ping/pong. WfRay = origin,tMin,dir,tMax,
|
- `wfRaysA`,`wfRaysB`: array<WfRay>, ping/pong. WfRay = origin,tMin,dir,tMax,
|
||||||
pixel,flags,cullMask,missIndex,sbtOffset,payloadSlot,kind,_pad.
|
pixel,flags,cullMask,missIndex,sbtOffset,payloadSlot,kind,_pad. Each holds
|
||||||
|
one bounce's rays; `raysPerPixel` (default 1) scales them so closest-hit
|
||||||
|
can emit several rays per pixel in one bounce before rtEmitRay drops.
|
||||||
- `wfHits`: array<HitResult> (sized = ray capacity).
|
- `wfHits`: array<HitResult> (sized = ray capacity).
|
||||||
- `wfPayload`: array<Payload> — declared in CODEGEN region after user Payload.
|
- `wfPayload`: array<Payload> — declared in CODEGEN region after user Payload.
|
||||||
- `wfAccum`: array<vec4<f32>> per pixel (W*H).
|
- `wfAccum`: array<atomic<u32>>, 4 slots per pixel (W*H, RGBA as f32 bit
|
||||||
|
patterns — 16 B/pixel). `rtAccumulate` CASes each channel, so a SHADE
|
||||||
|
invocation may emit several rays for the same pixel in one bounce (one
|
||||||
|
shadow ray per light, issue #30) without the accumulates racing.
|
||||||
- `wfCounters`: atomic counters: emitA, emitB, trace dispatch args, etc.
|
- `wfCounters`: atomic counters: emitA, emitB, trace dispatch args, etc.
|
||||||
- `wfIndirect`: INDIRECT dispatch-args buffer.
|
- `wfIndirect`: INDIRECT dispatch-args buffer.
|
||||||
|
|
||||||
|
|
@ -74,6 +79,15 @@ maxDepth=1 (primary only). Sponza maxDepth=2 (primary + shadow).
|
||||||
- [x] device limits (maxBufferSize / maxStorageBufferBindingSize /
|
- [x] device limits (maxBufferSize / maxStorageBufferBindingSize /
|
||||||
maxComputeWorkgroupsPerDimension) + timestamp-query feature
|
maxComputeWorkgroupsPerDimension) + timestamp-query feature
|
||||||
- [x] megakernel dead path removed (RT pipeline builds only wavefront)
|
- [x] megakernel dead path removed (RT pipeline builds only wavefront)
|
||||||
|
- [x] atomic pixel accumulator (#30) — `rtAccumulate` is a per-channel
|
||||||
|
f32 CAS over `array<atomic<u32>>`, lifting the old one-ray-per-pixel-
|
||||||
|
per-bounce cap so closest-hit can emit one shadow ray per light in a
|
||||||
|
single bounce (multi-light shadowing; 3DForts #153). Same 16 B/pixel
|
||||||
|
footprint; uncontended CAS succeeds first try, so single-ray scenes
|
||||||
|
(VulkanTriangle/Sponza/RTStress) are unaffected. Capacity side:
|
||||||
|
`RTPass::raysPerPixel` (default 1) scales the ray/hit/payload buffers
|
||||||
|
so those N rays/pixel actually fit a bounce instead of being dropped
|
||||||
|
by rtEmitRay's capacity guard. Exercised by `examples/RTMultiShadow`.
|
||||||
- [~] binding packing (Phase 7): SKIPPED — target device reports 64 storage
|
- [~] binding packing (Phase 7): SKIPPED — target device reports 64 storage
|
||||||
buffers/stage (≥12), so the merge is unnecessary (issue makes it
|
buffers/stage (≥12), so the merge is unnecessary (issue makes it
|
||||||
conditional on <12). NOTE: this only holds because dom-webgpu.js now
|
conditional on <12). NOTE: this only holds because dom-webgpu.js now
|
||||||
|
|
|
||||||
|
|
@ -1714,7 +1714,11 @@ struct BvhNode {
|
||||||
@group(1) @binding(10) var<storage,read_write> wfRaysA : array<WfRay>;
|
@group(1) @binding(10) var<storage,read_write> wfRaysA : array<WfRay>;
|
||||||
@group(1) @binding(11) var<storage,read_write> wfRaysB : array<WfRay>;
|
@group(1) @binding(11) var<storage,read_write> wfRaysB : array<WfRay>;
|
||||||
@group(1) @binding(12) var<storage,read_write> wfHits : array<HitResult>;
|
@group(1) @binding(12) var<storage,read_write> wfHits : array<HitResult>;
|
||||||
@group(1) @binding(13) var<storage,read_write> wfAccum : array<vec4<f32>>;
|
// wfAccum: 4 atomic<u32> slots per pixel (RGBA as f32 bit patterns — same
|
||||||
|
// 16 B/pixel footprint as the old array<vec4<f32>>). Atomic so a SHADE
|
||||||
|
// invocation may emit N rays for the same pixel in one bounce (e.g. one
|
||||||
|
// shadow ray per light) and their rtAccumulate calls don't race (#30).
|
||||||
|
@group(1) @binding(13) var<storage,read_write> wfAccum : array<atomic<u32>>;
|
||||||
@group(1) @binding(14) var<storage,read_write> wfCounters : array<atomic<u32>>;
|
@group(1) @binding(14) var<storage,read_write> wfCounters : array<atomic<u32>>;
|
||||||
// @group(1) @binding(15) wfPayload : array<Payload> — emitted by codegen.
|
// @group(1) @binding(15) wfPayload : array<Payload> — emitted by codegen.
|
||||||
|
|
||||||
|
|
@ -1735,11 +1739,29 @@ fn _wfCurCount() -> u32 {
|
||||||
return min(raw, wfParams.rayCapacity);
|
return min(raw, wfParams.rayCapacity);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Add linear radiance to the pixel this SHADE/GENERATE thread owns. Safe
|
// Atomic float add via compare-exchange on the f32 bit pattern. WGSL has
|
||||||
// without atomics: at most one ray per pixel per bounce, and bounces run
|
// no native atomic<f32>, so each channel CASes until its read-add-write
|
||||||
// in separate passes (implicit barrier between them).
|
// wins. Uncontended (the common ≤1 ray/pixel/bounce case) the exchange
|
||||||
|
// succeeds first try; contention only occurs when several rays for the
|
||||||
|
// same pixel resolve in one SHADE pass.
|
||||||
|
fn _wfAtomicAddF32(slot: u32, v: f32) {
|
||||||
|
var old = atomicLoad(&wfAccum[slot]);
|
||||||
|
loop {
|
||||||
|
let sum = bitcast<u32>(bitcast<f32>(old) + v);
|
||||||
|
let r = atomicCompareExchangeWeak(&wfAccum[slot], old, sum);
|
||||||
|
if (r.exchanged) { break; }
|
||||||
|
old = r.old_value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add linear radiance to the pixel this SHADE/GENERATE thread owns. Atomic,
|
||||||
|
// so a closest-hit may emit any number of rays for the same pixel within a
|
||||||
|
// single bounce (multi-light shadowing) and their accumulates won't race.
|
||||||
fn rtAccumulate(rgb: vec3<f32>) {
|
fn rtAccumulate(rgb: vec3<f32>) {
|
||||||
wfAccum[_wfPixel] = wfAccum[_wfPixel] + vec4<f32>(rgb, 0.0);
|
let base = _wfPixel * 4u;
|
||||||
|
_wfAtomicAddF32(base + 0u, rgb.x);
|
||||||
|
_wfAtomicAddF32(base + 1u, rgb.y);
|
||||||
|
_wfAtomicAddF32(base + 2u, rgb.z);
|
||||||
}
|
}
|
||||||
|
|
||||||
// raygen → emit the pixel's primary ray. Bounce 0's current buffer is
|
// raygen → emit the pixel's primary ray. Bounce 0's current buffer is
|
||||||
|
|
@ -3030,8 +3052,9 @@ function wfNextPow2(n) {
|
||||||
return p;
|
return p;
|
||||||
}
|
}
|
||||||
|
|
||||||
function ensureWavefrontBuffers(W, H) {
|
function ensureWavefrontBuffers(W, H, raysPerPixel) {
|
||||||
const cap = W * H;
|
const rpp = Math.max(1, raysPerPixel | 0);
|
||||||
|
const cap = W * H * rpp; // per-bounce ray capacity (= WfParams.rayCapacity)
|
||||||
rtState.wf = rtState.wf || { cap: 0 };
|
rtState.wf = rtState.wf || { cap: 0 };
|
||||||
const wf = rtState.wf;
|
const wf = rtState.wf;
|
||||||
if (wf.cap === cap && wf.raysA) return wf;
|
if (wf.cap === cap && wf.raysA) return wf;
|
||||||
|
|
@ -3041,7 +3064,8 @@ function ensureWavefrontBuffers(W, H) {
|
||||||
wf.raysA = device.createBuffer({ size: cap * 64, usage: S, label: "wf-raysA" });
|
wf.raysA = device.createBuffer({ size: cap * 64, usage: S, label: "wf-raysA" });
|
||||||
wf.raysB = device.createBuffer({ size: cap * 64, usage: S, label: "wf-raysB" });
|
wf.raysB = device.createBuffer({ size: cap * 64, usage: S, label: "wf-raysB" });
|
||||||
wf.hits = device.createBuffer({ size: cap * 112, usage: S, label: "wf-hits" });
|
wf.hits = device.createBuffer({ size: cap * 112, usage: S, label: "wf-hits" });
|
||||||
wf.accum = device.createBuffer({ size: cap * 16, usage: S, label: "wf-accum" });
|
// Accumulator is per *pixel*, not per ray: 4 atomic<u32> (16 B).
|
||||||
|
wf.accum = device.createBuffer({ size: W * H * 16, usage: S, label: "wf-accum" });
|
||||||
wf.payload = device.createBuffer({ size: 2 * cap * WF_PAYLOAD_BYTES, usage: S, label: "wf-payload" });
|
wf.payload = device.createBuffer({ size: 2 * cap * WF_PAYLOAD_BYTES, usage: S, label: "wf-payload" });
|
||||||
wf.counters = device.createBuffer({ size: 64,
|
wf.counters = device.createBuffer({ size: 64,
|
||||||
usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST | GPUBufferUsage.COPY_SRC, label: "wf-counters" });
|
usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST | GPUBufferUsage.COPY_SRC, label: "wf-counters" });
|
||||||
|
|
@ -3270,7 +3294,8 @@ function wfUserBindGroups(pipe, handlesPtr, handlesCount) {
|
||||||
|
|
||||||
env.wgpuDispatchRT = (pipelineHandle, pushPtr, pushBytes,
|
env.wgpuDispatchRT = (pipelineHandle, pushPtr, pushBytes,
|
||||||
tlasBufHandle, instanceCount, gx, gy,
|
tlasBufHandle, instanceCount, gx, gy,
|
||||||
handlesPtr, handlesCount, maxDepth, outTexHandle) => {
|
handlesPtr, handlesCount, maxDepth, outTexHandle,
|
||||||
|
raysPerPixel) => {
|
||||||
if (!state.encoder) return;
|
if (!state.encoder) return;
|
||||||
const pipe = rtPipelines.get(pipelineHandle);
|
const pipe = rtPipelines.get(pipelineHandle);
|
||||||
const tlas = buffers.get(tlasBufHandle);
|
const tlas = buffers.get(tlasBufHandle);
|
||||||
|
|
@ -3296,9 +3321,9 @@ env.wgpuDispatchRT = (pipelineHandle, pushPtr, pushBytes,
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const W = state.width, H = state.height;
|
const W = state.width, H = state.height;
|
||||||
const cap = W * H;
|
|
||||||
const depth = Math.max(1, maxDepth | 0);
|
const depth = Math.max(1, maxDepth | 0);
|
||||||
const wf = ensureWavefrontBuffers(W, H);
|
const wf = ensureWavefrontBuffers(W, H, raysPerPixel);
|
||||||
|
const cap = wf.cap; // per-bounce ray capacity = raysPerPixel·W·H
|
||||||
|
|
||||||
// ── Per-pass WfParams ring. queue.writeBuffer lands before submit, so
|
// ── Per-pass WfParams ring. queue.writeBuffer lands before submit, so
|
||||||
// we can't mutate the uniform between passes — instead we pre-write one
|
// we can't mutate the uniform between passes — instead we pre-write one
|
||||||
|
|
|
||||||
|
|
@ -270,7 +270,13 @@ void PipelineRTWebGPU::Init(WebGPUCommandEncoderRef /*cmd*/,
|
||||||
wgsl += "fn wfGenerate(@builtin(global_invocation_id) gid: vec3<u32>) {\n";
|
wgsl += "fn wfGenerate(@builtin(global_invocation_id) gid: vec3<u32>) {\n";
|
||||||
wgsl += " if (gid.x >= wfParams.surfaceW || gid.y >= wfParams.surfaceH) { return; }\n";
|
wgsl += " if (gid.x >= wfParams.surfaceW || gid.y >= wfParams.surfaceH) { return; }\n";
|
||||||
wgsl += " let pixel = gid.y * wfParams.surfaceW + gid.x;\n";
|
wgsl += " let pixel = gid.y * wfParams.surfaceW + gid.x;\n";
|
||||||
wgsl += " wfAccum[pixel] = vec4<f32>(0.0, 0.0, 0.0, 0.0);\n";
|
// wfAccum is 4 atomic<u32> slots per pixel (f32 bit patterns; see the
|
||||||
|
// bindings prelude JS-side). bitcast<u32>(0.0) == 0u, so plain zeroes.
|
||||||
|
wgsl += " let accumBase = pixel * 4u;\n";
|
||||||
|
wgsl += " atomicStore(&wfAccum[accumBase + 0u], 0u);\n";
|
||||||
|
wgsl += " atomicStore(&wfAccum[accumBase + 1u], 0u);\n";
|
||||||
|
wgsl += " atomicStore(&wfAccum[accumBase + 2u], 0u);\n";
|
||||||
|
wgsl += " atomicStore(&wfAccum[accumBase + 3u], 0u);\n";
|
||||||
wgsl += " _wfPixel = pixel;\n";
|
wgsl += " _wfPixel = pixel;\n";
|
||||||
wgsl += " ";
|
wgsl += " ";
|
||||||
wgsl += raygenEntryFn;
|
wgsl += raygenEntryFn;
|
||||||
|
|
@ -300,7 +306,16 @@ void PipelineRTWebGPU::Init(WebGPUCommandEncoderRef /*cmd*/,
|
||||||
wgsl += "fn wfResolve(@builtin(global_invocation_id) gid: vec3<u32>) {\n";
|
wgsl += "fn wfResolve(@builtin(global_invocation_id) gid: vec3<u32>) {\n";
|
||||||
wgsl += " if (gid.x >= wfParams.surfaceW || gid.y >= wfParams.surfaceH) { return; }\n";
|
wgsl += " if (gid.x >= wfParams.surfaceW || gid.y >= wfParams.surfaceH) { return; }\n";
|
||||||
wgsl += " let pixel = gid.y * wfParams.surfaceW + gid.x;\n";
|
wgsl += " let pixel = gid.y * wfParams.surfaceW + gid.x;\n";
|
||||||
wgsl += " let outc = runResolve(gid.xy, wfAccum[pixel]);\n";
|
// Reassemble the vec4<f32> the resolve hook expects from the pixel's 4
|
||||||
|
// atomic<u32> accumulator slots (no contention here — RESOLVE runs in
|
||||||
|
// its own pass — but atomics may only be accessed atomically in WGSL).
|
||||||
|
wgsl += " let accumBase = pixel * 4u;\n";
|
||||||
|
wgsl += " let hdr = vec4<f32>(\n";
|
||||||
|
wgsl += " bitcast<f32>(atomicLoad(&wfAccum[accumBase + 0u])),\n";
|
||||||
|
wgsl += " bitcast<f32>(atomicLoad(&wfAccum[accumBase + 1u])),\n";
|
||||||
|
wgsl += " bitcast<f32>(atomicLoad(&wfAccum[accumBase + 2u])),\n";
|
||||||
|
wgsl += " bitcast<f32>(atomicLoad(&wfAccum[accumBase + 3u])));\n";
|
||||||
|
wgsl += " let outc = runResolve(gid.xy, hdr);\n";
|
||||||
wgsl += " textureStore(outImage, vec2<i32>(i32(gid.x), i32(gid.y)), outc);\n";
|
wgsl += " textureStore(outImage, vec2<i32>(i32(gid.x), i32(gid.y)), outc);\n";
|
||||||
wgsl += "}\n";
|
wgsl += "}\n";
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -103,6 +103,13 @@ export namespace Crafter {
|
||||||
// its own composite→swapchain pass afterwards. Ignored (0) for the
|
// its own composite→swapchain pass afterwards. Ignored (0) for the
|
||||||
// default canvas path.
|
// default canvas path.
|
||||||
std::uint32_t outTexHandle = 0;
|
std::uint32_t outTexHandle = 0;
|
||||||
|
// Per-bounce ray budget as a multiple of the pixel count. The
|
||||||
|
// wavefront ray/hit/payload buffers hold raysPerPixel·W·H rays, so
|
||||||
|
// a closest-hit may emit up to raysPerPixel rays per pixel within
|
||||||
|
// one bounce (e.g. one shadow ray per light — rtAccumulate is
|
||||||
|
// atomic, issue #30) before rtEmitRay starts dropping. Memory
|
||||||
|
// scales linearly; keep at 1 for single-ray-per-pixel pipelines.
|
||||||
|
std::uint32_t raysPerPixel = 1;
|
||||||
|
|
||||||
RTPass(PipelineRTWebGPU* p) : pipeline(p) {}
|
RTPass(PipelineRTWebGPU* p) : pipeline(p) {}
|
||||||
|
|
||||||
|
|
@ -121,7 +128,8 @@ export namespace Crafter {
|
||||||
handlesPtr,
|
handlesPtr,
|
||||||
static_cast<std::int32_t>(handlesCount),
|
static_cast<std::int32_t>(handlesCount),
|
||||||
static_cast<std::int32_t>(maxDepth),
|
static_cast<std::int32_t>(maxDepth),
|
||||||
outTexHandle);
|
outTexHandle,
|
||||||
|
static_cast<std::int32_t>(raysPerPixel));
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -222,6 +222,11 @@ namespace Crafter::WebGPU {
|
||||||
// `outTexHandle` is the destination texture for an HDR-output pipeline
|
// `outTexHandle` is the destination texture for an HDR-output pipeline
|
||||||
// (one loaded with hdrOutputFormat != 0); ignored (pass 0) for the
|
// (one loaded with hdrOutputFormat != 0); ignored (pass 0) for the
|
||||||
// default canvas path.
|
// default canvas path.
|
||||||
|
// `raysPerPixel` scales the wavefront ray/hit/payload buffers to
|
||||||
|
// raysPerPixel·W·H rays per bounce, so closest-hit can emit up to that
|
||||||
|
// many rays per pixel within one bounce (e.g. one shadow ray per light)
|
||||||
|
// without rtEmitRay dropping past capacity. 1 = the historical
|
||||||
|
// single-ray-per-pixel footprint.
|
||||||
__attribute__((import_module("env"), import_name("wgpuDispatchRT")))
|
__attribute__((import_module("env"), import_name("wgpuDispatchRT")))
|
||||||
extern "C" void wgpuDispatchRT(std::uint32_t pipelineHandle,
|
extern "C" void wgpuDispatchRT(std::uint32_t pipelineHandle,
|
||||||
const void* pushPtr, std::int32_t pushBytes,
|
const void* pushPtr, std::int32_t pushBytes,
|
||||||
|
|
@ -230,7 +235,8 @@ namespace Crafter::WebGPU {
|
||||||
std::int32_t gx, std::int32_t gy,
|
std::int32_t gx, std::int32_t gy,
|
||||||
const void* handlesPtr, std::int32_t handlesCount,
|
const void* handlesPtr, std::int32_t handlesCount,
|
||||||
std::int32_t maxDepth,
|
std::int32_t maxDepth,
|
||||||
std::uint32_t outTexHandle);
|
std::uint32_t outTexHandle,
|
||||||
|
std::int32_t raysPerPixel);
|
||||||
|
|
||||||
// GPU TLAS-build dispatch. Two sequential compute passes:
|
// GPU TLAS-build dispatch. Two sequential compute passes:
|
||||||
// 1. tlasBuildMain — per-instance world AABB + identity permutation
|
// 1. tlasBuildMain — per-instance world AABB + identity permutation
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue