Dynamic descriptor_heap indexing faults the device in ray-tracing (closest-hit) shaders on NVIDIA #23

New issue

Closed

opened 2026-06-03 21:51:05 +02:00 by catbot · 0 comments

catbot commented

2026-06-03 21:51:05 +02:00

Member

Summary

Indexing a layout(descriptor_heap) array with a runtime (dynamic) index inside a ray-tracing closest-hit shader device-losts on the NVIDIA proprietary driver (VK_ERROR_DEVICE_LOST, instruction-pointer / READ_INVALID device-fault) with validation off. GPU-Assisted Validation masks it (the scene runs fine under GPU-AV, which is why it isn't caught in a validated run).

This is the same family as #15/#7 (descriptor-heap AS reads) and #21/#22 (RT recursion depth + per-shader TLAS push for compute), but for plain SSBO and sampled-texture descriptors read with a non-constant heap index.

Isolation (verified, NVIDIA RTX 4090, driver 610.43.02)

Driving the 3DForts native Game scene headlessly and bisecting closesthit.glsl:

A closest-hit that reads only lightHeap[lightSlot] where lightSlot is a spec constant → survives indefinitely. ✅
The same shader reading indexHeap[assetIndexStart + gl_InstanceCustomIndexEXT] / vertexHeap[...] (heap index offset by a runtime value) → device-lost on the first geometry hit. ❌
Reading a texture dynamically — textureHeap[assetColorStart + gl_InstanceCustomIndexEXT] — also device-losts. ❌ (so it's SSBO and sampled-image descriptors.)
nonuniformEXT() on the dynamic index does not help.
The identical dynamic-heap-index pattern works fine in fragment shaders (the UIRenderer indexes uiTextures[]/ui*Heap[] by per-item runtime slots) — so this is RT-stage-specific, not a general descriptor_heap problem.
Raygen reading a spec-constant-indexed SSBO (camera) works; only the dynamic index in the hit stage faults.

Why no example catches it

VulkanTriangle traces from raygen only (hardcoded camera, no hit-shader heap reads). Sponza's closest-hit indexes albedo[albedoSlot] with a spec constant. No shipped RT example does dynamic heap indexing in a hit shader, so this path is untested.

Impact

Any RT renderer with bindless per-mesh geometry/material (the standard pattern: per-instance vertex/index/texture arrays indexed by gl_InstanceCustomIndexEXT) cannot run on the descriptor_heap path on NVIDIA. 3DForts' native Game scene is blocked on this (the WebGPU path side-steps it with bucketed texture arrays + a single buffer).

Ask

Either support dynamic descriptor_heap indexing in RT shaders on the NVIDIA path (likely an analogous SPIR-V rewrite to the AS-read workaround, or a descriptor-buffer addressing fix), or document the limitation and the recommended pattern (buffer_reference for buffers? a single texture_2d_array for materials?) so consumers can avoid dynamic heap indexing in hit shaders.

Reported from 3DForts #87 (native Game scene device-loss). 3DForts-side prerequisites already fixed there: wall any-hit hit groups 8/9/10, shadow recursion depth 2, and the compute-shader TLAS-address push for splash/builder-pick.

## Summary Indexing a `layout(descriptor_heap)` array with a **runtime (dynamic)** index inside a ray-tracing **closest-hit** shader device-losts on the NVIDIA proprietary driver (`VK_ERROR_DEVICE_LOST`, instruction-pointer / READ_INVALID device-fault) with validation **off**. GPU-Assisted Validation masks it (the scene runs fine under GPU-AV, which is why it isn't caught in a validated run). This is the same family as #15/#7 (descriptor-heap AS reads) and #21/#22 (RT recursion depth + per-shader TLAS push for compute), but for **plain SSBO and sampled-texture descriptors** read with a non-constant heap index. ## Isolation (verified, NVIDIA RTX 4090, driver 610.43.02) Driving the 3DForts native Game scene headlessly and bisecting `closesthit.glsl`: - A closest-hit that reads **only** `lightHeap[lightSlot]` where `lightSlot` is a **spec constant** → survives indefinitely. ✅ - The same shader reading `indexHeap[assetIndexStart + gl_InstanceCustomIndexEXT]` / `vertexHeap[...]` (heap index offset by a **runtime** value) → device-lost on the first geometry hit. ❌ - Reading a **texture** dynamically — `textureHeap[assetColorStart + gl_InstanceCustomIndexEXT]` — also device-losts. ❌ (so it's SSBO *and* sampled-image descriptors.) - `nonuniformEXT()` on the dynamic index does **not** help. - The identical dynamic-heap-index pattern works fine in **fragment** shaders (the UIRenderer indexes `uiTextures[]`/`ui*Heap[]` by per-item runtime slots) — so this is **RT-stage-specific**, not a general descriptor_heap problem. - Raygen reading a spec-constant-indexed SSBO (camera) works; only the *dynamic* index in the hit stage faults. ## Why no example catches it `VulkanTriangle` traces from raygen only (hardcoded camera, no hit-shader heap reads). `Sponza`'s closest-hit indexes `albedo[albedoSlot]` with a **spec constant**. No shipped RT example does dynamic heap indexing in a hit shader, so this path is untested. ## Impact Any RT renderer with bindless per-mesh geometry/material (the standard pattern: per-instance vertex/index/texture arrays indexed by `gl_InstanceCustomIndexEXT`) cannot run on the descriptor_heap path on NVIDIA. 3DForts' native Game scene is blocked on this (the WebGPU path side-steps it with bucketed texture arrays + a single buffer). ## Ask Either support dynamic `descriptor_heap` indexing in RT shaders on the NVIDIA path (likely an analogous SPIR-V rewrite to the AS-read workaround, or a descriptor-buffer addressing fix), or document the limitation and the recommended pattern (buffer_reference for buffers? a single texture_2d_array for materials?) so consumers can avoid dynamic heap indexing in hit shaders. Reported from 3DForts #87 (native Game scene device-loss). 3DForts-side prerequisites already fixed there: wall any-hit hit groups 8/9/10, shadow recursion depth 2, and the compute-shader TLAS-address push for splash/builder-pick.