diff --git a/AGENTS.consumer.md b/AGENTS.consumer.md index e0bb2a8..aa22182 100644 --- a/AGENTS.consumer.md +++ b/AGENTS.consumer.md @@ -29,16 +29,16 @@ window.StartSync(); // the event loop - Include `vulkan/vulkan.h` yourself, before the imports (the examples wrap it in `#ifndef CRAFTER_GRAPHICS_WINDOW_DOM`): the library includes it only in its own global module fragments, and neither Vulkan's types nor its macros (`VK_TRUE`, the `VK_*` flags) cross an `import`. On the DOM build `Device::Initialize()` takes no arguments and `DeviceFeatures` does not exist, which is why the dual-target examples keep a separate DOM `main`. - `StartSync` runs the platform loop until `open` is false (`onClose` fires first). Each pass of the loop dispatches platform events into the `Window` events, runs `Gamepad::Tick()` and `onBeforeUpdate`, and runs `Update()` for a window that is due. On Wayland a window on the compositor's clock updates from its frame callback during that event dispatch, before the pass's `onBeforeUpdate`, and the loop wakes for every event, so `onBeforeUpdate` fires several times per frame; on Win32 and in the browser `onBeforeUpdate` precedes the same pass's `Update`. `updateInterval`, set before `StartUpdate`, makes a window update every so often whether visible or not; pair it with `SetPresentMode(PresentMode::Mailbox)` so a present never waits for the display. `AddPollFd(fd, callback)` (Wayland) wakes the loop for a socket or an eventfd. -- `Update()` computes `mouseDelta` and calls `Render()`, which acquires a swapchain image, waits for that image's fence (three frames in flight, `Window::numFrames`), reclaims deferred deletions, begins `drawCmdBuffers[currentBuffer]`, binds the heap (or the descriptor set), fires `onUpdate(FrameTime)`, records every pass in `passes` in order with a barrier on the swapchain image between them, transitions the image to present, submits and presents. Per-frame GPU work of your own (a TLAS rebuild, a physics dispatch) is recorded from `onUpdate` into `window.drawCmdBuffers[window.currentBuffer]`; the heaps are bound by then and the passes come after. `onResize` fires after a swapchain rebuild; anything holding descriptors of `window.imageViews[]` rewrites them there. -- A `RenderPass` is `Record(cmd, frameIdx, window)` plus `SwapchainStage()`, which must return the stage that touches the swapchain image (`COMPUTE_SHADER`, `RAY_TRACING_SHADER` or `TRANSFER`): the frame's barriers take their stage and access masks from it. The image stays in `VK_IMAGE_LAYOUT_GENERAL` through the whole pass list, with `STORAGE | TRANSFER_SRC` usage and `TRANSFER_DST` where the surface allows it (`window.swapchainUsage`). Nothing clears it: draw a background quad, or clear from a transfer pass where `TRANSFER_DST` is there. +- `Update()` computes `mouseDelta` and calls `Render()`, which waits for the fence of its frame slot (three frames in flight, `Window::numFrames`; the slot is `window.currentFrame`), acquires a swapchain image (`window.currentImage`; the swapchain has as many images as its present mode needs, not necessarily three), writes it into this window's swapchain slots of the heap, reclaims deferred deletions, begins `drawCmdBuffers[currentFrame]`, binds the heap (or the descriptor set), fires `onUpdate(FrameTime)`, records every pass in `passes` in order with a barrier on the swapchain image between them, transitions the image to present, submits and presents. Per-frame GPU work of your own (a TLAS rebuild, a physics dispatch) is recorded from `onUpdate` into `window.drawCmdBuffers[window.currentFrame]`; the heaps are bound by then and the passes come after. A `Resize` or `SetPresentMode` from a listener of the frame (an input action ticked in `onUpdate`, say) takes effect after its present. `onResize` fires after a swapchain rebuild (a new size or present mode); the swapchain slots follow by themselves, and anything else that holds the old images or is sized after the window is made again there. +- A `RenderPass` is `Record(cmd, frameIdx, window)` plus `SwapchainStage()`, which must return the stage that touches the swapchain image (`COMPUTE_SHADER`, `RAY_TRACING_SHADER` or `TRANSFER`): the frame's barriers take their stage and access masks from it. `frameIdx` is the frame slot, `window.currentFrame`, not the image: the frame's image is `window.images[window.currentImage]`. The image stays in `VK_IMAGE_LAYOUT_GENERAL` through the whole pass list, with `STORAGE | TRANSFER_SRC` usage and `TRANSFER_DST` where the surface allows it (`window.swapchainUsage`). Nothing clears it: draw a background quad, or clear from a transfer pass where `TRANSFER_DST` is there. - `window.width` and `height` are physical pixels, as is `currentMousePos`. `CaptureNextFrame(path)` saves the next frame as a PNG. `SetTitle`, `SetAppId`, `SetCursorImage`, `SetDefaultCursor`; `SetTextInput(active, x, y, w, h, purpose)` tells the compositor's input method a text field has focus, which opens an on-screen keyboard on a phone. -- Several windows can exist (on Wayland `Device::windows` lists them; the other backends keep no list), but they share one frame clock: one rendering window is the assumption behind the deletion queue and the staging rings. +- Several windows can exist (on Wayland `Device::windows` lists them; the other backends keep no list), and may share a heap, each writing only its own swapchain slots, but they share one frame clock: one rendering window is the assumption behind the deletion queue and the staging rings. ### Device and features - `Device::Initialize(required = {}, optional = {})` creates the instance, weighs every GPU and creates the device. `DeviceFeatures` holds the five core feature structs, `vulkan10` to `vulkan14`; only their `VkBool32`s are read, so `{.vulkan12 = {.shaderFloat16 = VK_TRUE}}` is a complete argument. The library enables what its own shaders use and nothing more: a `uint16_t` spec constant needs `.vulkan10.shaderInt16`, fp16 arithmetic `.vulkan12.shaderFloat16`, 16-bit storage `.vulkan11.storageBuffer16BitAccess`, scalar layout `.vulkan12.scalarBlockLayout`. A GPU lacking a required feature is skipped; when none is left the call throws naming each GPU's gaps, and may be called again asking for less. Optional features are enabled where the GPU has them; `Device::enabledFeatures` says which, `supportedFeatures` what the GPU has, `apiVersion` the version in use (`vulkan14` features need a 1.4 GPU; `maintenance5` is always on). `CRAFTER_GRAPHICS_DEVICE=` picks GPU `n`. - `VulkanShader` checks every module's SPIR-V capabilities against what was enabled and throws naming the file and the field to pass (`vulkan10.shaderInt16`), or saying the GPU lacks it. Most drivers accept such a module silently, so this is the error you get here instead of a wrong frame on another machine. -- Optional groups, decided once at `Initialize`: `Device::descriptorHeapSupported` (`VK_EXT_descriptor_heap`), `rayTracingSupported` (needs the heap too), `memoryDecompressionSupported` (`VK_EXT_memory_decompression`: `.cmesh` and `.ctex` decompress on the GPU; otherwise the library decodes them on the CPU), `deviceFaultSupported`. The RT entry points throw through `Device::RequireRayTracing` without ray tracing; the compute/UI path runs everywhere. +- Optional groups, decided once at `Initialize`: `Device::descriptorHeapSupported` (`VK_EXT_descriptor_heap`), `rayTracingSupported` (needs the heap too), `memoryDecompressionSupported` (`VK_EXT_memory_decompression`: `.cmesh` and `.ctex` decompress on the GPU; otherwise the library decodes them on the CPU), `deviceFaultSupported`, `swapchainMaintenance1Supported` (`VK_KHR_swapchain_maintenance1`, with the instance's `surface_maintenance1`, `surfaceMaintenance1Supported`: a swapchain names its present mode and gets that mode's own image count; otherwise the stricter of the mode's and the surface's). The RT entry points throw through `Device::RequireRayTracing` without ray tracing; the compute/UI path runs everywhere. - `Device::device`, `queue`, `commandPool`, `physDevice`, `instance`, `memoryProperties`, `deviceProperties`, `rayTracingProperties` and `descriptorHeapProperties` are public statics for raw Vulkan. `Device::GetMemoryType(typeBits, required, preferred = 0)` and `PreferDirectDeviceWrite(bytes)` are the allocation helpers. - `Device::pipelineCache` is shared by every pipeline and written to `Device::pipelineCachePath` (`pipeline_cache.bin` in the working directory) at exit. Set the path before `Initialize`, and gitignore the file. @@ -55,10 +55,10 @@ Frames are pipelined three deep and nothing waits for the GPU at steady state, s ### The bindless heap -`DescriptorHeapVulkan::Initialize(images, buffers, samplers)` sizes one resource heap and one sampler heap per frame (`resourceHeap[Window::numFrames]`, `samplerHeap[...]`, mapped `VulkanBuffer`s) and the slot allocators. `AllocateImageSlots(n)`, `AllocateBufferSlots(n)` and `AllocateSamplerSlots(n)` return `{firstElement, count}` ranges (contiguous; freed runs are reused), `Free*Slots` returns them, and running out throws naming the pool and the counts. `Initialize` throws too when its buffer slots would need shader-visible indices past 16 bits. +`DescriptorHeapVulkan::Initialize(images, buffers, samplers)` sizes one resource heap and one sampler heap per frame in flight (`resourceHeap[Window::numFrames]`, `samplerHeap[...]`, mapped `VulkanBuffer`s) and the slot allocators. `AllocateImageSlots(n)`, `AllocateBufferSlots(n)` and `AllocateSamplerSlots(n)` return `{firstElement, count}` ranges (contiguous; freed runs are reused), `Free*Slots` returns them, and running out throws naming the pool and the counts. `Initialize` throws too when its buffer slots would need shader-visible indices past 16 bits. - **Buffer slot indices are relative.** `AllocateBufferSlots` indexes the heap's buffer region, which starts after the image region, while a GLSL `descriptor_heap` SSBO array indexes buffer descriptors from heap byte 0. The shader-visible index is `heap.bufferStartElement + raw`. The RAII `BufferSlot` converts to exactly that (its `operator std::uint16_t()`), `UIRenderer::RegisterBuffer` returns one, and raw code passes `heap.bufferStartElement` as a spec constant: the RT examples declare `layout(constant_id = 0) const uint16_t bufferStart = 0us;` and index `topLevelAS[bufferStart]`, which is why they ask for `shaderInt16`. Image slots index the heap as they are, in image-descriptor units; samplers live in their own heap. A wrong buffer index reads zeros, which looks like every field of every item being zero. In descriptor-set mode `bufferStartElement` is 0, so code that goes through the conversion is right in both. -- Writing a descriptor by hand: `Device::vkWriteResourceDescriptorsEXT` into `heap.resourceHeap[f].value + heap.ImageByteOffset(imageRange.firstElement)` or `heap.BufferByteOffset(bufferRange.firstElement)`, `Device::descriptorHeapProperties.imageDescriptorSize` or `bufferDescriptorSize` bytes long, for each frame `f`, then `heap.resourceHeap[f].FlushDevice()`. `BufferByteOffset` takes the raw index `AllocateBufferSlots` returned and adds the region start itself; handing it a `BufferSlot`, which converts to the absolute index, adds the start twice and puts the descriptor in the wrong place. `ImageByteOffset` takes an image slot as it is. Per-frame resources (the swapchain image `window.imageViews[f]`, `tlases[f]`) are written into frame `f`'s heap; a shared resource into every frame's. `examples/VulkanTriangle/main.cpp` is the pattern. The `UIRenderer::Register*` helpers do this for you in both binding models. +- Writing a descriptor by hand: `Device::vkWriteResourceDescriptorsEXT` into `heap.resourceHeap[f].value + heap.ImageByteOffset(imageRange.firstElement)` or `heap.BufferByteOffset(bufferRange.firstElement)`, `Device::descriptorHeapProperties.imageDescriptorSize` or `bufferDescriptorSize` bytes long, for each frame `f`, then `heap.resourceHeap[f].FlushDevice()`. `BufferByteOffset` takes the raw index `AllocateBufferSlots` returned and adds the region start itself; handing it a `BufferSlot`, which converts to the absolute index, adds the start twice and puts the descriptor in the wrong place. `ImageByteOffset` takes an image slot as it is. Per-frame resources (`tlases[f]`) are written into frame `f`'s heap; a shared resource into every frame's. The swapchain image is neither, since frames and images pair up differently from frame to frame: list its image slot once with `heap.AddSwapchainSlot(slot, window)`, and the window writes the image each frame draws into there, a storage image in `GENERAL`, in both binding models and across resizes (a slot listed during a frame is written for that frame). `examples/VulkanTriangle/main.cpp` is the pattern. The `UIRenderer::Register*` helpers do this for you in both binding models. - Binding the heap outside the frame loop: `Window::Render` binds `window.descriptorHeap` in every frame it records. A command buffer it does not record that dispatches against the heap (`window.StartInit()`'s, say) calls `heap.Bind(cmd, frame)`, which binds that frame's heaps with their reserved ranges, or in descriptor-set mode its descriptor set. Don't build the `VkBindHeapInfoEXT` by hand: where the reserved range sits is the heap's to decide. - **Descriptor-set fallback.** On a driver without `VK_EXT_descriptor_heap` (Mesa Turnip on Adreno) `Device::descriptorHeapSupported` is false and the same slots live in one bindless descriptor set per frame; slot handles, item structs, `UIDispatchHeader` and the `UIRenderer` API are unchanged, so application code does not branch. Ray tracing is unavailable there. `CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP` forces this path on a driver that has the heap, to test it. Raw `vkWriteResourceDescriptorsEXT` code is heap-only; a consumer that must run on such a driver stays on the `UIRenderer::Register*` helpers. @@ -119,9 +119,9 @@ Heap-only: check `Device::rayTracingSupported`, or expect `std::runtime_error` f - Shaders: `VulkanShader shader("raygen.spv", "main", VK_SHADER_STAGE_RAYGEN_BIT_KHR, &specInfo);` (`nullptr` for no specialisation); `ShaderBindingTableVulkan table; table.Init(shaders);`; `PipelineRTVulkan pipeline; pipeline.Init(cmd, raygenGroups, missGroups, hitGroups, table, maxRecursionDepth = 1);` with spans of `VkRayTracingShaderGroupCreateInfoKHR` indexing the shader array. **`maxRecursionDepth` counts the raygen as 1**: a closest-hit that traces a shadow ray needs 2, and tracing past the pipeline's depth loses the device. - Geometry: `Mesh mesh; mesh.Build(std::span> vertices, std::span indices, cmd, RTBuildOptions{});` builds a BLAS and sets `mesh.blasAddr`; `Build(const CompressedMeshAsset&, cmd)` builds from a `.cmesh`, decompressed on the GPU where supported; `BuildProcedural(aabbs, opaque, cmd)` builds from `RTAabb` boxes for a `VK_RAY_TRACING_SHADER_GROUP_TYPE_PROCEDURAL_HIT_GROUP_KHR` with an intersection shader, and has an overload taking a device address a compute pass wrote. `RTBuildOptions` is `.preference` (`FastTrace`, `FastBuild`), `.allowUpdate` and `.noDuplicateAnyHit`; with `allowUpdate`, `Refit(vertices, indices, cmd)` and its dirty-range overload update in place and keep `blasAddr`. A rebuild into storage that still fits keeps `blasAddr` too; read it again after a rebuild that grows. -- Instances: a `RenderingElement3D` holds a `VkAccelerationStructureInstanceKHR instance` (`transform`, `mask`, `instanceCustomIndex`, `instanceShaderBindingTableRecordOffset`, `flags`, `accelerationStructureReference = mesh.blasAddr`; Crafter.Math's `MatrixRowMajor::Store(reinterpret_cast(instance.transform.matrix))` writes the transform), `userMetadata` (copied per instance id into `tlases[f].metadataBuffer`) and `transformOwnedByGpu`. `RenderingElement3D::Add(&e)` and `Remove(&e)` register it. `RenderingElement3D::BuildTLAS(cmd, frameIndex, preference = FastTrace)` builds or refits `tlases[frameIndex]` from every registered element: once per frame slot at setup (`0`, `1`, `2`), and per frame from `onUpdate` for `window.currentBuffer` when instances move. `e.MarkHostDataDirty()` after a change to an element's host fields lets the per-frame copy skip unchanged slots. `tlases[f].address` goes into a `VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR` descriptor of frame `f` (see "The bindless heap") and changes only when the TLAS outgrows its storage. +- Instances: a `RenderingElement3D` holds a `VkAccelerationStructureInstanceKHR instance` (`transform`, `mask`, `instanceCustomIndex`, `instanceShaderBindingTableRecordOffset`, `flags`, `accelerationStructureReference = mesh.blasAddr`; Crafter.Math's `MatrixRowMajor::Store(reinterpret_cast(instance.transform.matrix))` writes the transform), `userMetadata` (copied per instance id into `tlases[f].metadataBuffer`) and `transformOwnedByGpu`. `RenderingElement3D::Add(&e)` and `Remove(&e)` register it. `RenderingElement3D::BuildTLAS(cmd, frameIndex, preference = FastTrace)` builds or refits `tlases[frameIndex]` from every registered element: once per frame slot at setup (`0`, `1`, `2`), and per frame from `onUpdate` for `window.currentFrame` when instances move. `e.MarkHostDataDirty()` after a change to an element's host fields lets the per-frame copy skip unchanged slots. `tlases[f].address` goes into a `VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR` descriptor of frame `f` (see "The bindless heap") and changes only when the TLAS outgrows its storage. - The pass: `RTPass rtPass(&pipeline); rtPass.pushPtr = &camera; rtPass.pushBytes = sizeof(camera);` pushes your data at offset 0 of the shaders' push block before every trace (the bytes are read at record time, so updating `camera` before the frame is enough); `width` and `height` trace at a size other than the window's. `window.passes.push_back(&rtPass);`. -- In GLSL, with the extensions the examples enable (`GL_EXT_ray_tracing`, `GL_EXT_descriptor_heap`, `GL_EXT_nonuniform_qualifier` for a non-constant heap index, `GL_EXT_shader_explicit_arithmetic_types_int16` for a `uint16_t` spec constant, and `GL_EXT_shader_image_load_formatted` or a `writeonly` image): `layout(descriptor_heap) uniform accelerationStructureEXT topLevelAS[];`, `layout(descriptor_heap) uniform writeonly image2D image[];`, indices from spec constants or push data. The output image is the swapchain image at its image slot, in `VK_IMAGE_LAYOUT_GENERAL`. Start from `examples/VulkanTriangle/raygen.glsl`. +- In GLSL, with the extensions the examples enable (`GL_EXT_ray_tracing`, `GL_EXT_descriptor_heap`, `GL_EXT_nonuniform_qualifier` for a non-constant heap index, `GL_EXT_shader_explicit_arithmetic_types_int16` for a `uint16_t` spec constant, and `GL_EXT_shader_image_load_formatted` or a `writeonly` image): `layout(descriptor_heap) uniform accelerationStructureEXT topLevelAS[];`, `layout(descriptor_heap) uniform writeonly image2D image[];`, indices from spec constants or push data. The output image is the swapchain image at an image slot listed with `AddSwapchainSlot`, in `VK_IMAGE_LAYOUT_GENERAL`. Start from `examples/VulkanTriangle/raygen.glsl`. - **NVIDIA.** On the proprietary driver (still on `615.78.08`) reading an acceleration structure through the heap loses the device. The library works around it transparently: `VulkanShader` rewrites the SPIR-V so the heap AS read becomes a TLAS-address load from the push block plus `OpConvertUToAccelerationStructureKHR`, and `RTPass` and `ComputeShader` push the address. Nothing changes in your shaders or code, except that the rewrite appends an 8-byte member after your push block, and that it turns `shaderInt64` on for itself only: a shader of yours that declares `Int64` still has to ask for `shaderInt64`, or it runs on NVIDIA alone. `CRAFTER_GRAPHICS_NO_AS_WORKAROUND` disables the rewrite, to probe a new driver. Second note: on `610.43.02`, indexing a `descriptor_heap` array with a runtime value inside a hit shader faulted (fixed in `615.78.08`). The portable pattern, which the Sponza example uses, is one descriptor at a spec-constant slot and dynamic indexing inside it: all geometry in one SSBO indexed by element offset, materials in one `texture2DArray` indexed by layer. `examples/Sponza/README.md` has the GLSL. ### Input diff --git a/AGENTS.md b/AGENTS.md index 0cde585..05c0479 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,7 +31,7 @@ Partitions, by what they are for. Every partition file exists on every target; o crafter-build is the only build tool: the Flatpak (`alias crafter-build='flatpak run net.catcrafts.CrafterBuild'`; a locally built one runs with `--branch=local`) or a native build of the sibling checkout. `crafter-build --version` prints the compiler it resolved. Loading `project.cpp` clones Crafter.Event, Crafter.Math and Crafter.Asset into the crafter-build cache and, since they are unpinned, `git pull`s them on every invocation, and clones the two Khronos header repositories, so a build needs the network. `--local` takes the sibling checkouts (`../Crafter.Event`, `../Crafter.Math`, `../Crafter.Asset`) instead and hands the flag on, so Crafter.Asset picks the sibling Crafter.Math too; two copies of one module on the path do not link. -The host is Linux on Wayland. The native Linux backend is Wayland only (no X11), so every test that calls `Device::Initialize` needs `WAYLAND_DISPLAY`: it connects to the display right after creating the Vulkan instance, window or no window. The build links `wayland-client`, `xkbcommon`, `udev` and the Vulkan loader from the host or the sysroot; libevdev is vendored and built as its own static library, `Crafter.Graphics-libevdev`, once per build. A Vulkan 1.3 driver is the floor; `VK_EXT_descriptor_heap` (with `shaderUntypedPointers`) and ray tracing are optional groups detected at start-up. Recent Mesa's llvmpipe advertises the heap, so a machine without a GPU runs the UI path. The Khronos validation layer is optional and off by default. +The host is Linux on Wayland. The native Linux backend is Wayland only (no X11), so every test that calls `Device::Initialize` needs `WAYLAND_DISPLAY`: it connects to the display right after creating the Vulkan instance, window or no window. The build links `wayland-client`, `xkbcommon`, `udev` and the Vulkan loader from the host or the sysroot; libevdev is vendored and built as its own static library, `Crafter.Graphics-libevdev`, once per build. A Vulkan 1.3 driver is the floor; `VK_EXT_descriptor_heap` (with `shaderUntypedPointers`), ray tracing and `swapchain_maintenance1` (with the instance's `surface_maintenance1`, for per-present-mode image counts) are optional groups detected at start-up. Recent Mesa's llvmpipe advertises the heap, so a machine without a GPU runs the UI path. The Khronos validation layer is optional and off by default. Targets, picked with `--target=` and passed down to the dependencies: @@ -70,9 +70,9 @@ Tests are declared in `project.cpp`, not discovered, through the `LibraryTest(na A test is an `int main()`: exit 0 passes, nonzero fails, 77 skips (for a missing capability, never for a failed check). Each runs in its own process, in parallel, with a 60 s default timeout, and the output of a failed, crashed or timed-out test is in `build/test-logs/.log`; a later pass does not delete the log, so check its timestamp. What the tests need, in three groups: -- Pure logic, no GPU: they stage `Device`'s statics or feed synthetic data to the decision functions that were factored out for them (`Device::ResolveFeatures`, `ChooseDevice`, `CheckShaderCapabilities`, `GetMemoryType`, `PreferDirectDeviceWrite`, `PipelineCacheDataCompatible`, `AlignMappedFlushRange`, `BuildMipChainFinalBarriers`, `SwapchainStageUnion`, `BuildTlasWriteBarrier`, `UIRenderer::ClipFlags`, `WorkaroundNvidiaAS::Patch`, `DescriptorHeapVulkan::ComputeLayout`, the font and input-field helpers). `PushConstantRewrite` and `UIFusedShader` compile GLSL with glslang and check the SPIR-V with spirv-val at runtime, and `ExampleShaders` compiles every GLSL shader under `examples/` the way crafter-build would, the descriptor-set build included, so a shader contract change that breaks an example fails here without building the example; `MouseScroll` and `PointerEnterLeave` drive the Wayland listener callbacks directly. +- Pure logic, no GPU: they stage `Device`'s statics or feed synthetic data to the decision functions that were factored out for them (`Device::ResolveFeatures`, `ChooseDevice`, `CheckShaderCapabilities`, `GetMemoryType`, `PreferDirectDeviceWrite`, `PipelineCacheDataCompatible`, `AlignMappedFlushRange`, `BuildMipChainFinalBarriers`, `SwapchainStageUnion`, `BuildTlasWriteBarrier`, `UIRenderer::ClipFlags`, `Window::SwapchainMinImageCount`, `WorkaroundNvidiaAS::Patch`, `DescriptorHeapVulkan::ComputeLayout`, the font and input-field helpers). `PushConstantRewrite` and `UIFusedShader` compile GLSL with glslang and check the SPIR-V with spirv-val at runtime, and `ExampleShaders` compiles every GLSL shader under `examples/` the way crafter-build would, the descriptor-set build included, so a shader contract change that breaks an example fails here without building the example; `MouseScroll` and `PointerEnterLeave` drive the Wayland listener callbacks directly. - A headless device: `Device::Initialize()` and no `Window` (`HeapSlotReuse`, `DeferredDeletion`, `MeshLifetime`, `ImageStagingRelease`, `BLASBuildOptions`, the `TLAS*` tests, ...). Headless still means a Wayland display on Linux. Most of the RT ones check `Device::rayTracingSupported` and return 77 without it; `BLASBuildOptions`, `TLASInstanceDirtyTracking` and `MeshDecompressStagingRelease` do not, and crash on a GPU without ray tracing (`Device::RequireRayTracing` throws out of `main`). `DeviceSelection` skips with one GPU. -- A window and a live compositor: `FrameLoopSync`, `SetupCmdBufferReuse`, `UIFallbackParity`, `SwapchainTransferDst`, `WindowUpdateClock`, `HeapASRead`, `HeapDynamicIndexHit`, `RTPassPushData`, `WindowOptionalGlobals` (hides xdg-decoration, viewporter and fractional scale from the window in turn), and `FeatureRequests` for its retry case. They render, read the frame back with `Window::CaptureNextFrame` (a PNG, decoded with a private `STB_IMAGE_STATIC` copy of stb_image) and check pixels. +- A window and a live compositor: `FrameLoopSync`, `SetupCmdBufferReuse`, `UIFallbackParity`, `SwapchainTransferDst`, `WindowUpdateClock`, `HeapASRead`, `HeapDynamicIndexHit`, `RTPassPushData`, `WindowOptionalGlobals` (hides xdg-decoration, viewporter and fractional scale from the window in turn), `SwapchainImageCount` (every present mode, and swapchains with more images than frames in flight, in child processes on llvmpipe where the host's GPU never makes one), and `FeatureRequests` for its retry case. They render, read the frame back with `Window::CaptureNextFrame` (a PNG, decoded with a private `STB_IMAGE_STATIC` copy of stb_image) or a transfer pass of their own, and check pixels. Conventions the existing tests follow: @@ -109,7 +109,7 @@ These were decided deliberately. Do not undo them without the maintainer's say-s 4. **The UI contracts are frozen.** `UIDispatchHeader` is 48 bytes and starts every UI push block; `QuadItem` is 64, `CircleItem` 48, `ImageItem` 64 and `GlyphItem` 48 bytes, byte for byte with `ui-shared.glsl` under `std430`. They grow only through the reserved `flags` bits and `_pad`. `UIFusedHeader` (128 bytes) is a separate contract for `DispatchFused`. 5. **Three tiers, and the source is the customisation API.** Tier 1 is `ComputeShader` and `UIRenderer::Dispatch`; Tier 2 the standard kernels, item structs and `UIRenderer` helpers; Tier 3 stateless functions that append items to user-owned buffers and never dispatch. No library `Theme` type (colours come as `*Colors` aggregates), no widget tree, no layout engine beyond `Rect::SubRect` and `Inset`, no hit-testing, no focus management; state lives in user-owned PODs (`InputField` is the model). A component that does not fit is copied and edited, not hooked: no virtual overrides, no extension points. Additive changes (another kernel, another stateless component) are fine. 6. **Standard kernels dispatch one workgroup per 8x8 screen tile**, each thread owning one pixel and iterating every item in array order into a local accumulator, stored once. Array order is draw order and there is no inter-workgroup race. Never go back to a workgroup per item: Vulkan orders nothing between the workgroups of one dispatch, and overlapping items then race on the image (striped fills, stale pixels). The cost is `O(W*H*N)` and documented as such; the shared-memory tile cull in `ui-shared.glsl` must stay a strict superset of the per-pixel test. -7. **Three frames in flight, a fence per image, no steady-state wait-idle.** The owners of buffers, acceleration structures, shader modules and pipelines (`VulkanBuffer`, `Mesh`, `TlasWithBuffer`, `VulkanShader`, `ComputeShader`, `PipelineRTVulkan`) release them through `Device::EnqueueDeletion`, which has overloads for exactly those handle kinds; `ReclaimDeletions` frees an entry `framesInFlight` frames later and `DrainDeletions` after a wait-idle. Owners of handles the queue has no overload for wait for the GPU first and then destroy directly: `~DescriptorHeapVulkan` in descriptor-set mode, `~Window`, `~FontAtlas` through `ImageVulkan::Destroy`. A destructor never destroys a handle without one or the other: a frame in flight may still use it, and a static owner of a queued kind is destroyed after the driver has gone, which only the queue survives. `Clear()` is for a GPU known to be idle. The deletion queue, its mutex and `RenderingElement3D::tlases` are leaked on purpose for the same reason. Staged re-uploads go through the per-frame staging ring in `VulkanBuffer::Stage`. +7. **Three frames in flight, a fence per frame, no steady-state wait-idle.** Frames in flight are not swapchain images: the swapchain has as many images as its present mode needs, what lives per frame (command buffer, fence, acquire semaphore, heap copy, TLAS) is keyed by `Window::currentFrame` and what lives per image by `currentImage`, and the image a frame draws into reaches shaders through the heap's swapchain slots (`DescriptorHeapVulkan::AddSwapchainSlot`, #160). The owners of buffers, acceleration structures, shader modules and pipelines (`VulkanBuffer`, `Mesh`, `TlasWithBuffer`, `VulkanShader`, `ComputeShader`, `PipelineRTVulkan`) release them through `Device::EnqueueDeletion`, which has overloads for exactly those handle kinds; `ReclaimDeletions` frees an entry `framesInFlight` frames later and `DrainDeletions` after a wait-idle. Owners of handles the queue has no overload for wait for the GPU first and then destroy directly: `~DescriptorHeapVulkan` in descriptor-set mode, `~Window`, `~FontAtlas` through `ImageVulkan::Destroy`. A destructor never destroys a handle without one or the other: a frame in flight may still use it, and a static owner of a queued kind is destroyed after the driver has gone, which only the queue survives. `Clear()` is for a GPU known to be idle. The deletion queue, its mutex and `RenderingElement3D::tlases` are leaked on purpose for the same reason. Staged re-uploads go through the per-frame staging ring in `VulkanBuffer::Stage`. 8. **The library enables only the device features its own shaders use.** Everything an application's shaders need (`shaderInt16` for a `uint16_t` spec constant, fp16, 16-bit storage, `scalarBlockLayout`) is the application's to ask for through `Device::Initialize(required, optional)`, and `VulkanShader` refuses a module whose capabilities nothing enabled, naming the field. Do not add a feature to the `library` set for an example's convenience; have the example ask. 9. **Barriers are scoped and derived.** A `RenderPass` reports its `SwapchainStage()`; the frame loop's acquire, inter-pass and present barriers take their stage and access masks from the passes (`SwapchainStageUnion`, `SwapchainWriterAccess`, `BuildSwapchainInterPassBarrier`), masked by `Device::stageMask` so the RT stage never appears on a device without it. In the frame loop there is no `ALL_COMMANDS` and no queue-wide memory barrier, and a ranged flush rounds to `nonCoherentAtomSize`. The acceleration-structure builds and the GPU decompression do use global `VkMemoryBarrier`s on purpose (`BuildTlasWriteBarrier`, `BuildTlasReadBarrier`, `Decompress::DecompressOnGPU`): execution order alone never makes a build's writes visible to the shaders that read them. The window never clears the swapchain image (`TRANSFER_DST` is not on every surface); passes draw their own background. 10. **Driver workarounds are fenced and written to be deleted.** The NVIDIA descriptor-heap AS-read rewrite is one block in `ShaderVulkan.cppm` gated on `Device::workaroundDescriptorHeapAS`, with its push-data hook in `RTPass` and `ComputeShader`, a kill switch (`CRAFTER_GRAPHICS_NO_AS_WORKAROUND`), a probe test and a status note in the README and `examples/VulkanTriangle/README.md`. A new workaround gets the same: a `Device` flag, one fenced block, a probe, a note. A fault the library cannot hide (dynamic descriptor indexing in hit shaders) is documented with the consumer-side pattern instead (`examples/Sponza/README.md`). @@ -141,11 +141,11 @@ These were decided deliberately. Do not undo them without the maintainer's say-s - `Device::Initialize` on the Wayland backend connects to the display and throws without one; "headless" tests still need a compositor. - Buffer slot indices: `AllocateBufferSlots` returns an index relative to the heap's buffer region, while GLSL `descriptor_heap` SSBO arrays index in buffer-descriptor units from heap byte 0. `BufferSlot` converts to `bufferStartElement + raw`; raw code (the RT examples) passes `heap.bufferStartElement` as a `uint16_t` spec constant, which needs `shaderInt16`. In descriptor-set mode `bufferStartElement` is 0. Image and sampler slots are used as they are. A wrong index reads zeros, which looks like "every item field is zero". - `UIRenderer::RegisterBuffer` refuses a buffer created without `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT`. Freed slots are handed out again at once, so a slot may only be freed when no frame in flight reads it (after `vkQueueWaitIdle`, or at a scene switch). In descriptor-set mode a slot write destroys the previous view immediately. -- The swapchain image stays in `VK_IMAGE_LAYOUT_GENERAL` across the pass list and is never cleared; `Window::clearColor` is unused. `swapchainUsage` says whether `TRANSFER_DST` was available. `onResize` fires after a swapchain rebuild; anything holding descriptors of `window.imageViews[]` rewrites them there (`UIRenderer` does its own). +- The swapchain image stays in `VK_IMAGE_LAYOUT_GENERAL` across the pass list and is never cleared; `Window::clearColor` is unused. `swapchainUsage` says whether `TRANSFER_DST` was available. `onResize` fires after a swapchain rebuild; slots listed with `AddSwapchainSlot` follow by themselves, and anything else holding the old images is made again there. A rebuild asked for during a frame (`Resize` or `SetPresentMode` from a listener) waits for that frame's present, since the rest of the frame indexes the swapchain it acquired from. `RenderPass::Record`'s `frameIdx` is the frame slot, not the image. - `Device::frameCounter` is one clock for every window, so the deletion queue and the staging rings assume one rendering window. - `ImageVulkan::Create`'s default `consumerStage` is the RT stage, corrected to compute on a device without ray tracing; the font atlas passes compute explicitly. `VulkanTransition`'s `GENERAL` mapping does the same. - `UIRenderer::ShapeText` and `FontAtlas` key their caches on the `Font*`: call `UIRenderer::InvalidateFont` before destroying a `Font`, or a new one at the same address aliases stale runs. -- `RenderingElement3D::elements` stores raw pointers; a `std::vector` that grows after `Add` dangles them (Sponza reserves up front). `BuildTLAS` on an index whose earlier submission has not completed overwrites a TLAS in use; building `tlases[window.currentBuffer]` from `onUpdate` is safe, and `tlases[i].address` changes when a TLAS outgrows its storage. +- `RenderingElement3D::elements` stores raw pointers; a `std::vector` that grows after `Add` dangles them (Sponza reserves up front). `BuildTLAS` on an index whose earlier submission has not completed overwrites a TLAS in use; building `tlases[window.currentFrame]` from `onUpdate` is safe, and `tlases[i].address` changes when a TLAS outgrows its storage. - `PipelineRTVulkan::Init` defaults `maxRecursionDepth` to 1; a closest-hit that traces needs 2, and tracing deeper than the pipeline allows loses the device. - `BuildTlasWriteBarrier`'s source access mask carries no read bit on purpose: synchronization validation 1.4.363 otherwise reports two builds in one command buffer as a write-after-write hazard. - NVIDIA, as of driver `615.78.08`: reading a TLAS through the heap still faults (worked around); dynamic descriptor indexing in hit shaders works there and faulted on `610.43.02`, so the Sponza pattern (a spec-constant slot, a dynamic index within the resource) stays the portable one. `HeapDynamicIndexHit` skips on the older drivers. diff --git a/README.md b/README.md index cbaba4e..78afdab 100644 --- a/README.md +++ b/README.md @@ -91,18 +91,36 @@ shaded through an intersection shader with an any-hit cut-out. compositor has them. Without xdg-decoration (GNOME) the window has no title bar, as the library draws no client-side decorations; when either of the other two is missing it draws at scale 1, and the compositor - scales it up on a HiDPI output. + scales it up on a HiDPI output. Natively the frame loop keeps three + frames in flight (`Window::numFrames`) over however many images the + swapchain has: as many as its present mode needs. Where the instance has + `surface_maintenance1` and the device `swapchain_maintenance1`, that is + the mode's own count, which on Mesa's Wayland is below the surface's + count for all modes (FIFO three, Mailbox four); without, the stricter + of the two. `window.currentFrame` picks what lives per frame in flight + (the command buffer, the descriptor-heap copy, + `RenderingElement3D::tlases`, the `frameIdx` a pass is recorded with); + `window.currentImage` is the swapchain image the frame draws into, + `window.images[window.currentImage]`. A shader reads that image from a + heap slot listed with `heap.AddSwapchainSlot(slot, window)`, which the + window rewrites with the acquired image every frame, so nothing is + rewritten after a resize, and two windows can share a heap. A `Resize` + or `SetPresentMode` from a listener of the frame being recorded takes + effect after that frame's present. - **Device** *(Vulkan only)* — single-instance bring-up targeting `VK_EXT_descriptor_heap`; pipelines are created with `VK_PIPELINE_CREATE_2_DESCRIPTOR_HEAP_BIT_EXT` so there are no descriptor-set layouts and push constants travel via - `vkCmdPushDataEXT`. Needs Vulkan 1.3. The heap, ray tracing and - device-fault extensions are optional groups, detected at start-up - (`Device::descriptorHeapSupported`, `rayTracingSupported`, - `deviceFaultSupported`). Device features split by who uses them: the - ones the binding model and the library's own shaders need are the - library's to enable; arithmetic, storage and layout features an - application's shaders use (`shaderInt16` for a `uint16_t` spec + `vkCmdPushDataEXT`. Needs Vulkan 1.3. The heap, ray tracing, + device-fault and swapchain-maintenance extensions are optional groups, + detected at start-up (`Device::descriptorHeapSupported`, + `rayTracingSupported`, `deviceFaultSupported`, + `swapchainMaintenance1Supported`), as is the instance's + `surface_maintenance1` (`surfaceMaintenance1Supported`). Device + features split by who uses them: the ones the binding model and the + library's own shaders need are the library's to enable; arithmetic, + storage and layout features an application's shaders use + (`shaderInt16` for a `uint16_t` spec constant, `scalarBlockLayout`, fp16, 16-bit storage) are the application's to ask for, as core feature structs: `Device::Initialize({.vulkan10 = {.shaderInt16 = VK_TRUE}})`. A second diff --git a/examples/README.md b/examples/README.md index 1bd619d..ef2fda7 100644 --- a/examples/README.md +++ b/examples/README.md @@ -21,8 +21,9 @@ Ray-traced single triangle through `vkCmdTraceRaysKHR`. Shows the full ray-tracing setup: `DescriptorHeapVulkan` with image and buffer slots, `PipelineRTVulkan` from raygen / miss / closesthit SPIR-V, BLAS via `Mesh::Build`, TLAS via `RenderingElement3D::BuildTLAS`, direct -`vkWriteResourceDescriptorsEXT` for swapchain views, `RTPass` on -`window.passes`. Smallest test of the bindless ray-tracing path. +`vkWriteResourceDescriptorsEXT` for the per-frame TLASes, +`AddSwapchainSlot` for the output image, `RTPass` on `window.passes`. +Smallest test of the bindless ray-tracing path. ### [HelloUI](HelloUI/) Compute-shader UI demo using all three UI tiers: diff --git a/examples/RTVolume/main.cpp b/examples/RTVolume/main.cpp index bf4b160..5c33e4a 100644 --- a/examples/RTVolume/main.cpp +++ b/examples/RTVolume/main.cpp @@ -113,7 +113,7 @@ int main() { // descriptor-heap compute pipeline still expects a heap bound for its // push-data path — the render loop binds it every frame, but the init // command buffer hasn't, so bind it here before dispatching. - descriptorHeap.Bind(cmd, window.currentBuffer); + descriptorHeap.Bind(cmd, window.currentFrame); ComputeShader aabbWriter; aabbWriter.Load("aabbs.comp.spv", descriptorHeap); @@ -167,23 +167,20 @@ int main() { window.FinishInit(); - // Write descriptors: TLAS at bufSlots[0], output image at imgSlots[0]. - // Per-frame replicated — same pattern as the Sponza example. + // Write descriptors: each frame's TLAS at bufSlots[0] of that frame's + // copy of the heap — same pattern as the Sponza example. The output image + // at imgSlots[0] is the swapchain's, which the window writes every frame + // with the image that frame draws into. + descriptorHeap.AddSwapchainSlot(imgSlots.firstElement, window); VkDeviceAddressRangeKHR tlasRanges[Window::numFrames]; - VkImageDescriptorInfoEXT outImgInfos[Window::numFrames]; for (std::uint32_t f = 0; f < Window::numFrames; ++f) { tlasRanges[f] = { .address = RenderingElement3D::tlases[f].address }; - outImgInfos[f] = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[f], - .layout = VK_IMAGE_LAYOUT_GENERAL, - }; } std::vector resources; std::vector destinations; - resources.reserve(Window::numFrames * 2); - destinations.reserve(Window::numFrames * 2); + resources.reserve(Window::numFrames); + destinations.reserve(Window::numFrames); for (std::uint32_t f = 0; f < Window::numFrames; ++f) { resources.push_back({ .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, @@ -195,16 +192,6 @@ int main() { + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize, }); - resources.push_back({ - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &outImgInfos[f] }, - }); - destinations.push_back({ - .address = descriptorHeap.resourceHeap[f].value - + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize, - }); } Device::vkWriteResourceDescriptorsEXT(Device::device, static_cast(resources.size()), diff --git a/examples/Sponza/main.cpp b/examples/Sponza/main.cpp index 9e906c3..761ec27 100644 --- a/examples/Sponza/main.cpp +++ b/examples/Sponza/main.cpp @@ -165,11 +165,13 @@ int main() { window.FinishInit(); - // Write descriptors: TLAS at bufSlots[0], output image at imgSlots[0], - // albedo (a sampled image) at imgSlots[1], and its sampler at - // samplerSlots[0] in the sampler heap. Per-frame replicated. + // Write descriptors: TLAS at bufSlots[0] and albedo (a sampled image) at + // imgSlots[1], into every frame's copy of the heap, each frame with its + // own TLAS, and the albedo's sampler at samplerSlots[0] in the sampler + // heap. The output image at imgSlots[0] is the swapchain's, which the + // window writes every frame with the image that frame draws into. + descriptorHeap.AddSwapchainSlot(imgSlots.firstElement, window); VkDeviceAddressRangeKHR tlasRanges[Window::numFrames]; - VkImageDescriptorInfoEXT outImgInfos[Window::numFrames]; // The heap takes an image view's create-info, not a VkImageView. const VkImageViewCreateInfo albedoView { .sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO, @@ -189,17 +191,12 @@ int main() { }; for (std::uint32_t f = 0; f < Window::numFrames; ++f) { tlasRanges[f] = { .address = RenderingElement3D::tlases[f].address }; - outImgInfos[f] = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[f], - .layout = VK_IMAGE_LAYOUT_GENERAL, - }; } std::vector resources; std::vector destinations; - resources.reserve(Window::numFrames * 3); - destinations.reserve(Window::numFrames * 3); + resources.reserve(Window::numFrames * 2); + destinations.reserve(Window::numFrames * 2); for (std::uint32_t f = 0; f < Window::numFrames; ++f) { resources.push_back({ .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, @@ -211,16 +208,6 @@ int main() { + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize, }); - resources.push_back({ - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &outImgInfos[f] }, - }); - destinations.push_back({ - .address = descriptorHeap.resourceHeap[f].value - + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize, - }); resources.push_back({ .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE, diff --git a/examples/SponzaBench/main.cpp b/examples/SponzaBench/main.cpp index 377a474..614d5e6 100644 --- a/examples/SponzaBench/main.cpp +++ b/examples/SponzaBench/main.cpp @@ -190,29 +190,26 @@ int main() { albedo.Update(loadedTex, cmd, VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL); // The albedo is uploaded to keep the texture-decompress + staging // setup path in the measured "setup" window, but is not bound — the - // closest-hit shades from barycentrics, so only the TLAS + per-frame - // output image need descriptors. + // closest-hit shades from barycentrics, so only the per-frame TLAS and + // the output image need descriptors. for (std::uint32_t f = 0; f < Window::numFrames; ++f) RenderingElement3D::BuildTLAS(cmd, f); window.FinishInit(); + // The output image at imgSlots[0] is the swapchain's, which the window + // writes every frame with the image that frame draws into. + descriptorHeap.AddSwapchainSlot(imgSlots.firstElement, window); VkDeviceAddressRangeKHR tlasRanges[Window::numFrames]; - VkImageDescriptorInfoEXT outImgInfos[Window::numFrames]; for (std::uint32_t f = 0; f < Window::numFrames; ++f) { tlasRanges[f] = { .address = RenderingElement3D::tlases[f].address }; - outImgInfos[f] = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[f], - .layout = VK_IMAGE_LAYOUT_GENERAL, - }; } std::vector resources; std::vector destinations; - resources.reserve(Window::numFrames * 2); - destinations.reserve(Window::numFrames * 2); + resources.reserve(Window::numFrames); + destinations.reserve(Window::numFrames); for (std::uint32_t f = 0; f < Window::numFrames; ++f) { resources.push_back({ .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, @@ -224,16 +221,6 @@ int main() { + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize, }); - resources.push_back({ - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &outImgInfos[f] }, - }); - destinations.push_back({ - .address = descriptorHeap.resourceHeap[f].value - + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize, - }); } Device::vkWriteResourceDescriptorsEXT(Device::device, static_cast(resources.size()), diff --git a/examples/VulkanTriangle/README.md b/examples/VulkanTriangle/README.md index b416c82..a479ca7 100644 --- a/examples/VulkanTriangle/README.md +++ b/examples/VulkanTriangle/README.md @@ -17,7 +17,9 @@ The minimal ray-traced example. Renders a single static triangle through - `Mesh::Build` constructing a BLAS and `RenderingElement3D::BuildTLAS` the per-frame TLAS. - Direct descriptor writes via `vkWriteResourceDescriptorsEXT` for the - swapchain views and TLAS device addresses. + per-frame TLAS device addresses, and `AddSwapchainSlot` for the output + image, which the window writes with the swapchain image each frame + draws into. - `RTPass{&pipeline}` plugged into `window.passes` — the canonical way to add ray tracing to a window in this library. diff --git a/examples/VulkanTriangle/main.cpp b/examples/VulkanTriangle/main.cpp index af0cd5b..cd99ef8 100644 --- a/examples/VulkanTriangle/main.cpp +++ b/examples/VulkanTriangle/main.cpp @@ -105,6 +105,12 @@ int main() { auto imgSlots = descriptorHeap.AllocateImageSlots(1); auto bufSlots = descriptorHeap.AllocateBufferSlots(1); + // The output image is the swapchain image the frame draws into, which + // changes from frame to frame: the window writes it into the heap at this + // slot every frame. + descriptorHeap.AddSwapchainSlot(imgSlots.firstElement, window); + + // Each frame in flight has its own TLAS and its own copy of the heap. VkDeviceAddressRangeKHR tlasRange0 = { .address = RenderingElement3D::tlases[0].address, }; @@ -117,85 +123,40 @@ int main() { .address = RenderingElement3D::tlases[2].address, }; - VkImageDescriptorInfoEXT imageInfo0 = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[0], - .layout = VK_IMAGE_LAYOUT_GENERAL - }; - - VkImageDescriptorInfoEXT imageInfo1 = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[1], - .layout = VK_IMAGE_LAYOUT_GENERAL - }; - - VkImageDescriptorInfoEXT imageInfo2 = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[2], - .layout = VK_IMAGE_LAYOUT_GENERAL - }; - - VkResourceDescriptorInfoEXT resources[6] = { + VkResourceDescriptorInfoEXT resources[3] = { { .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, .data = { .pAddressRange = &tlasRange0} }, - { - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &imageInfo0 } - }, { .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, .data = { .pAddressRange = &tlasRange1} }, - { - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &imageInfo1 } - }, { .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, .data = { .pAddressRange = &tlasRange2} }, - { - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &imageInfo2 } - }, }; - - VkHostAddressRangeEXT destinations[6] = { + + VkHostAddressRangeEXT destinations[3] = { { .address = descriptorHeap.resourceHeap[0].value + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize }, - { - .address = descriptorHeap.resourceHeap[0].value + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize - }, { .address = descriptorHeap.resourceHeap[1].value + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize }, - { - .address = descriptorHeap.resourceHeap[1].value + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize - }, { .address = descriptorHeap.resourceHeap[2].value + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize }, - { - .address = descriptorHeap.resourceHeap[2].value + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize - }, }; - Device::vkWriteResourceDescriptorsEXT(Device::device, 6, resources, destinations); + Device::vkWriteResourceDescriptorsEXT(Device::device, 3, resources, destinations); descriptorHeap.resourceHeap[0].FlushDevice(); descriptorHeap.resourceHeap[1].FlushDevice(); diff --git a/implementations/Crafter.Graphics-Device.cpp b/implementations/Crafter.Graphics-Device.cpp index 9abc596..a5876d3 100644 --- a/implementations/Crafter.Graphics-Device.cpp +++ b/implementations/Crafter.Graphics-Device.cpp @@ -42,6 +42,12 @@ const char* const instanceExtensionNames[] = { "VK_KHR_wayland_surface" #endif }; +namespace { + // The surface_maintenance1 Initialize enabled on the instance, the KHR + // form or the EXT one, or null. A device's swapchain_maintenance1 has to + // be the same form: each requires its own. + const char* instanceSurfaceMaintenance1 = nullptr; +} // Device extensions are split by what breaks without them. Only the first // group is needed to put UI pixels on screen, and a GPU without it is skipped, // as is one below Vulkan 1.4 without VK_KHR_maintenance5 (core from 1.4). @@ -778,6 +784,31 @@ void Device::seat_handle_capabilities(void*, wl_seat* seat, uint32_t capabilitie #endif +std::vector> Device::DistinctGpus(VkInstance instance) { + std::uint32_t physDeviceCount = 0; + CheckVkResult(vkEnumeratePhysicalDevices(instance, &physDeviceCount, nullptr)); + std::vector physDevices(physDeviceCount); + CheckVkResult(vkEnumeratePhysicalDevices(instance, &physDeviceCount, physDevices.data())); + + std::vector> gpus; + std::vector> seen; + for (VkPhysicalDevice device : physDevices) { + VkPhysicalDeviceIDProperties id { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ID_PROPERTIES }; + VkPhysicalDeviceProperties2 properties { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2, .pNext = &id }; + vkGetPhysicalDeviceProperties2(device, &properties); + std::array uuid; + std::ranges::copy(id.deviceUUID, uuid.begin()); + const auto at = std::ranges::find(seen, uuid); + if (at == seen.end()) { + seen.push_back(uuid); + gpus.push_back({ device }); + } else { + gpus[at - seen.begin()].push_back(device); + } + } + return gpus; +} + namespace { // What Initialize learns about one GPU before choosing it. struct Candidate { @@ -800,6 +831,10 @@ namespace { bool faultVendorBinary = false; bool decompression = false; bool workaroundAS = false; + // VK_KHR_swapchain_maintenance1 or VK_EXT_swapchain_maintenance1, + // whichever matches the instance's surface_maintenance1, with its + // feature bit; null without. Becomes swapchainMaintenance1Supported. + const char* swapchainMaintenance1 = nullptr; // The extensions every device needs, maintenance5 among them below // Vulkan 1.4. Initialize adds the groups'. std::vector requiredExtensions; @@ -880,6 +915,14 @@ namespace { const bool rayTracingExtensions = hasAll(rayTracingExtensionNames); const bool faultExtension = availableExtensions.contains(VK_EXT_DEVICE_FAULT_EXTENSION_NAME); const bool decompressionExtension = availableExtensions.contains(VK_EXT_MEMORY_DECOMPRESSION_EXTENSION_NAME); + const char* swapchainMaintenance1Extension = + instanceSurfaceMaintenance1 == nullptr ? nullptr + : std::string_view(instanceSurfaceMaintenance1) == VK_KHR_SURFACE_MAINTENANCE_1_EXTENSION_NAME + ? VK_KHR_SWAPCHAIN_MAINTENANCE_1_EXTENSION_NAME + : VK_EXT_SWAPCHAIN_MAINTENANCE_1_EXTENSION_NAME; + if (swapchainMaintenance1Extension != nullptr && !availableExtensions.contains(swapchainMaintenance1Extension)) { + swapchainMaintenance1Extension = nullptr; + } DeviceFeatures& supported = candidate.supported; VkPhysicalDeviceMaintenance5Features maintenance5Query { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MAINTENANCE_5_FEATURES, @@ -905,6 +948,10 @@ namespace { VkPhysicalDeviceMemoryDecompressionFeaturesEXT decompressionQuery { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_DECOMPRESSION_FEATURES_EXT, }; + // The EXT form's struct is the same one under another name. + VkPhysicalDeviceSwapchainMaintenance1FeaturesKHR swapchainMaintenance1Query { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SWAPCHAIN_MAINTENANCE_1_FEATURES_KHR, + }; VkPhysicalDeviceFeatures2 featureQuery { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, }; @@ -935,6 +982,9 @@ namespace { if (decompressionExtension) { ask(decompressionQuery); } + if (swapchainMaintenance1Extension != nullptr) { + ask(swapchainMaintenance1Query); + } vkGetPhysicalDeviceFeatures2(physical, &featureQuery); supported.vulkan10 = featureQuery.features; // The chain pointed at this function's locals. @@ -959,6 +1009,7 @@ namespace { candidate.fault = faultExtension && faultQuery.deviceFault; candidate.faultVendorBinary = faultQuery.deviceFaultVendorBinary; candidate.decompression = decompressionExtension && decompressionQuery.memoryDecompression; + candidate.swapchainMaintenance1 = swapchainMaintenance1Query.swapchainMaintenance1 ? swapchainMaintenance1Extension : nullptr; // NVIDIA's brand-new VK_EXT_descriptor_heap acceleration-structure read // path faults (see #7); enable the SPIR-V rewrite workaround there. // Other drivers (and any future fixed NVIDIA driver, once this gate is @@ -1043,33 +1094,7 @@ namespace { // The GPU Initialize uses, or a throw naming why there is none. Candidate ChooseGpu(VkInstance instance, std::uint32_t instanceVersion, const DeviceFeatures& required, const DeviceFeatures& optional) { - std::uint32_t physDeviceCount = 0; - Device::CheckVkResult(vkEnumeratePhysicalDevices(instance, &physDeviceCount, nullptr)); - std::vector physDevices(physDeviceCount); - Device::CheckVkResult(vkEnumeratePhysicalDevices(instance, &physDeviceCount, physDevices.data())); - - // The handles, grouped by GPU: a GPU two drivers both list (a Flatpak - // sees the runtime's and the GL extension's manifests; RADV beside - // AMDVLK) has one device UUID and two handles. The numbers below count - // GPUs, so one means the same GPU inside a sandbox and out. - std::vector> gpus; - { - std::vector> seen; - for (VkPhysicalDevice device : physDevices) { - VkPhysicalDeviceIDProperties id { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ID_PROPERTIES }; - VkPhysicalDeviceProperties2 properties { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2, .pNext = &id }; - vkGetPhysicalDeviceProperties2(device, &properties); - std::array uuid; - std::ranges::copy(id.deviceUUID, uuid.begin()); - const auto at = std::ranges::find(seen, uuid); - if (at == seen.end()) { - seen.push_back(uuid); - gpus.push_back({ device }); - } else { - gpus[at - seen.begin()].push_back(device); - } - } - } + const std::vector> gpus = Device::DistinctGpus(instance); // A GPU's first handle that can run the application, else its first // handle, gaps and all. const auto evaluate = [&](const std::vector& handles) { @@ -1153,8 +1178,36 @@ void Device::Initialize(const DeviceFeatures& required, const DeviceFeatures& op instanceCreateInfo.sType = VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO; instanceCreateInfo.pNext = enableValidation ? (const void*)&validationFeatures : nullptr; instanceCreateInfo.pApplicationInfo = &app; - instanceCreateInfo.enabledExtensionCount = sizeof(instanceExtensionNames) / sizeof(const char*); - instanceCreateInfo.ppEnabledExtensionNames = instanceExtensionNames; + + // The extensions above, plus optional ones where the loader has them: + // surface_maintenance1 answers a surface's capabilities for one present + // mode, through vkGetPhysicalDeviceSurfaceCapabilities2KHR, so it comes + // with get_surface_capabilities2 or not at all. The KHR form is the EXT + // one promoted; either will do. + std::vector enabledInstanceExtensions(std::begin(instanceExtensionNames), std::end(instanceExtensionNames)); + { + std::uint32_t count = 0; + CheckVkResult(vkEnumerateInstanceExtensionProperties(nullptr, &count, nullptr)); + std::vector available(count); + CheckVkResult(vkEnumerateInstanceExtensionProperties(nullptr, &count, available.data())); + const auto offered = [&](const char* name) { + return std::ranges::any_of(available, [&](const VkExtensionProperties& extension) { + return std::strcmp(extension.extensionName, name) == 0; + }); + }; + const char* surfaceMaintenance1 = + offered(VK_KHR_SURFACE_MAINTENANCE_1_EXTENSION_NAME) ? VK_KHR_SURFACE_MAINTENANCE_1_EXTENSION_NAME + : offered(VK_EXT_SURFACE_MAINTENANCE_1_EXTENSION_NAME) ? VK_EXT_SURFACE_MAINTENANCE_1_EXTENSION_NAME + : nullptr; + surfaceMaintenance1Supported = surfaceMaintenance1 != nullptr && offered(VK_KHR_GET_SURFACE_CAPABILITIES_2_EXTENSION_NAME); + instanceSurfaceMaintenance1 = surfaceMaintenance1Supported ? surfaceMaintenance1 : nullptr; + if (surfaceMaintenance1Supported) { + enabledInstanceExtensions.push_back(VK_KHR_GET_SURFACE_CAPABILITIES_2_EXTENSION_NAME); + enabledInstanceExtensions.push_back(surfaceMaintenance1); + } + } + instanceCreateInfo.enabledExtensionCount = static_cast(enabledInstanceExtensions.size()); + instanceCreateInfo.ppEnabledExtensionNames = enabledInstanceExtensions.data(); size_t foundInstanceLayers = 0; @@ -1182,6 +1235,14 @@ void Device::Initialize(const DeviceFeatures& required, const DeviceFeatures& op } CheckVkResult(vkCreateInstance(&instanceCreateInfo, NULL, &instance)); + if (surfaceMaintenance1Supported) { + vkGetPhysicalDeviceSurfaceCapabilities2KHR = reinterpret_cast( + vkGetInstanceProcAddr(instance, "vkGetPhysicalDeviceSurfaceCapabilities2KHR")); + surfaceMaintenance1Supported = vkGetPhysicalDeviceSurfaceCapabilities2KHR != nullptr; + if (!surfaceMaintenance1Supported) { + instanceSurfaceMaintenance1 = nullptr; + } + } VkDebugUtilsMessengerCreateInfoEXT debugUtilsMessengerCreateInfo = {}; debugUtilsMessengerCreateInfo.sType = VK_STRUCTURE_TYPE_DEBUG_UTILS_MESSENGER_CREATE_INFO_EXT; @@ -1239,6 +1300,7 @@ void Device::Initialize(const DeviceFeatures& required, const DeviceFeatures& op rayTracingSupported = chosen.rayTracing; deviceFaultSupported = chosen.fault; memoryDecompressionSupported = chosen.decompression; + swapchainMaintenance1Supported = chosen.swapchainMaintenance1 != nullptr; workaroundDescriptorHeapAS = chosen.workaroundAS; shaderInt64ForWorkaroundOnly = chosen.workaroundAS && required.vulkan10.shaderInt64 == VK_FALSE && optional.vulkan10.shaderInt64 == VK_FALSE; @@ -1344,6 +1406,11 @@ void Device::Initialize(const DeviceFeatures& required, const DeviceFeatures& op .memoryDecompression = VK_TRUE, }; + VkPhysicalDeviceSwapchainMaintenance1FeaturesKHR swapchainMaintenance1Features { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SWAPCHAIN_MAINTENANCE_1_FEATURES_KHR, + .swapchainMaintenance1 = VK_TRUE, + }; + VkPhysicalDeviceDescriptorHeapFeaturesEXT desciptorHeapFeatures { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DESCRIPTOR_HEAP_FEATURES_EXT, .descriptorHeap = VK_TRUE, @@ -1379,6 +1446,9 @@ void Device::Initialize(const DeviceFeatures& required, const DeviceFeatures& op if (memoryDecompressionSupported) { link(memoryDecompressionFeatures); } + if (swapchainMaintenance1Supported) { + link(swapchainMaintenance1Features); + } if (rayTracingSupported) { link(physicalDeviceRayQueryFeatures); link(physicalDeviceRayTracingPipelineFeatures); @@ -1417,6 +1487,9 @@ void Device::Initialize(const DeviceFeatures& required, const DeviceFeatures& op if (memoryDecompressionSupported) { enabledDeviceExtensions.push_back(VK_EXT_MEMORY_DECOMPRESSION_EXTENSION_NAME); } + if (swapchainMaintenance1Supported) { + enabledDeviceExtensions.push_back(chosen.swapchainMaintenance1); + } VkDeviceCreateInfo deviceCreateInfo = {}; deviceCreateInfo.sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO; @@ -1610,9 +1683,11 @@ namespace { void Device::ReclaimDeletions() { std::lock_guard lock(deletionMutex); // Free every entry whose retire frame has been reached. An entry tagged at - // frame F retires at F + framesInFlight: by the time the CPU has begun that - // many later frames (each gated by a per-image fence wait), single-queue - // submission order guarantees all GPU work from frame F is complete. + // frame F retires at F + framesInFlight: Window::Render calls this after + // waiting for its frame slot's fence, and that slot's last submission is + // frame F itself once framesInFlight frames have passed, so all GPU work + // from frame F is complete (and, by single-queue submission order, + // everything before it). // // frameCounter is monotonic, so retireAfter is non-decreasing down the // queue and the ready entries are a contiguous prefix: pop them off the diff --git a/implementations/Crafter.Graphics-UI-Shared.cpp b/implementations/Crafter.Graphics-UI-Shared.cpp index 112e1d5..ce5fa13 100644 --- a/implementations/Crafter.Graphics-UI-Shared.cpp +++ b/implementations/Crafter.Graphics-UI-Shared.cpp @@ -37,7 +37,7 @@ UIDispatchHeader UIRenderer::FillHeader(std::uint32_t itemBufferSlot, h.clipH = clipRectPx[3]; h.itemCount = itemCount; #ifndef CRAFTER_GRAPHICS_WINDOW_DOM - h.frameIdx = window_->currentBuffer; + h.frameIdx = window_->currentFrame; #else h.frameIdx = 0; #endif diff --git a/implementations/Crafter.Graphics-UI-WebGPU.cpp b/implementations/Crafter.Graphics-UI-WebGPU.cpp index ba58067..bd92ea6 100644 --- a/implementations/Crafter.Graphics-UI-WebGPU.cpp +++ b/implementations/Crafter.Graphics-UI-WebGPU.cpp @@ -53,10 +53,8 @@ void UIRenderer::Initialize(Window& window, GraphicsDescriptorHeap& heap, Graphi fontAtlasSamplerSlot_ = RegisterLinearClampSampler(); } - resizeSub_.SetEvent(&window.onResize, [this]() { - // Storage textures + bind groups are recreated JS-side at next - // wgpuFrameBegin (via the ensureSized path). Nothing to do here. - }); + // Nothing to redo on a resize: storage textures and bind groups are + // recreated JS-side at the next wgpuFrameBegin (the ensureSized path). } void UIRenderer::Record(GraphicsCommandBuffer cmd, std::uint32_t frameIdx, Window& window) { diff --git a/implementations/Crafter.Graphics-UI.cpp b/implementations/Crafter.Graphics-UI.cpp index b2ce600..420d6e8 100644 --- a/implementations/Crafter.Graphics-UI.cpp +++ b/implementations/Crafter.Graphics-UI.cpp @@ -36,12 +36,13 @@ void UIRenderer::Initialize(Window& window, GraphicsDescriptorHeap& heap, Graphi drawText.Load(textSpv, heap); drawFused.Load(fusedSpv, heap); - // Allocate one image slot for the swapchain output. Each per-frame heap - // copy will hold ITS frame's image at this slot. + // One image slot for the swapchain output. The window writes the image + // each frame draws into there, in that frame's heap copy (or set), so it + // follows resizes and present-mode changes by itself; called during a + // frame, it is written for that frame too. auto outRange = heap_->AllocateImageSlots(1); outImageSlot_ = ImageSlot{heap_, outRange.firstElement}; - - WriteSwapchainDescriptors(); + heap_->AddSwapchainSlot(outRange.firstElement, window); // Optional font-atlas registration (user must have called atlas->Initialize // already before reaching here, so atlas->image is live). @@ -52,17 +53,9 @@ void UIRenderer::Initialize(Window& window, GraphicsDescriptorHeap& heap, Graphi WriteFontAtlasDescriptor(); } - // No whole-heap flush here: WriteSwapchainDescriptors / WriteFontAtlasDescriptor - // / RegisterSampler each flush their own written byte range above, so the GPU - // already sees everything written during setup. - - // Re-bind the swapchain image descriptors after every resize. Window - // already drained the queue and rebuilt the imageViews[] before - // firing the event, so vkWriteResourceDescriptorsEXT is safe here. - // WriteSwapchainDescriptors flushes its own range, so no extra flush needed. - resizeSub_.SetEvent(&window.onResize, [this]() { - WriteSwapchainDescriptors(); - }); + // No whole-heap flush here: WriteFontAtlasDescriptor / RegisterSampler each + // flush their own written byte range above, so the GPU already sees + // everything written during setup. (void)initCmd; // reserved for future image-layout tweaks } @@ -215,7 +208,7 @@ void UIRenderer::DispatchFused(GraphicsCommandBuffer cmd, | clipBit(images, 0x4u) | clipBit(text, 0x8u); pc.surfaceWidth = window_->width; pc.surfaceHeight = window_->height; - pc.frameIdx = window_->currentBuffer; + pc.frameIdx = window_->currentFrame; pc._pad = 0; std::copy(quads.clipRectPx.begin(), quads.clipRectPx.end(), pc.clipQuads); std::copy(circles.clipRectPx.begin(), circles.clipRectPx.end(), pc.clipCircles); @@ -254,7 +247,7 @@ void UIRenderer::Dispatch(GraphicsCommandBuffer cmd, const GraphicsComputeShader .newLayout = VK_IMAGE_LAYOUT_GENERAL, .srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, - .image = window_->images[window_->currentBuffer], + .image = window_->images[window_->currentImage], .subresourceRange = { .aspectMask = VK_IMAGE_ASPECT_COLOR_BIT, .baseMipLevel = 0, @@ -274,50 +267,6 @@ void UIRenderer::Dispatch(GraphicsCommandBuffer cmd, const GraphicsComputeShader // ─── descriptor writes ───────────────────────────────────────────────── -void UIRenderer::WriteSwapchainDescriptors() { - if (!Device::descriptorHeapSupported) { - // Sets mode: each frame's descriptor set holds ITS swapchain image at - // outImageSlot_, through a view the heap object creates and owns. - for (std::uint8_t f = 0; f < Window::numFrames; ++f) { - heap_->WriteImageSet(f, outImageSlot_, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - window_->imageViews[f], VK_IMAGE_LAYOUT_GENERAL); - } - return; - } - - // Each per-frame heap holds ITS swapchain image at outImageSlot_. - std::array infos{}; - std::array resources{}; - std::array destinations{}; - - for (std::uint32_t f = 0; f < Window::numFrames; ++f) { - infos[f] = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window_->imageViews[f], - .layout = VK_IMAGE_LAYOUT_GENERAL, - }; - resources[f] = { - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &infos[f] }, - }; - destinations[f] = { - .address = heap_->resourceHeap[f].value - + heap_->ImageByteOffset(outImageSlot_), - .size = Device::descriptorHeapProperties.imageDescriptorSize, - }; - } - Device::vkWriteResourceDescriptorsEXT( - Device::device, Window::numFrames, resources.data(), destinations.data() - ); - // Only the one image descriptor at outImageSlot_ changed in each per-frame - // heap (same slot offset in every frame), so flush just that byte range - // rather than the whole multi-KB heap. - const std::uint32_t off = heap_->ImageByteOffset(outImageSlot_); - const std::uint32_t sz = Device::descriptorHeapProperties.imageDescriptorSize; - for (auto& h : heap_->resourceHeap) h.FlushDevice(off, sz); -} - void UIRenderer::WriteFontAtlasDescriptor() { atlasViewCreateInfo_ = { .sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO, diff --git a/implementations/Crafter.Graphics-Window.cpp b/implementations/Crafter.Graphics-Window.cpp index 810ec29..0b1aaea 100644 --- a/implementations/Crafter.Graphics-Window.cpp +++ b/implementations/Crafter.Graphics-Window.cpp @@ -538,20 +538,18 @@ Window::Window(std::uint32_t width, std::uint32_t height) : width(width), height Device::CheckVkResult(vkAllocateCommandBuffers(Device::device, &cmdBufAllocateInfo, drawCmdBuffers)); // Per-frame-in-flight synchronisation objects (issue #40). The fences are - // created signaled so the first wait on each image's fence — before that - // image has ever been submitted — returns immediately instead of + // created signaled so the first wait on each frame slot's fence — before + // that slot has ever been submitted — returns immediately instead of // deadlocking. The submitInfo is rebuilt per frame in Render() now that - // the wait/signal semaphores vary by frame. + // the wait/signal semaphores vary by frame. The per-image render-finished + // semaphores come with the images, in CreateSwapchain. VkSemaphoreCreateInfo semaphoreCreateInfo {}; semaphoreCreateInfo.sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO; VkFenceCreateInfo fenceCreateInfo {}; fenceCreateInfo.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO; fenceCreateInfo.flags = VK_FENCE_CREATE_SIGNALED_BIT; for (std::uint8_t i = 0; i < numFrames; i++) { - Device::CheckVkResult(vkCreateSemaphore(Device::device, &semaphoreCreateInfo, nullptr, &renderFinishedSemaphores[i])); Device::CheckVkResult(vkCreateFence(Device::device, &fenceCreateInfo, nullptr, &waitFences[i])); - } - for (std::uint8_t i = 0; i < numAcquireSemaphores; i++) { Device::CheckVkResult(vkCreateSemaphore(Device::device, &semaphoreCreateInfo, nullptr, &imageAcquiredSemaphores[i])); } @@ -562,7 +560,7 @@ Window::Window(std::uint32_t width, std::uint32_t height) : width(width), height // Per-frame info structs: everything that never varies between frames is // set here once. Render() patches the barriers' image/oldLayout and the - // sync handles that follow currentBuffer/frameCounter (submitInfo's + // sync handles that follow currentFrame/currentImage (submitInfo's // wait/signal semaphores + command buffer, presentInfo's wait semaphore). cmdBufInfo = {}; cmdBufInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; @@ -617,15 +615,15 @@ Window::Window(std::uint32_t width, std::uint32_t height) : width(width), height }; // pImageIndices/pSwapchains point at members, so they follow value changes - // (currentBuffer each frame, swapChain across recreation) on their own. The + // (currentImage each frame, swapChain across recreation) on their own. The // wait semaphore is the per-image render-finished semaphore, patched per - // frame in Render() (it follows currentBuffer). + // frame in Render() (it follows currentImage). presentInfo = {}; presentInfo.sType = VK_STRUCTURE_TYPE_PRESENT_INFO_KHR; presentInfo.pNext = NULL; presentInfo.swapchainCount = 1; presentInfo.pSwapchains = &swapChain; - presentInfo.pImageIndices = ¤tBuffer; + presentInfo.pImageIndices = ¤tImage; presentInfo.waitSemaphoreCount = 1; } @@ -636,12 +634,17 @@ Window::~Window() { if (Device::device != VK_NULL_HANDLE) { if (drawCmdBuffers[0] != VK_NULL_HANDLE) vkFreeCommandBuffers(Device::device, Device::commandPool, numFrames, drawCmdBuffers); for (std::uint8_t i = 0; i < numFrames; i++) { - if (renderFinishedSemaphores[i]) vkDestroySemaphore(Device::device, renderFinishedSemaphores[i], nullptr); if (waitFences[i]) vkDestroyFence(Device::device, waitFences[i], nullptr); - } - for (std::uint8_t i = 0; i < numAcquireSemaphores; i++) { if (imageAcquiredSemaphores[i]) vkDestroySemaphore(Device::device, imageAcquiredSemaphores[i], nullptr); } + for (VkSemaphore semaphore : renderFinishedSemaphores) { + vkDestroySemaphore(Device::device, semaphore, nullptr); + } + renderFinishedSemaphores.clear(); + for (VkImageView view : imageViews) { + vkDestroyImageView(Device::device, view, nullptr); + } + imageViews.clear(); if (swapChain != VK_NULL_HANDLE) vkDestroySwapchainKHR(Device::device, swapChain, nullptr); swapChain = VK_NULL_HANDLE; } @@ -726,6 +729,15 @@ void Window::Resize(std::uint32_t newWidth, std::uint32_t newHeight) { // Skip degenerate resizes. Win32 minimised windows give (0, 0); Wayland // sometimes echoes the current size in a configure. if (newWidth == 0 || newHeight == 0) return; + // A listener of the frame being recorded asked: the rest of that frame + // still draws into an image of the current swapchain at the current + // size, so the size changes once the frame has been presented. The last + // size asked for wins, the current one too. + if (recording) { + deferredWidth = newWidth; + deferredHeight = newHeight; + return; + } if (newWidth == width && newHeight == height) return; // Win32 fires WM_SIZE synchronously inside CreateWindowEx (before the // constructor's CreateSwapchain). Defer the first resize to that @@ -739,9 +751,13 @@ void Window::Resize(std::uint32_t newWidth, std::uint32_t newHeight) { width = newWidth; height = newHeight; + RebuildSwapchain(); +} - // Caller (configure handler / WM_SIZE) runs between frames, but be - // defensive: ensure no in-flight commands reference the old swapchain. +void Window::RebuildSwapchain() { + // Configure handlers and WM_SIZE run between frames, and a frame that + // asked waited for its present, but frames may still be in flight: none + // may still use the old swapchain. Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); // The queue is idle, so every deferred resource is now safe to free // regardless of its retire frame (issue #101). @@ -771,15 +787,17 @@ void Window::RecreateSwapchainAndImages() { void Window::SetPresentMode(PresentMode mode) { presentMode = mode; - presentModeUnfit = false; // A window without a swapchain yet picks the mode up when it makes one. if (swapChain == VK_NULL_HANDLE) return; // The mode is fixed when a swapchain is made, so make a new one, as - // Resize does: nothing in flight may still use the old images. - Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); - Device::DrainDeletions(); - RecreateSwapchainAndImages(); - onResize.Invoke(); + // Resize does: after the present when a frame is being recorded (a + // VSync key ticked from onUpdate), since the rest of that frame draws + // into the old swapchain's image. + if (recording) { + swapchainRebuildPending = true; + return; + } + RebuildSwapchain(); } void Window::SetTitle(const std::string_view title) { @@ -1255,37 +1273,61 @@ namespace { } void Window::Render() { + // This frame's slot among the numFrames in flight. Its command buffer, + // descriptor-heap copy and acquire semaphore were last used numFrames + // frames ago: wait for that submission's fence before touching any of + // them. This — not a full-queue wait-idle — is what bounds + // frames-in-flight and gives the CPU/GPU overlap. The wait comes before + // the acquire and is keyed by frame, not by the acquired image as when + // the two were one index, so the swapchain may have any number of images + // (#160) and a frame still waits for exactly the one numFrames before it, + // which the deletion queue and the staging rings count on. + currentFrame = static_cast(frameCounter % numFrames); + Device::CheckVkResult(vkWaitForFences(Device::device, 1, &waitFences[currentFrame], VK_TRUE, UINT64_MAX)); + // Acquire the next image from the swap chain. If the surface has // changed size out from under us (compositor/Win32 resize delivered // between Render calls), recreate and retry once. - // Pick this frame's acquire semaphore from a free-running CPU counter: the - // image index (currentBuffer) isn't known until acquire returns, so the - // acquire's signal semaphore can't be keyed by the image. - const std::uint32_t acquireSlot = static_cast(frameCounter % numAcquireSemaphores); - VkSemaphore imageAcquired = imageAcquiredSemaphores[acquireSlot]; + VkSemaphore imageAcquired = imageAcquiredSemaphores[currentFrame]; { VkResult acquire = vkAcquireNextImageKHR(Device::device, swapChain, UINT64_MAX, - imageAcquired, (VkFence)nullptr, ¤tBuffer); + imageAcquired, (VkFence)nullptr, ¤tImage); if (acquire == VK_ERROR_OUT_OF_DATE_KHR) { Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); Device::DrainDeletions(); RecreateSwapchainAndImages(); onResize.Invoke(); acquire = vkAcquireNextImageKHR(Device::device, swapChain, UINT64_MAX, - imageAcquired, (VkFence)nullptr, ¤tBuffer); + imageAcquired, (VkFence)nullptr, ¤tImage); } if (acquire != VK_SUBOPTIMAL_KHR) { Device::CheckVkResult(acquire); } } - // drawCmdBuffers[currentBuffer] and this image's descriptor-heap slot are - // about to be re-recorded. Wait on the fence guarding this image's - // previous submission (keyed by image index, since acquire returns an - // arbitrary index), then reset it for this submit. This — not a full-queue - // wait-idle — is what bounds frames-in-flight and gives the CPU/GPU overlap. - Device::CheckVkResult(vkWaitForFences(Device::device, 1, &waitFences[currentBuffer], VK_TRUE, UINT64_MAX)); - Device::CheckVkResult(vkResetFences(Device::device, 1, &waitFences[currentBuffer])); + // From here to the present the frame owns images[currentImage] and its + // heap copy: a Resize or SetPresentMode from a listener waits for the + // present (see the end of this function), and a swapchain slot listed + // now is written at once. The guard clears it on the way out of a throw + // too, so a failed frame doesn't hold every later rebuild back. + recording = true; + struct RecordingGuard { + bool& recording; + ~RecordingGuard() { recording = false; } + } recordingGuard { recording }; + + // The image this frame draws into, at this window's swapchain slots, in + // this frame's copy of the heap (or its descriptor set), which the fence + // wait above freed. Before onUpdate and the passes, which record against + // it. + if (descriptorHeap) { + descriptorHeap->WriteSwapchainImage(*this); + } + + // Reset for this frame's submit after the acquire, which can fail (and + // recreate the swapchain on the way), and the slot writes: a fence reset + // and then never submitted would hang the next wait on this slot. + Device::CheckVkResult(vkResetFences(Device::device, 1, &waitFences[currentFrame])); // Now that a frame's fence has been waited, free any deferred resources // whose retire frame has been reached (issue #101). frameCounter is bumped @@ -1293,19 +1335,19 @@ void Window::Render() { // value entries enqueued this frame are tagged with. Device::ReclaimDeletions(); - Device::CheckVkResult(vkBeginCommandBuffer(drawCmdBuffers[currentBuffer], &cmdBufInfo)); + Device::CheckVkResult(vkBeginCommandBuffer(drawCmdBuffers[currentFrame], &cmdBufInfo)); // On an image's first use after (re)creating the swapchain it is still in // VK_IMAGE_LAYOUT_UNDEFINED; every subsequent frame leaves it in // PRESENT_SRC_KHR. Transitioning from the wrong oldLayout is a validation // error, so pick based on whether this image has been rendered before. // The image was just acquired above, so touching it here is legal. - const bool firstUse = !imageInitialised[currentBuffer]; - imageInitialised[currentBuffer] = true; + const bool firstUse = !imageInitialised[currentImage]; + imageInitialised[currentImage] = true; acquireBarrier.oldLayout = firstUse ? VK_IMAGE_LAYOUT_UNDEFINED : VK_IMAGE_LAYOUT_PRESENT_SRC_KHR; - acquireBarrier.image = images[currentBuffer]; + acquireBarrier.image = images[currentImage]; // Real per-pass stage union for this frame's passes — the swapchain image // is only ever written by compute/RT (and possibly transfer), never by the @@ -1327,7 +1369,7 @@ void Window::Render() { // The acquire->GENERAL transition only needs to be visible before the first // pass writes the image, i.e. at the writer stages — not ALL_COMMANDS. - vkCmdPipelineBarrier(drawCmdBuffers[currentBuffer], VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, swapWriterStages, 0, 0, nullptr, 0, nullptr, 1, &acquireBarrier); + vkCmdPipelineBarrier(drawCmdBuffers[currentFrame], VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, swapWriterStages, 0, 0, nullptr, 0, nullptr, 1, &acquireBarrier); // Synthesise key-repeat events before listeners run, so the focused // widget's OnTextInput / OnKeyDown sees them in the same frame. @@ -1340,7 +1382,7 @@ void Window::Render() { // Pass-side dispatches still run with the same heaps bound — moving // the bind earlier doesn't change anything for them. if (descriptorHeap) { - descriptorHeap->Bind(drawCmdBuffers[currentBuffer], currentBuffer); + descriptorHeap->Bind(drawCmdBuffers[currentFrame], currentFrame); } onUpdate.Invoke({startTime, startTime-lastFrameBegin}); @@ -1360,7 +1402,7 @@ void Window::Render() { (void)clearColor; for (std::size_t i = 0; i < passes.size(); ++i) { - passes[i]->Record(drawCmdBuffers[currentBuffer], currentBuffer, *this); + passes[i]->Record(drawCmdBuffers[currentFrame], currentFrame, *this); if (i + 1 < passes.size()) { // Make pass i's writes to the swapchain image visible to pass i+1. @@ -1376,13 +1418,13 @@ void Window::Render() { // default stage reports a union that names RAY_TRACING_SHADER. const VkPipelineStageFlags srcStages = passes[i]->SwapchainStage() & Device::stageMask; const VkPipelineStageFlags dstStages = passes[i + 1]->SwapchainStage() & Device::stageMask; - VkImageMemoryBarrier ib = BuildSwapchainInterPassBarrier(images[currentBuffer], srcStages, dstStages); - vkCmdPipelineBarrier(drawCmdBuffers[currentBuffer], srcStages, dstStages, + VkImageMemoryBarrier ib = BuildSwapchainInterPassBarrier(images[currentImage], srcStages, dstStages); + vkCmdPipelineBarrier(drawCmdBuffers[currentFrame], srcStages, dstStages, 0, 0, nullptr, 0, nullptr, 1, &ib); } } - presentBarrier.image = images[currentBuffer]; + presentBarrier.image = images[currentImage]; // Mirror of the acquire barrier: srcAccessMask must be supported by the src // stage mask (swapWriterStages). SHADER_WRITE alone is valid for COMPUTE/RT @@ -1395,7 +1437,7 @@ void Window::Render() { std::optional capture; VkPipelineStageFlags presentWaitStages = swapWriterStages; if (pendingCapture) { - capture.emplace(RecordFrameCapture(drawCmdBuffers[currentBuffer], images[currentBuffer], width, height, swapWriterStages, presentBarrier.srcAccessMask)); + capture.emplace(RecordFrameCapture(drawCmdBuffers[currentFrame], images[currentImage], width, height, swapWriterStages, presentBarrier.srcAccessMask)); presentWaitStages |= VK_PIPELINE_STAGE_TRANSFER_BIT; capture->path = std::move(*pendingCapture); pendingCapture.reset(); @@ -1404,36 +1446,31 @@ void Window::Render() { // The ->PRESENT_SRC transition only needs to wait on the stages that // actually wrote the image (the per-pass union), not ALL_COMMANDS; present // itself is gated by the render-finished semaphore, so dst stays BOTTOM. - vkCmdPipelineBarrier(drawCmdBuffers[currentBuffer], presentWaitStages, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0, 0, nullptr, 0, nullptr, 1, &presentBarrier); + vkCmdPipelineBarrier(drawCmdBuffers[currentFrame], presentWaitStages, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0, 0, nullptr, 0, nullptr, 1, &presentBarrier); - Device::CheckVkResult(vkEndCommandBuffer(drawCmdBuffers[currentBuffer])); + Device::CheckVkResult(vkEndCommandBuffer(drawCmdBuffers[currentFrame])); // Patch the per-frame fields of the cached submitInfo/presentInfo (the // invariant fields were set once in the constructor): submit waits on this // frame's acquire semaphore and signals this image's render-finished // semaphore, present waits on that same render-finished semaphore. The - // fence (keyed by image index) lets a future Render() reuse this image's - // command buffer + heap slot safely. The semaphore handles live in member - // arrays, so taking their addresses here is stable for the call. - submitInfo.pWaitSemaphores = &imageAcquiredSemaphores[acquireSlot]; - submitInfo.pSignalSemaphores = &renderFinishedSemaphores[currentBuffer]; - submitInfo.pCommandBuffers = &drawCmdBuffers[currentBuffer]; - Device::CheckVkResult(vkQueueSubmit(Device::queue, 1, &submitInfo, waitFences[currentBuffer])); + // fence (keyed by frame slot) lets the Render() numFrames later reuse this + // slot's command buffer, heap copy and acquire semaphore safely. The + // semaphore handles live in member arrays, so taking their addresses here + // is stable for the call. + submitInfo.pWaitSemaphores = &imageAcquiredSemaphores[currentFrame]; + submitInfo.pSignalSemaphores = &renderFinishedSemaphores[currentImage]; + submitInfo.pCommandBuffers = &drawCmdBuffers[currentFrame]; + Device::CheckVkResult(vkQueueSubmit(Device::queue, 1, &submitInfo, waitFences[currentFrame])); - presentInfo.pWaitSemaphores = &renderFinishedSemaphores[currentBuffer]; + presentInfo.pWaitSemaphores = &renderFinishedSemaphores[currentImage]; VkResult result = vkQueuePresentKHR(Device::queue, &presentInfo); + // The frame is handed over: a swapchain rebuild may go ahead now. + recording = false; if (result == VK_SUBOPTIMAL_KHR || result == VK_ERROR_OUT_OF_DATE_KHR) { - // Surface size changed mid-present. Drain the queue (the only place we - // still wait-idle, alongside Resize() and the acquire OUT_OF_DATE - // path), rebuild the swapchain, and let dependents (descriptors - // holding old image handles) re-bind via onResize before the next - // frame. The wait-idle here also re-settles every per-frame fence to a - // signaled state, so the next Render()'s fence wait passes through. - Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); - Device::DrainDeletions(); - RecreateSwapchainAndImages(); - onResize.Invoke(); + // Surface size changed mid-present: rebuild below. + swapchainRebuildPending = true; } else { Device::CheckVkResult(result); } @@ -1442,8 +1479,8 @@ void Window::Render() { // while the GPU is still executing frame N. frameCounter++; // Advance the global deferred-deletion clock too (issue #101). Kept - // separate from the per-window frameCounter above (which also keys the - // acquire-semaphore slot) so non-window code can enqueue against it. + // separate from the per-window frameCounter above (which also picks the + // frame slot) so non-window code can enqueue against it. Device::frameCounter++; if (capture) { @@ -1452,6 +1489,25 @@ void Window::Render() { Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); WriteFrameCapture(*capture, colorFormat); } + + // The rebuild the frame held back: the present's, or a Resize or + // SetPresentMode from one of its listeners. RebuildSwapchain drains the + // queue (the only other wait-idles are Resize's and SetPresentMode's own + // and the acquire's OUT_OF_DATE path) and fires onResize before the next + // frame; the wait also leaves every per-frame fence signaled, so the next + // Render()'s fence wait passes through. + bool rebuild = swapchainRebuildPending; + if (deferredWidth != 0) { + rebuild = rebuild || deferredWidth != width || deferredHeight != height; + width = deferredWidth; + height = deferredHeight; + deferredWidth = 0; + deferredHeight = 0; + } + if (rebuild) { + swapchainRebuildPending = false; + RebuildSwapchain(); + } } #ifdef CRAFTER_TIMING @@ -1493,14 +1549,86 @@ void Window::LogTiming() { } #endif +VkSurfaceCapabilitiesKHR Window::SwapchainCapabilities(VkPhysicalDevice gpu, VkSurfaceKHR surface, + VkPresentModeKHR mode, bool modePassedAlong) { + VkSurfaceCapabilitiesKHR allModes {}; + if (!Device::surfaceMaintenance1Supported || !modePassedAlong) { + Device::CheckVkResult(vkGetPhysicalDeviceSurfaceCapabilitiesKHR(gpu, surface, &allModes)); + if (!Device::surfaceMaintenance1Supported) { + return allModes; + } + } + VkSurfacePresentModeKHR modeInfo { + .sType = VK_STRUCTURE_TYPE_SURFACE_PRESENT_MODE_KHR, + .presentMode = mode, + }; + const VkPhysicalDeviceSurfaceInfo2KHR surfaceInfo { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SURFACE_INFO_2_KHR, + .pNext = &modeInfo, + .surface = surface, + }; + VkSurfaceCapabilities2KHR forMode { .sType = VK_STRUCTURE_TYPE_SURFACE_CAPABILITIES_2_KHR }; + Device::CheckVkResult(Device::vkGetPhysicalDeviceSurfaceCapabilities2KHR(gpu, &surfaceInfo, &forMode)); + VkSurfaceCapabilitiesKHR capabilities = forMode.surfaceCapabilities; + if (!modePassedAlong) { + // VUID-VkSwapchainCreateInfoKHR-presentMode-02839 and the + // forward-progress rule of a UINT64_MAX acquire count against the + // surface's minimum for all modes unless the swapchain names its + // mode, so that minimum binds too. + capabilities.minImageCount = std::max(capabilities.minImageCount, allModes.minImageCount); + if (allModes.maxImageCount != 0) { + capabilities.maxImageCount = capabilities.maxImageCount == 0 + ? allModes.maxImageCount + : std::min(capabilities.maxImageCount, allModes.maxImageCount); + } + } + return capabilities; +} + +std::uint32_t Window::SwapchainMinImageCount(const VkSurfaceCapabilitiesKHR& capabilities) { + std::uint32_t count = std::max(numFrames, capabilities.minImageCount); + if (capabilities.maxImageCount != 0) { + count = std::min(count, capabilities.maxImageCount); + } + return count; +} + void Window::CreateSwapchain() { // Store the current swap chain handle so we can use it later on to ease up recreation VkSwapchainKHR oldSwapchain = swapChain; - // Get physical device surface properties and formats - VkSurfaceCapabilitiesKHR surfCaps; - Device::CheckVkResult(vkGetPhysicalDeviceSurfaceCapabilitiesKHR(Device::physDevice, vulkanSurface, &surfCaps)); + // VK_PRESENT_MODE_FIFO_KHR ("v-sync") is guaranteed available per spec, so + // it is the fallback for a mode the surface lacks. For benchmarking we + // want frame pacing uncapped from the display refresh, so when + // CRAFTER_PRESENT_IMMEDIATE is set in the environment we prefer IMMEDIATE + // (then MAILBOX) whatever presentMode asks for. Chosen first, because how + // many images the swapchain needs depends on it. + { + std::uint32_t presentModeCount = 0; + vkGetPhysicalDeviceSurfacePresentModesKHR(Device::physDevice, vulkanSurface, &presentModeCount, nullptr); + std::vector presentModes(presentModeCount); + vkGetPhysicalDeviceSurfacePresentModesKHR(Device::physDevice, vulkanSurface, &presentModeCount, presentModes.data()); + auto offered = [&](VkPresentModeKHR mode) { return std::ranges::find(presentModes, mode) != presentModes.end(); }; + swapchainPresentMode = VK_PRESENT_MODE_FIFO_KHR; + if (std::getenv("CRAFTER_PRESENT_IMMEDIATE")) { + if (offered(VK_PRESENT_MODE_IMMEDIATE_KHR)) swapchainPresentMode = VK_PRESENT_MODE_IMMEDIATE_KHR; + else if (offered(VK_PRESENT_MODE_MAILBOX_KHR)) swapchainPresentMode = VK_PRESENT_MODE_MAILBOX_KHR; + } else if (presentMode == PresentMode::Mailbox && offered(VK_PRESENT_MODE_MAILBOX_KHR)) { + swapchainPresentMode = VK_PRESENT_MODE_MAILBOX_KHR; + } else if (presentMode == PresentMode::Immediate && offered(VK_PRESENT_MODE_IMMEDIATE_KHR)) { + swapchainPresentMode = VK_PRESENT_MODE_IMMEDIATE_KHR; + } + } + + // Get physical device surface properties, for that present mode. With + // swapchain_maintenance1 the swapchain names its mode, so that mode's + // image counts are the ones that apply, which on Mesa's Wayland are below + // the surface's for all modes (FIFO three where MAILBOX needs four, #160). + // This is the last capabilities query before vkCreateSwapchainKHR, as the + // validation layer checks minImageCount against the surface's last answer. + const VkSurfaceCapabilitiesKHR surfCaps = SwapchainCapabilities(Device::physDevice, vulkanSurface, + swapchainPresentMode, Device::swapchainMaintenance1Supported); VkExtent2D swapchainExtent = {}; // If width (and height) equals the special value 0xFFFFFFFF, the size of the surface will be set by the swapchain @@ -1534,7 +1662,7 @@ void Window::CreateSwapchain() VkSwapchainCreateInfoKHR swapchainCI = {}; swapchainCI.sType = VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR; swapchainCI.surface = vulkanSurface; - swapchainCI.minImageCount = numFrames; + swapchainCI.minImageCount = SwapchainMinImageCount(surfCaps); swapchainCI.imageFormat = colorFormat; swapchainCI.imageColorSpace = colorSpace; swapchainCI.imageExtent = { swapchainExtent.width, swapchainExtent.height }; @@ -1545,75 +1673,66 @@ void Window::CreateSwapchain() swapchainCI.imageArrayLayers = 1; swapchainCI.imageSharingMode = VK_SHARING_MODE_EXCLUSIVE; swapchainCI.queueFamilyIndexCount = 0; - // VK_PRESENT_MODE_FIFO_KHR ("v-sync") is guaranteed available per spec, so - // it is the fallback for a mode the surface lacks. For benchmarking we - // want frame pacing uncapped from the display refresh, so when - // CRAFTER_PRESENT_IMMEDIATE is set in the environment we prefer IMMEDIATE - // (then MAILBOX) whatever presentMode asks for. - { - std::uint32_t presentModeCount = 0; - vkGetPhysicalDeviceSurfacePresentModesKHR(Device::physDevice, vulkanSurface, &presentModeCount, nullptr); - std::vector presentModes(presentModeCount); - vkGetPhysicalDeviceSurfacePresentModesKHR(Device::physDevice, vulkanSurface, &presentModeCount, presentModes.data()); - auto offered = [&](VkPresentModeKHR mode) { return std::ranges::find(presentModes, mode) != presentModes.end(); }; - swapchainPresentMode = VK_PRESENT_MODE_FIFO_KHR; - if (presentModeUnfit) { - // Stays FIFO; see below. - } else if (std::getenv("CRAFTER_PRESENT_IMMEDIATE")) { - if (offered(VK_PRESENT_MODE_IMMEDIATE_KHR)) swapchainPresentMode = VK_PRESENT_MODE_IMMEDIATE_KHR; - else if (offered(VK_PRESENT_MODE_MAILBOX_KHR)) swapchainPresentMode = VK_PRESENT_MODE_MAILBOX_KHR; - } else if (presentMode == PresentMode::Mailbox && offered(VK_PRESENT_MODE_MAILBOX_KHR)) { - swapchainPresentMode = VK_PRESENT_MODE_MAILBOX_KHR; - } else if (presentMode == PresentMode::Immediate && offered(VK_PRESENT_MODE_IMMEDIATE_KHR)) { - swapchainPresentMode = VK_PRESENT_MODE_IMMEDIATE_KHR; - } - } swapchainCI.presentMode = swapchainPresentMode; + // The mode, named again for swapchain_maintenance1: what makes the mode's + // own image counts the ones that apply, at creation and to the acquires. + const VkSwapchainPresentModesCreateInfoKHR presentModes { + .sType = VK_STRUCTURE_TYPE_SWAPCHAIN_PRESENT_MODES_CREATE_INFO_KHR, + .presentModeCount = 1, + .pPresentModes = &swapchainPresentMode, + }; + if (Device::swapchainMaintenance1Supported) { + swapchainCI.pNext = &presentModes; + } // Setting oldSwapChain to the saved handle of the previous swapchain aids in resource reuse and makes sure that we can still present already acquired images swapchainCI.oldSwapchain = oldSwapchain; // Setting clipped to VK_TRUE allows the implementation to discard rendering outside of the surface area swapchainCI.clipped = VK_TRUE; swapchainCI.compositeAlpha = compositeAlpha; + // The old images' views go before the images do. Nothing uses them: the + // caller waited for the queue, and a heap copy or descriptor set that + // still names one is written again before its frame binds it. + for (VkImageView view : imageViews) { + vkDestroyImageView(Device::device, view, nullptr); + } + imageViews.clear(); + Device::CheckVkResult(vkCreateSwapchainKHR(Device::device, &swapchainCI, nullptr, &swapChain)); // If an existing swap chain is re-created, destroy the old swap chain and the ressources owned by the application (image views, images are owned by the swap chain) if (oldSwapchain != VK_NULL_HANDLE) { vkDestroySwapchainKHR(Device::device, oldSwapchain, nullptr); } + // The swap chain images: as many as the driver made, which may be more + // than minImageCount asked for. Everything kept per image is sized from + // this count; what is kept per frame in flight stays numFrames, so any + // count works (#160). uint32_t imageCount{ 0 }; Device::CheckVkResult(vkGetSwapchainImagesKHR(Device::device, swapChain, &imageCount, nullptr)); - // images[] and every other per-frame array hold exactly numFrames - // entries. A driver may create more images than minImageCount asked for - // (a surface whose minimum is above numFrames must, and Mesa gives - // MAILBOX at least four). MAILBOX and IMMEDIATE are wishes: FIFO, with - // numFrames, is the frame loop's own. If that has other than numFrames - // too, writing them into images[] would run off its end, so refuse. - if (imageCount != numFrames && swapchainPresentMode != VK_PRESENT_MODE_FIFO_KHR) { - std::println(std::cerr, "Window: the {} swapchain has {} images, not the frame loop's {}; using FIFO", - swapchainPresentMode == VK_PRESENT_MODE_MAILBOX_KHR ? "MAILBOX" : "IMMEDIATE", imageCount, numFrames); - vkDestroySwapchainKHR(Device::device, swapChain, nullptr); - presentModeUnfit = true; - swapchainPresentMode = VK_PRESENT_MODE_FIFO_KHR; - swapchainCI.presentMode = swapchainPresentMode; - swapchainCI.oldSwapchain = VK_NULL_HANDLE; - Device::CheckVkResult(vkCreateSwapchainKHR(Device::device, &swapchainCI, nullptr, &swapChain)); - Device::CheckVkResult(vkGetSwapchainImagesKHR(Device::device, swapChain, &imageCount, nullptr)); - } - if (imageCount != numFrames) { - throw std::runtime_error(std::format("Window: the swapchain has {} images, but the frame loop is built for exactly {}", imageCount, numFrames)); - } - - // Get the swap chain images - Device::CheckVkResult(vkGetSwapchainImagesKHR(Device::device, swapChain, &imageCount, images)); + images.resize(imageCount); + Device::CheckVkResult(vkGetSwapchainImagesKHR(Device::device, swapChain, &imageCount, images.data())); // Brand-new swapchain images are in VK_IMAGE_LAYOUT_UNDEFINED; none have // been rendered/presented yet. Render() consults this to pick the correct // oldLayout for each image's first post-acquire transition. - imageInitialised.fill(false); + imageInitialised.assign(imageCount, false); - for (std::uint8_t i = 0; i < numFrames; i++) { - imageViews[i] = { + // A render-finished semaphore per image. The ones the old swapchain's + // images used are kept: its last presents may still be waiting on them. + const VkSemaphoreCreateInfo semaphoreCreateInfo { .sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO }; + while (renderFinishedSemaphores.size() < imageCount) { + VkSemaphore semaphore = VK_NULL_HANDLE; + Device::CheckVkResult(vkCreateSemaphore(Device::device, &semaphoreCreateInfo, nullptr, &semaphore)); + renderFinishedSemaphores.push_back(semaphore); + } + + static std::atomic swapchainsMade = 0; + swapchainGeneration = ++swapchainsMade; + + imageViewInfos.resize(imageCount); + for (std::uint32_t i = 0; i < imageCount; i++) { + imageViewInfos[i] = { .sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO, .flags = 0, .image = images[i], @@ -1634,13 +1753,19 @@ void Window::CreateSwapchain() }, }; } + // The views made, for descriptor sets, once per swapchain rather than + // once per frame whose image changed. + imageViews.resize(imageCount); + for (std::uint32_t i = 0; i < imageCount; i++) { + Device::CheckVkResult(vkCreateImageView(Device::device, &imageViewInfos[i], nullptr, &imageViews[i])); + } } VkCommandBuffer Window::StartInit() { VkCommandBufferBeginInfo cmdBufInfo {}; cmdBufInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; - // StartInit reuses drawCmdBuffers[currentBuffer] — the same buffer the - // steady-state Render() loop submits, guarded by waitFences[currentBuffer]. + // StartInit reuses drawCmdBuffers[currentFrame] — the same buffer the + // steady-state Render() loop submits, guarded by waitFences[currentFrame]. // When setup runs mid-session (e.g. building map meshes / acceleration // structures on a scene transition) the loop's last submission of this // buffer may still be in flight: re-beginning it would trip @@ -1653,7 +1778,7 @@ VkCommandBuffer Window::StartInit() { // (not the per-frame path), so a full wait-idle here is fine — FinishInit // already wait-idles at the end. Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); - Device::CheckVkResult(vkBeginCommandBuffer(drawCmdBuffers[currentBuffer], &cmdBufInfo)); + Device::CheckVkResult(vkBeginCommandBuffer(drawCmdBuffers[currentFrame], &cmdBufInfo)); // The swapchain images are deliberately NOT transitioned here. They are // presentable images and may only be touched after vkAcquireNextImageKHR @@ -1662,15 +1787,15 @@ VkCommandBuffer Window::StartInit() { // VK_IMAGE_LAYOUT_UNDEFINED on its first acquired use (see // imageInitialised). This command buffer is for the caller's one-time // setup work (uploads, acceleration-structure builds, etc.). - return drawCmdBuffers[currentBuffer]; + return drawCmdBuffers[currentFrame]; } void Window::FinishInit() { VkSubmitInfo submitInfo{}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.commandBufferCount = 1; - submitInfo.pCommandBuffers = &drawCmdBuffers[currentBuffer]; - Device::CheckVkResult(vkEndCommandBuffer(drawCmdBuffers[currentBuffer])); + submitInfo.pCommandBuffers = &drawCmdBuffers[currentFrame]; + Device::CheckVkResult(vkEndCommandBuffer(drawCmdBuffers[currentFrame])); Device::CheckVkResult(vkQueueSubmit(Device::queue, 1, &submitInfo, VK_NULL_HANDLE)); Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); } @@ -1679,11 +1804,11 @@ VkCommandBuffer Window::GetCmd() { VkCommandBufferBeginInfo cmdBufInfo {}; cmdBufInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; // Same lifecycle hazard as StartInit: GetCmd re-begins and EndCmd - // re-submits the shared drawCmdBuffers[currentBuffer], so drain any + // re-submits the shared drawCmdBuffers[currentFrame], so drain any // in-flight submission of it (and free the resources it references) before // re-recording (issue #153). Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); - Device::CheckVkResult(vkBeginCommandBuffer(drawCmdBuffers[currentBuffer], &cmdBufInfo)); + Device::CheckVkResult(vkBeginCommandBuffer(drawCmdBuffers[currentFrame], &cmdBufInfo)); VkImageSubresourceRange range{}; range.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; @@ -1691,16 +1816,16 @@ VkCommandBuffer Window::GetCmd() { range.levelCount = VK_REMAINING_MIP_LEVELS; range.baseArrayLayer = 0; range.layerCount = VK_REMAINING_ARRAY_LAYERS; - - return drawCmdBuffers[currentBuffer]; + + return drawCmdBuffers[currentFrame]; } void Window::EndCmd(VkCommandBuffer cmd) { VkSubmitInfo submitInfo{}; submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submitInfo.commandBufferCount = 1; - submitInfo.pCommandBuffers = &drawCmdBuffers[currentBuffer]; - Device::CheckVkResult(vkEndCommandBuffer(drawCmdBuffers[currentBuffer])); + submitInfo.pCommandBuffers = &drawCmdBuffers[currentFrame]; + Device::CheckVkResult(vkEndCommandBuffer(drawCmdBuffers[currentFrame])); Device::CheckVkResult(vkQueueSubmit(Device::queue, 1, &submitInfo, VK_NULL_HANDLE)); Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); } @@ -1974,7 +2099,7 @@ void Window::Render() { if (!open) return; Crafter::WebGPU::wgpuFrameBegin(); for (RenderPass* p : passes) { - if (p) p->Record(/*cmd*/ 0u, currentBuffer, *this); + if (p) p->Record(/*cmd*/ 0u, currentFrame, *this); } Crafter::WebGPU::wgpuFrameEnd(); } diff --git a/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm b/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm index 6f72e7b..cf05276 100644 --- a/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm +++ b/interfaces/Crafter.Graphics-DescriptorHeapVulkan.cppm @@ -61,7 +61,10 @@ export namespace Crafter { // All bindings are PARTIALLY_BOUND | UPDATE_AFTER_BIND | // UPDATE_UNUSED_WHILE_PENDING: most slots are never written, and a // slot is written into every frame's set at once while earlier frames - // may still be executing with their set bound. + // may still be executing with their set bound. A swapchain slot too + // is written into a set another window's frame in flight is bound + // to, and when listed during a frame, into the set that frame has + // already bound. static constexpr std::uint32_t storageImageBinding = 0; static constexpr std::uint32_t sampledImageBinding = 1; static constexpr std::uint32_t samplerBinding = 2; @@ -77,12 +80,116 @@ export namespace Crafter { VkDescriptorSet sets[Window::numFrames] {}; // Descriptor sets reference VkImageView / VkSampler objects, where the // heap takes create-infos, so in sets mode this struct creates and - // owns them. setImageViews[frame][slot] is the view frame's set uses - // at that image slot; a view shared by all frames is stored under - // frame 0 only. setSamplers[slot] likewise. - std::vector setImageViews[Window::numFrames]; + // owns them: setImageViews[slot] is the view every frame's set uses + // at that image slot, setSamplers[slot] likewise. The swapchain's + // views are its window's (Window::imageViews). + std::vector setImageViews; std::vector setSamplers; + // ─── Swapchain slots ──────────────────────────────────────────── + // Image slots that name a window's swapchain image. A window's + // swapchain images are not its frames: it keeps Window::numFrames + // frames in flight over however many images the driver made, and + // which one a frame draws into is known only once it has been + // acquired (#160). So each frame, after the acquire and before + // onUpdate, Window::Render writes the image it acquired into the slots + // listed here for it, as a storage image in GENERAL, in that frame's + // heap copy (or descriptor set), through WriteSwapchainImage. A shader + // reads the image it draws into from one fixed slot, and nothing has + // to be written again after a resize. Each slot is one window's, so + // two windows can share a heap: neither writes the other's slots, + // which the other's frames in flight may be reading. UIRenderer lists + // its output slot; a pass of your own lists its own with + // AddSwapchainSlot. Freeing a slot takes it off the list. + struct SwapchainSlot { + std::uint16_t slot = 0; + // Compared, never dereferenced: a window gone takes nothing down. + const Window* window = nullptr; + // What each frame's copy holds at the slot, so a frame whose image + // hasn't changed writes nothing: the image's index in the + // swapchain of that Window::swapchainGeneration, which starts at + // 1, so generation 0 means nothing yet. + struct Written { + std::uint64_t generation = 0; + std::uint32_t image = 0; + }; + std::array written {}; + }; + std::vector swapchainSlots; + + // Lists an image slot, raw as AllocateImageSlots returned it, as one + // that names `window`'s swapchain image. Every frame's copy gets it + // from that frame on; a slot listed during a frame `window` records + // with this heap gets that frame's image at once, so a pass set up + // mid-frame draws into it the same frame. Listing a slot again moves + // it to `window`. Throws std::invalid_argument for a slot past the + // heap's image slots, which an empty ImageSlot (0xFFFF) is. + void AddSwapchainSlot(std::uint16_t slot, const Window& window) { + if (slot >= imageCapacity) { + throw std::invalid_argument(std::format( + "DescriptorHeapVulkan::AddSwapchainSlot: image slot {} is not one of the heap's {} image slots", slot, imageCapacity)); + } + std::erase_if(swapchainSlots, [slot](const SwapchainSlot& listed) { return listed.slot == slot; }); + swapchainSlots.push_back({ .slot = slot, .window = &window }); + if (window.recording && window.descriptorHeap == this) { + WriteSwapchainImage(window); + } + } + + // Writes window.imageViewInfos[window.currentImage] (in sets mode, + // window.imageViews[window.currentImage]) into window's swapchain + // slots in the copy of window.currentFrame. Window::Render calls it + // once the frame's fence has been waited, so none of window's frames + // still reads that copy, and other windows' frames don't read these + // slots. + void WriteSwapchainImage(const Window& window) { + for (SwapchainSlot& listed : swapchainSlots) { + if (listed.window != &window) { + continue; + } + SwapchainSlot::Written& written = listed.written[window.currentFrame]; + if (written.generation == window.swapchainGeneration && written.image == window.currentImage) { + continue; + } + if (Device::descriptorHeapSupported) { + const VkImageDescriptorInfoEXT imageInfo { + .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, + .pView = &window.imageViewInfos[window.currentImage], + .layout = VK_IMAGE_LAYOUT_GENERAL, + }; + const VkResourceDescriptorInfoEXT resource { + .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, + .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, + .data = { .pImage = &imageInfo }, + }; + VulkanBuffer& heap = resourceHeap[window.currentFrame]; + const std::uint32_t size = Device::descriptorHeapProperties.imageDescriptorSize; + const VkHostAddressRangeEXT destination { + .address = heap.value + ImageByteOffset(listed.slot), + .size = size, + }; + Device::vkWriteResourceDescriptorsEXT(Device::device, 1, &resource, &destination); + heap.FlushDevice(ImageByteOffset(listed.slot), size); + } else { + const VkDescriptorImageInfo imageInfo { + .imageView = window.imageViews[window.currentImage], + .imageLayout = VK_IMAGE_LAYOUT_GENERAL, + }; + const VkWriteDescriptorSet write { + .sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, + .dstSet = sets[window.currentFrame], + .dstBinding = storageImageBinding, + .dstArrayElement = listed.slot, + .descriptorCount = 1, + .descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, + .pImageInfo = &imageInfo, + }; + vkUpdateDescriptorSets(Device::device, 1, &write, 0, nullptr); + } + written = { .generation = window.swapchainGeneration, .image = window.currentImage }; + } + } + DescriptorHeapVulkan() = default; DescriptorHeapVulkan(const DescriptorHeapVulkan&) = delete; DescriptorHeapVulkan& operator=(const DescriptorHeapVulkan&) = delete; @@ -105,12 +212,10 @@ export namespace Crafter { // Safe on a partly built heap: destroying VK_NULL_HANDLE is a no-op. void DestroySets() noexcept { vkDestroyDescriptorPool(Device::device, setPool, nullptr); - for (std::vector& views : setImageViews) { - for (VkImageView view : views) { - vkDestroyImageView(Device::device, view, nullptr); - } - views.clear(); + for (VkImageView view : setImageViews) { + vkDestroyImageView(Device::device, view, nullptr); } + setImageViews.clear(); for (VkSampler sampler : setSamplers) { vkDestroySampler(Device::device, sampler, nullptr); } @@ -224,6 +329,8 @@ export namespace Crafter { bufferFreelist.reserve(buffers); samplerFreelist.clear(); samplerFreelist.reserve(samplers); + // Every slot is unallocated again, and the new heaps hold nothing. + swapchainSlots.clear(); // The heaps are written by the CPU every frame (HOST_VISIBLE is // mandatory) and read by the GPU, so DEVICE_LOCAL is a perf @@ -272,6 +379,8 @@ export namespace Crafter { bufferFreelist.reserve(buffers); samplerFreelist.clear(); samplerFreelist.reserve(samplers); + // Every slot is unallocated again, as in heap mode. + swapchainSlots.clear(); // At least one descriptor per binding even when a pool was sized // zero (Initialize(1, 1, 0) is common): a zero-count pool size is @@ -347,9 +456,7 @@ export namespace Crafter { }; Device::CheckVkResult(vkAllocateDescriptorSets(Device::device, &allocInfo, sets)); - for (std::vector& views : setImageViews) { - views.assign(imageCount, VK_NULL_HANDLE); - } + setImageViews.assign(imageCount, VK_NULL_HANDLE); setSamplers.assign(samplerCount, VK_NULL_HANDLE); rollback.heap = nullptr; } @@ -364,30 +471,8 @@ export namespace Crafter { // destroys it immediately. That relies on the same contract as slot // reuse in heap mode: no frame in flight may still reference the slot // (true after the queue drain that precedes Window::onResize, and for - // a slot that was freed at a scene-switch barrier). - - // Writes `type` (STORAGE_IMAGE or SAMPLED_IMAGE) at an image slot of - // one frame's set, for images that differ per frame: the swapchain. - // A slot written this way must be written for every frame. - void WriteImageSet(std::uint8_t frame, std::uint16_t slot, VkDescriptorType type, - const VkImageViewCreateInfo& viewInfo, VkImageLayout layout) { - VkImageView view = VK_NULL_HANDLE; - Device::CheckVkResult(vkCreateImageView(Device::device, &viewInfo, nullptr, &view)); - vkDestroyImageView(Device::device, setImageViews[frame][slot], nullptr); - setImageViews[frame][slot] = view; - - const VkDescriptorImageInfo imageInfo { .imageView = view, .imageLayout = layout }; - const VkWriteDescriptorSet write { - .sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, - .dstSet = sets[frame], - .dstBinding = type == VK_DESCRIPTOR_TYPE_STORAGE_IMAGE ? storageImageBinding : sampledImageBinding, - .dstArrayElement = slot, - .descriptorCount = 1, - .descriptorType = type, - .pImageInfo = &imageInfo, - }; - vkUpdateDescriptorSets(Device::device, 1, &write, 0, nullptr); - } + // a slot that was freed at a scene-switch barrier). The swapchain + // image is written per frame by WriteSwapchainImage instead. // Writes the same image at an image slot of every frame's set, through // one shared view. @@ -395,11 +480,8 @@ export namespace Crafter { const VkImageViewCreateInfo& viewInfo, VkImageLayout layout) { VkImageView view = VK_NULL_HANDLE; Device::CheckVkResult(vkCreateImageView(Device::device, &viewInfo, nullptr, &view)); - for (std::vector& views : setImageViews) { - vkDestroyImageView(Device::device, views[slot], nullptr); - views[slot] = VK_NULL_HANDLE; - } - setImageViews[0][slot] = view; + vkDestroyImageView(Device::device, setImageViews[slot], nullptr); + setImageViews[slot] = view; const VkDescriptorImageInfo imageInfo { .imageView = view, .imageLayout = layout }; VkWriteDescriptorSet writes[Window::numFrames]; @@ -543,7 +625,12 @@ export namespace Crafter { // freeing. Multi-slot ranges are decomposed into their individual slots, // each put in its sorted place (TakeRun looks for runs). // noexcept: capacity is reserved at Initialize so inserting never reallocates. + // A freed swapchain slot stops being one, so the window doesn't write + // the swapchain image over the slot's next owner. void FreeImageSlots(ImageSlotRange r) noexcept { + std::erase_if(swapchainSlots, [r](const SwapchainSlot& listed) { + return listed.slot >= r.firstElement && listed.slot - r.firstElement < r.count; + }); Release(imageFreelist, r.firstElement, r.count); } diff --git a/interfaces/Crafter.Graphics-Device.cppm b/interfaces/Crafter.Graphics-Device.cppm index bf89c20..99aa859 100644 --- a/interfaces/Crafter.Graphics-Device.cppm +++ b/interfaces/Crafter.Graphics-Device.cppm @@ -295,6 +295,20 @@ export namespace Crafter { inline static bool rayTracingSupported = false; // VK_EXT_device_fault: the VK_ERROR_DEVICE_LOST report in CheckVkResult. inline static bool deviceFaultSupported = false; + // VK_KHR_surface_maintenance1 (or its EXT form) with + // VK_KHR_get_surface_capabilities2, instance extensions enabled where + // the loader has them: a surface's capabilities for one present mode. + // Its image counts can differ from the surface's answer for all modes, + // and Window::CreateSwapchain asks for the mode it uses (#160). + inline static bool surfaceMaintenance1Supported = false; + inline static PFN_vkGetPhysicalDeviceSurfaceCapabilities2KHR vkGetPhysicalDeviceSurfaceCapabilities2KHR = nullptr; + // VK_KHR_swapchain_maintenance1 (or its EXT form) with its + // swapchainMaintenance1 feature, a device group enabled where the + // driver has it. With it a swapchain is made naming its present mode + // (VkSwapchainPresentModesCreateInfoKHR), which is what lets it have + // that mode's image count where the surface's count for all modes is + // higher (Turnip's FIFO: 3 where MAILBOX needs 4). + inline static bool swapchainMaintenance1Supported = false; // Pipeline stages that are valid on this device: everything, minus // RAY_TRACING_SHADER when ray tracing is unsupported. AND it into any // stage mask built from a constant that names the RT stage. @@ -340,6 +354,12 @@ export namespace Crafter { // over, and throws listing every GPU's gaps when none qualifies. Pure // logic apart from that print, driven by tests/DeviceFeatures. static std::size_t ChooseDevice(std::span choices); + // `instance`'s physical devices grouped by GPU, in the loader's order: + // a GPU two drivers both list (a Flatpak sees the runtime's and the GL + // extension's manifests; RADV beside AMDVLK) has one device UUID and + // two handles. CRAFTER_GRAPHICS_DEVICE numbers these groups, so one + // number means the same GPU inside a sandbox and out. + static std::vector> DistinctGpus(VkInstance instance); // Throws when `spirv` declares, directly or implicitly (Int64Atomics // declares Int64), a SPIR-V capability that nothing enabled: a core // feature that is off, a subgroup operation the GPU lacks, or one of diff --git a/interfaces/Crafter.Graphics-RenderPass.cppm b/interfaces/Crafter.Graphics-RenderPass.cppm index bbc621a..58461a2 100644 --- a/interfaces/Crafter.Graphics-RenderPass.cppm +++ b/interfaces/Crafter.Graphics-RenderPass.cppm @@ -26,6 +26,12 @@ export namespace Crafter { #endif // !CRAFTER_GRAPHICS_WINDOW_DOM struct RenderPass { + // `frameIdx` is the frame's slot among the frames in flight + // (Window::currentFrame), for what lives per frame: the heap copy + // bound, RenderingElement3D::tlases. It is not the swapchain image, + // which is window.images[window.currentImage] natively: the two stopped + // being one index once the swapchain could have more images than + // frames in flight (#160). virtual void Record(GraphicsCommandBuffer cmd, std::uint32_t frameIdx, Window& window) = 0; virtual ~RenderPass() = default; #ifndef CRAFTER_GRAPHICS_WINDOW_DOM diff --git a/interfaces/Crafter.Graphics-RenderingElement3D.cppm b/interfaces/Crafter.Graphics-RenderingElement3D.cppm index d180a25..0e4fd5b 100644 --- a/interfaces/Crafter.Graphics-RenderingElement3D.cppm +++ b/interfaces/Crafter.Graphics-RenderingElement3D.cppm @@ -151,11 +151,11 @@ export namespace Crafter { // The build writes tlases[index] in place: its instance and metadata // buffers on the host as it records, its acceleration structure and // scratch on the GPU. So call it only once every earlier submission - // that used tlases[index] has completed. Window::Render runs onUpdate - // after waiting for the fence of the image it renders, so building - // tlases[currentBuffer] there is safe. Earlier builds, traces and - // compute ray queries in the same command buffer are ordered before - // the build by the build itself. + // that used tlases[index] has completed. The TLASes are per frame in + // flight, and Window::Render runs onUpdate after waiting for the fence + // of its frame slot, so building tlases[currentFrame] there is safe. + // Earlier builds, traces and compute ray queries in the same command + // buffer are ordered before the build by the build itself. // // A rebuild for a new instance count or preference builds into the // same acceleration structure while it fits its storage, so diff --git a/interfaces/Crafter.Graphics-UI.cppm b/interfaces/Crafter.Graphics-UI.cppm index 91d964a..ecba244 100644 --- a/interfaces/Crafter.Graphics-UI.cppm +++ b/interfaces/Crafter.Graphics-UI.cppm @@ -429,10 +429,7 @@ export namespace Crafter { bool firstDispatchThisFrame_ = true; #endif - Crafter::EventListener resizeSub_; - #ifndef CRAFTER_GRAPHICS_WINDOW_DOM - void WriteSwapchainDescriptors(); void WriteFontAtlasDescriptor(); void WriteBufferDescriptor(std::uint16_t slot, VkBuffer buffer, VkDeviceAddress address, std::uint32_t size); void WriteSampledImageDescriptor(std::uint16_t slot, diff --git a/interfaces/Crafter.Graphics-Window.cppm b/interfaces/Crafter.Graphics-Window.cppm index d307c91..a1d3097 100644 --- a/interfaces/Crafter.Graphics-Window.cppm +++ b/interfaces/Crafter.Graphics-Window.cppm @@ -61,10 +61,10 @@ export namespace Crafter { // has. Mailbox replaces a frame still waiting to be shown, so presenting // never waits, which suits a window that updates on its own clock // (Window::updateInterval). Immediate shows each frame at once and may - // tear. A mode the surface lacks falls back to Fifo, as does one whose - // swapchain the driver makes with more images than the frame loop's - // Window::numFrames (Mesa's Mailbox has at least four). In the browser - // the page presents the canvas and the mode has no effect. + // tear. A mode the surface lacks falls back to Fifo. The mode decides how + // many images the swapchain has (Mesa's Mailbox needs at least four), + // which the frame loop takes as they come. In the browser the page + // presents the canvas and the mode has no effect. enum class PresentMode { Fifo, Mailbox, @@ -79,10 +79,12 @@ export namespace Crafter { Event onClose; Event onBeforeUpdate; Event onUpdate; - // Fires when the swapchain has been recreated for a new size. - // width/height already reflect the new size. Passes that hold - // descriptors referring to window.imageViews[] must re-write them - // here (the old VkImage handles have been destroyed). + // Fires when the swapchain has been recreated, for a new size or a + // new present mode. width/height already reflect the new size, and + // the old images have been destroyed: anything that holds one of + // them, or is sized after the window, is made again here. The heap + // slots that name the swapchain image follow by themselves (see + // DescriptorHeapVulkan::AddSwapchainSlot). Event onResize; bool open = true; bool updating = false; @@ -208,7 +210,10 @@ export namespace Crafter { void StartUpdate(); void StopUpdate(); // Makes the swapchain again with the new mode and fires onResize, as - // a resize does, so call it outside StartInit/FinishInit. The + // a resize does, so call it outside StartInit/FinishInit. Called while + // a frame is being recorded (from onUpdate, a pass, an input action + // ticked there), the new swapchain is made once that frame has been + // presented: the frame still draws into an image of the old one. The // environment variable CRAFTER_PRESENT_IMMEDIATE overrides the mode, // for benchmarks. void SetPresentMode(PresentMode mode); @@ -221,6 +226,9 @@ export namespace Crafter { // pixels, or that none has. Remembered: it is applied again whenever // the keyboard focus comes back to this window. void SetTextInput(bool active, std::int32_t x = 0, std::int32_t y = 0, std::int32_t width = 0, std::int32_t height = 0, TextInputPurpose purpose = TextInputPurpose::Normal); + // Like SetPresentMode, a resize asked for while a frame is being + // recorded takes effect once that frame has been presented, width and + // height included, which the rest of the frame still draws at. void Resize(std::uint32_t width, std::uint32_t height); void Render(); void Update(); @@ -337,13 +345,35 @@ export namespace Crafter { void FinishInit(); VkCommandBuffer GetCmd(); void EndCmd(VkCommandBuffer cmd); + // Makes the swapchain, after the old one's images have stopped being + // used: every caller has waited for the queue (or has no frames yet). void CreateSwapchain(); - // Tear-and-rebuild helper used by Resize() and the OUT_OF_DATE - // recovery in Render(). Calls CreateSwapchain() and re-issues the - // initial PRESENT_SRC_KHR layout transition for the new images so - // Render()'s barriers (which assume oldLayout = PRESENT_SRC_KHR) - // stay valid. Does NOT fire onResize — callers do that. + // The capabilities a swapchain of `mode` on `surface` is sized by. With + // surface_maintenance1 they are the surface's for that mode, which may + // differ from its answer for all modes (Mesa on Wayland: FIFO 3, all + // modes 4, because MAILBOX needs 4). Those are the ones that apply when + // the swapchain is made naming its mode, which `modePassedAlong` says + // (swapchain_maintenance1's VkSwapchainPresentModesCreateInfoKHR). + // Without that the counts for all modes bind as well, so the minimum + // and maximum are the stricter of the two. Asks for the mode last, as + // the validation layer checks minImageCount against the surface's + // last answer. Static so a test sizes swapchains the way the window + // does. + static VkSurfaceCapabilitiesKHR SwapchainCapabilities(VkPhysicalDevice gpu, VkSurfaceKHR surface, + VkPresentModeKHR mode, bool modePassedAlong); + // The minImageCount CreateSwapchain asks for, from those capabilities: + // numFrames, or the surface's minimum where that is more, and never + // more than its maximum (0 means no maximum). The driver may still + // make more. Static so a test can feed it the numbers drivers report. + static std::uint32_t SwapchainMinImageCount(const VkSurfaceCapabilitiesKHR& capabilities); + // Calls CreateSwapchain(). The new images start in UNDEFINED, and + // Render() transitions each on its first acquire (imageInitialised). + // Does NOT fire onResize — callers do that. void RecreateSwapchainAndImages(); + // What Resize and SetPresentMode do once no frame is being recorded: + // waits for the queue, frees the deletion queue, makes the swapchain + // again at width x height and fires onResize. + void RebuildSwapchain(); // Saves the frame the next Render() draws to a PNG at `path`, for // visual regression tests and screenshots: that Render copies the @@ -353,9 +383,46 @@ export namespace Crafter { // acquired again. void CaptureNextFrame(std::filesystem::path path); std::optional pendingCapture; + + // ── Frames and images ─────────────────────────────────────────── + // The frame loop keeps numFrames frames in flight, and that number is + // its own: the swapchain has however many images the driver made for + // the present mode, which may be fewer or more (Mesa's Mailbox has at + // least four; issue #160). Two indices follow from that, set by + // Render() before onUpdate: + // + // currentFrame this frame's slot, frameCounter % numFrames. It + // picks everything that lives per frame in flight: + // the command buffer, the fence and acquire + // semaphore, the descriptor heap copy (or set) the + // frame binds, RenderingElement3D::tlases, the + // frameIdx a RenderPass is recorded with. + // currentImage the swapchain image the frame acquired and draws + // into: images[currentImage]. + // + // The two go together differently from frame to frame, so a heap copy + // can't hold "its" swapchain image: Render() writes the acquired one + // into the frame's copy, at the slots DescriptorHeapVulkan:: + // AddSwapchainSlot lists for this window, before onUpdate. static constexpr std::uint8_t numFrames = 3; + std::uint32_t currentFrame = 0; + std::uint32_t currentImage = 0; + // True from a frame's acquire to its present: images[currentImage] + // and the frame's heap copy are this frame's, and a swapchain rebuild + // (Resize, SetPresentMode) waits for the present. What is waiting: + // a new present mode, or a present that found the swapchain out of + // date; and the last size a resize asked for, 0 when none did. + bool recording = false; + bool swapchainRebuildPending = false; + std::uint32_t deferredWidth = 0; + std::uint32_t deferredHeight = 0; VkSurfaceKHR vulkanSurface = VK_NULL_HANDLE; VkSwapchainKHR swapChain = VK_NULL_HANDLE; + // Bumped from a process-wide count whenever CreateSwapchain makes a + // swapchain, so it names one swapchain for good: a new swapchain's + // handle and images may reuse the old one's handle values. The heap + // keys what it wrote into each copy on it. + std::uint64_t swapchainGeneration = 0; // What the swapchain images were created for: STORAGE | TRANSFER_SRC, // plus TRANSFER_DST where the surface allows it, so a pass can blit or // clear into the image instead of only writing it from a shader. @@ -363,10 +430,6 @@ export namespace Crafter { // The mode the swapchain was made with: presentMode, unless the // surface lacks it or CRAFTER_PRESENT_IMMEDIATE is set. VkPresentModeKHR swapchainPresentMode = VK_PRESENT_MODE_FIFO_KHR; - // The mode presentMode asks for made a swapchain with more images than - // numFrames once, so swapchains are FIFO until SetPresentMode is - // called again, rather than being made twice at every resize. - bool presentModeUnfit = false; VkFormat colorFormat; VkColorSpaceKHR colorSpace; // Supported composite-alpha mode for vulkanSurface. A surface property @@ -374,8 +437,14 @@ export namespace Crafter { // once at construction rather than re-derived on every CreateSwapchain // (i.e. on every resize configure). VkCompositeAlphaFlagBitsKHR compositeAlpha = VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR; - VkImage images[numFrames]; - VkImageViewCreateInfo imageViews[numFrames]; + // One entry per swapchain image, as many as vkGetSwapchainImagesKHR + // returned; remade with the swapchain. imageViewInfos are the views a + // heap descriptor takes (VkImageDescriptorInfoEXT::pView); imageViews + // the same views made, which a descriptor set takes, owned by the + // window and destroyed with its swapchain. + std::vector images; + std::vector imageViewInfos; + std::vector imageViews; // Tracks whether each swapchain image has been rendered (and thus // left in PRESENT_SRC_KHR) at least once. Freshly created swapchain // images start in VK_IMAGE_LAYOUT_UNDEFINED, so the first per-frame @@ -383,17 +452,18 @@ export namespace Crafter { // Reset in CreateSwapchain(). A presentable image may only be touched // after it has been acquired, so this initial transition happens // lazily in Render() (post-acquire) rather than up front. - std::array imageInitialised{}; + std::vector imageInitialised; std::thread thread; + // Per frame in flight. VkCommandBuffer drawCmdBuffers[numFrames] = {}; // Per-frame Vulkan info structs whose contents are almost entirely // invariant. Initialised once in the constructor; Render() patches only // the few fields that change each frame (the two barriers' image and // the acquire barrier's oldLayout, and — for the multi-frame-in-flight // sync below — submitInfo/presentInfo's wait/signal semaphores plus - // submitInfo's command buffer, which follow currentBuffer/frameCounter). + // submitInfo's command buffer, which follow currentFrame/currentImage). // presentInfo.pImageIndices and pSwapchains point at the - // currentBuffer/swapChain members, so they track value changes + // currentImage/swapChain members, so they track value changes // (including swapchain recreation) automatically. VkSubmitInfo submitInfo; VkCommandBufferBeginInfo cmdBufInfo; @@ -402,14 +472,9 @@ export namespace Crafter { VkPresentInfoKHR presentInfo; // ── Multi-frame-in-flight synchronisation (issue #40) ────────────── - // The renderer keeps up to numFrames frames in flight, so a single - // semaphore pair + a per-frame wait-idle is no longer enough. Note - // that drawCmdBuffers, the per-frame descriptor heap slots, and the - // swapchain images are ALL keyed by the acquired image index - // (currentBuffer) — WriteSwapchainDescriptors bakes heap slot i to - // write imageViews[i], so the index that selects a command buffer / - // heap slot must equal the acquired image index. Everything below - // follows from that. + // Render() waits for its frame slot's fence before it acquires, so + // the command buffer, the heap copy and the acquire semaphore of that + // slot are free again, whichever image the acquire then returns. // Signaled by the queue submit, waited by vkQueuePresentKHR. Keyed by // the acquired IMAGE index: the presentation engine keeps this @@ -417,29 +482,27 @@ export namespace Crafter { // semaphore is the only safe key. Keying it per-CPU-frame trips // VUID-vkQueueSubmit-pSignalSemaphores-00067. // https://docs.vulkan.org/guide/latest/swapchain_semaphore_reuse.html - std::array renderFinishedSemaphores{}; + // Grown by CreateSwapchain to the image count and never shrunk: a + // present of the old swapchain may still wait on one, and nothing + // reports when that wait is done, so they live as long as the window. + std::vector renderFinishedSemaphores; // Signaled by the queue submit, waited + reset before - // drawCmdBuffers[image] and that image's descriptor-heap slot are - // re-recorded. Keyed by image index. Created signaled so the first - // wait on each image passes through instead of deadlocking. + // drawCmdBuffers[frame] and that frame's descriptor-heap copy are + // re-recorded. Keyed by frame slot. Created signaled so the first + // wait on each slot passes through instead of deadlocking. std::array waitFences{}; // Signaled by vkAcquireNextImageKHR, waited by the queue submit. Keyed - // by a free-running CPU frame counter, because the image index isn't - // known until acquire returns. Sized numFrames+1: at most numFrames - // frames are ever in flight (each image's command buffer is gated by - // its own fence, and there are numFrames images), so numFrames+1 - // distinct acquire semaphores guarantees the one being reused has no - // pending operation — required by - // VUID-vkAcquireNextImageKHR-semaphore-01779. - static constexpr std::uint8_t numAcquireSemaphores = numFrames + 1; - std::array imageAcquiredSemaphores{}; + // by frame slot: the slot's fence is waited before the acquire, and + // that fence follows the submit that waited on the semaphore, so the + // semaphore has no pending operation when it is reused + // (VUID-vkAcquireNextImageKHR-semaphore-01779). + std::array imageAcquiredSemaphores{}; - // Free-running count of Render() calls; drives the acquire-semaphore - // slot (frameCounter % numAcquireSemaphores). + // Free-running count of Render() calls; currentFrame is this modulo + // numFrames. std::uint64_t frameCounter = 0; - std::uint32_t currentBuffer = 0; VkPipelineStageFlags submitPipelineStages = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT; std::vector passes; DescriptorHeapVulkan* descriptorHeap = nullptr; @@ -450,7 +513,7 @@ export namespace Crafter { // holds the per-Window state Crafter::Window users expect: // a list of render passes and a pointer to the descriptor heap. static constexpr std::uint8_t numFrames = 1; - std::uint32_t currentBuffer = 0; + std::uint32_t currentFrame = 0; std::vector passes; DescriptorHeapWebGPU* descriptorHeap = nullptr; std::optional> clearColor; diff --git a/project.cpp b/project.cpp index b289507..3f26162 100644 --- a/project.cpp +++ b/project.cpp @@ -351,8 +351,9 @@ extern "C" Configuration CrafterBuildProject(std::span a // Drives the wl_pointer callbacks directly, as MouseScroll does. cfg.tests.push_back(LibraryTest("PointerEnterLeave")); - // Issue #40: multi-frame-in-flight frame pacing (per-image fences - // + per-frame semaphores, no steady-state wait-idle). Drives the + // Issue #40: multi-frame-in-flight frame pacing (per-frame fences + // and acquire semaphores, per-image render-finished semaphores, no + // steady-state wait-idle). Drives the // real frame loop against a live Wayland compositor for many more // frames than there are in-flight slots and asserts the validation // layer stays silent — the old singleton-semaphore design only @@ -392,6 +393,19 @@ extern "C" Configuration CrafterBuildProject(std::span a parityTest.config.files.emplace_back(fs::path("examples/HelloUI/font.ttf")); cfg.tests.push_back(std::move(parityTest)); + // Issue #160: a swapchain has as many images as its present mode + // needs, and the frame loop takes any count. Every offered mode + // is kept (no fallback to FIFO), and every frame presents what it + // drew, read back a texel per frame. Where this process's GPU + // never makes more than numFrames images, the frame checks run + // again in child processes on a GPU that does (llvmpipe), on the + // heap and on descriptor sets. Loads the UI kernels like + // UIFallbackParity. Needs the Wayland backend + a real compositor. + Test imageCountTest = LibraryTest("SwapchainImageCount"); + imageCountTest.config.shaders = cfg.shaders; + imageCountTest.config.buildFiles = cfg.buildFiles; + cfg.tests.push_back(std::move(imageCountTest)); + // Reading the TLAS from the descriptor heap (#7/#15): renders a // triangle through traceRayEXT and an inline ray query and checks // the read-back pixels, through the engine's normal path (so on diff --git a/tests/FrameLoopSync/main.cpp b/tests/FrameLoopSync/main.cpp index ca9be69..eafe232 100644 --- a/tests/FrameLoopSync/main.cpp +++ b/tests/FrameLoopSync/main.cpp @@ -7,13 +7,16 @@ // semaphore pair with zero fences for the Window's whole lifetime. This // reworks the pacing model: // -// - one VkFence per swapchain slot, keyed by the ACQUIRED IMAGE INDEX, -// passed to vkQueueSubmit and waited on (then reset) before that image's -// command buffer is re-recorded; -// - one semaphore pair per in-flight frame, keyed by a free-running CPU -// frame counter % numFrames (the image index isn't known until acquire -// returns), so frame N's render-complete and frame N+1's acquire can be -// pending at once; +// - one VkFence per in-flight frame slot (a free-running CPU frame counter +// % numFrames), passed to vkQueueSubmit and waited on before the slot's +// command buffer is re-recorded — before the acquire, so the slot's +// acquire semaphore is free again too. Until #160 the fences were keyed +// by the acquired image index, which tied the frame loop to a swapchain +// of exactly numFrames images; +// - one acquire semaphore per frame slot, and one render-complete +// semaphore per swapchain image (the present holds it until the image +// comes back), so frame N's render-complete and frame N+1's acquire can +// be pending at once; // - the steady-state vkQueueWaitIdle is gone (kept only on resize / // OUT_OF_DATE / teardown). // @@ -25,9 +28,10 @@ // many more frames than there are in-flight slots, then asserts: // // - every Render() completed and the CPU frame counter advanced by the -// expected amount (no deadlock on the per-image fence wait); -// - the acquired image index rotated across more than one slot (proof the -// swapchain is genuinely multi-buffered, not pinned to image 0); +// expected amount (no deadlock on the per-frame fence wait); +// - the frame slot followed the frame counter, and the acquired image +// index rotated across more than one image (proof the swapchain is +// genuinely multi-buffered, not pinned to image 0); // - the Vulkan validation layer reported ZERO errors over the whole run. // This is the load-bearing check: the old singleton-pair design only // avoided being an active race because the wait-idle masked it. Reusing a @@ -64,22 +68,24 @@ int main() { // per-frame fences/semaphores under test. Window window(640, 480, "FrameLoopSync test"); - // Render() with no passes still exercises the full pacing path: acquire - // (signals frameSemaphores[frame].presentComplete) → per-image fence - // wait+reset → barriers → submit (signals renderComplete, fences the - // command buffer) → present. Run well past numFrames so every semaphore - // slot and every per-image fence is reused several times — exactly the - // condition the old single-pair / wait-idle design could not survive - // without draining the queue every frame. + // Render() with no passes still exercises the full pacing path: per-frame + // fence wait → acquire (signals the slot's acquire semaphore) → fence + // reset → barriers → submit (signals the image's render-complete + // semaphore, fences the command buffer) → present. Run well past + // numFrames so every semaphore and every per-frame fence is reused + // several times — exactly the condition the old single-pair / wait-idle + // design could not survive without draining the queue every frame. constexpr int kFrames = 60; static_assert(kFrames > Window::numFrames * 3, "must loop enough to reuse each in-flight slot multiple times"); - std::array imageSeen{}; + std::vector imageSeen(window.images.size()); + bool framesFollowCounter = true; for (int i = 0; i < kFrames; ++i) { window.Render(); - if (window.currentBuffer < Window::numFrames) { - imageSeen[window.currentBuffer] = true; + framesFollowCounter = framesFollowCounter && window.currentFrame == static_cast(i % Window::numFrames); + if (window.currentImage < imageSeen.size()) { + imageSeen[window.currentImage] = true; } } @@ -91,11 +97,13 @@ int main() { std::format("CPU frame counter advanced once per Render() ({} of {})", window.frameCounter, kFrames)); + Check(framesFollowCounter, "each Render() used frame slot frameCounter % numFrames"); + int distinctImages = 0; for (bool seen : imageSeen) if (seen) ++distinctImages; Check(distinctImages > 1, - std::format("swapchain rotated across multiple images ({} of {} slots used)", - distinctImages, Window::numFrames)); + std::format("swapchain rotated across multiple images ({} of {} images used)", + distinctImages, window.images.size())); Check(Device::validationErrorCount == 0, std::format("no Vulkan validation errors across {} frames ({} seen)", diff --git a/tests/HeapASRead/main.cpp b/tests/HeapASRead/main.cpp index 885a914..e159b02 100644 --- a/tests/HeapASRead/main.cpp +++ b/tests/HeapASRead/main.cpp @@ -34,7 +34,6 @@ #include "../../lib/stb_image.h" import Crafter.Graphics; -import Crafter.Event; import Crafter.Math; import std; @@ -203,24 +202,18 @@ int Render(Path path, const fs::path& out) { window.FinishInit(); - // Per frame: the TLAS at buffer slot 0 and that frame's swapchain image at - // image slot 0, each in its own copy of the heap. Written again whenever - // the swapchain is recreated, since the image descriptors point at its - // images. + // Per frame: the TLAS at buffer slot 0, in each frame's own copy of the + // heap. The swapchain image at image slot 0, which the window writes + // every frame with the image that frame draws into. auto imgSlots = descriptorHeap.AllocateImageSlots(1); auto bufSlots = descriptorHeap.AllocateBufferSlots(1); - auto writeDescriptors = [&] { + descriptorHeap.AddSwapchainSlot(imgSlots.firstElement, window); + { std::array tlasRanges{}; - std::array imageInfos{}; std::vector resources; std::vector destinations; for (std::uint32_t f = 0; f < Window::numFrames; ++f) { tlasRanges[f] = { .address = RenderingElement3D::tlases[f].address }; - imageInfos[f] = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[f], - .layout = VK_IMAGE_LAYOUT_GENERAL, - }; resources.push_back({ .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, @@ -230,22 +223,11 @@ int Render(Path path, const fs::path& out) { .address = descriptorHeap.resourceHeap[f].value + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize, }); - resources.push_back({ - .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &imageInfos[f] }, - }); - destinations.push_back({ - .address = descriptorHeap.resourceHeap[f].value + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize, - }); } Device::vkWriteResourceDescriptorsEXT(Device::device, static_cast(resources.size()), resources.data(), destinations.data()); for (auto& heap : descriptorHeap.resourceHeap) heap.FlushDevice(); - }; - writeDescriptors(); - EventListener resizeListener(&window.onResize, writeDescriptors); + } window.descriptorHeap = &descriptorHeap; RTPass rtPass(&pipeline); diff --git a/tests/HeapDynamicIndexHit/main.cpp b/tests/HeapDynamicIndexHit/main.cpp index a46d583..1552bab 100644 --- a/tests/HeapDynamicIndexHit/main.cpp +++ b/tests/HeapDynamicIndexHit/main.cpp @@ -32,7 +32,6 @@ #include "../../lib/stb_image.h" import Crafter.Graphics; -import Crafter.Event; import Crafter.Math; import std; @@ -264,11 +263,11 @@ int RenderScene(const fs::path& out) { colourRanges[i] = { .address = colourBuffers[i].address, .size = 4 * sizeof(float) }; } - // Written again whenever the swapchain is recreated, since the output - // image descriptors point at its images. - auto writeDescriptors = [&] { + // The output image at image slot 0 is the swapchain's, which the window + // writes every frame with the image that frame draws into. + descriptorHeap.AddSwapchainSlot(imgSlots.firstElement, window); + { std::array tlasRanges{}; - std::array outInfos{}; std::vector resources; std::vector destinations; auto add = [&](std::uint32_t f, VkResourceDescriptorInfoEXT resource, std::uint32_t offset, std::uint32_t size) { @@ -279,19 +278,10 @@ int RenderScene(const fs::path& out) { const std::uint32_t bufferSize = Device::descriptorHeapProperties.bufferDescriptorSize; for (std::uint32_t f = 0; f < Window::numFrames; ++f) { tlasRanges[f] = { .address = RenderingElement3D::tlases[f].address }; - outInfos[f] = { - .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, - .pView = &window.imageViews[f], - .layout = VK_IMAGE_LAYOUT_GENERAL, - }; add(f, { .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, .data = { .pAddressRange = &tlasRanges[f] } }, descriptorHeap.BufferByteOffset(bufSlots.firstElement), bufferSize); - add(f, { .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, - .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &outInfos[f] } }, - descriptorHeap.ImageByteOffset(imgSlots.firstElement), imageSize); for (std::uint16_t i = 0; i < 2; ++i) { add(f, { .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE, @@ -306,9 +296,7 @@ int RenderScene(const fs::path& out) { Device::vkWriteResourceDescriptorsEXT(Device::device, static_cast(resources.size()), resources.data(), destinations.data()); for (auto& heap : descriptorHeap.resourceHeap) heap.FlushDevice(); - }; - writeDescriptors(); - EventListener resizeListener(&window.onResize, writeDescriptors); + } VkSamplerCreateInfo samplerInfo = { .sType = VK_STRUCTURE_TYPE_SAMPLER_CREATE_INFO, diff --git a/tests/RTPassPushData/main.cpp b/tests/RTPassPushData/main.cpp index d939db7..3d8ce18 100644 --- a/tests/RTPassPushData/main.cpp +++ b/tests/RTPassPushData/main.cpp @@ -27,7 +27,6 @@ #include "../../lib/stb_image.h" import Crafter.Graphics; -import Crafter.Event; import Crafter.Math; import std; @@ -51,7 +50,7 @@ struct ClearPass : RenderPass { void Record(VkCommandBuffer cmd, std::uint32_t, Window& window) override { const VkClearColorValue magenta = {{1.0f, 0.0f, 1.0f, 1.0f}}; const VkImageSubresourceRange range = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}; - vkCmdClearColorImage(cmd, window.images[window.currentBuffer], VK_IMAGE_LAYOUT_GENERAL, &magenta, 1, &range); + vkCmdClearColorImage(cmd, window.images[window.currentImage], VK_IMAGE_LAYOUT_GENERAL, &magenta, 1, &range); } VkPipelineStageFlags SwapchainStage() const override { return VK_PIPELINE_STAGE_TRANSFER_BIT; } }; @@ -121,30 +120,25 @@ int main() { for (std::uint32_t f = 0; f < Window::numFrames; ++f) RenderingElement3D::BuildTLAS(cmd, f); window.FinishInit(); + // Each frame's TLAS at buffer slot 0 of that frame's heap copy; the + // swapchain image at image slot 0, which the window writes every frame. auto imgSlots = descriptorHeap.AllocateImageSlots(1); auto bufSlots = descriptorHeap.AllocateBufferSlots(1); - auto writeDescriptors = [&] { + descriptorHeap.AddSwapchainSlot(imgSlots.firstElement, window); + { std::array tlasRanges{}; - std::array imageInfos{}; std::vector resources; std::vector destinations; for (std::uint32_t f = 0; f < Window::numFrames; ++f) { tlasRanges[f] = { .address = RenderingElement3D::tlases[f].address }; - imageInfos[f] = { .sType = VK_STRUCTURE_TYPE_IMAGE_DESCRIPTOR_INFO_EXT, .pView = &window.imageViews[f], .layout = VK_IMAGE_LAYOUT_GENERAL }; resources.push_back({ .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_ACCELERATION_STRUCTURE_KHR, .data = { .pAddressRange = &tlasRanges[f] } }); destinations.push_back({ .address = descriptorHeap.resourceHeap[f].value + descriptorHeap.BufferByteOffset(bufSlots.firstElement), .size = Device::descriptorHeapProperties.bufferDescriptorSize }); - resources.push_back({ .sType = VK_STRUCTURE_TYPE_RESOURCE_DESCRIPTOR_INFO_EXT, .type = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, - .data = { .pImage = &imageInfos[f] } }); - destinations.push_back({ .address = descriptorHeap.resourceHeap[f].value + descriptorHeap.ImageByteOffset(imgSlots.firstElement), - .size = Device::descriptorHeapProperties.imageDescriptorSize }); } Device::vkWriteResourceDescriptorsEXT(Device::device, static_cast(resources.size()), resources.data(), destinations.data()); for (auto& heap : descriptorHeap.resourceHeap) heap.FlushDevice(); - }; - writeDescriptors(); - EventListener resizeListener(&window.onResize, writeDescriptors); + } window.descriptorHeap = &descriptorHeap; ClearPass clear; diff --git a/tests/SetupCmdBufferReuse/main.cpp b/tests/SetupCmdBufferReuse/main.cpp index ae25736..9a0dab1 100644 --- a/tests/SetupCmdBufferReuse/main.cpp +++ b/tests/SetupCmdBufferReuse/main.cpp @@ -17,7 +17,7 @@ // narrows SwapchainStage() to COMPUTE_SHADER, exactly like UIRenderer. // // Problem 2 — mid-session StartInit/FinishInit reuse the shared -// drawCmdBuffers[currentBuffer]. With no steady-state wait-idle the loop's +// drawCmdBuffers[currentFrame]. With no steady-state wait-idle the loop's // last submission of that buffer is still in flight when setup runs, so the // old code re-began it (VUID-vkBeginCommandBuffer-commandBuffer-00049) and // re-submitted it (VUID-vkQueueSubmit-pCommandBuffers-00071) while pending, @@ -82,7 +82,7 @@ int main() { // ── Problem 2: mid-session setup while a frame is in flight ────────────── // After Render() returns there is no steady-state wait-idle, so the last - // submission of drawCmdBuffers[currentBuffer] is still pending. Calling + // submission of drawCmdBuffers[currentFrame] is still pending. Calling // StartInit()/FinishInit() now (as a scene transition would, to build map // meshes / acceleration structures) reuses that exact command buffer. The // setup buffer is left empty on purpose: it's the begin/submit lifecycle diff --git a/tests/SwapchainImageCount/main.cpp b/tests/SwapchainImageCount/main.cpp new file mode 100644 index 0000000..9667282 --- /dev/null +++ b/tests/SwapchainImageCount/main.cpp @@ -0,0 +1,577 @@ +//SPDX-License-Identifier: LGPL-3.0-only +//SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts® + +// Issue #160: the swapchain gets as many images as its present mode needs, +// and the frame loop takes however many that is. CreateSwapchain used to ask +// for exactly Window::numFrames (3) images whatever the surface said, read +// them into a three-element array, and key everything a frame in flight +// owns, its command buffer, its fence and the descriptor-heap copy holding +// "its" swapchain image, by the acquired image index. A swapchain of any +// other size fell back to FIFO (asked for MAILBOX or IMMEDIATE) or threw, +// and a surface whose minimum is above three got a create info below it. +// Mesa's MAILBOX needs at least four images: Turnip on phoc, and llvmpipe on +// any Wayland compositor. +// +// Checked here: +// +// - Window::SwapchainMinImageCount on the numbers drivers report. +// - For every present mode the surface offers, SetPresentMode makes a +// swapchain of that mode, with at least the images the window asks for +// (Window::SwapchainCapabilities, the mode's own count where the device +// has swapchain_maintenance1). +// - Every frame draws into the image it presents. A UI pass fills the +// window with a colour that differs from frame to frame, and a transfer +// pass after it copies the middle texel of the frame's swapchain image +// into the frame slot's buffer, read back when the slot comes round +// again. With an image count other than numFrames, frames and images +// pair up differently from frame to frame: a heap copy that kept one +// image would have the UI draw into another frame's image, and the texel +// would hold an older colour. +// - A SetPresentMode or Resize from a listener of the frame being recorded +// takes effect after that frame's present: the rest of the frame indexes +// the swapchain it acquired from. +// - A swapchain slot listed during a frame is written for that frame (heap +// mode, where the descriptor can be read back and compared with the UI's). +// - Two windows sharing one heap each draw their own colours: a window +// writes only its own swapchain slots. +// - No Vulkan validation errors, which only means something with +// CRAFTER_GRAPHICS_VALIDATION set and the layer loaded. +// +// The GPU this process picks may give every mode numFrames images (NVIDIA +// does), which leaves the case that matters untried, and a process runs one +// binding model. So the frame checks also run in child processes, picked +// with CRAFTER_GRAPHICS_DEVICE, on a GPU whose surface needs more images for +// some mode (this process's own, else llvmpipe, which most Linux machines +// have), in each binding model the parent didn't run there: on the +// descriptor heap, and with CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP on descriptor +// sets, the path Turnip takes. Without such a GPU that part is a note, not a +// failure. `--frames heap|sets` runs the frame checks alone on whichever GPU +// the environment picks. +// +// Needs a live Wayland compositor and a Vulkan device, like FrameLoopSync. + +#include "vulkan/vulkan.h" +#include "vulkan/vk_enum_string_helper.h" +#include +#include + +import Crafter.Graphics; +import Crafter.Event; +import std; + +using namespace Crafter; +namespace fs = std::filesystem; + +namespace { + +int failures = 0; + +void Check(bool ok, std::string_view what) { + std::println("{} {}", ok ? "PASS" : "FAIL", what); + if (!ok) ++failures; +} + +// The test binary's own directory: the build places the UI kernels (both +// variants) next to it, and the test runner's working directory is not that +// directory. +fs::path ExeDir() { + return fs::read_symlink("/proc/self/exe").parent_path(); +} + +int RunCommand(const std::string& cmd) { + int status = std::system(cmd.c_str()); + if (status == -1) return -1; + return (status & 0x7f) == 0 ? ((status >> 8) & 0xff) : 128 + (status & 0x7f); +} + +struct ModeCase { + PresentMode mode; + VkPresentModeKHR vk; + std::string_view name; +}; +constexpr ModeCase kModes[] = { + { PresentMode::Fifo, VK_PRESENT_MODE_FIFO_KHR, "FIFO" }, + { PresentMode::Mailbox, VK_PRESENT_MODE_MAILBOX_KHR, "MAILBOX" }, + { PresentMode::Immediate, VK_PRESENT_MODE_IMMEDIATE_KHR, "IMMEDIATE" }, +}; + +std::vector OfferedModes(VkPhysicalDevice gpu, VkSurfaceKHR surface) { + std::uint32_t count = 0; + Device::CheckVkResult(vkGetPhysicalDeviceSurfacePresentModesKHR(gpu, surface, &count, nullptr)); + std::vector modes(count); + Device::CheckVkResult(vkGetPhysicalDeviceSurfacePresentModesKHR(gpu, surface, &count, modes.data())); + return modes; +} + +bool Offers(const std::vector& offered, VkPresentModeKHR mode) { + return std::ranges::find(offered, mode) != offered.end(); +} + +// The most images any present mode under test asks for, for a swapchain of +// `surface` on `gpu`, sized as a window on it would size it. +std::uint32_t MostImagesAsked(VkPhysicalDevice gpu, VkSurfaceKHR surface, bool modePassedAlong) { + const std::vector offered = OfferedModes(gpu, surface); + std::uint32_t most = 0; + for (const ModeCase& m : kModes) { + if (Offers(offered, m.vk)) { + most = std::max(most, Window::SwapchainMinImageCount( + Window::SwapchainCapabilities(gpu, surface, m.vk, modePassedAlong))); + } + } + return most; +} + +void CheckMinImageCount() { + const auto asked = [](std::uint32_t minImageCount, std::uint32_t maxImageCount) { + VkSurfaceCapabilitiesKHR capabilities{}; + capabilities.minImageCount = minImageCount; + capabilities.maxImageCount = maxImageCount; + return Window::SwapchainMinImageCount(capabilities); + }; + Check(asked(2, 8) == Window::numFrames, "a minimum below numFrames asks for numFrames (NVIDIA: 2, at most 8)"); + Check(asked(3, 0) == 3, "a minimum of numFrames asks for numFrames (Mesa's FIFO: 3, no maximum)"); + Check(asked(4, 0) == 4, "a minimum above numFrames asks for the minimum (Mesa's MAILBOX: 4, no maximum)"); + Check(asked(5, 6) == 5, "a minimum above numFrames within the maximum asks for the minimum"); + Check(asked(1, 2) == 2, "a maximum below numFrames caps the request"); + Check(asked(2, 0) == Window::numFrames, "no maximum leaves numFrames as it is"); +} + +// A colour for frame `n` that differs from every frame's within 200 of it, +// exact in eight bits. +std::array Colour(std::uint64_t n) { + return { + static_cast(40 + (n * 7) % 200), + static_cast(250 - (n * 13) % 200), + static_cast((n * 29) % 256), + }; +} + +// Copies the middle texel of the frame's swapchain image into the frame +// slot's buffer, after the UI pass drew it: what the frame presents. Added +// only for an 8-bit RGBA or BGRA swapchain, whose texel fits the buffer. +struct ReadbackPass : RenderPass { + std::array, Window::numFrames> texels; + + void Record(VkCommandBuffer cmd, std::uint32_t frameIdx, Window& window) override { + const VkBufferImageCopy region { + .imageSubresource = { VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1 }, + .imageOffset = { static_cast(window.width / 2), static_cast(window.height / 2), 0 }, + .imageExtent = { 1, 1, 1 }, + }; + vkCmdCopyImageToBuffer(cmd, window.images[window.currentImage], VK_IMAGE_LAYOUT_GENERAL, + texels[frameIdx].buffer, 1, ®ion); + const VkBufferMemoryBarrier copied { + .sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER, + .srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT, + .dstAccessMask = VK_ACCESS_HOST_READ_BIT, + .srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, + .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, + .buffer = texels[frameIdx].buffer, + .offset = 0, + .size = VK_WHOLE_SIZE, + }; + vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, + 0, 0, nullptr, 1, &copied, 0, nullptr); + } + + VkPipelineStageFlags SwapchainStage() const override { + return VK_PIPELINE_STAGE_TRANSFER_BIT; + } +}; + +// One window filled with a colour per frame through a UIRenderer on `heap`, +// and what each frame presented read back. Holds listeners that capture it: +// build it in place and don't move it. +struct ColourScene { + Window& window; + GraphicsDescriptorHeap& heap; + std::string_view name; + // Added to the frame number: two scenes sharing a heap draw different + // colours, so one that drew into the other's image is seen. + std::uint64_t colourOffset; + + UIRenderer ui; + // One quad buffer per frame slot: the colour changes every frame, and a + // buffer the frames shared would be overwritten under the frames still + // in flight. + std::array, Window::numFrames> quads; + std::array quadSlots; + ReadbackPass readback; + bool bgra = false; + bool checkTexels = false; + + // The frame each slot's texel buffer holds, -1 for none, and what the + // texel checks found since Reset. + std::array frameOfSlot; + std::uint64_t frame = 0; + std::uint64_t framesChecked = 0; + std::uint32_t wrong = 0; + std::string firstWrong; + std::set imagesUsed; + std::set> pairsUsed; + // Run once at the start of the next frame's build, then dropped. + std::function duringNextFrame; + EventListener buildSub; + + ColourScene(Window& window, GraphicsDescriptorHeap& heap, std::string_view name, std::uint64_t colourOffset) + : window(window), heap(heap), name(name), colourOffset(colourOffset) { + const fs::path dir = ExeDir(); + VkCommandBuffer init = window.StartInit(); + ui.Initialize(window, heap, init, + dir / "ui-quads.comp.spv", dir / "ui-circles.comp.spv", + dir / "ui-images.comp.spv", dir / "ui-text.comp.spv", + dir / "ui-fused.comp.spv"); + for (std::uint32_t f = 0; f < Window::numFrames; ++f) { + quads[f].Create(VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, 1, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT); + quadSlots[f] = ui.RegisterBuffer(quads[f]); + } + bgra = window.colorFormat == VK_FORMAT_B8G8R8A8_UNORM; + checkTexels = bgra || window.colorFormat == VK_FORMAT_R8G8B8A8_UNORM; + window.passes.push_back(&ui); + if (checkTexels) { + for (VulkanBuffer& texel : readback.texels) { + texel.Create(VK_BUFFER_USAGE_TRANSFER_DST_BIT, + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, 1); + } + window.passes.push_back(&readback); + } else { + std::println("NOTE {}: the swapchain format {} isn't 8-bit RGBA or BGRA: its colours aren't checked", + name, string_VkFormat(window.colorFormat)); + } + window.FinishInit(); + Reset(); + + buildSub.SetEvent(&ui.onBuild, [this](UIBuildArgs a) { + if (duringNextFrame) { + std::function run = std::move(duringNextFrame); + duringNextFrame = nullptr; + run(); + } + // The slot's fence has been waited: its buffer holds what the + // frame numFrames ago presented. This frame's copy comes after. + CheckSlot(a.frameIdx); + frameOfSlot[a.frameIdx] = static_cast(frame); + imagesUsed.insert(this->window.currentImage); + pairsUsed.insert({ a.frameIdx, this->window.currentImage }); + + const std::array colour = Colour(frame + this->colourOffset); + quads[a.frameIdx].value[0] = QuadItem{ 0, 0, 4096, 4096, + colour[0] / 255.0f, colour[1] / 255.0f, colour[2] / 255.0f, 1.0f, + 0, 0, 0, 0, 0, 0, 0, 0 }; + quads[a.frameIdx].FlushDevice(a.cmd, VK_ACCESS_SHADER_READ_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT); + ui.DispatchQuads(a.cmd, quadSlots[a.frameIdx], 1); + }); + } + + ~ColourScene() { + std::erase(window.passes, &ui); + std::erase(window.passes, &readback); + } + + void Reset() { + frameOfSlot.fill(-1); + framesChecked = 0; + wrong = 0; + firstWrong.clear(); + imagesUsed.clear(); + pairsUsed.clear(); + } + + void CheckSlot(std::uint32_t slot) { + if (!checkTexels || frameOfSlot[slot] < 0) return; + const std::uint32_t texel = readback.texels[slot].value[0]; + const std::array rgb = bgra + ? std::array{ static_cast(texel >> 16), static_cast(texel >> 8), static_cast(texel) } + : std::array{ static_cast(texel), static_cast(texel >> 8), static_cast(texel >> 16) }; + const std::array expected = Colour(static_cast(frameOfSlot[slot]) + colourOffset); + bool same = true; + for (std::size_t c = 0; c < 3; ++c) { + same = same && std::abs(static_cast(rgb[c]) - static_cast(expected[c])) <= 1; + } + ++framesChecked; + if (!same && wrong++ == 0) { + firstWrong = std::format("frame {} presented ({}, {}, {}), drew ({}, {}, {})", frameOfSlot[slot], + rgb[0], rgb[1], rgb[2], expected[0], expected[1], expected[2]); + } + } + + // One frame of this scene's window. + void Render() { + window.Render(); + ++frame; + } + + // After the frames since Reset: settles the queue, reads the slots not + // read yet, and reports. + void Report(std::string_view what) { + Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); + for (std::uint32_t slot = 0; slot < Window::numFrames; ++slot) { + CheckSlot(slot); + frameOfSlot[slot] = -1; + } + if (checkTexels) { + Check(wrong == 0, std::format("{}: each of {} frames presented the colour it drew{}", what, framesChecked, + wrong == 0 ? std::string() : std::format(" ({} didn't; {})", wrong, firstWrong))); + } + } +}; + +// The present modes on the scene's window, then a rebuild asked for during a +// frame, then a slot listed during one. Sets `moreImagesThanFrames` when some +// mode's swapchain had more images than numFrames. +void CheckModes(ColourScene& scene, bool& moreImagesThanFrames) { + Window& window = scene.window; + const std::vector offered = OfferedModes(Device::physDevice, window.vulkanSurface); + for (const ModeCase& m : kModes) { + if (!Offers(offered, m.vk)) { + std::println("NOTE the surface doesn't offer {}", m.name); + continue; + } + // Asked before the swapchain is made, which asks again: the + // validation layer checks the image count against the last answer. + const VkSurfaceCapabilitiesKHR capabilities = Window::SwapchainCapabilities( + Device::physDevice, window.vulkanSurface, m.vk, Device::swapchainMaintenance1Supported); + const std::uint32_t asked = Window::SwapchainMinImageCount(capabilities); + window.SetPresentMode(m.mode); + Check(window.swapchainPresentMode == m.vk, + std::format("{}: the swapchain was made with it, not a fallback ({})", m.name, string_VkPresentModeKHR(window.swapchainPresentMode))); + const std::size_t images = window.images.size(); + Check(images >= asked && images >= capabilities.minImageCount, + std::format("{}: {} images, at least the {} asked for and the mode's minimum of {}", m.name, images, asked, capabilities.minImageCount)); + Check(window.imageViewInfos.size() == images && window.imageViews.size() == images + && window.imageInitialised.size() == images && window.renderFinishedSemaphores.size() >= images, + std::format("{}: the per-image state covers all {} images", m.name, images)); + moreImagesThanFrames = moreImagesThanFrames || images > Window::numFrames; + + scene.Reset(); + // Enough frames for every frame slot to meet several images. + const std::uint64_t frames = 4 * std::max(images, Window::numFrames); + for (std::uint64_t i = 0; i < frames; ++i) { + scene.Render(); + } + scene.Report(m.name); + Check(scene.imagesUsed.size() > 1, std::format("{}: the frames went round {} of the {} images", m.name, scene.imagesUsed.size(), images)); + std::println(" {}: {} images, {} pairings of frame slot and image", m.name, images, scene.pairsUsed.size()); + } + + // A present mode set, and a resize asked for, from a listener of the + // frame being recorded: the frame goes on with the swapchain it + // acquired from, and the new one is made after its present. + const ModeCase* first = nullptr; + const ModeCase* second = nullptr; + for (const ModeCase& m : kModes) { + if (!Offers(offered, m.vk)) continue; + (first == nullptr ? first : second) = &m; + if (second != nullptr) break; + } + if (second != nullptr) { + window.SetPresentMode(first->mode); + const VkSwapchainKHR before = window.swapChain; + const std::uint32_t widthBefore = window.width; + const std::uint32_t heightBefore = window.height; + bool unchangedInFrame = false; + scene.duringNextFrame = [&] { + window.SetPresentMode(second->mode); + window.Resize(widthBefore + 16, heightBefore + 8); + unchangedInFrame = window.swapChain == before && window.swapchainPresentMode == first->vk + && window.width == widthBefore && window.height == heightBefore && window.currentImage < window.images.size(); + }; + scene.Reset(); + scene.Render(); + Check(unchangedInFrame, std::format("a {} -> {} switch and a resize asked for during a frame leave that frame's swapchain and size alone", + first->name, second->name)); + Check(window.swapChain != before && window.swapchainPresentMode == second->vk, + std::format("after the frame the swapchain is made again, with {} ({})", second->name, string_VkPresentModeKHR(window.swapchainPresentMode))); + Check(window.width == widthBefore + 16 && window.height == heightBefore + 8, + std::format("after the frame the window has the size asked for ({}x{})", window.width, window.height)); + for (int i = 0; i < 2 * Window::numFrames; ++i) { + scene.Render(); + } + scene.Report("after the rebuild a frame asked for"); + } else { + std::println("NOTE the surface offers one present mode: no mid-frame switch to check"); + } + + // A swapchain slot listed during a frame holds that frame's image at + // once: in heap mode the descriptor bytes match the UI output slot's, + // which the frame wrote before onUpdate. + if (Device::descriptorHeapSupported) { + const ImageSlotRange spare = scene.heap.AllocateImageSlots(1); + bool same = false; + scene.duringNextFrame = [&] { + scene.heap.AddSwapchainSlot(spare.firstElement, window); + const std::uint8_t* copy = scene.heap.resourceHeap[window.currentFrame].value; + same = std::memcmp(copy + scene.heap.ImageByteOffset(spare.firstElement), + copy + scene.heap.ImageByteOffset(scene.ui.OutImageSlot()), + Device::descriptorHeapProperties.imageDescriptorSize) == 0; + }; + scene.Render(); + Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); + scene.heap.FreeImageSlots(spare); + Check(same, "a swapchain slot listed during a frame holds that frame's image at once"); + } +} + +// Renders on this process's GPU: every offered mode on one window, then two +// windows sharing the heap. Sets `moreImagesThanFrames` when some mode's +// swapchain had more images than numFrames. +int RenderFrames(bool& moreImagesThanFrames) { + // Two scenes' output slots, one spare for the mid-frame slot, and each + // scene's quad buffers. + GraphicsDescriptorHeap heap; + heap.Initialize(/*images*/ 3, /*buffers*/ 2 * Window::numFrames, /*samplers*/ 1); + { + Window window(160, 120, "SwapchainImageCount test"); + window.descriptorHeap = &heap; + { + ColourScene scene(window, heap, "first window", 0); + CheckModes(scene, moreImagesThanFrames); + + // A second window on the same heap. Interleaved frames: one + // window's slot writes land in copies the other's frames in + // flight are bound to, and must not touch its slots. + Window second(160, 120, "SwapchainImageCount second window"); + second.descriptorHeap = &heap; + { + ColourScene other(second, heap, "second window", 100); + scene.Reset(); + for (int i = 0; i < 4 * Window::numFrames; ++i) { + scene.Render(); + other.Render(); + } + scene.Report("two windows on one heap, the first"); + other.Report("two windows on one heap, the second"); + } + Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); + } + Device::CheckVkResult(vkQueueWaitIdle(Device::queue)); + } + + if (std::getenv("CRAFTER_GRAPHICS_VALIDATION") != nullptr) { + Check(Device::validationErrorCount == 0, + std::format("no Vulkan validation errors ({} seen)", Device::validationErrorCount)); + } + return failures == 0 ? EXIT_SUCCESS : EXIT_FAILURE; +} + +// A child's run of the frame checks in binding model `model` ("heap" or +// "sets") on whatever GPU its environment picks. +int FramesOnly(std::string_view model) { + try { + Device::Initialize(); + } catch (const std::runtime_error& error) { + // A missing capability is a skip; anything else that failed is a + // failure of the run that was asked for. + if (std::string_view(error.what()).find("can't run this application") != std::string_view::npos) { + std::println("SKIP this GPU can't run the library: {}", error.what()); + return 77; + } + std::println("FAIL Device::Initialize: {}", error.what()); + return EXIT_FAILURE; + } + std::println(" on {}, {}", std::string_view(Device::deviceProperties.deviceName), + Device::descriptorHeapSupported ? "descriptor heap" : "descriptor sets"); + if (model == "heap" && !Device::descriptorHeapSupported) { + std::println("SKIP this GPU has no descriptor heap"); + return 77; + } + Check(Device::descriptorHeapSupported == (model == "heap"), + std::format("the run is on {}, as asked", model == "heap" ? "the descriptor heap" : "descriptor sets")); + bool moreImagesThanFrames = false; + return RenderFrames(moreImagesThanFrames); +} + +// The number CRAFTER_GRAPHICS_DEVICE gives the GPU `gpu` is one of. +std::optional GpuNumber(const std::vector>& gpus, VkPhysicalDevice gpu) { + for (std::size_t i = 0; i < gpus.size(); ++i) { + if (std::ranges::find(gpus[i], gpu) != gpus[i].end()) return i; + } + return std::nullopt; +} + +bool PresentsTo(VkPhysicalDevice gpu, VkSurfaceKHR surface) { + std::uint32_t families = 0; + vkGetPhysicalDeviceQueueFamilyProperties(gpu, &families, nullptr); + for (std::uint32_t family = 0; family < families; ++family) { + VkBool32 supported = VK_FALSE; + if (vkGetPhysicalDeviceSurfaceSupportKHR(gpu, family, surface, &supported) == VK_SUCCESS && supported) { + return true; + } + } + return false; +} + +// The number of a GPU, other than `own`, whose swapchain for `surface` would +// have more images than numFrames in some mode. Whether it has +// swapchain_maintenance1 isn't known without a device, so its counts are +// taken as without: at most an overestimate, and that GPU's MAILBOX needs the +// most either way. +std::optional GpuWithMoreImages(const std::vector>& gpus, + std::size_t own, VkSurfaceKHR surface) { + for (std::size_t i = 0; i < gpus.size(); ++i) { + if (i == own || !PresentsTo(gpus[i].front(), surface)) continue; + if (MostImagesAsked(gpus[i].front(), surface, /*modePassedAlong*/ false) > Window::numFrames) { + return i; + } + } + return std::nullopt; +} + +int Drive() { + CheckMinImageCount(); + + Device::Initialize(); + const bool ownHeap = Device::descriptorHeapSupported; + std::println(" on {}, {}, swapchain_maintenance1 {}", std::string_view(Device::deviceProperties.deviceName), + ownHeap ? "descriptor heap" : "descriptor sets", Device::swapchainMaintenance1Supported ? "on" : "off"); + bool moreImagesThanFrames = false; + RenderFrames(moreImagesThanFrames); + + // Where frames and images pair up differently, on each binding model: + // this process covered its own GPU's model if that GPU makes more images + // than frames; the children cover the rest. + const std::vector> gpus = Device::DistinctGpus(Device::instance); + const std::optional own = GpuNumber(gpus, Device::physDevice); + std::optional target = moreImagesThanFrames ? own : std::nullopt; + if (!target && own) { + Window probe(160, 120, "SwapchainImageCount probe"); + target = GpuWithMoreImages(gpus, *own, probe.vulkanSurface); + } + if (!target) { + std::println("NOTE no GPU here makes a swapchain with more than {} images, " + "so frames and images were never paired differently", Window::numFrames); + return failures == 0 ? EXIT_SUCCESS : EXIT_FAILURE; + } + + const fs::path exe = fs::read_symlink("/proc/self/exe"); + const auto child = [&](std::string_view model) { + // `env -u` so a CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP inherited from + // the caller's environment can't turn the heap run into a sets run. + // LP_NUM_THREADS keeps llvmpipe, the usual target, off most of the + // machine's cores while the other tests run, timing ones among them. + // env reads its options before the first assignment. + const int exit = RunCommand(std::format("env {} LP_NUM_THREADS=4 CRAFTER_GRAPHICS_DEVICE={} '{}' --frames {}", + model == "heap" ? "-u CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP" : "CRAFTER_GRAPHICS_NO_DESCRIPTOR_HEAP=1", + *target, exe.string(), model)); + if (exit == 77) { + std::println("NOTE GPU {} skipped the run on {} (above)", *target, model); + } else { + Check(exit == 0, std::format("GPU {} draws the frames right on {} (exit {})", *target, model, exit)); + } + }; + const bool coveredHere = moreImagesThanFrames; + if (!(coveredHere && ownHeap)) child("heap"); + if (!(coveredHere && !ownHeap)) child("sets"); + return failures == 0 ? EXIT_SUCCESS : EXIT_FAILURE; +} + +} // namespace + +int main(int argc, char** argv) { + // It would override every present mode under test. + unsetenv("CRAFTER_PRESENT_IMMEDIATE"); + const int result = argc == 3 && std::string_view(argv[1]) == "--frames" ? FramesOnly(argv[2]) : Drive(); + std::println("{}", result == EXIT_SUCCESS ? "all checks passed" : result == 77 ? "skipped" : "check(s) failed"); + return result; +} diff --git a/tests/SwapchainTransferDst/main.cpp b/tests/SwapchainTransferDst/main.cpp index ced31da..d66f806 100644 --- a/tests/SwapchainTransferDst/main.cpp +++ b/tests/SwapchainTransferDst/main.cpp @@ -42,7 +42,7 @@ struct ClearPass : RenderPass { void Record(VkCommandBuffer cmd, std::uint32_t, Window& window) override { const VkImageSubresourceRange range = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}; - vkCmdClearColorImage(cmd, window.images[window.currentBuffer], VK_IMAGE_LAYOUT_GENERAL, &colour, 1, &range); + vkCmdClearColorImage(cmd, window.images[window.currentImage], VK_IMAGE_LAYOUT_GENERAL, &colour, 1, &range); } VkPipelineStageFlags SwapchainStage() const override { diff --git a/tests/UIFallbackParity/main.cpp b/tests/UIFallbackParity/main.cpp index b2d342e..a465419 100644 --- a/tests/UIFallbackParity/main.cpp +++ b/tests/UIFallbackParity/main.cpp @@ -20,7 +20,8 @@ // registered again before the first frame, so the second write has to // replace the first one's view / sampler. A stale descriptor shows the // wrong texture and fails the comparison; -// - a resize, which rewrites the per-frame swapchain descriptors; +// - a resize, after which every frame's copy (heap or set) has to be +// given the new swapchain's images: in sets mode through new views; // - an image created with ImageVulkan::Create's default consumer stage, // which has to be resolved to a stage the device actually has; // - a user-style ui.Dispatch whose push payload is not a whole number of @@ -293,9 +294,10 @@ int RenderScene(const fs::path& out) { window.FinishInit(); // More frames than there are in flight, so every per-frame heap copy / - // descriptor set is used; then a resize, which recreates the swapchain - // and rewrites its descriptors; then enough frames for the glyphs shaped - // on the first frame to have reached the atlas texture. + // descriptor set is used; then a resize, which recreates the swapchain, + // whose new images the frames then write into their copies; then enough + // frames for the glyphs shaped on the first frame to have reached the + // atlas texture. for (int i = 0; i < Window::numFrames * 2; ++i) window.Render(); window.Resize(kResizedWidth, kResizedHeight); for (int i = 0; i < Window::numFrames * 3; ++i) window.Render();