feat: windows on their own clock, deferred deletion for every handle owner, staged re-uploads, and RTPass push data
Window - updateInterval runs Update on the window's own clock while the compositor sends no frame callbacks (hidden, minimized): the Wayland loop is a poll() loop, and the Win32 loop sleeps until the next update. Both read the interval each pass. - SetPresentMode (FIFO, Mailbox, Immediate). A mode the surface lacks, or whose swapchain has other than numFrames images (Mesa's Mailbox has four), falls back to FIFO until the mode is set again. - AddPollFd / RemovePollFd wake the Wayland loop on the app's descriptors. - The swapchain has TRANSFER_DST where the surface allows it. - CaptureNextFrame replaces SaveFrame: it records the copy in the frame, before the present, where SaveFrame read an image already presented. A frame unwound past before the write still frees the readback buffer. - Inter-pass barriers on the swapchain image carry the access masks of the stages the passes use. Deferred deletion (issue #101) - VulkanBuffer, Mesh, VulkanShader, ComputeShader and PipelineRTVulkan hand their handles to the deletion queue from their destructors and moves, as a frame in flight may still use them; a static owner goes after the driver has torn down. VulkanShader and PipelineRTVulkan move and can't be copied; re-Load and re-Init queue the old pipeline. - The queue and its mutex are never destroyed, and the queue takes handles from any thread; Device::frameCounter is atomic. Both the queue and the staging rings count frames as one window renders them (documented). - DrainDeletions also lets every buffer's staging ring start over: without a Window nothing else reclaims either, so an app without one calls it after waiting for its work. Meshes and buffers - A Mesh rebuild queues the old acceleration structure and builds into the same storage after a barrier, so the BLAS address stays put when it fits. The refit and rebuild barriers cover compute ray queries too. - RTBuildOptions::noDuplicateAnyHit. Mesh::geometryFlags replaces opaque, and the refit and rebuild fallbacks keep the flags. - A re-upload into a buffer a frame in flight may read (UploadDeviceLocal of the same size, UploadDeviceLocalRange) copies on the GPU, after those reads, through a per-frame staging ring; on ReBAR it used to write the buffer in place. Empty uploads record nothing. - A compressed Mesh::Build that keeps its buffers decompresses into them after the reads of frames in flight, and the hit shaders and ray queries wait for the decompress as well as the BLAS build. Its vertex buffer takes a ranged refit's copies. - Mesh move-assigns (queueing the BLAS it replaces), so it can be replaced in place or erased from a vector. - VulkanBuffer::Create asks for a device address only for buffers with SHADER_DEVICE_ADDRESS usage, as vkGetBufferDeviceAddress requires; the others have address 0. UIRenderer::RegisterBuffer refuses those. - Build inputs are synchronized as shader reads. Other - RTPass pushes the app's own push data and traces at an explicit size. - DescriptorHeapVulkan hands freed slot ranges out again (they leaked), and keeps its free lists sorted. - ImageVulkan's mip chain clamps each level to 1 texel: a non-square image's shorter side went to 0. - VulkanShader throws on a missing file instead of failing in bad_alloc. - CRAFTER_GRAPHICS_DEVICE picks the GPU by its number among the distinct GPUs Vulkan lists; anything but such a number is an error. - The debug messenger prints info messages only with CRAFTER_GRAPHICS_VALIDATION set. - Windows: the WinRT gamepad code moved to a C file (mingw-w64's WinRT headers don't compile as C++), -lcombase dropped, a reference leak fixed. - project.cpp honours --local for Crafter.Event, Math and Asset, as Asset does for Math, and builds every test through one helper, projectArgs included. - Tests: DebugMessengerSeverity, DeletionQueueThreads, DeviceSelection, HeapSlotReuse, MeshLifetime, MipChainNonSquare, PipelineLifetime, RTPassPushData, SwapchainTransferDst, UploadInFlight, VulkanBufferAddress, VulkanShaderLifetime, WindowUpdateClock; others extended. Breaking: SaveFrame is CaptureNextFrame; Mesh::opaque is geometryFlags; a buffer without SHADER_DEVICE_ADDRESS usage has no address; frameCounter is std::atomic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
449d5da553
commit
079b6b0e1f
GPG key ID:
2937E59CDCC1BCFB
47 changed files with 3572 additions and 1430 deletions
|
|
@ -51,6 +51,9 @@ export namespace Crafter {
|
|||
~ComputeShader();
|
||||
|
||||
// Loads a SPIR-V compute shader from disk and creates its pipeline.
|
||||
// Loading again replaces the pipeline. The old one, like the one a
|
||||
// ComputeShader holds when it's destroyed or moved over, goes through
|
||||
// the deferred-deletion queue: a frame in flight may still dispatch it.
|
||||
//
|
||||
// `spvPath` always names the descriptor-heap build (<name>.spv). On a
|
||||
// device without the heap, the <name>.sets.spv build next to it is
|
||||
|
|
|
|||
|
|
@ -391,12 +391,35 @@ export namespace Crafter {
|
|||
vkUpdateDescriptorSets(Device::device, Window::numFrames, writes, 0, nullptr);
|
||||
}
|
||||
|
||||
// Takes `count` consecutive slots out of `freelist`, the first of
|
||||
// them returned, if it holds a run that long: freed ranges and freed
|
||||
// single slots alike come back as ranges, rather than only fresh
|
||||
// slots serving a range. The freelists are kept sorted (Free), which
|
||||
// puts neighbours side by side. A freed slot is handed out again at
|
||||
// once, which is why the Free* calls below require that no frame in
|
||||
// flight still uses it.
|
||||
static std::optional<std::uint16_t> TakeRun(std::vector<std::uint16_t>& freelist, std::uint16_t count) {
|
||||
if (count == 0 || freelist.size() < count) return std::nullopt;
|
||||
for (std::size_t i = 0; i + count <= freelist.size(); ++i) {
|
||||
// Sorted and without repeats: the ends `count - 1` apart are a run.
|
||||
if (freelist[i + count - 1] - freelist[i] == count - 1) {
|
||||
const std::uint16_t first = freelist[i];
|
||||
freelist.erase(freelist.begin() + static_cast<std::ptrdiff_t>(i), freelist.begin() + static_cast<std::ptrdiff_t>(i + count));
|
||||
return first;
|
||||
}
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
ImageSlotRange AllocateImageSlots(std::uint16_t count) {
|
||||
if (count == 1 && !imageFreelist.empty()) {
|
||||
std::uint16_t s = imageFreelist.back();
|
||||
imageFreelist.pop_back();
|
||||
return {s, 1};
|
||||
}
|
||||
if (const std::optional<std::uint16_t> first = TakeRun(imageFreelist, count)) {
|
||||
return {*first, count};
|
||||
}
|
||||
if (imageNext + count > imageCapacity) {
|
||||
std::uint16_t remaining = static_cast<std::uint16_t>((imageCapacity - imageNext) + imageFreelist.size());
|
||||
throw std::runtime_error(std::format("DescriptorHeapVulkan: out of image slots ({} requested, {} remaining of {})", count, remaining, imageCapacity));
|
||||
|
|
@ -412,6 +435,9 @@ export namespace Crafter {
|
|||
bufferFreelist.pop_back();
|
||||
return {s, 1};
|
||||
}
|
||||
if (const std::optional<std::uint16_t> first = TakeRun(bufferFreelist, count)) {
|
||||
return {*first, count};
|
||||
}
|
||||
if (bufferNext + count > bufferCapacity) {
|
||||
std::uint16_t remaining = static_cast<std::uint16_t>((bufferCapacity - bufferNext) + bufferFreelist.size());
|
||||
throw std::runtime_error(std::format("DescriptorHeapVulkan: out of buffer slots ({} requested, {} remaining of {})", count, remaining, bufferCapacity));
|
||||
|
|
@ -427,6 +453,9 @@ export namespace Crafter {
|
|||
samplerFreelist.pop_back();
|
||||
return {s, 1};
|
||||
}
|
||||
if (const std::optional<std::uint16_t> first = TakeRun(samplerFreelist, count)) {
|
||||
return {*first, count};
|
||||
}
|
||||
if (samplerNext + count > samplerCapacity) {
|
||||
std::uint16_t remaining = static_cast<std::uint16_t>((samplerCapacity - samplerNext) + samplerFreelist.size());
|
||||
throw std::runtime_error(std::format("DescriptorHeapVulkan: out of sampler slots ({} requested, {} remaining of {})", count, remaining, samplerCapacity));
|
||||
|
|
@ -441,23 +470,26 @@ export namespace Crafter {
|
|||
// via WriteSampledImageDescriptor / WriteBufferDescriptor / sampler write.
|
||||
// Caller MUST ensure no in-flight GPU frame still references the freed
|
||||
// slots — typically vkDeviceWaitIdle (or a scene-switch barrier) before
|
||||
// freeing. Multi-slot ranges are decomposed into their individual slots.
|
||||
// noexcept: capacity is reserved at Initialize so push_back never reallocates.
|
||||
// freeing. Multi-slot ranges are decomposed into their individual slots,
|
||||
// each put in its sorted place (TakeRun looks for runs).
|
||||
// noexcept: capacity is reserved at Initialize so inserting never reallocates.
|
||||
void FreeImageSlots(ImageSlotRange r) noexcept {
|
||||
for (std::uint16_t i = 0; i < r.count; ++i) {
|
||||
imageFreelist.push_back(static_cast<std::uint16_t>(r.firstElement + i));
|
||||
}
|
||||
Release(imageFreelist, r.firstElement, r.count);
|
||||
}
|
||||
|
||||
void FreeBufferSlots(BufferSlotRange r) noexcept {
|
||||
for (std::uint16_t i = 0; i < r.count; ++i) {
|
||||
bufferFreelist.push_back(static_cast<std::uint16_t>(r.firstElement + i));
|
||||
}
|
||||
Release(bufferFreelist, r.firstElement, r.count);
|
||||
}
|
||||
|
||||
void FreeSamplerSlots(SamplerSlotRange r) noexcept {
|
||||
for (std::uint16_t i = 0; i < r.count; ++i) {
|
||||
samplerFreelist.push_back(static_cast<std::uint16_t>(r.firstElement + i));
|
||||
Release(samplerFreelist, r.firstElement, r.count);
|
||||
}
|
||||
|
||||
static void Release(std::vector<std::uint16_t>& freelist, std::uint16_t first, std::uint16_t count) noexcept {
|
||||
const auto at = std::ranges::lower_bound(freelist, first);
|
||||
const std::ptrdiff_t index = at - freelist.begin();
|
||||
for (std::uint16_t i = 0; i < count; ++i) {
|
||||
freelist.insert(freelist.begin() + index + i, static_cast<std::uint16_t>(first + i));
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -378,17 +378,30 @@ export namespace Crafter {
|
|||
// instead, tagged with the current frameCounter; ReclaimDeletions frees
|
||||
// an entry only once framesInFlight frames have elapsed, by which point
|
||||
// single-queue submission order guarantees all GPU work that could
|
||||
// reference it has completed.
|
||||
// reference it has completed. Destructors that own Vulkan handles
|
||||
// (VulkanBuffer, Mesh, VulkanShader, ComputeShader, PipelineRTVulkan)
|
||||
// queue them too: they can't know
|
||||
// whether a frame in flight still uses the handle, and a static owner
|
||||
// is destroyed after the driver and its layers have torn down, when no
|
||||
// Vulkan call may run. An entry holds one kind of handle; the others
|
||||
// stay VK_NULL_HANDLE.
|
||||
struct PendingDeletion {
|
||||
std::uint64_t retireAfter; // frameCounter value at enqueue time
|
||||
VkBuffer buffer;
|
||||
VkDeviceMemory memory;
|
||||
VkBuffer buffer = VK_NULL_HANDLE;
|
||||
VkDeviceMemory memory = VK_NULL_HANDLE;
|
||||
VkAccelerationStructureKHR accelerationStructure = VK_NULL_HANDLE;
|
||||
VkShaderModule shaderModule = VK_NULL_HANDLE;
|
||||
VkPipeline pipeline = VK_NULL_HANDLE;
|
||||
};
|
||||
// Monotonic recorded-frame index, bumped once per Window::Render. Not
|
||||
// the same as Window::frameCounter (which is per-window and also drives
|
||||
// the acquire-semaphore slot) — this one is global so Mesh/Device code
|
||||
// can enqueue too, not just the window loop.
|
||||
inline static std::uint64_t frameCounter = 0;
|
||||
// can enqueue too, not just the window loop. Atomic: a destructor on
|
||||
// another thread reads it as it queues. It counts frames as one
|
||||
// window renders them: with several windows rendering, framesInFlight
|
||||
// of its counts pass sooner than each window's frames in flight, so
|
||||
// the deletion queue and the staging rings assume one window.
|
||||
inline static std::atomic<std::uint64_t> frameCounter = 0;
|
||||
// = Window::numFrames; set at window init. The number of frames that
|
||||
// must elapse after an enqueue before the resource is safe to free.
|
||||
// Zero until set, which only makes ReclaimDeletions more eager — and it
|
||||
|
|
@ -397,18 +410,36 @@ export namespace Crafter {
|
|||
// A deque, not a vector: frameCounter is monotonic, so retireAfter is
|
||||
// non-decreasing down the queue and the entries ready to reclaim are
|
||||
// always a prefix. ReclaimDeletions pops that prefix off the front in
|
||||
// O(ready) instead of walking and compacting the whole queue.
|
||||
inline static std::deque<PendingDeletion> deletionQueue;
|
||||
// O(ready) instead of walking and compacting the whole queue. Never
|
||||
// destroyed: static owners queue their handles from their destructors
|
||||
// at exit, and those may run after this queue's own destructor would.
|
||||
inline static std::deque<PendingDeletion>& deletionQueue = *new std::deque<PendingDeletion>();
|
||||
// Held by the calls below: handle owners go on whichever thread drops
|
||||
// them (a loader's, say) while the render loop reclaims. Never
|
||||
// destroyed, for the same reason as the queue.
|
||||
inline static std::mutex& deletionMutex = *new std::mutex();
|
||||
// Tag {buffer, memory} for deletion after framesInFlight more frames.
|
||||
// A VK_NULL_HANDLE buffer is ignored (nothing to free).
|
||||
static void EnqueueDeletion(VkBuffer buffer, VkDeviceMemory memory);
|
||||
// The same for an acceleration structure, a shader module and a
|
||||
// pipeline. Queue an acceleration structure before the buffer it lives
|
||||
// in: entries are freed in queue order. A VK_NULL_HANDLE is ignored.
|
||||
static void EnqueueDeletion(VkAccelerationStructureKHR accelerationStructure);
|
||||
static void EnqueueDeletion(VkShaderModule shaderModule);
|
||||
static void EnqueueDeletion(VkPipeline pipeline);
|
||||
// Free every entry whose retire frame has been reached
|
||||
// (retireAfter + framesInFlight <= frameCounter). Call once per frame
|
||||
// after waiting that frame's fence.
|
||||
static void ReclaimDeletions();
|
||||
// Free every queued entry unconditionally. Call only after a wait-idle
|
||||
// (resize / teardown), when no in-flight GPU work can reference them.
|
||||
// Free every queued entry unconditionally, and let every buffer's
|
||||
// staging ring start over. Call only after a wait-idle (resize /
|
||||
// teardown), when no in-flight GPU work can reference them. Without a
|
||||
// Window nothing else reclaims them: an app that has none calls this
|
||||
// after waiting for its work, or both only grow.
|
||||
static void DrainDeletions();
|
||||
// How many DrainDeletions have run: the staging rings start over
|
||||
// when it changes.
|
||||
inline static std::atomic<std::uint64_t> drains = 0;
|
||||
|
||||
// ─── Wayland key repeat ────────────────────────────────────────
|
||||
// TickKeyRepeats fires onRawKeyDown / onRawKeyHold / onTextInput on
|
||||
|
|
|
|||
|
|
@ -177,46 +177,7 @@ export namespace Crafter {
|
|||
®ion
|
||||
);
|
||||
|
||||
if(mipLevels > 1) {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, 0, 1);
|
||||
|
||||
for (std::uint16_t i = 1; i < mipLevels; ++i) {
|
||||
std::uint16_t mipWidth = width >> i;
|
||||
std::uint16_t mipHeight = height >> i;
|
||||
|
||||
std::uint16_t previousMipWidth = width >> (i - std::uint16_t(1));
|
||||
std::uint16_t previousMipHeight = height >> (i - std::uint16_t(1));
|
||||
|
||||
VkImageBlit blit = {};
|
||||
blit.srcSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
|
||||
blit.srcSubresource.mipLevel = i - 1;
|
||||
blit.srcSubresource.baseArrayLayer = 0;
|
||||
blit.srcSubresource.layerCount = 1;
|
||||
blit.srcOffsets[0] = { 0, 0, 0 };
|
||||
blit.srcOffsets[1] = { (int32_t)previousMipWidth, (int32_t)previousMipHeight, 1 };
|
||||
|
||||
blit.dstSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
|
||||
blit.dstSubresource.mipLevel = i;
|
||||
blit.dstSubresource.baseArrayLayer = 0;
|
||||
blit.dstSubresource.layerCount = 1;
|
||||
blit.dstOffsets[0] = { 0, 0, 0 };
|
||||
blit.dstOffsets[1] = { (int32_t)mipWidth, (int32_t)mipHeight, 1 };
|
||||
|
||||
vkCmdBlitImage(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &blit, VK_FILTER_LINEAR);
|
||||
// The final blit's destination is never read again, so it
|
||||
// skips the DST->SRC barrier and is taken straight to the
|
||||
// consumer layout by the batched final transition below.
|
||||
if (i + 1 < mipLevels) {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, i, 1);
|
||||
}
|
||||
}
|
||||
|
||||
std::array<VkImageMemoryBarrier, 2> finalBarriers;
|
||||
std::uint32_t finalCount = BuildMipChainFinalBarriers(image, layout, mipLevels, finalBarriers);
|
||||
vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_TRANSFER_BIT, consumerStage, 0, 0, nullptr, 0, nullptr, finalCount, finalBarriers.data());
|
||||
} else {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, layout, VK_PIPELINE_STAGE_TRANSFER_BIT, consumerStage, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_READ_BIT, 0, mipLevels);
|
||||
}
|
||||
BuildMipChain(cmd, layout);
|
||||
ReleaseStaging();
|
||||
}
|
||||
|
||||
|
|
@ -336,44 +297,51 @@ export namespace Crafter {
|
|||
region.imageExtent = { width, height, 1 };
|
||||
vkCmdCopyBufferToImage(cmd, buffer.buffer, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, ®ion);
|
||||
|
||||
if (mipLevels > 1) {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, 0, 1);
|
||||
for (std::uint16_t i = 1; i < mipLevels; ++i) {
|
||||
std::uint16_t mipWidth = width >> i;
|
||||
std::uint16_t mipHeight = height >> i;
|
||||
std::uint16_t previousMipWidth = width >> (i - std::uint16_t(1));
|
||||
std::uint16_t previousMipHeight = height >> (i - std::uint16_t(1));
|
||||
|
||||
VkImageBlit blit = {};
|
||||
blit.srcSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
|
||||
blit.srcSubresource.mipLevel = i - 1;
|
||||
blit.srcSubresource.baseArrayLayer = 0;
|
||||
blit.srcSubresource.layerCount = 1;
|
||||
blit.srcOffsets[0] = { 0, 0, 0 };
|
||||
blit.srcOffsets[1] = { (int32_t)previousMipWidth, (int32_t)previousMipHeight, 1 };
|
||||
blit.dstSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
|
||||
blit.dstSubresource.mipLevel = i;
|
||||
blit.dstSubresource.baseArrayLayer = 0;
|
||||
blit.dstSubresource.layerCount = 1;
|
||||
blit.dstOffsets[0] = { 0, 0, 0 };
|
||||
blit.dstOffsets[1] = { (int32_t)mipWidth, (int32_t)mipHeight, 1 };
|
||||
vkCmdBlitImage(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &blit, VK_FILTER_LINEAR);
|
||||
// The final blit's destination is never read again, so it
|
||||
// skips the DST->SRC barrier and is taken straight to the
|
||||
// consumer layout by the batched final transition below.
|
||||
if (i + 1 < mipLevels) {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, i, 1);
|
||||
}
|
||||
}
|
||||
std::array<VkImageMemoryBarrier, 2> finalBarriers;
|
||||
std::uint32_t finalCount = BuildMipChainFinalBarriers(image, layout, mipLevels, finalBarriers);
|
||||
vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_TRANSFER_BIT, consumerStage, 0, 0, nullptr, 0, nullptr, finalCount, finalBarriers.data());
|
||||
} else {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, layout, VK_PIPELINE_STAGE_TRANSFER_BIT, consumerStage, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_READ_BIT, 0, mipLevels);
|
||||
}
|
||||
BuildMipChain(cmd, layout);
|
||||
ReleaseStaging();
|
||||
}
|
||||
|
||||
// After level 0 was copied in (in TRANSFER_DST): blits each level from
|
||||
// the one above it, then takes the whole image to `layout` for
|
||||
// consumerStage. A level is max(1, size >> level) on each side, as
|
||||
// Vulkan sizes it: a non-square image's shorter side reaches 1 before
|
||||
// the chain ends, and a blit to 0 would write nothing.
|
||||
void BuildMipChain(VkCommandBuffer cmd, VkImageLayout layout) {
|
||||
if (mipLevels <= 1) {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, layout, VK_PIPELINE_STAGE_TRANSFER_BIT, consumerStage, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_READ_BIT, 0, mipLevels);
|
||||
return;
|
||||
}
|
||||
auto levelSize = [](std::uint16_t size, std::uint32_t level) {
|
||||
return std::max<std::int32_t>(1, static_cast<std::int32_t>(size >> level));
|
||||
};
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, 0, 1);
|
||||
for (std::uint32_t i = 1; i < mipLevels; ++i) {
|
||||
VkImageBlit blit = {};
|
||||
blit.srcSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
|
||||
blit.srcSubresource.mipLevel = i - 1;
|
||||
blit.srcSubresource.baseArrayLayer = 0;
|
||||
blit.srcSubresource.layerCount = 1;
|
||||
blit.srcOffsets[0] = { 0, 0, 0 };
|
||||
blit.srcOffsets[1] = { levelSize(width, i - 1), levelSize(height, i - 1), 1 };
|
||||
blit.dstSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
|
||||
blit.dstSubresource.mipLevel = i;
|
||||
blit.dstSubresource.baseArrayLayer = 0;
|
||||
blit.dstSubresource.layerCount = 1;
|
||||
blit.dstOffsets[0] = { 0, 0, 0 };
|
||||
blit.dstOffsets[1] = { levelSize(width, i), levelSize(height, i), 1 };
|
||||
vkCmdBlitImage(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &blit, VK_FILTER_LINEAR);
|
||||
// The final blit's destination is never read again, so it
|
||||
// skips the DST->SRC barrier and is taken straight to the
|
||||
// consumer layout by the batched final transition below.
|
||||
if (i + 1 < mipLevels) {
|
||||
TransitionImageLayout(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, i, 1);
|
||||
}
|
||||
}
|
||||
std::array<VkImageMemoryBarrier, 2> finalBarriers;
|
||||
std::uint32_t finalCount = BuildMipChainFinalBarriers(image, layout, mipLevels, finalBarriers);
|
||||
vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_TRANSFER_BIT, consumerStage, 0, 0, nullptr, 0, nullptr, finalCount, finalBarriers.data());
|
||||
}
|
||||
|
||||
void Destroy() {
|
||||
vkDestroyImageView(Device::device, imageView, nullptr);
|
||||
vkDestroyImage(Device::device, image, nullptr);
|
||||
|
|
|
|||
|
|
@ -49,6 +49,12 @@ export namespace Crafter {
|
|||
// PREFER_FAST_BUILD are mutually exclusive, so `preference` selects
|
||||
// exactly one — allowUpdate is layered on top of whichever is set.
|
||||
bool allowUpdate = false;
|
||||
// Set VK_GEOMETRY_NO_DUPLICATE_ANY_HIT_INVOCATION_BIT_KHR: an any-hit
|
||||
// shader then runs at most once per primitive along a ray, which a
|
||||
// shader that accumulates along it (transmittance through tinted
|
||||
// glass) relies on. Without it the implementation may call it twice
|
||||
// for one primitive. Can cost some trace speed.
|
||||
bool noDuplicateAnyHit = false;
|
||||
};
|
||||
|
||||
class Mesh {
|
||||
|
|
@ -75,12 +81,14 @@ export namespace Crafter {
|
|||
// has cleared — no longer pinned for the mesh's life. Between Builds the
|
||||
// handle is null; the next Build's Resize re-creates it.
|
||||
VulkanBuffer<std::byte, true> compressedStaging;
|
||||
VkAccelerationStructureGeometryTrianglesDataKHR blasData;
|
||||
VkAccelerationStructureGeometryKHR blas;
|
||||
VkAccelerationStructureKHR accelerationStructure = VK_NULL_HANDLE;
|
||||
VkDeviceAddress blasAddr;
|
||||
bool opaque;
|
||||
VkDeviceAddress blasAddr = 0;
|
||||
// ─── Build-option / refit state ──────────────────────────────────
|
||||
// The geometry's flags at the last (re)build: OPAQUE for triangles
|
||||
// and for procedural geometry built opaque, plus
|
||||
// NO_DUPLICATE_ANY_HIT_INVOCATION when asked for. Kept because an
|
||||
// UPDATE must describe its geometry exactly as the build did.
|
||||
VkGeometryFlagsKHR geometryFlags = VK_GEOMETRY_OPAQUE_BIT_KHR;
|
||||
// Flags the BLAS was last (re)built with — carried so Refit can
|
||||
// re-issue an identical-flags UPDATE build (the FAST_TRACE/FAST_BUILD
|
||||
// preference plus, when opted in, ALLOW_UPDATE).
|
||||
|
|
@ -95,6 +103,23 @@ export namespace Crafter {
|
|||
// Whether ALLOW_UPDATE was set on the last build, i.e. whether an
|
||||
// in-place refit is possible at all.
|
||||
bool allowUpdate = false;
|
||||
|
||||
Mesh() = default;
|
||||
// Moving hands the BLAS over: the moved-from Mesh keeps no handle, so
|
||||
// only the last owner releases it. Assigning queues the BLAS this one
|
||||
// held, as the destructor would.
|
||||
Mesh(Mesh&& other);
|
||||
Mesh& operator=(Mesh&& other);
|
||||
// Queues the BLAS for deferred deletion, ahead of the buffers it lives
|
||||
// in (they queue themselves): a frame in flight may still trace it.
|
||||
~Mesh();
|
||||
|
||||
// A rebuild (any Build on a Mesh that already has a BLAS) queues the
|
||||
// old handle for deferred deletion, since this command buffer or a
|
||||
// frame in flight may still use it, and builds into the same storage
|
||||
// after a barrier when the new BLAS fits, so blasAddr normally stays
|
||||
// the same. A BLAS that outgrows its storage moves, so read blasAddr
|
||||
// again after a rebuild.
|
||||
void Build(std::span<Vector<float, 3, 3>> verticies, std::span<std::uint32_t> indicies, VkCommandBuffer cmd, RTBuildOptions options = {});
|
||||
// GPU path: decompresses vertex (region 0) and index (region 1) streams
|
||||
// from asset.blob into vertexBuffer / indexBuffer using
|
||||
|
|
@ -126,7 +151,11 @@ export namespace Crafter {
|
|||
// and VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, its boxes laid out as
|
||||
// VkAabbPositionsKHR (`stride`, default sizeof(RTAabb) == 24), and the
|
||||
// caller must pipeline-barrier the producing writes (e.g. a compute
|
||||
// dispatch) before the build on `cmd` reads them. The buffer's
|
||||
// dispatch) before the build on `cmd` reads them, with
|
||||
// VK_ACCESS_SHADER_READ_BIT at the acceleration-structure build stage:
|
||||
// a build reads its geometry as a shader read, and the
|
||||
// ACCELERATION_STRUCTURE access bits cover only the scratch and the
|
||||
// acceleration structures themselves. The buffer's
|
||||
// lifetime is the caller's responsibility — it is never copied, so it
|
||||
// must outlive the build submitted on `cmd`. Everything else (opaque
|
||||
// bit, hit-group contract, instance wiring) matches the host-span
|
||||
|
|
@ -151,8 +180,8 @@ export namespace Crafter {
|
|||
// Dirty-range refit: same as Refit above, but only the contiguous block
|
||||
// of vertices in [dirtyVertexOffset, dirtyVertexOffset + dirtyVertexCount)
|
||||
// actually moved, so on the in-place UPDATE path only that sub-range is
|
||||
// re-uploaded — host-write + flush + barrier (direct) or re-stage + copy
|
||||
// (staged) scale with the moved vertices, not the whole array (#119). Use
|
||||
// re-uploaded — the staged copy scales with the moved vertices, not the
|
||||
// whole array (#119). Use
|
||||
// this for a deforming mesh that nudges a small, known window each frame.
|
||||
// `verticies` / `indicies` are still the *full* arrays (same contract and
|
||||
// topology rule as the full-span Refit — the index buffer is read only
|
||||
|
|
@ -229,6 +258,8 @@ export namespace Crafter {
|
|||
struct RTBuildOptions {
|
||||
RTBuildPreference preference = RTBuildPreference::FastTrace;
|
||||
bool allowUpdate = false;
|
||||
// No effect here either; for API symmetry like the rest.
|
||||
bool noDuplicateAnyHit = false;
|
||||
};
|
||||
|
||||
class Mesh {
|
||||
|
|
|
|||
|
|
@ -14,14 +14,19 @@ import :ShaderBindingTableVulkan;
|
|||
import :Types;
|
||||
|
||||
export namespace Crafter {
|
||||
// Owns a ray-tracing pipeline and its shader binding table. Both go to
|
||||
// the deferred-deletion queue when it is destroyed, moved over or
|
||||
// initialised again: a frame in flight may still trace with them, and a
|
||||
// static one is destroyed after the driver. It moves, and can't be copied:
|
||||
// two copies would both release the pipeline.
|
||||
struct PipelineRTVulkan {
|
||||
VkPipeline pipeline;
|
||||
std::vector<std::uint8_t> shaderHandles;
|
||||
VkPipeline pipeline = VK_NULL_HANDLE;
|
||||
std::vector<std::uint8_t> shaderHandles;
|
||||
VulkanBuffer<std::uint8_t, true> sbtBuffer;
|
||||
VkStridedDeviceAddressRegionKHR raygenRegion;
|
||||
VkStridedDeviceAddressRegionKHR missRegion;
|
||||
VkStridedDeviceAddressRegionKHR hitRegion;
|
||||
VkStridedDeviceAddressRegionKHR callableRegion;
|
||||
VkStridedDeviceAddressRegionKHR raygenRegion {};
|
||||
VkStridedDeviceAddressRegionKHR missRegion {};
|
||||
VkStridedDeviceAddressRegionKHR hitRegion {};
|
||||
VkStridedDeviceAddressRegionKHR callableRegion {};
|
||||
|
||||
// NVIDIA descriptor-heap AS-read workaround (issue #15 / #7): copied
|
||||
// from the shader table at Init so RTPass can push the active TLAS
|
||||
|
|
@ -30,6 +35,35 @@ export namespace Crafter {
|
|||
bool workaroundNeedsTlas = false;
|
||||
std::uint32_t workaroundTlasPushOffset = 0;
|
||||
|
||||
PipelineRTVulkan() = default;
|
||||
PipelineRTVulkan(const PipelineRTVulkan&) = delete;
|
||||
PipelineRTVulkan& operator=(const PipelineRTVulkan&) = delete;
|
||||
|
||||
PipelineRTVulkan(PipelineRTVulkan&& other) noexcept
|
||||
: pipeline(std::exchange(other.pipeline, VK_NULL_HANDLE)),
|
||||
shaderHandles(std::move(other.shaderHandles)),
|
||||
sbtBuffer(std::move(other.sbtBuffer)),
|
||||
raygenRegion(other.raygenRegion),
|
||||
missRegion(other.missRegion),
|
||||
hitRegion(other.hitRegion),
|
||||
callableRegion(other.callableRegion),
|
||||
workaroundNeedsTlas(other.workaroundNeedsTlas),
|
||||
workaroundTlasPushOffset(other.workaroundTlasPushOffset) {}
|
||||
|
||||
PipelineRTVulkan& operator=(PipelineRTVulkan&& other) noexcept {
|
||||
if (this == &other) return *this;
|
||||
Device::EnqueueDeletion(std::exchange(pipeline, std::exchange(other.pipeline, VK_NULL_HANDLE)));
|
||||
shaderHandles = std::move(other.shaderHandles);
|
||||
sbtBuffer = std::move(other.sbtBuffer);
|
||||
raygenRegion = other.raygenRegion;
|
||||
missRegion = other.missRegion;
|
||||
hitRegion = other.hitRegion;
|
||||
callableRegion = other.callableRegion;
|
||||
workaroundNeedsTlas = other.workaroundNeedsTlas;
|
||||
workaroundTlasPushOffset = other.workaroundTlasPushOffset;
|
||||
return *this;
|
||||
}
|
||||
|
||||
// maxRecursionDepth: the maximum ray-recursion depth the pipeline must
|
||||
// support — i.e. the deepest chain of nested traceRayEXT calls. The
|
||||
// raygen counts as depth 1, so a closest-hit shader that traces a shadow
|
||||
|
|
@ -67,7 +101,11 @@ export namespace Crafter {
|
|||
.layout = VK_NULL_HANDLE
|
||||
};
|
||||
|
||||
Device::CheckVkResult(Device::vkCreateRayTracingPipelinesKHR(Device::device, {}, Device::pipelineCache, 1, &rtPipelineInfo, nullptr, &pipeline));
|
||||
VkPipeline created = VK_NULL_HANDLE;
|
||||
Device::CheckVkResult(Device::vkCreateRayTracingPipelinesKHR(Device::device, {}, Device::pipelineCache, 1, &rtPipelineInfo, nullptr, &created));
|
||||
// Initialising again replaces the pipeline (and below, the binding
|
||||
// table), which a frame in flight may still trace with.
|
||||
Device::EnqueueDeletion(std::exchange(pipeline, created));
|
||||
|
||||
std::size_t dataSize = Device::rayTracingProperties.shaderGroupHandleSize * rtPipelineInfo.groupCount;
|
||||
shaderHandles.resize(dataSize);
|
||||
|
|
@ -111,6 +149,7 @@ export namespace Crafter {
|
|||
// direct-write type lacking HOST_COHERENT is still flushed correctly.
|
||||
VkMemoryPropertyFlags sbtPreferred = Device::PreferDirectDeviceWrite(bufferSize)
|
||||
? VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT : 0;
|
||||
sbtBuffer.DeferredClear();
|
||||
sbtBuffer.Create(VK_BUFFER_USAGE_2_SHADER_BINDING_TABLE_BIT_KHR | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, bufferSize, sbtPreferred);
|
||||
|
||||
std::uint8_t* offset = sbtBuffer.value;
|
||||
|
|
@ -138,7 +177,7 @@ export namespace Crafter {
|
|||
}
|
||||
|
||||
~PipelineRTVulkan() {
|
||||
vkDestroyPipeline(Device::device, pipeline, nullptr);
|
||||
Device::EnqueueDeletion(pipeline);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -17,6 +17,17 @@ import :RenderingElement3D;
|
|||
export namespace Crafter {
|
||||
struct RTPass : RenderPass {
|
||||
PipelineRTVulkan* pipeline;
|
||||
// The application's push data, pushed at offset 0 before every trace
|
||||
// (e.g. a camera), as the DOM backend's RTPass takes it: it starts the
|
||||
// shaders' push block. The pointed-to bytes are read at Record time,
|
||||
// so updating them before Render() is enough. Null means none.
|
||||
const void* pushPtr = nullptr;
|
||||
std::uint32_t pushBytes = 0;
|
||||
// The trace size; 0 means the window's current size, as before. Set
|
||||
// it to trace into an image of its own size rather than the
|
||||
// swapchain's.
|
||||
std::uint32_t width = 0;
|
||||
std::uint32_t height = 0;
|
||||
|
||||
RTPass(PipelineRTVulkan* p) : pipeline(p) {}
|
||||
|
||||
|
|
@ -29,6 +40,14 @@ export namespace Crafter {
|
|||
|
||||
void Record(VkCommandBuffer cmd, std::uint32_t frameIdx, Window& window) override {
|
||||
vkCmdBindPipeline(cmd, VK_PIPELINE_BIND_POINT_RAY_TRACING_KHR, pipeline->pipeline);
|
||||
if (pushBytes != 0) {
|
||||
VkPushDataInfoEXT userPush {
|
||||
.sType = VK_STRUCTURE_TYPE_PUSH_DATA_INFO_EXT,
|
||||
.offset = 0,
|
||||
.data = { .address = pushPtr, .size = pushBytes },
|
||||
};
|
||||
Device::vkCmdPushDataEXT(cmd, &userPush);
|
||||
}
|
||||
// NVIDIA descriptor-heap AS-read workaround (issue #15 / #7): feed
|
||||
// the active frame's TLAS device address into the push-constant
|
||||
// block that VulkanShader synthesizes, so the rewritten raygen can
|
||||
|
|
@ -53,7 +72,7 @@ export namespace Crafter {
|
|||
&pipeline->missRegion,
|
||||
&pipeline->hitRegion,
|
||||
&pipeline->callableRegion,
|
||||
window.width, window.height, 1);
|
||||
width != 0 ? width : window.width, height != 0 ? height : window.height, 1);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -74,19 +74,35 @@ export namespace Crafter {
|
|||
return access;
|
||||
}
|
||||
|
||||
// Read and write access for the stages a pass touches the swapchain image
|
||||
// in: the next pass after an inter-pass barrier may read the image as well
|
||||
// as write it, through shaders or through transfers.
|
||||
inline VkAccessFlags SwapchainPassAccess(VkPipelineStageFlags stages) {
|
||||
VkAccessFlags access = 0;
|
||||
if (stages & (VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT
|
||||
| VK_PIPELINE_STAGE_RAY_TRACING_SHADER_BIT_KHR))
|
||||
access |= VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT;
|
||||
if (stages & VK_PIPELINE_STAGE_TRANSFER_BIT)
|
||||
access |= VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT;
|
||||
return access;
|
||||
}
|
||||
|
||||
// The inter-pass swapchain barrier (replacing the old queue-wide
|
||||
// VkMemoryBarrier at the #115 location). Scoped to the swapchain image's
|
||||
// single colour subresource so only that image's shader caches round-trip —
|
||||
// unrelated buffers/images stay resident — exactly as the intra-pass UI
|
||||
// barrier already does. The image is bound as a storage image and stays in
|
||||
// VK_IMAGE_LAYOUT_GENERAL, so this is a pure memory dependency (no layout
|
||||
// transition). The execution scope (stage masks) is supplied per pass pair
|
||||
// by the caller via vkCmdPipelineBarrier.
|
||||
inline VkImageMemoryBarrier BuildSwapchainInterPassBarrier(VkImage image) {
|
||||
// transition). The stages are the two passes' SwapchainStage(), which the
|
||||
// caller also hands vkCmdPipelineBarrier, and the access masks follow
|
||||
// them: a transfer pass (a clear, a blit) writes with TRANSFER_WRITE, which
|
||||
// SHADER_WRITE would not cover, and an access a stage can't perform is
|
||||
// invalid in its barrier (VUID-vkCmdPipelineBarrier-pImageMemoryBarriers-02818/02820).
|
||||
inline VkImageMemoryBarrier BuildSwapchainInterPassBarrier(VkImage image, VkPipelineStageFlags srcStages, VkPipelineStageFlags dstStages) {
|
||||
return {
|
||||
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
|
||||
.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT,
|
||||
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT,
|
||||
.srcAccessMask = SwapchainWriterAccess(srcStages),
|
||||
.dstAccessMask = SwapchainPassAccess(dstStages),
|
||||
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
|
||||
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
|
||||
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
|
||||
|
|
|
|||
|
|
@ -336,11 +336,10 @@ export namespace WorkaroundNvidiaAS {
|
|||
export namespace Crafter {
|
||||
class VulkanShader {
|
||||
public:
|
||||
std::vector<VkSpecializationMapEntry> specilizations;
|
||||
VkSpecializationInfo* specilizationInfo;
|
||||
VkShaderStageFlagBits stage;
|
||||
std::string entrypoint;
|
||||
VkShaderModule shader;
|
||||
VkShaderModule shader = VK_NULL_HANDLE;
|
||||
// NVIDIA descriptor-heap AS-read workaround (issue #15 / #7): set when
|
||||
// this module read an acceleration structure and was rewritten to fetch
|
||||
// the TLAS device address from a push constant. `tlasPushOffset` is the
|
||||
|
|
@ -362,23 +361,29 @@ export namespace Crafter {
|
|||
if ((stage & rayTracingStages) != 0) {
|
||||
Device::RequireRayTracing();
|
||||
}
|
||||
// A file that can't be read throws, naming it: carrying on would
|
||||
// size the code from a failed tellg() (-1) and die in the
|
||||
// allocation, or hand the driver no code at all.
|
||||
std::ifstream file(path, std::ios::binary);
|
||||
if (!file) {
|
||||
std::cerr << "Error: Could not open file " << path << std::endl;
|
||||
throw std::runtime_error(std::format("VulkanShader: could not open {}", path.string()));
|
||||
}
|
||||
|
||||
|
||||
// Move to the end of the file to determine its size
|
||||
file.seekg(0, std::ios::end);
|
||||
std::streamsize size = file.tellg();
|
||||
file.seekg(0, std::ios::beg);
|
||||
|
||||
if (size <= 0 || size % sizeof(std::uint32_t) != 0) {
|
||||
throw std::runtime_error(std::format("VulkanShader: {} is not SPIR-V ({} bytes)", path.string(), size));
|
||||
}
|
||||
|
||||
std::vector<std::uint32_t> spirv(size / sizeof(std::uint32_t));
|
||||
|
||||
|
||||
// Read the data into the vector
|
||||
if (!file.read(reinterpret_cast<char*>(spirv.data()), size)) {
|
||||
std::cerr << "Error: Could not read data from file" << std::endl;
|
||||
throw std::runtime_error(std::format("VulkanShader: could not read {}", path.string()));
|
||||
}
|
||||
|
||||
|
||||
file.close();
|
||||
|
||||
// NVIDIA descriptor-heap AS-read workaround (issue #15 / #7).
|
||||
|
|
@ -397,6 +402,35 @@ export namespace Crafter {
|
|||
|
||||
Device::CheckVkResult(vkCreateShaderModule(Device::device, &module_info, nullptr, &shader));
|
||||
}
|
||||
|
||||
// The module is released through the deferred-deletion queue, so a
|
||||
// static VulkanShader makes no Vulkan call at exit. Copies would both
|
||||
// release it; a move hands it over.
|
||||
VulkanShader(const VulkanShader&) = delete;
|
||||
VulkanShader& operator=(const VulkanShader&) = delete;
|
||||
VulkanShader(VulkanShader&& other) noexcept
|
||||
: specilizationInfo(other.specilizationInfo)
|
||||
, stage(other.stage)
|
||||
, entrypoint(std::move(other.entrypoint))
|
||||
, shader(std::exchange(other.shader, VK_NULL_HANDLE))
|
||||
, patchedAS(other.patchedAS)
|
||||
, tlasPushOffset(other.tlasPushOffset) {
|
||||
}
|
||||
VulkanShader& operator=(VulkanShader&& other) noexcept {
|
||||
if (this != &other) {
|
||||
Device::EnqueueDeletion(shader);
|
||||
specilizationInfo = other.specilizationInfo;
|
||||
stage = other.stage;
|
||||
entrypoint = std::move(other.entrypoint);
|
||||
shader = std::exchange(other.shader, VK_NULL_HANDLE);
|
||||
patchedAS = other.patchedAS;
|
||||
tlasPushOffset = other.tlasPushOffset;
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
~VulkanShader() {
|
||||
Device::EnqueueDeletion(shader);
|
||||
}
|
||||
};
|
||||
}
|
||||
#endif // !CRAFTER_GRAPHICS_WINDOW_DOM
|
||||
|
|
|
|||
|
|
@ -273,7 +273,10 @@ export namespace Crafter {
|
|||
std::uint32_t gx, std::uint32_t gy = 1, std::uint32_t gz = 1);
|
||||
|
||||
// Allocates a heap slot for the buffer and registers the GPU handle.
|
||||
// Returns a move-only BufferSlot RAII handle.
|
||||
// Returns a move-only BufferSlot RAII handle. The slot points at the
|
||||
// buffer by its device address, so it needs one: a buffer made without
|
||||
// VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT is refused
|
||||
// (std::invalid_argument).
|
||||
template<typename T, bool Mapped>
|
||||
BufferSlot RegisterBuffer(GraphicsBuffer<T, Mapped>& buffer);
|
||||
|
||||
|
|
@ -441,6 +444,11 @@ export namespace Crafter {
|
|||
// ─── template-method implementations ────────────────────────────────
|
||||
template<typename T, bool Mapped>
|
||||
BufferSlot UIRenderer::RegisterBuffer(GraphicsBuffer<T, Mapped>& buffer) {
|
||||
#ifndef CRAFTER_GRAPHICS_WINDOW_DOM
|
||||
if (buffer.address == 0) {
|
||||
throw std::invalid_argument("UIRenderer::RegisterBuffer: the buffer has no device address (create it with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT)");
|
||||
}
|
||||
#endif
|
||||
auto range = heap_->AllocateBufferSlots(1);
|
||||
#ifndef CRAFTER_GRAPHICS_WINDOW_DOM
|
||||
WriteBufferDescriptor(range.firstElement, buffer.buffer, buffer.address, buffer.size);
|
||||
|
|
|
|||
|
|
@ -109,11 +109,17 @@ namespace Crafter {
|
|||
.memoryTypeIndex = memoryTypeIndex
|
||||
};
|
||||
|
||||
// Only a buffer created for SHADER_DEVICE_ADDRESS may have its
|
||||
// address queried (VUID-VkBufferDeviceAddressInfo-buffer-02601);
|
||||
// the others keep address 0.
|
||||
const bool addressable = (usageFlags & VK_BUFFER_USAGE_2_SHADER_DEVICE_ADDRESS_BIT) != 0;
|
||||
VkMemoryAllocateFlagsInfoKHR allocFlagsInfo {
|
||||
.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_FLAGS_INFO_KHR,
|
||||
.flags = VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT_KHR,
|
||||
};
|
||||
memAlloc.pNext = &allocFlagsInfo;
|
||||
if (addressable) {
|
||||
memAlloc.pNext = &allocFlagsInfo;
|
||||
}
|
||||
Device::CheckVkResult(vkAllocateMemory(Device::device, &memAlloc, nullptr, &memory));
|
||||
|
||||
Device::CheckVkResult(vkBindBufferMemory(Device::device, buffer, memory, 0));
|
||||
|
|
@ -122,14 +128,12 @@ namespace Crafter {
|
|||
.sType = VK_STRUCTURE_TYPE_BUFFER_DEVICE_ADDRESS_INFO,
|
||||
.buffer = buffer
|
||||
};
|
||||
address = vkGetBufferDeviceAddress(Device::device, &addressInfo);
|
||||
address = addressable ? vkGetBufferDeviceAddress(Device::device, &addressInfo) : 0;
|
||||
|
||||
// Record the allocation's byte size (≥ `size`) for every buffer,
|
||||
// not just mapped ones: UploadDeviceLocalRange's direct path flushes
|
||||
// only the dirty sub-range and clamps its rounded-up upper bound to
|
||||
// this (the nonCoherentAtomSize-aligned-end exception, see
|
||||
// AlignMappedFlushRange). A non-mapped buffer never maps persistently,
|
||||
// but its memory is still the allocation a transient map writes into.
|
||||
// Record the allocation's byte size (≥ `size`): a ranged
|
||||
// FlushDevice flushes only the dirty sub-range and clamps its
|
||||
// rounded-up upper bound to this (the nonCoherentAtomSize-aligned-end
|
||||
// exception, see AlignMappedFlushRange).
|
||||
mappedSize = memReqs.size;
|
||||
if constexpr(Mapped) {
|
||||
Device::CheckVkResult(vkMapMemory(Device::device, memory, 0, memReqs.size, 0, reinterpret_cast<void**>(&(VulkanBufferMappedConditional<T, true>::value))));
|
||||
|
|
@ -227,97 +231,62 @@ namespace Crafter {
|
|||
|
||||
// Upload `count` host elements from `src` into this buffer as
|
||||
// device-local, GPU-read geometry, choosing placement at runtime via
|
||||
// the #89 upload strategy (Device::PreferDirectDeviceWrite) and then
|
||||
// recording a barrier from the upload to (dstStageMask, dstAccessMask)
|
||||
// on `cmd`:
|
||||
// the #89 upload strategy (Device::PreferDirectDeviceWrite), and order
|
||||
// the upload before (dstStageMask, dstAccessMask) on `cmd`:
|
||||
// ReBAR / UMA (direct) — allocate HOST_VISIBLE with DEVICE_LOCAL as a
|
||||
// best-effort preference (GetMemoryType lands the host-visible
|
||||
// device-local type that the strategy already proved exists), map
|
||||
// transiently, memcpy, flush if the chosen type isn't coherent,
|
||||
// unmap. No staging buffer: it would be pure overhead on a bar where
|
||||
// the device-local heap is itself host-writable.
|
||||
// No / small BAR (staged) — allocate pure DEVICE_LOCAL (+ TRANSFER_DST),
|
||||
// fill a transient HOST_VISIBLE staging buffer, vkCmdCopyBuffer into
|
||||
// this buffer, then hand the staging buffer to the fence-keyed
|
||||
// deferred-deletion queue (#101/#102) so it outlives the copy submit.
|
||||
// Requires !Mapped: the destination must be free to be device-local-only
|
||||
// (a persistent map would force HOST_VISIBLE), so the direct path maps
|
||||
// just long enough to write. A same-size re-upload reuses the allocation
|
||||
// (Resize), so the device address stays stable across an in-place AS
|
||||
// UPDATE refit that re-calls this.
|
||||
// No / small BAR (staged) — allocate pure DEVICE_LOCAL and copy from
|
||||
// the staging ring on the GPU (CopyThroughStaging).
|
||||
// The direct write is only for a fresh allocation, which nothing can be
|
||||
// reading yet. A same-size re-upload reuses the allocation (Resize), so
|
||||
// the device address stays stable across an in-place AS UPDATE refit
|
||||
// that re-calls this, but a frame in flight may still read it: that
|
||||
// upload always copies, on the GPU timeline, after those reads.
|
||||
// dstStageMask must name every stage that reads the buffer, since the
|
||||
// next upload waits for them. Requires !Mapped: the destination must be
|
||||
// free to be device-local-only (a persistent map would force
|
||||
// HOST_VISIBLE), so the direct path maps just long enough to write.
|
||||
void UploadDeviceLocal(VkBufferUsageFlags2 usageFlags, const T* src, std::uint32_t count, VkCommandBuffer cmd, VkAccessFlags dstAccessMask, VkPipelineStageFlags dstStageMask) requires(!Mapped) {
|
||||
VkDeviceSize bytes = static_cast<VkDeviceSize>(count) * sizeof(T);
|
||||
VkAccessFlags srcAccessMask;
|
||||
VkPipelineStageFlags srcStageMask;
|
||||
if (Device::PreferDirectDeviceWrite(bytes)) {
|
||||
// Nothing to put anywhere: no buffer of no bytes, no copy of none.
|
||||
if (count == 0) return;
|
||||
const VkDeviceSize bytes = static_cast<VkDeviceSize>(count) * sizeof(T);
|
||||
// Any upload may be followed by one that copies in.
|
||||
usageFlags |= VK_BUFFER_USAGE_TRANSFER_DST_BIT;
|
||||
const bool direct = Device::PreferDirectDeviceWrite(bytes);
|
||||
const VkBuffer previous = buffer;
|
||||
if (direct) {
|
||||
Resize(usageFlags, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, count, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
|
||||
void* mapped = nullptr;
|
||||
Device::CheckVkResult(vkMapMemory(Device::device, memory, 0, VK_WHOLE_SIZE, 0, &mapped));
|
||||
std::memcpy(mapped, src, bytes);
|
||||
// Match FlushDevice()'s gate: a non-coherent type needs an
|
||||
// explicit flush before the device reads the written range.
|
||||
if (!(memoryPropertyFlagsChosen & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)) {
|
||||
VkMappedMemoryRange range {
|
||||
.sType = VK_STRUCTURE_TYPE_MAPPED_MEMORY_RANGE,
|
||||
.memory = memory,
|
||||
.offset = 0,
|
||||
.size = VK_WHOLE_SIZE
|
||||
};
|
||||
vkFlushMappedMemoryRanges(Device::device, 1, &range);
|
||||
}
|
||||
vkUnmapMemory(Device::device, memory);
|
||||
srcAccessMask = VK_ACCESS_HOST_WRITE_BIT;
|
||||
srcStageMask = VK_PIPELINE_STAGE_HOST_BIT;
|
||||
} else {
|
||||
Resize(usageFlags | VK_BUFFER_USAGE_TRANSFER_DST_BIT, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, count);
|
||||
// Persistent staging instead of a Create+DeferredClear per call
|
||||
// (issue #120): on no-/small-BAR hardware Mesh::Refit /
|
||||
// RecordProceduralBuild hit this branch every frame for deforming
|
||||
// meshes, so a fresh alloc+map+free cycle per upload was pure
|
||||
// churn on exactly the hardware this path targets. Keep a small
|
||||
// ring of staging buffers, each grown to a high-water mark and
|
||||
// reused — reallocated (by Resize, which defers the outgrown
|
||||
// allocation) only when a larger upload arrives.
|
||||
//
|
||||
// The ring is grown lazily, on first entry into this staged
|
||||
// branch, so ReBAR/UMA hardware (which always takes the direct
|
||||
// `if` branch above) never constructs a staging allocation it
|
||||
// does not use.
|
||||
//
|
||||
// Why a ring and not one shared buffer: the vkCmdCopyBuffer below
|
||||
// still reads the staging buffer after this call returns — the
|
||||
// reason the old code used DeferredClear. With frames pipelined
|
||||
// framesInFlight deep, overwriting one shared staging buffer next
|
||||
// frame would clobber data the previous frame's copy is still
|
||||
// reading. Indexing by frameCounter % framesInFlight gives each
|
||||
// in-flight frame its own slot; a slot is rewritten only after
|
||||
// framesInFlight frames have elapsed — the exact window after
|
||||
// which single-queue submission order (the same guarantee the
|
||||
// #101 deletion queue relies on) ensures that copy has completed.
|
||||
const std::uint32_t ringSize =
|
||||
Device::framesInFlight ? Device::framesInFlight : 1;
|
||||
if (stagingRing.size() < ringSize) {
|
||||
stagingRing.resize(ringSize);
|
||||
}
|
||||
VulkanBuffer<T, true>& staging =
|
||||
stagingRing[Device::frameCounter % ringSize];
|
||||
// SHADER_DEVICE_ADDRESS: Create always queries the buffer device
|
||||
// address (and allocates with the device-address bit); the
|
||||
// staging buffer's own address is otherwise unused. Resize reuses
|
||||
// the existing allocation (and its mapped pointer) when the
|
||||
// upload still fits the slot's high-water capacity.
|
||||
staging.Resize(VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, count);
|
||||
std::memcpy(staging.value, src, bytes);
|
||||
staging.FlushDevice();
|
||||
VkBufferCopy region { .srcOffset = 0, .dstOffset = 0, .size = bytes };
|
||||
vkCmdCopyBuffer(cmd, staging.buffer, buffer, 1, ®ion);
|
||||
srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
|
||||
srcStageMask = VK_PIPELINE_STAGE_TRANSFER_BIT;
|
||||
Resize(usageFlags, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, count);
|
||||
}
|
||||
if (!direct || buffer == previous) {
|
||||
CopyThroughStaging(src, 0, count, cmd, dstAccessMask, dstStageMask);
|
||||
return;
|
||||
}
|
||||
|
||||
void* mapped = nullptr;
|
||||
Device::CheckVkResult(vkMapMemory(Device::device, memory, 0, VK_WHOLE_SIZE, 0, &mapped));
|
||||
std::memcpy(mapped, src, bytes);
|
||||
// Match FlushDevice()'s gate: a non-coherent type needs an
|
||||
// explicit flush before the device reads the written range.
|
||||
if (!(memoryPropertyFlagsChosen & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)) {
|
||||
VkMappedMemoryRange range {
|
||||
.sType = VK_STRUCTURE_TYPE_MAPPED_MEMORY_RANGE,
|
||||
.memory = memory,
|
||||
.offset = 0,
|
||||
.size = VK_WHOLE_SIZE
|
||||
};
|
||||
vkFlushMappedMemoryRanges(Device::device, 1, &range);
|
||||
}
|
||||
vkUnmapMemory(Device::device, memory);
|
||||
|
||||
VkBufferMemoryBarrier barrier = {
|
||||
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
|
||||
.srcAccessMask = srcAccessMask,
|
||||
.srcAccessMask = VK_ACCESS_HOST_WRITE_BIT,
|
||||
.dstAccessMask = dstAccessMask,
|
||||
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
|
||||
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
|
||||
|
|
@ -325,79 +294,55 @@ namespace Crafter {
|
|||
.offset = 0,
|
||||
.size = VK_WHOLE_SIZE
|
||||
};
|
||||
vkCmdPipelineBarrier(cmd, srcStageMask, dstStageMask, 0, 0, NULL, 1, &barrier, 0, NULL);
|
||||
vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_HOST_BIT, dstStageMask, 0, 0, NULL, 1, &barrier, 0, NULL);
|
||||
}
|
||||
|
||||
// Re-upload only the half-open sub-range [offset, offset+count) of an
|
||||
// already-allocated device-local buffer — the dirty-range counterpart of
|
||||
// Re-upload only the half-open sub-range [offset, offset+count) of a
|
||||
// buffer UploadDeviceLocal made — the dirty-range counterpart of
|
||||
// UploadDeviceLocal. `src` points at the first changed element (not the
|
||||
// start of the whole array); `offset` is that element's index in this
|
||||
// buffer. The buffer must already exist at its full size (a prior
|
||||
// UploadDeviceLocal / Build sized it): this never Resizes, so the device
|
||||
// address stays stable and the untouched elements keep their last
|
||||
// contents. Only the dirty bytes are written + flushed (direct path) or
|
||||
// staged + copied (staged path), and the post-upload barrier covers only
|
||||
// that sub-range — so a deforming-mesh refit that nudges a few vertices
|
||||
// pays for those vertices, not a full-array re-upload + flush + copy (#119).
|
||||
//
|
||||
// The direct-vs-staged choice is read from the memory type the buffer was
|
||||
// actually allocated with (HOST_VISIBLE → map + write in place;
|
||||
// device-local-only → stage + GPU copy), NOT from the sub-range size: the
|
||||
// allocation is fixed, so a small dirty range must not be mis-routed to a
|
||||
// map of a non-host-visible buffer the way UploadDeviceLocal's size-based
|
||||
// PreferDirectDeviceWrite check would. The staged path's GPU copy needs
|
||||
// VK_BUFFER_USAGE_TRANSFER_DST_BIT on the destination — already present,
|
||||
// since UploadDeviceLocal only allocates device-local-only memory on the
|
||||
// staged path, where it adds that bit.
|
||||
// buffer. This never Resizes, so the device address stays stable and the
|
||||
// untouched elements keep their last contents. Only the dirty elements
|
||||
// are staged and copied, and the barriers cover only that sub-range — so
|
||||
// a deforming-mesh refit that nudges a few vertices pays for those
|
||||
// vertices, not a full-array re-upload (#119). The buffer is one a frame
|
||||
// in flight may still read, so the copy waits for its readers
|
||||
// (dstStageMask, as in UploadDeviceLocal) on the GPU, whatever memory
|
||||
// type it lives in.
|
||||
void UploadDeviceLocalRange(const T* src, std::uint32_t offset, std::uint32_t count, VkCommandBuffer cmd, VkAccessFlags dstAccessMask, VkPipelineStageFlags dstStageMask) requires(!Mapped) {
|
||||
VkDeviceSize byteOffset = static_cast<VkDeviceSize>(offset) * sizeof(T);
|
||||
VkDeviceSize bytes = static_cast<VkDeviceSize>(count) * sizeof(T);
|
||||
VkAccessFlags srcAccessMask;
|
||||
VkPipelineStageFlags srcStageMask;
|
||||
if (memoryPropertyFlagsChosen & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) {
|
||||
void* mapped = nullptr;
|
||||
Device::CheckVkResult(vkMapMemory(Device::device, memory, 0, VK_WHOLE_SIZE, 0, &mapped));
|
||||
std::memcpy(static_cast<std::byte*>(mapped) + byteOffset, src, bytes);
|
||||
// Non-coherent memory needs an explicit flush — but only of the
|
||||
// sub-range actually written, rounded outward to
|
||||
// nonCoherentAtomSize (and clamped to the allocation size).
|
||||
if (!(memoryPropertyFlagsChosen & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)) {
|
||||
MappedFlushRange r = AlignMappedFlushRange(
|
||||
byteOffset, bytes, Device::nonCoherentAtomSize, mappedSize);
|
||||
VkMappedMemoryRange range {
|
||||
.sType = VK_STRUCTURE_TYPE_MAPPED_MEMORY_RANGE,
|
||||
.memory = memory,
|
||||
.offset = r.offset,
|
||||
.size = r.size
|
||||
};
|
||||
vkFlushMappedMemoryRanges(Device::device, 1, &range);
|
||||
}
|
||||
vkUnmapMemory(Device::device, memory);
|
||||
srcAccessMask = VK_ACCESS_HOST_WRITE_BIT;
|
||||
srcStageMask = VK_PIPELINE_STAGE_HOST_BIT;
|
||||
} else {
|
||||
// Device-local-only: stage just the dirty elements and copy them
|
||||
// into place at byteOffset. The staging buffer outlives the queued
|
||||
// copy via the fence-keyed deletion queue (#101/#102), exactly as
|
||||
// the full-buffer staged path does.
|
||||
VulkanBuffer<T, true> staging;
|
||||
staging.Create(VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, count);
|
||||
std::memcpy(staging.value, src, bytes);
|
||||
staging.FlushDevice();
|
||||
VkBufferCopy region { .srcOffset = 0, .dstOffset = byteOffset, .size = bytes };
|
||||
vkCmdCopyBuffer(cmd, staging.buffer, buffer, 1, ®ion);
|
||||
staging.DeferredClear();
|
||||
srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
|
||||
srcStageMask = VK_PIPELINE_STAGE_TRANSFER_BIT;
|
||||
}
|
||||
CopyThroughStaging(src, offset, count, cmd, dstAccessMask, dstStageMask);
|
||||
}
|
||||
|
||||
// Order only the written sub-range before the consumer reads it. The
|
||||
// untouched bytes were made visible by their own prior upload's
|
||||
// barrier and are not written in this submit, so they need no fresh
|
||||
// dependency even though the build reads the whole buffer.
|
||||
VkBufferMemoryBarrier barrier = {
|
||||
// Copy `count` elements from `src` to element `offset` of this buffer on
|
||||
// the GPU timeline: after the reads by `dstStageMask` recorded earlier
|
||||
// or still in flight, and before the ones that follow. The bytes go
|
||||
// through this frame's slot of the staging ring first.
|
||||
void CopyThroughStaging(const T* src, std::uint32_t offset, std::uint32_t count, VkCommandBuffer cmd, VkAccessFlags dstAccessMask, VkPipelineStageFlags dstStageMask) requires(!Mapped) {
|
||||
// A copy or barrier of no bytes is invalid usage.
|
||||
if (count == 0) return;
|
||||
const VkDeviceSize byteOffset = static_cast<VkDeviceSize>(offset) * sizeof(T);
|
||||
const VkDeviceSize bytes = static_cast<VkDeviceSize>(count) * sizeof(T);
|
||||
|
||||
// Write after read: an execution dependency on the earlier readers.
|
||||
VkBufferMemoryBarrier readsDone = {
|
||||
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
|
||||
.srcAccessMask = srcAccessMask,
|
||||
.srcAccessMask = 0,
|
||||
.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
|
||||
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
|
||||
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
|
||||
.buffer = buffer,
|
||||
.offset = byteOffset,
|
||||
.size = bytes
|
||||
};
|
||||
vkCmdPipelineBarrier(cmd, dstStageMask, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0, NULL, 1, &readsDone, 0, NULL);
|
||||
|
||||
const auto [staging, stagingOffset] = Stage(src, count);
|
||||
VkBufferCopy region { .srcOffset = stagingOffset, .dstOffset = byteOffset, .size = bytes };
|
||||
vkCmdCopyBuffer(cmd, staging, buffer, 1, ®ion);
|
||||
|
||||
VkBufferMemoryBarrier copied = {
|
||||
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
|
||||
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
|
||||
.dstAccessMask = dstAccessMask,
|
||||
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
|
||||
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
|
||||
|
|
@ -405,7 +350,47 @@ namespace Crafter {
|
|||
.offset = byteOffset,
|
||||
.size = bytes
|
||||
};
|
||||
vkCmdPipelineBarrier(cmd, srcStageMask, dstStageMask, 0, 0, NULL, 1, &barrier, 0, NULL);
|
||||
vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_TRANSFER_BIT, dstStageMask, 0, 0, NULL, 1, &copied, 0, NULL);
|
||||
}
|
||||
|
||||
// Put `count` elements in this frame's slot of the staging ring, after
|
||||
// whatever this frame staged before, and say where: the slot's buffer
|
||||
// and the byte offset. A copy still in flight never reads the slot, as
|
||||
// a slot is used again only framesInFlight frames later — the window
|
||||
// after which single-queue submission order (the same guarantee the
|
||||
// #101 deletion queue relies on) ensures that copy has completed. Each
|
||||
// slot keeps its high-water size (issue #120); outgrowing it mid-frame
|
||||
// hands the old allocation, with this frame's earlier bytes, to the
|
||||
// deletion queue. Without a window no frame passes; after a
|
||||
// Device::DrainDeletions (which follows a wait for the queue) nothing
|
||||
// reads any slot, so they all start over.
|
||||
std::pair<VkBuffer, VkDeviceSize> Stage(const T* src, std::uint32_t count) requires(!Mapped) {
|
||||
const std::uint32_t ringSize = Device::framesInFlight ? Device::framesInFlight : 1;
|
||||
if (stagingRing.size() < ringSize) {
|
||||
stagingRing.resize(ringSize);
|
||||
stagingFrame.resize(ringSize, std::numeric_limits<std::uint64_t>::max());
|
||||
stagingUsed.resize(ringSize, 0);
|
||||
}
|
||||
if (stagingDrains != Device::drains) {
|
||||
stagingDrains = Device::drains;
|
||||
std::ranges::fill(stagingUsed, 0);
|
||||
}
|
||||
const std::size_t slot = Device::frameCounter % ringSize;
|
||||
if (stagingFrame[slot] != Device::frameCounter) {
|
||||
stagingFrame[slot] = Device::frameCounter;
|
||||
stagingUsed[slot] = 0;
|
||||
}
|
||||
VulkanBuffer<T, true>& staging = stagingRing[slot];
|
||||
const std::uint32_t needed = stagingUsed[slot] + count;
|
||||
if (staging.buffer == VK_NULL_HANDLE || static_cast<VkDeviceSize>(needed) * sizeof(T) > staging.capacity) {
|
||||
staging.Resize(VK_BUFFER_USAGE_TRANSFER_SRC_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, needed);
|
||||
stagingUsed[slot] = 0;
|
||||
}
|
||||
const std::uint32_t first = stagingUsed[slot];
|
||||
std::memcpy(staging.value + first, src, static_cast<std::size_t>(count) * sizeof(T));
|
||||
staging.FlushDevice(static_cast<VkDeviceSize>(first) * sizeof(T), static_cast<VkDeviceSize>(count) * sizeof(T));
|
||||
stagingUsed[slot] = first + count;
|
||||
return { staging.buffer, static_cast<VkDeviceSize>(first) * sizeof(T) };
|
||||
}
|
||||
|
||||
void FlushDevice() requires(Mapped) {
|
||||
|
|
@ -516,18 +501,31 @@ namespace Crafter {
|
|||
vkInvalidateMappedMemoryRanges(Device::device, 1, &range);
|
||||
}
|
||||
|
||||
// Persistent per-frame-in-flight staging ring for the staged
|
||||
// UploadDeviceLocal path (issue #120). Empty — and so holding zero
|
||||
// host-visible allocations — until the first upload that actually
|
||||
// stages, which only happens on no-/small-BAR hardware (ReBAR/UMA always
|
||||
// takes the direct-write branch and never touches this, so the lazy ring
|
||||
// keeps the change a no-op there). Sized to Device::framesInFlight so
|
||||
// each in-flight frame copies from its own slot; each slot grows to a
|
||||
// high-water mark via Resize and is reused across frames rather than
|
||||
// reallocated every call — mirroring the TLAS instance/metadata reuse.
|
||||
// Persistent per-frame-in-flight staging ring for the copies Stage
|
||||
// records (issue #120). Empty — and so holding zero host-visible
|
||||
// allocations — until the first upload that stages: on ReBAR/UMA only
|
||||
// re-uploads do, as a first upload writes directly. Sized to
|
||||
// Device::framesInFlight so each in-flight frame copies from its own
|
||||
// slot; each slot grows to a high-water mark via Resize and is reused
|
||||
// across frames rather than reallocated every call — mirroring the TLAS
|
||||
// instance/metadata reuse. stagingFrame and stagingUsed say, per slot,
|
||||
// which frame last staged into it and how many elements it has staged;
|
||||
// stagingDrains, the Device::drains it last saw.
|
||||
std::vector<VulkanBuffer<T, true>> stagingRing;
|
||||
std::vector<std::uint64_t> stagingFrame;
|
||||
std::vector<std::uint32_t> stagingUsed;
|
||||
std::uint64_t stagingDrains = 0;
|
||||
|
||||
VulkanBuffer(VulkanBuffer&& other) {
|
||||
VulkanBuffer(VulkanBuffer&& other) noexcept {
|
||||
*this = std::move(other);
|
||||
}
|
||||
|
||||
// Takes `other`'s buffer over. The one this held goes to the deletion
|
||||
// queue, as the destructor would send it: a frame in flight may still
|
||||
// read it.
|
||||
VulkanBuffer& operator=(VulkanBuffer&& other) noexcept {
|
||||
if (this == &other) return *this;
|
||||
DeferredClear();
|
||||
buffer = other.buffer;
|
||||
memory = other.memory;
|
||||
size = other.size;
|
||||
|
|
@ -538,15 +536,21 @@ namespace Crafter {
|
|||
other.buffer = VK_NULL_HANDLE;
|
||||
address = other.address;
|
||||
stagingRing = std::move(other.stagingRing);
|
||||
stagingFrame = std::move(other.stagingFrame);
|
||||
stagingUsed = std::move(other.stagingUsed);
|
||||
stagingDrains = other.stagingDrains;
|
||||
if constexpr(Mapped) {
|
||||
VulkanBufferMappedConditional<T, true>::value = other.VulkanBufferMappedConditional<T, true>::value;
|
||||
}
|
||||
};
|
||||
return *this;
|
||||
}
|
||||
|
||||
// Deferred, like DeferredClear: the owner may go while a frame in
|
||||
// flight still reads the buffer, and a static owner goes after the
|
||||
// driver has torn down. Call Clear() first to free at once when the
|
||||
// GPU is known to be idle. The staging ring's buffers defer the same way.
|
||||
~VulkanBuffer() {
|
||||
if(buffer != VK_NULL_HANDLE) {
|
||||
Clear();
|
||||
}
|
||||
DeferredClear();
|
||||
}
|
||||
|
||||
VulkanBuffer(VulkanBuffer&) = delete;
|
||||
|
|
|
|||
|
|
@ -56,6 +56,21 @@ export namespace Crafter {
|
|||
struct DescriptorHeapWebGPU;
|
||||
#endif
|
||||
|
||||
// How a window hands finished frames to the display. Fifo waits for the
|
||||
// display's refresh; it is the default and the one mode every surface
|
||||
// has. Mailbox replaces a frame still waiting to be shown, so presenting
|
||||
// never waits, which suits a window that updates on its own clock
|
||||
// (Window::updateInterval). Immediate shows each frame at once and may
|
||||
// tear. A mode the surface lacks falls back to Fifo, as does one whose
|
||||
// swapchain the driver makes with more images than the frame loop's
|
||||
// Window::numFrames (Mesa's Mailbox has at least four). In the browser
|
||||
// the page presents the canvas and the mode has no effect.
|
||||
enum class PresentMode {
|
||||
Fifo,
|
||||
Mailbox,
|
||||
Immediate,
|
||||
};
|
||||
|
||||
struct Window {
|
||||
FrameTime currentFrameTime;
|
||||
std::uint32_t width;
|
||||
|
|
@ -71,6 +86,17 @@ export namespace Crafter {
|
|||
Event<void> onResize;
|
||||
bool open = true;
|
||||
bool updating = false;
|
||||
// Zero (the default): Update runs when the compositor's frame
|
||||
// callback says the window is about to show a new frame, so it
|
||||
// follows the display and stops while the window is hidden or
|
||||
// minimized. Non-zero: Update runs on this clock instead, visible or
|
||||
// not, for an app whose output goes somewhere besides the screen.
|
||||
// Pair it with PresentMode::Mailbox so a present never waits on the
|
||||
// display. Set it before StartUpdate. Native only: in the browser,
|
||||
// requestAnimationFrame drives Update.
|
||||
std::chrono::nanoseconds updateInterval{0};
|
||||
// The mode SetPresentMode last asked for.
|
||||
PresentMode presentMode = PresentMode::Fifo;
|
||||
// Currently-pressed raw key codes. The runtime stores raw platform
|
||||
// codes only (Win32 PS/2 scancode + extended bit; Wayland kernel
|
||||
// keycode). Cross-platform default bindings use `Key(CrafterKeys::X)`
|
||||
|
|
@ -156,6 +182,11 @@ export namespace Crafter {
|
|||
|
||||
void StartUpdate();
|
||||
void StopUpdate();
|
||||
// Makes the swapchain again with the new mode and fires onResize, as
|
||||
// a resize does, so call it outside StartInit/FinishInit. The
|
||||
// environment variable CRAFTER_PRESENT_IMMEDIATE overrides the mode,
|
||||
// for benchmarks.
|
||||
void SetPresentMode(PresentMode mode);
|
||||
void SetTitle(const std::string_view title);
|
||||
// The xdg app id, which the compositor matches to a .desktop file
|
||||
// for the icon, the task switcher and activation.
|
||||
|
|
@ -208,6 +239,21 @@ export namespace Crafter {
|
|||
#endif
|
||||
|
||||
#ifdef CRAFTER_GRAPHICS_WINDOW_WAYLAND
|
||||
// Wakes the event loop when `fd` becomes readable and calls
|
||||
// `onReadable` from it, which must consume what made the descriptor
|
||||
// readable, or the loop wakes again at once. For a socket, or an
|
||||
// eventfd another thread signals, so new data can drive an Update
|
||||
// right away (call Update from the callback) instead of waiting for
|
||||
// the next one. StartSync's loop serves the descriptors of every
|
||||
// window. Remove a descriptor before closing it.
|
||||
void AddPollFd(int fd, std::function<void()> onReadable);
|
||||
void RemovePollFd(int fd);
|
||||
struct PollFd {
|
||||
int fd;
|
||||
std::function<void()> onReadable;
|
||||
};
|
||||
std::vector<PollFd> pollFds;
|
||||
|
||||
float scale = 1.0f;
|
||||
bool configured = false;
|
||||
// Pending size from the most recent xdg_toplevel.configure, in
|
||||
|
|
@ -273,14 +319,28 @@ export namespace Crafter {
|
|||
// stay valid. Does NOT fire onResize — callers do that.
|
||||
void RecreateSwapchainAndImages();
|
||||
|
||||
// Save the current swapchain image (state after Render() returns) to
|
||||
// a PNG file. Allocates a one-shot staging buffer + command buffer,
|
||||
// copies image-to-buffer, waits idle, then writes PNG via stb. Useful
|
||||
// for visual regression tests and screenshotting from headless code.
|
||||
void SaveFrame(const std::filesystem::path& path);
|
||||
// Saves the frame the next Render() draws to a PNG at `path`, for
|
||||
// visual regression tests and screenshots: that Render copies the
|
||||
// swapchain image into a readback buffer before presenting it, waits
|
||||
// for the frame, and writes the file. The copy can't come later: a
|
||||
// presented image belongs to the presentation engine until it is
|
||||
// acquired again.
|
||||
void CaptureNextFrame(std::filesystem::path path);
|
||||
std::optional<std::filesystem::path> pendingCapture;
|
||||
static constexpr std::uint8_t numFrames = 3;
|
||||
VkSurfaceKHR vulkanSurface = VK_NULL_HANDLE;
|
||||
VkSwapchainKHR swapChain = VK_NULL_HANDLE;
|
||||
// What the swapchain images were created for: STORAGE | TRANSFER_SRC,
|
||||
// plus TRANSFER_DST where the surface allows it, so a pass can blit or
|
||||
// clear into the image instead of only writing it from a shader.
|
||||
VkImageUsageFlags swapchainUsage = 0;
|
||||
// The mode the swapchain was made with: presentMode, unless the
|
||||
// surface lacks it or CRAFTER_PRESENT_IMMEDIATE is set.
|
||||
VkPresentModeKHR swapchainPresentMode = VK_PRESENT_MODE_FIFO_KHR;
|
||||
// The mode presentMode asks for made a swapchain with more images than
|
||||
// numFrames once, so swapchains are FIFO until SetPresentMode is
|
||||
// called again, rather than being made twice at every resize.
|
||||
bool presentModeUnfit = false;
|
||||
VkFormat colorFormat;
|
||||
VkColorSpaceKHR colorSpace;
|
||||
// Supported composite-alpha mode for vulkanSurface. A surface property
|
||||
|
|
|
|||
Loading…
Reference in a new issue