[perf] Runtime upload-strategy helper: direct HOST_VISIBLE|DEVICE_LOCAL vs staging (ReBAR-aware)

jorijnvdgraaf commented

2026-06-16 18:55:02 +02:00

Owner

claude:done

Subsystem: Device / VulkanBuffer — upload strategy
Impact: Enabler (unifies the memory-placement cluster) · Effort: Medium
Depends on: #59 (GetMemoryType preferred/fallback) · Unblocks: #65, #72, #73, #75

Problem

Several issues want to put a CPU-written, GPU-read buffer "where the GPU reads it fastest." The right strategy is platform-dependent and currently each issue re-derives it:

On ReBAR / UMA, the entire DEVICE_LOCAL heap is HOST_VISIBLE, so you map a DEVICE_LOCAL|HOST_VISIBLE allocation and write straight into VRAM — a staging buffer + copy is pure overhead.
On a non-ReBAR discrete GPU, DEVICE_LOCAL|HOST_VISIBLE is a small (~256 MB) BAR window on a separate heap; allocating large buffers there competes for the window and throws (#58). There, staging into pure DEVICE_LOCAL is mandatory.

Verified on this dev machine (discrete GPU): memory type 4 is DEVICE_LOCAL|HOST_VISIBLE|HOST_COHERENT on heap 0 — the full 24 GiB device-local heap (ReBAR on). The CPU/UMA device is a single 125 GiB heap with all flags. So on both, #73's proposed staging buffer would be wasted; but the decision must be made at runtime, not hardcoded.

Proposed fix — runtime upload-strategy helper

// true  -> allocate HOST_VISIBLE|DEVICE_LOCAL, map, write directly (no staging)
// false -> allocate pure DEVICE_LOCAL + HOST_VISIBLE staging + vkCmdCopyBuffer
bool Device::PreferDirectDeviceWrite(VkDeviceSize size);

Logic:

Find a DEVICE_LOCAL|HOST_VISIBLE type (via #59). None -> return false (must stage).
barHeap = heaps[type.heapIndex].size; vram = largest DEVICE_LOCAL heap.
ReBAR/UMA (barHeap >= ~0.9 * vram) -> true (direct map+write).
Small BAR window (barHeap << vram) -> true only for small/hot buffers under a budget threshold; false for large buffers -> stage.

API shape — three outcomes, not two (caller must constrain the fallback)

A bool direct-vs-stage return is insufficient. There are three outcomes, and which fallback is correct when no combined type exists depends on the caller's write pattern, which the helper does not know:

enum class UploadStrategy { DirectDeviceVisible, Staged, PlainHostVisible };
// allowStaging=false forbids the Staged branch for per-frame-mutated / co-written buffers
UploadStrategy Device::PlanUpload(VkDeviceSize size, bool allowStaging);

Write-once buffers (e.g. SBT #72) pass allowStaging=true — staging into DEVICE_LOCAL is a clean one-time fallback.
Per-frame-rewritten / co-write buffers (#65 instance co-write, #75 metadata, #73 deforming) pass allowStaging=false — when no combined type exists they must fall back to PlainHostVisible, because a per-frame stage+copy defeats the purpose (and clobbers GPU co-writes for #65).
Readback (#74) does not use this helper at all (wants HOST_CACHED).

Ideally a thin abstraction in VulkanBuffer (an upload-intent Create/Resize that hides the staging entirely from callers), so #65/#72/#73/#75 just declare intent and the buffer picks direct-vs-staged.

Correctness caveats

Budget, not just heap size. For real robustness, consult VK_EXT_memory_budget for remaining budget on a small BAR window rather than total size, so a busy window is not over-committed.
This helper is for the upload direction (CPU-write -> GPU-read). It is the wrong tool for readback (#74), which wants HOST_CACHED (type 3 here) — the ReBAR type is coherent-not-cached, so CPU reads from it are slow. Keep the two paths distinct.
The staged path carries the usual lifetime hazard (staging buffer must outlive the copy submit — fence tracking), same as #67.
Composes with #59: #59 selects the memory type; this helper selects the strategy.

0 comments

claude:done **Subsystem:** Device / VulkanBuffer — upload strategy **Impact:** Enabler (unifies the memory-placement cluster) · **Effort:** Medium **Depends on:** #59 (GetMemoryType preferred/fallback) · **Unblocks:** #65, #72, #73, #75 ### Problem Several issues want to put a CPU-written, GPU-read buffer "where the GPU reads it fastest." The right strategy is platform-dependent and currently each issue re-derives it: - On **ReBAR / UMA**, the entire `DEVICE_LOCAL` heap is `HOST_VISIBLE`, so you map a `DEVICE_LOCAL|HOST_VISIBLE` allocation and write straight into VRAM — a staging buffer + copy is **pure overhead**. - On a **non-ReBAR discrete GPU**, `DEVICE_LOCAL|HOST_VISIBLE` is a small (~256 MB) BAR window on a *separate* heap; allocating large buffers there competes for the window and throws (#58). There, staging into pure `DEVICE_LOCAL` is **mandatory**. Verified on this dev machine (discrete GPU): memory type 4 is `DEVICE_LOCAL|HOST_VISIBLE|HOST_COHERENT` on heap 0 — the **full 24 GiB** device-local heap (ReBAR on). The CPU/UMA device is a single 125 GiB heap with all flags. So on both, #73's proposed staging buffer would be wasted; but the decision must be made at runtime, not hardcoded. ### Proposed fix — runtime upload-strategy helper ```cpp // true -> allocate HOST_VISIBLE|DEVICE_LOCAL, map, write directly (no staging) // false -> allocate pure DEVICE_LOCAL + HOST_VISIBLE staging + vkCmdCopyBuffer bool Device::PreferDirectDeviceWrite(VkDeviceSize size); ``` Logic: 1. Find a `DEVICE_LOCAL|HOST_VISIBLE` type (via #59). None -> return false (must stage). 2. `barHeap = heaps[type.heapIndex].size`; `vram = largest DEVICE_LOCAL heap`. 3. ReBAR/UMA (`barHeap >= ~0.9 * vram`) -> true (direct map+write). 4. Small BAR window (`barHeap << vram`) -> true only for small/hot buffers under a budget threshold; false for large buffers -> stage. ### API shape — three outcomes, not two (caller must constrain the fallback) A `bool` direct-vs-stage return is insufficient. There are **three** outcomes, and which fallback is correct when no combined type exists depends on the caller's write pattern, which the helper does not know: ```cpp enum class UploadStrategy { DirectDeviceVisible, Staged, PlainHostVisible }; // allowStaging=false forbids the Staged branch for per-frame-mutated / co-written buffers UploadStrategy Device::PlanUpload(VkDeviceSize size, bool allowStaging); ``` - **Write-once buffers** (e.g. SBT #72) pass `allowStaging=true` — staging into `DEVICE_LOCAL` is a clean one-time fallback. - **Per-frame-rewritten / co-write buffers** (#65 instance co-write, #75 metadata, #73 deforming) pass `allowStaging=false` — when no combined type exists they must fall back to `PlainHostVisible`, because a per-frame stage+copy defeats the purpose (and clobbers GPU co-writes for #65). - **Readback** (#74) does not use this helper at all (wants `HOST_CACHED`). Ideally a thin abstraction in `VulkanBuffer` (an upload-intent `Create`/`Resize` that hides the staging entirely from callers), so #65/#72/#73/#75 just declare intent and the buffer picks direct-vs-staged. ### Correctness caveats 1. **Budget, not just heap size.** For real robustness, consult `VK_EXT_memory_budget` for *remaining* budget on a small BAR window rather than total size, so a busy window is not over-committed. 2. This helper is for the **upload** direction (CPU-write -> GPU-read). It is the wrong tool for **readback** (#74), which wants `HOST_CACHED` (type 3 here) — the ReBAR type is coherent-not-cached, so CPU reads from it are slow. Keep the two paths distinct. 3. The staged path carries the usual lifetime hazard (staging buffer must outlive the copy submit — fence tracking), same as #67. 4. Composes with #59: #59 selects the memory *type*; this helper selects the *strategy*. 0 comments

jorijnvdgraaf referenced this issue

2026-06-16 18:55:23 +02:00

[perf] Mesh vertex/index/aabb are HOST_VISIBLE — place device-local (direct on ReBAR, staged otherwise) #73

jorijnvdgraaf added the

claude:ready

label

2026-06-16 19:09:48 +02:00