CRAFTER_GRAPHICS_VALIDATION without the validation layer runs unvalidated without a word, so every validation check passes with 0 errors #189

New issue

Open

opened 2026-11-01 01:01:28 +00:00 by jorijnvdgraaf · 0 comments

jorijnvdgraaf commented

2026-11-01 01:01:28 +00:00

Owner

Summary

Device::Initialize treats CRAFTER_GRAPHICS_VALIDATION as optional. If VK_LAYER_KHRONOS_validation is not among the instance layers, it creates the instance without it and reports nothing. Device::validationErrorCount then stays at 0, and every test that gates a validation check on the variable passes without anything having been validated.

That is the situation with every released crafter-build today. Stable and beta 5.0.0 have no Khronos layer in their sandbox, and flatpak run clears VK_LAYER_PATH and VK_ADD_LAYER_PATH, so a layer installed on the host cannot stand in for it. A distro without its validation-layers package gives the same silent result.

Line numbers refer to 3f646ec.

Where

implementations/Crafter.Graphics-Device.cpp:1184 reads the variable, and :1196 chains VkValidationFeaturesEXT (GPU-AV) into the instance create info.
:1232 enumerates the instance layers and counts the ones in layerNames.
:1248 enables them only when enableValidation && foundInstanceLayers >= ... holds. Otherwise vkCreateInstance at :1254 runs with no layer, and the chained validation features do nothing.

Reproduction

Measured on 2026-11-01 (RTX 4090, NVIDIA 615.78.08, KDE Wayland). Reintroduce #157, a readback recorded after vkQueuePresentKHR, then run CRAFTER_GRAPHICS_VALIDATION=1 crafter-build test SwapchainTransferDst:

crafter-build stable or beta 5.0.0: the test passes, with "0 seen".
A crafter-build that bundles the layer: the test fails with FAIL no Vulkan validation errors while clearing and capturing (1 seen), and the layer reports vkQueueSubmit(): pSubmits[0] performs a layout transition on presentable VkImage 0xe000000000e, but the image has not been acquired from VkSwapchainKHR 0xd000000000d (either never or since the last present operation)., the same as a run on the host. That build is crafter-build branch claude/flatpak-debug-tools, installed locally as Flatpak branch local and not released yet.

Expected

When the variable is set and the layer cannot be enabled, Device::Initialize should fail with a message that names the layer, rather than run unvalidated. The variable asks for validation, and an unvalidated run cannot answer that question.

AGENTS.template.md assumes the layer is there:

line 61, CRAFTER_GRAPHICS_VALIDATION=1 crafter-build test;
line 66, the definition of done: "crafter-build test passes on the host with CRAFTER_GRAPHICS_VALIDATION set";
line 82: checks are gated on the variable "since without the layer the count is always zero".

crafter-build is adding the layer, GPU-AV included, to its Flatpak (branch claude/flatpak-debug-tools). That makes the variable work inside the sandbox once released, but does not change what happens wherever the layer is missing.

## Summary `Device::Initialize` treats `CRAFTER_GRAPHICS_VALIDATION` as optional. If `VK_LAYER_KHRONOS_validation` is not among the instance layers, it creates the instance without it and reports nothing. `Device::validationErrorCount` then stays at 0, and every test that gates a validation check on the variable passes without anything having been validated. That is the situation with every released crafter-build today. Stable and beta 5.0.0 have no Khronos layer in their sandbox, and `flatpak run` clears `VK_LAYER_PATH` and `VK_ADD_LAYER_PATH`, so a layer installed on the host cannot stand in for it. A distro without its validation-layers package gives the same silent result. Line numbers refer to `3f646ec`. ## Where - `implementations/Crafter.Graphics-Device.cpp:1184` reads the variable, and `:1196` chains `VkValidationFeaturesEXT` (GPU-AV) into the instance create info. - `:1232` enumerates the instance layers and counts the ones in `layerNames`. - `:1248` enables them only when `enableValidation && foundInstanceLayers >= ...` holds. Otherwise `vkCreateInstance` at `:1254` runs with no layer, and the chained validation features do nothing. ## Reproduction Measured on 2026-11-01 (RTX 4090, NVIDIA 615.78.08, KDE Wayland). Reintroduce #157, a readback recorded after `vkQueuePresentKHR`, then run `CRAFTER_GRAPHICS_VALIDATION=1 crafter-build test SwapchainTransferDst`: - crafter-build stable or beta 5.0.0: the test passes, with "0 seen". - A crafter-build that bundles the layer: the test fails with `FAIL no Vulkan validation errors while clearing and capturing (1 seen)`, and the layer reports `vkQueueSubmit(): pSubmits[0] performs a layout transition on presentable VkImage 0xe000000000e, but the image has not been acquired from VkSwapchainKHR 0xd000000000d (either never or since the last present operation).`, the same as a run on the host. That build is crafter-build branch `claude/flatpak-debug-tools`, installed locally as Flatpak branch `local` and not released yet. ## Expected When the variable is set and the layer cannot be enabled, `Device::Initialize` should fail with a message that names the layer, rather than run unvalidated. The variable asks for validation, and an unvalidated run cannot answer that question. `AGENTS.template.md` assumes the layer is there: - line 61, `CRAFTER_GRAPHICS_VALIDATION=1 crafter-build test`; - line 66, the definition of done: "`crafter-build test` passes on the host with `CRAFTER_GRAPHICS_VALIDATION` set"; - line 82: checks are gated on the variable "since without the layer the count is always zero". ## Related crafter-build is adding the layer, GPU-AV included, to its Flatpak (branch `claude/flatpak-debug-tools`). That makes the variable work inside the sandbox once released, but does not change what happens wherever the layer is missing.