Repository navigation
[Bug] Vulkan Backend Produces Distorted / Gibrish Images with --vae-conv-direct in Higher Resolutions #1673
Description
Git commit
stable-diffusion.cpp version master-709-92a3b73, commit 92a3b73
Operating System & Version
Ubuntu 24.04
GGML backends
Vulkan
Command-line arguments used
./sd-cli --mode img_gen --diffusion-model /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf --vae /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors --llm /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf --lora-model-dir /home/tipu/AI/image/loras/ --lora-apply-mode immediately --cfg-scale 1 -H 1024 -W 1024 --steps 7 --seed 765675567 --rng cpu --scheduler smoothstep --sampling-method euler --flow-shift 2 -o output_$(date +%Y-%m-%d_%H-%M-%S).png --fa --vae-conv-direct -p "A cozy study with walls covered in flowing ink sketches and soft watercolor washes. At the center, a circular tear in the paper wall opens upward like a magical portal, revealing a lush 3D meadow bathed in gentle morning sunlight. The curled edge of the torn paper casts soft shadows onto the drawn walls. A playful fox leaps out from the portal: its front legs and head fully in the 3D meadow, mid-air body partially transitioning into sketch lines, tail still a delicate ink drawing. Everyday objects in the room—like a drawn teacup and floating books—cast subtle realistic shadows, blending the dimensions. Soft pastel palette, visible paper texture, high detail, whimsical yet believable '2D-to-3D transition' realism, dynamic motion, enchanting and tactile atmosphere."
Steps to reproduce
First I produced image which was 512*512 with --vae-conv-direct all good.
./sd-cli --mode img_gen --diffusion-model /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf --vae /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors --llm /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf --lora-model-dir /home/tipu/AI/image/loras/ --lora-apply-mode immediately --cfg-scale 1 -H 512 -W 512 --steps 7 --seed 765675567 --rng cpu --scheduler smoothstep --sampling-method euler --flow-shift 2 -o output_$(date +%Y-%m-%d_%H-%M-%S).png --fa --vae-conv-direct -p "A cozy study with walls covered in flowing ink sketches and soft watercolor washes. At the center, a circular tear in the paper wall opens upward like a magical portal, revealing a lush 3D meadow bathed in gentle morning sunlight. The curled edge of the torn paper casts soft shadows onto the drawn walls. A playful fox leaps out from the portal: its front legs and head fully in the 3D meadow, mid-air body partially transitioning into sketch lines, tail still a delicate ink drawing. Everyday objects in the room—like a drawn teacup and floating books—cast subtle realistic shadows, blending the dimensions. Soft pastel palette, visible paper texture, high detail, whimsical yet believable '2D-to-3D transition' realism, dynamic motion, enchanting and tactile atmosphere."
Increased the resolution to 1024*1024 with --vae-conv-direct image becomes all distorted
./sd-cli --mode img_gen --diffusion-model /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf --vae /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors --llm /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf --lora-model-dir /home/tipu/AI/image/loras/ --lora-apply-mode immediately --cfg-scale 1 -H 1024 -W 1024 --steps 7 --seed 765675567 --rng cpu --scheduler smoothstep --sampling-method euler --flow-shift 2 -o output_$(date +%Y-%m-%d_%H-%M-%S).png --fa --vae-conv-direct -p "A cozy study with walls covered in flowing ink sketches and soft watercolor washes. At the center, a circular tear in the paper wall opens upward like a magical portal, revealing a lush 3D meadow bathed in gentle morning sunlight. The curled edge of the torn paper casts soft shadows onto the drawn walls. A playful fox leaps out from the portal: its front legs and head fully in the 3D meadow, mid-air body partially transitioning into sketch lines, tail still a delicate ink drawing. Everyday objects in the room—like a drawn teacup and floating books—cast subtle realistic shadows, blending the dimensions. Soft pastel palette, visible paper texture, high detail, whimsical yet believable '2D-to-3D transition' realism, dynamic motion, enchanting and tactile atmosphere."
Keep the resolution to 1024*1024. Remove the --vae-conv-direct and replace with --vae-on-cpu output image is as required
./sd-cli --mode img_gen --diffusion-model /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf --vae /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors --llm /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf --lora-model-dir /home/tipu/AI/image/loras/ --lora-apply-mode immediately --cfg-scale 1 -H 1024 -W 1024 --steps 7 --seed 765675567 --rng cpu --scheduler smoothstep --sampling-method euler --flow-shift 2 -o output_$(date +%Y-%m-%d_%H-%M-%S).png --fa --vae-on-cpu -p "A cozy study with walls covered in flowing ink sketches and soft watercolor washes. At the center, a circular tear in the paper wall opens upward like a magical portal, revealing a lush 3D meadow bathed in gentle morning sunlight. The curled edge of the torn paper casts soft shadows onto the drawn walls. A playful fox leaps out from the portal: its front legs and head fully in the 3D meadow, mid-air body partially transitioning into sketch lines, tail still a delicate ink drawing. Everyday objects in the room—like a drawn teacup and floating books—cast subtle realistic shadows, blending the dimensions. Soft pastel palette, visible paper texture, high detail, whimsical yet believable '2D-to-3D transition' realism, dynamic motion, enchanting and tactile atmosphere."
What you expected to happen
Image should be normal as it is by replacing --vae-conv-direct and replace with --vae-on-cpu
What actually happened
gibberish / distorted images are produced. As earlier shared in #1666
Logs / error messages / stack trace
1024*1024 generation with distorted image.
logs
bash ./sd-cli --mode img_gen --diffusion-model /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf --vae /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors --llm /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf --lora-model-dir /home/tipu/AI/image/loras/ --lora-apply-mode immediately --cfg-scale 1 -H 1024 -W 1024 --steps 7 --seed 765675567 --rng cpu --scheduler smoothstep --sampling-method euler --flow-shift 2 -o output_$(date +%Y-%m-%d_%H-%M-%S).png --fa --vae-conv-direct -p "A cozy study with walls covered in flowing ink sketches and soft watercolor washes. At the center, a circular tear in the paper wall opens upward like a magical portal, revealing a lush 3D meadow bathed in gentle morning sunlight. The curled edge of the torn paper casts soft shadows onto the drawn walls. A playful fox leaps out from the portal: its front legs and head fully in the 3D meadow, mid-air body partially transitioning into sketch lines, tail still a delicate ink drawing. Everyday objects in the room—like a drawn teacup and floating books—cast subtle realistic shadows, blending the dimensions. Soft pastel palette, visible paper texture, high detail, whimsical yet believable '2D-to-3D transition' realism, dynamic motion, enchanting and tactile atmosphere."
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV RENOIR) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
[INFO ] stable-diffusion.cpp:397 - loading diffusion model from '/home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf'
[INFO ] model_loader.cpp:235 - load /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:459 - loading llm from '/home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf'
[INFO ] model_loader.cpp:235 - load /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:473 - loading vae from '/home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors'
[INFO ] model_loader.cpp:238 - load /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:524 - Version: Z-Image
[INFO ] stable-diffusion.cpp:551 - Weight type stat: f32: 634 | q8_0: 433 | bf16: 28
[INFO ] stable-diffusion.cpp:552 - Conditioner weight type stat: f32: 145 | q8_0: 253
[INFO ] stable-diffusion.cpp:553 - Diffusion model weight type stat: f32: 245 | q8_0: 180 | bf16: 28
[INFO ] stable-diffusion.cpp:554 - VAE weight type stat: f32: 244
[INFO ] stable-diffusion.cpp:955 - using VAE for encoding / decoding
[INFO ] auto_encoder_kl.hpp:527 - vae decoder: ch = 128
[INFO ] stable-diffusion.cpp:991 - Using Conv2d direct in the vae model
[INFO ] stable-diffusion.cpp:1059 - Using flash attention
[INFO ] stable-diffusion.cpp:1073 - Using flash attention in the diffusion model
[INFO ] stable-diffusion.cpp:1181 - total params memory size = 11131.89MB (VRAM 11131.89MB, RAM 0.00MB): text_encoders 4076.43MB(VRAM), diffusion_model 6891.46MB(VRAM), vae 164.00MB(VRAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A)
[INFO ] stable-diffusion.cpp:1281 - running in FLOW mode
[INFO ] stable-diffusion.cpp:4432 - generate_image 1024x1024
[INFO ] denoiser.hpp:607 - get_sigmas with SmoothStep scheduler
[INFO ] stable-diffusion.cpp:3456 - sampling using Euler method
|##################################################| 386/386 - 2.42GB/s
[INFO ] model_loader.cpp:1224 - loading tensors completed, taking 1.60s (read: 0.15s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 1.17s)
[INFO ] stable-diffusion.cpp:4171 - get_learned_condition completed, taking 2.77s
[INFO ] stable-diffusion.cpp:4478 - generating image: 1/1 - seed 765675567
|##################################################| 452/452 - 3.04GB/s
[INFO ] model_loader.cpp:1224 - loading tensors completed, taking 2.21s (read: 0.29s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 1.54s)
|==================================================| 7/7 - 64.16s/it
[INFO ] stable-diffusion.cpp:4510 - sampling completed, taking 440.58s
[INFO ] stable-diffusion.cpp:4522 - generating 1 latent images completed, taking 440.58s
[INFO ] stable-diffusion.cpp:4195 - decoding 1 latents
|##################################################| 138/138 - 940.30MB/s
[INFO ] model_loader.cpp:1224 - loading tensors completed, taking 0.20s (read: 0.00s, memcpy: 0.00s, convert: 0.05s, copy_to_backend: 0.10s)
[INFO ] stable-diffusion.cpp:4215 - latent 1 decoded, taking 25.17s
[INFO ] stable-diffusion.cpp:4219 - decode_first_stage completed, taking 25.17s
[INFO ] stable-diffusion.cpp:4659 - generate_image completed in 468.55s
[INFO ] main.cpp:463 - save result image 0 to 'output_2026-06-17_10-20-18.png' (success)
[INFO ] main.cpp:535 - 1/1 images saved
1024*1024 Working image
Details
./sd-cli --mode img_gen --diffusion-model /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf --vae /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors --llm /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf --lora-model-dir /home/tipu/AI/image/loras/ --lora-apply-mode immediately --cfg-scale 1 -H 1024 -W 1024 --steps 7 --seed 765675567 --rng cpu --scheduler smoothstep --sampling-method euler --flow-shift 2 -o output_$(date +%Y-%m-%d_%H-%M-%S).png --fa --vae-on-cpu -p "A cozy study with walls covered in flowing ink sketches and soft watercolor washes. At the center, a circular tear in the paper wall opens upward like a magical portal, revealing a lush 3D meadow bathed in gentle morning sunlight. The curled edge of the torn paper casts soft shadows onto the drawn walls. A playful fox leaps out from the portal: its front legs and head fully in the 3D meadow, mid-air body partially transitioning into sketch lines, tail still a delicate ink drawing. Everyday objects in the room—like a drawn teacup and floating books—cast subtle realistic shadows, blending the dimensions. Soft pastel palette, visible paper texture, high detail, whimsical yet believable '2D-to-3D transition' realism, dynamic motion, enchanting and tactile atmosphere."
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV RENOIR) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
[INFO ] stable-diffusion.cpp:397 - loading diffusion model from '/home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf'
[INFO ] model_loader.cpp:235 - load /home/tipu/AI/image/unet/Z-Image-Turbo-Art-Q8_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:459 - loading llm from '/home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf'
[INFO ] model_loader.cpp:235 - load /home/tipu/AI/models/other/Qwen3-4b-z-engineer/Qwen3-4b-Z-Engineer-V2-Q8_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:473 - loading vae from '/home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors'
[INFO ] model_loader.cpp:238 - load /home/tipu/AI/image/vae/UltraFlux-v1_vae.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:524 - Version: Z-Image
[INFO ] stable-diffusion.cpp:551 - Weight type stat: f32: 634 | q8_0: 433 | bf16: 28
[INFO ] stable-diffusion.cpp:552 - Conditioner weight type stat: f32: 145 | q8_0: 253
[INFO ] stable-diffusion.cpp:553 - Diffusion model weight type stat: f32: 245 | q8_0: 180 | bf16: 28
[INFO ] stable-diffusion.cpp:554 - VAE weight type stat: f32: 244
[INFO ] stable-diffusion.cpp:955 - using VAE for encoding / decoding
[INFO ] auto_encoder_kl.hpp:527 - vae decoder: ch = 128
[INFO ] stable-diffusion.cpp:1059 - Using flash attention
[INFO ] stable-diffusion.cpp:1073 - Using flash attention in the diffusion model
[INFO ] stable-diffusion.cpp:1181 - total params memory size = 11131.89MB (VRAM 10967.89MB, RAM 164.00MB): text_encoders 4076.43MB(VRAM), diffusion_model 6891.46MB(VRAM), vae 164.00MB(RAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A)
[INFO ] stable-diffusion.cpp:1281 - running in FLOW mode
[INFO ] stable-diffusion.cpp:4432 - generate_image 1024x1024
[INFO ] denoiser.hpp:607 - get_sigmas with SmoothStep scheduler
[INFO ] stable-diffusion.cpp:3456 - sampling using Euler method
|##################################################| 386/386 - 2.76GB/s
[INFO ] model_loader.cpp:1224 - loading tensors completed, taking 1.41s (read: 0.13s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 1.07s)
[INFO ] stable-diffusion.cpp:4171 - get_learned_condition completed, taking 2.52s
[INFO ] stable-diffusion.cpp:4478 - generating image: 1/1 - seed 765675567
|##################################################| 452/452 - 3.35GB/s
[INFO ] model_loader.cpp:1224 - loading tensors completed, taking 2.01s (read: 0.25s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 1.48s)
|==================================================| 7/7 - 61.84s/it
[INFO ] stable-diffusion.cpp:4510 - sampling completed, taking 428.98s
[INFO ] stable-diffusion.cpp:4522 - generating 1 latent images completed, taking 428.98s
[INFO ] stable-diffusion.cpp:4195 - decoding 1 latents
|##################################################| 138/138 - 866.98MB/s
[INFO ] model_loader.cpp:1224 - loading tensors completed, taking 0.22s (read: 0.02s, memcpy: 0.00s, convert: 0.11s, copy_to_backend: 0.00s)
[INFO ] stable-diffusion.cpp:4215 - latent 1 decoded, taking 64.52s
[INFO ] stable-diffusion.cpp:4219 - decode_first_stage completed, taking 64.52s
[INFO ] stable-diffusion.cpp:4659 - generate_image completed in 496.04s
[INFO ] main.cpp:463 - save result image 0 to 'output_2026-06-17_10-29-24.png' (success)
[INFO ] main.cpp:535 - 1/1 images saved
No response
Activity
GreenShadows commentedon Jun 17, 2026
You're using a modified encoder and VAE. That in itself could be the problem.
As far as I know, FA doesn't work well with Vega iGPUs. It just slows everything down. Disable it.
Also, don't use a fixed seed or GGUF Q8, the former causes bugs, and using a quantization higher than Q5 will push you up against the driver's limits, which can only allocate a maximum of about ~4GB.
engrtipusultan commentedon Jun 17, 2026
It was just fine couple of months back, I tested this combination before saving. Actually the image shared was same that I generated couple of months back and recorded cli command that is how I found out that a problem has happened now.
Earlier some images was generated by me and shared in #1666
--vae-conv-directwas working just fine.It is iGPU and FA on it works just fine on llama.cpp that I have using for some time. I can generate upto 19 tps TG and 130 tps PP for Gemma 4 QAT through vulkan. I use Qwen3.6 and other models just fine through llama.cpp. Yes anything above A3B is slow though but this is the only option I have.
I used fixed seed to compare with image that I generated some months back and recorded command for later. Hence now there is one to one comparison that it was working in past but not working now. As said earlier I downloaded is GGUF Q8 quite some time back and it was working fine.
engrtipusultan commentedon Jun 17, 2026
I understand this driver's limits, which can only allocate a maximum of about ~4GB. In llama.cpp Jeff programmed something so that if 4GB plus buffer is required it falls back to CPU. To avoid that I was using
--vae-conv-directand now with-vae-on-cpuit is still working for generation.If that limit is hit program should give OOM error rather then producing faulty images.
wbruna commentedon Jun 17, 2026
@GreenShadows , could you point out an issue related to fixed seeds?
Also, current sd.cpp should be able to deal with the 4G allocation limit just fine, as long as the inference code doesn't have any huge individual op (e.g. non-tiled VAE for high resolutions).
And this is likely hardware dependent, since I use
--vae-conv-directwith my card all the time (RX 7600 XT, radv driver). @engrtipusultan, could you try changing resolutions little by little, to find out what is the largest you can generate without the distortion? Also, does--params-backend cpuchanges anything?GreenShadows commentedon Jun 17, 2026
Using a fixed seed causes artifacts for me in some distilled models. But it's kinda random.
I have a laptop with this iGPU uArch, and I tested it: with a quantization level higher than Q5, it gets super slow and causes allocation errors in Vulkan.
Maybe this will improve a little now that AMD is working on adding support for Vega in ROCm. ROCm/rocm-libraries#5282
engrtipusultan commentedon Jun 18, 2026
I added
--params-backend cpuit did not make any difference. I could not understand what it does?I started increase resolution with a step of 64 from 512*512 so image is fine until 960 × 960 pixels on 1024 × 1024 pixels and 1088 × 1088 pixels image is garbage as shared above following are their logs.
960x960
1024x1024
1088x1088
wbruna commentedon Jun 29, 2026
The garbage image looks similar to the ones mentioned in #1589 . One interesting aspect is that, on some of those examples,
--vae-conv-directavoids the issue, instead of triggering it.It keeps the weights in RAM, moving them to VRAM on demand. It was a test to see if total VRAM usage could change the result.
I've reported this as a backend issue on: ggml-org/llama.cpp#24943 .
wbruna commentedon Jun 29, 2026
@engrtipusultan , could you confirm if this issue still happens in the latest release (
master-737-3b6c9ca)? I cannot reproduce this anymore on my iGPU.engrtipusultan commentedon Jun 30, 2026
I am getting coredump now with same command as opening post.
Details
wbruna commentedon Jun 30, 2026
Interesting: the exact same error happened with my iGPU. Please try setting the env var
GGML_VK_MAX_NODES_PER_SUBMITto a low value, e.g.GGML_VK_MAX_NODES_PER_SUBMIT=1.engrtipusultan commentedon Jun 30, 2026
It works and image is also perfectly fine.
Can you kindly breifly tell what does
GGML_VK_MAX_NODES_PER_SUBMIT=1do ?Details
engrtipusultan commentedon Jun 30, 2026
I see you had #24872 in llama.cpp.
wbruna commentedon Jun 30, 2026
Yeah, ggml-org/llama.cpp#24872 is a workaround for an issue (ggml-org/llama.cpp#21724) that started to happen some time ago on llama.cpp, mainly on low-powered GPUs.
LostRuins commentedon Jun 30, 2026
Could you let me know what's the minimum value that works for you?
engrtipusultan commentedon Jun 30, 2026
I have tried 1 to 10 values all work on my vega8 gfx90c. I have set value to 5 in my bashrc
Maybe wbruna can shed some light how to find best suitable value.