Unable to allocate RAM for VRAM when VRAM is full

[Lightning444]

NVIDIA Open GPU Kernel Modules Version

nvidia-open-dkms 595.58.03

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Cachyos

Kernel Release

Linux-LTS 6.18.19

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

NVIDIA GeForce RTX 4060 Ti OC 8GB

Describe the bug

When running games or applications, if the VRAM becomes full apps will freeze up or crash. Also when trying to open anything with the VRAM full those programs will crash upon opening them. When checking the logs it appears that there is many failures related to the allocation of the RAM.

To Reproduce

Just try to use as much VRAM as possible by running things like Memtest Vulkan alongside something else like a game or other apps. I find that it triggers at 7.8-7.9 GB usage.

Bug Incidence

Sometimes

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

I have a friend who has an AMD GPU running on CachyOS that seems to handle full VRAM just fine, sure it has some graphical bugs here and there, but I feel that it would be better then crashing at least. Also unable to test this with the proprietary driver because CachyOS no longer properly supports it.

[damachine]

Hello,
I have been experiencing the same problem since the 595 driver series.
When running memory-intensive applications, the screen goes completely black and the system becomes entirely unresponsive — no keyboard or mouse input works anymore. The only way to recover is a hard reset.

Bug report:
nvidia-bug-report.log.gz

I am attaching a portion of the system log from the point where the system had already stopped responding completely.
journalctl_log.txt

Please note that the log generates approximately 10,000+ "NVRM errors" entries within just a few microseconds during the crash.

Cheers

[canyalccn]

I doubt this bug was introduced with the new 595 drivers. Shared memory allocation has always been an issue on Linux NVidia. You can check out recently developed NVidiaGreenBoost kernel modules, it promises to fix this bug. However, no vulkan integration for gaming yet.

[Shiroiame-Kusu]

Same problem when VRAM full, sometimes it freezes my games, and sometimes it crashes the whole kernel.