vLLM backend (cuda12-vllm / cuda13-vllm) fails to import: bundled libnuma.so.1 requires GLIBC_2.38, but latest-gpu-nvidia-cuda-13 ships Ubuntu 22.04 (glibc 2.35)

[Ubsefor]

LocalAI version:
Docker Hub Image Tag

Environment, CPU architecture, OS, and Version:

NVIDIA DGPU

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 610.43.02              KMD Version: 610.43.02     CUDA UMD Version: 13.3     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX PRO 4000 Blac...    On  |   00000000:08:00.0  On |                    0 |
| 30%   43C    P8             11W /   70W |       6MiB /  24467MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

AMD IGPU

+------------------------------------------------------------------------------+
| AMD-SMI            26.5.0+2b22ab01                                           |
| OS kernel Version: 7.0.14-4-pve                                              |
| ROCm Version:      7.14.0                                                    |
| VBIOS Version:     612127                                                    |
| Platform:          Linux Baremetal                                           |
|-------------------------------------+----------------------------------------|
| BDF                        GPU-Name | Mem-Uti   Temp   UEC       Power-Usage |
| GPU  HIP-ID  OAM-ID  Partition-Mode | GFX-Uti    Fan               Mem-Usage |
|=====================================+========================================|
| 0000:01:00.0    AMD Radeon Graphics | N/A        N/A   0                 N/A |
|   0       0     N/A             N/A | N/A        N/A             13/63483 MB |
+-------------------------------------+----------------------------------------+
+------------------------------------------------------------------------------+
| Processes:                                                                   |
|  GPU      PID  Process Name       GTT_MEM  VRAM_MEM  MEM_USAGE  CU %  SDMA   |
|==============================================================================|
|  No running processes found                                                  |
+------------------------------------------------------------------------------+

Its a debian-13 lxc-container, running in PVE-9.2 pve-manager/9.2.6/7f8d010005bd72cb (running kernel: 7.0.14-4-pve):

Container version string

PRETTY_NAME="Debian GNU/Linux 13 (trixie)"
NAME="Debian GNU/Linux"
VERSION_ID="13"
VERSION="13 (trixie)"
VERSION_CODENAME=trixie
DEBIAN_VERSION_FULL=13.6
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Device mount points have amd igpu (listed above) and nvidia dgpu (also listed above)

Docker: Docker version 29.6.2, build dfc4efb

Describe the bug

The cuda12-vllm and cuda13-vllm backends (among sglang and other similar ones) fail at import time with an OSError on GLIBC_2.38' not found, sourced from the backend-bundled libnuma.so.1 in /backends//lib, not the system's own libnuma. This happens on an unmodified localai/localai:latest-gpu-nvidia-cuda-13 container, pulled from dockerhub.
The main runtime image's base OS is Ubuntu 22.04.5 LTS (glibc 2.35). The libnuma.so.1 shipped inside the vLLM backend package appears to have been built/linked against a newer glibc baseline (>=2.38, roughly Ubuntu 23.10/Debian trixie-era), which the 22.04-based runtime image cannot satisfy. Upstream vLLM itself only requires glibc >=2.35, so this looks like a backend-packaging mismatch rather than an actual vLLM requirement.
Because LD_LIBRARY_PATH is set to prefer /backends//lib, the bundled (incompatible) copy shadows whatever system libnuma might otherwise satisfy the dependency.

To Reproduce

• Run localai/localai:latest-gpu-nvidia-cuda-13 via Docker Compose with an NVIDIA GPU reservation.
• Configure a model to use the vllm backend.
• Trigger model load. LocalAI attempts cuda12-vllm, which fails; retry by manually specifying cuda13-vllm for the loaded model, try to load again and it fails identically.

Expected behavior
The vLLM backend imports successfully and the model loads.

Logs
Full backend log via built-in trace mechanisms:
backend-logs-ThinkingCap-Qwen3.6-27B-NVFP4-2026-08-05-2.json

The 'relevant part':

GLIBC error

FFmpeg version 4:
Traceback (most recent call last):
  File "/backends/cuda13-vllm/venv/lib/python3.10/site-packages/torch/_ops.py", line 1503, in load_library
    ctypes.CDLL(path)
  File "/backends/cuda13-vllm/python/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /backends/cuda13-vllm/lib/libnuma.so.1)

Additional context
The libnuma.so.1 packaged alongside the vLLM backend (per the backend-packaging notes in .agents/adding-backends.md, which describe bundling libs like libnuma.so.1 for portability and testing against a fresh ubuntu:24.04) was apparently built in/for an environment with glibc >=2.38. The shipped latest-gpu-nvidia-cuda-13 runtime image is still Ubuntu 22.04 (glibc 2.35), so the bundled lib can never load there, even though upstream vLLM's actual floor is glibc 2.35.
This seems to be 'intended behavior' on the latest image, according to LABEL org.opencontainers.image.version=22.04 on the layer of the image in docker hub, but still I consider that one a bug, because this prevents using any backends, which depend on GLIBC > 2.35