Skip to content

Cannot import some HF models #11337

Description

@dinccey

LocalAI version:
Docker localai/localai:latest-nvidia-l4t-arm64-cuda-13 v4.7.1

Environment, CPU architecture, OS, and Version:
ARM64 nvidia ubuntu
Describe the bug
Trying to import the new deepseek model from unsloth doesn't work. Advanced import hf://unsloth/DeepSeek-V4-Flash-0731-GGUF and quantization UD-IQ3_XXS yields this error:

Failed to install model: DeepSeek-V4-Flash-0731-GGUF(could not fetch "/models/ds4flash.gguf": local file does not exist (stat /models/ds4flash.gguf: no such file or directory) and "hf://unsloth/DeepSeek-V4-Flash-0731-GGUF" is not a recognized downloadable URL (supported schemes: http://, https://, file://, huggingface://, hf://, ollama://, oci://, ocifile://, github://))
To Reproduce
try to import unsloth/DeepSeek-V4-Flash-0731-GGUF
Expected behavior
model is downloaded correctly and can be loaded with DS4 or other backends
Logs
Failed to install model: DeepSeek-V4-Flash-0731-GGUF(could not fetch "/models/ds4flash.gguf": local file does not exist (stat /models/ds4flash.gguf: no such file or directory) and "hf://unsloth/DeepSeek-V4-Flash-0731-GGUF" is not a recognized downloadable URL (supported schemes: http://, https://, file://, huggingface://, hf://, ollama://, oci://, ocifile://, github://))

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions