Has DeepSpeed supported Qwen 3.5 yet?

[rin2401]

I’m fine-tuning Qwen 3.5 9B with Axolotl. Using FSDP works fine, but when I use DeepSpeed, it throws an error.

[tohtana]

Hi @rin2401,

Can you share more about your environment? It worked on my environment with:

• PyTorch 2.9.1+cu128
• DeepSpeed 0.18.10+71a0a36a
• transformers 5.2.0.dev0

[rin2401]

Hi @tohtana,
I'm encountering an issue while running finetuning Qwen3.5 using the Docker image axolotlai/axolotl:0.15.0.

My environment details:

• torch 2.10.0+cu128
• transformers 5.3.0
• deepspeed 0.18.2

When i check Deepspeed code i don't see any layer about Qwen3.5

DeepSpeed/deepspeed/module_inject/auto_tp.py

Line 138 in 37e232f

"LPLayerNorm", "SharedEmbedding", "OPTLearnedPositionalEmbedding", "LlamaRMSNorm", "FalconLinear",