Wan 2.2 T2V A14B diffusers PTQ: two-expert NVFP4-SVDQuant HF checkpoint export

CHANGELOG.rst

@@ -6,6 +6,7 @@ Changelog
 
 **Backward Breaking Changes**
 
+- SVDQuant calibration now uses a fixed SmoothQuant-style outlier migration (``SVDQuantConfig.alpha``, default ``1.0`` = migrate fully to the weights; diffusers example flag ``--alpha``) instead of the AWQ-Lite alpha search. The search optimized pre-SVD, weight-only output MSE — a mismatched objective once the SVD low-rank branch absorbs the migrated outliers — and cost an extra forward-loop pass with one quantized GEMM per alpha candidate per linear. SVDQuant now runs two forward-loop passes (stats + max calibration) instead of three; existing ``svdquant`` recipes will produce different (paper-aligned) scales. Checkpoint format is unchanged.
 - Remove the ``examples/diffusers/eval`` image-quality evaluation example (ImageReward / CLIP-IQA / CLIP metrics) and its references in ``examples/diffusers/README.md``. The example was deprecated in 0.45 and is no longer maintained.
 - Remove the deprecated ``examples/llm_autodeploy`` example (deprecated in 0.45). Use TensorRT-LLM's `AutoDeploy <https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/auto_deploy>`_ directly together with ModelOpt PTQ in ``examples/llm_ptq``.
 
@@ -19,6 +20,7 @@ Changelog
 
 **New Features**
 
+- Add Wan 2.2 T2V A14B NVFP4-SVDQuant HF checkpoint export in the diffusers quantization example: ``--model wan2.2-t2v-14b`` now quantizes both experts (``transformer`` and ``transformer_2``) by default, and the default Wan recipe (first-3/last-3 of the 40 ``blocks`` excluded, nothing outside ``blocks`` quantized) is applied **before** calibration via the block-range mechanism so SVDQuant leaves excluded weights bit-identical. VAE backbones (``--backbone vae``) keep their dedicated recipe. See ``examples/diffusers/README.md``.
 - Add the **D-PACE** loss objective for DFlash speculative-decoding training (`arXiv:2605.18810 <https://arxiv.org/abs/2605.18810>`_) and make it the default (``dflash_loss_objective: dpace``). It replaces the static exponential position decay with dynamic, confidence-derived per-position weights that adapt to whichever block positions currently limit acceptance. Smoothing is controlled by ``dflash_dpace_alpha`` (default 0.5); set ``dflash_loss_objective: decay`` to restore the previous static schedule. Training-only and detached from the gradient (no architecture or inference change).
 - Add the ``day0-release`` agent skill (``.agents/skills/day0-release/``), a deterministic end-to-end driver that chains the PTQ → evaluation → comparison skills (the evaluation stage deploys the checkpoint itself) with an enforced gate after each stage and returns a publish decision (ACCEPT / REGRESSION / ANOMALOUS / INFEASIBLE). Ships three GPU-free, unit-tested gate scripts (``gate_ptq.py``, ``gate_run.py``, ``gate_compare.py``) that validate checkpoint coverage, evaluation-run completeness, and baseline-vs-candidate accuracy threshold. v1 reports and stops on regression; the recipe-search loop is deferred.
 - Add **streaming** speculative-decoding training (EAGLE3 / DFlash): the draft trains on base-model hidden states produced on the fly by a co-located ``vllm serve`` (no disk dump), moved trainer-side over NIXL RDMA, scaling to multi-node (dedicated serve replicas + DDP trainers). New launcher examples for NVFP4 Kimi-K2.5 / K2.6 on GB200/aarch64 under ``tools/launcher/examples/moonshotai/``.

examples/diffusers/README.md

@@ -116,6 +116,28 @@ python quantize.py \
     --hf-ckpt-dir ./hf_ckpt
 ```
 
+#### Wan 2.2 T2V A14B NVFP4 SVDQuant [Script](./quantization/quantize.py)
+
+Wan 2.2 T2V A14B is a two-expert pipeline (high-noise `transformer` + low-noise
+`transformer_2`); with `--model wan2.2-t2v-14b` both experts are quantized by
+default, `transformer` first so the low-noise expert calibrates against the
+already-quantized high-noise expert. The recipe quantizes only the linears under
+`blocks`, keeping the **first 3 and last 3** of the 40 blocks (and everything
+outside `blocks`: text encoder, VAE, embedders, `proj_out`, ...) in original
+precision. The exclusion is applied **before calibration** so that for SVDQuant
+the excluded blocks' weights stay bit-identical to the original. Every quantized
+linear keeps the full SVDQuant recipe (AWQ `pre_quant_scale` + low-rank
+`svdquant_lora_a/b`).
+
+```sh
+python quantize.py \
+    --model wan2.2-t2v-14b \
+    --model-dtype BFloat16 --trt-high-precision-dtype BFloat16 \
+    --format fp4 --quant-algo svdquant --lowrank {32|64|128} \
+    --batch-size 1 --calib-size 16 --n-steps 20 \
+    --hf-ckpt-dir ./hf_ckpt
+```
+
 #### Wan 2.2 VAE NVFP4 (Conv3D Implicit GEMM)
 
 The Wan 2.2 VAE (`AutoencoderKLWan`, shared between the 5B and 14B pipelines) is built from 3D convolutions. When quantizing the VAE with NVFP4, the `Conv3d` layers are automatically dispatched through a custom BF16 WMMA implicit-GEMM kernel with fused FP4 activation quantization. Requires SM80+ (Ampere or newer). See [`modelopt/torch/kernels/quantization/conv/README.md`](../../modelopt/torch/kernels/quantization/conv/README.md) for kernel details.

examples/diffusers/quantization/calibration.py

@@ -18,6 +18,7 @@
 from pathlib import Path
 from typing import Any
 
+import torch
 from models_utils import MODEL_DEFAULTS, ModelType
 from pipeline_manager import PipelineManager
 from quantize_config import CalibrationConfig
@@ -50,6 +51,7 @@ def __init__(
         self.config = config
         self.model_type = model_type
         self.logger = logger
+        self._minimax_h3_generator: torch.Generator | None = None
 
     def load_and_batch_prompts(self) -> list[list[str]]:
         """
@@ -95,6 +97,8 @@ def run_calibration(self, batched_prompts: list[list[str]]) -> None:
                 elif self.model_type in [ModelType.WAN22_T2V_14b, ModelType.WAN22_T2V_5b]:
                     # Special handling for WAN video models
                     self._run_wan_video_calibration(prompt_batch, extra_args)
+                elif self.model_type == ModelType.MINIMAX_H3:
+                    self._run_minimax_h3_calibration(prompt_batch, extra_args)
                 else:
                     common_args = {
                         "prompt": prompt_batch,
@@ -105,6 +109,43 @@ def run_calibration(self, batched_prompts: list[list[str]]) -> None:
                 self.logger.debug(f"Completed calibration batch {i + 1}/{self.config.num_batches}")
         self.logger.info("Calibration completed successfully")
 
+    def _run_minimax_h3_calibration(
+        self, prompt_batch: list[str], extra_args: dict[str, Any]
+    ) -> None:
+        """Calibrate MiniMax-H3 through its prompt-only T2V denoising path."""
+        if len(prompt_batch) != 1:
+            raise ValueError(
+                "MiniMax-H3's T2V ModularPipeline accepts one prompt string per call; "
+                "use --batch-size 1 for calibration."
+            )
+
+        extra_params = self.pipeline_manager.config.extra_params
+        height = int(extra_params.get("height", extra_args["height"]))
+        width = int(extra_params.get("width", extra_args["width"]))
+        num_frames = int(extra_params.get("num_frames", extra_args["num_frames"]))
+        seed = int(extra_params.get("seed", 0))
+        if self._minimax_h3_generator is None:
+            self._minimax_h3_generator = torch.Generator(device="cpu").manual_seed(seed)
+
+        self.logger.debug(
+            "MiniMax-H3 T2V calibration call: height=%d width=%d num_frames=%d "
+            "num_inference_steps=%d seed=%d",
+            height,
+            width,
+            num_frames,
+            self.config.n_steps,
+            seed,
+        )
+        self.pipe(
+            prompt=prompt_batch[0],
+            height=height,
+            width=width,
+            num_frames=num_frames,
+            num_inference_steps=self.config.n_steps,
+            generator=self._minimax_h3_generator,
+            output=["videos", "audio", "sampling_rate"],
+        )
+
     def _run_wan_video_calibration(
         self, prompt_batch: list[str], extra_args: dict[str, Any]
     ) -> None:

examples/diffusers/quantization/config.py

@@ -41,7 +41,11 @@ def set_quant_config_attr(quant_config, trt_high_precision_dtype, quant_algo, **
     elif quant_algo == "svdquant":
         if "lowrank" in kwargs:
             algo_cfg["lowrank"] = kwargs["lowrank"]
-        # Layers excluded from the SVDQuant algorithm (no AWQ smoothing, no
+        # SmoothQuant-style migration strength applied before the SVD
+        # (1.0 = migrate outliers fully to the weights).
+        if "alpha" in kwargs:
+            algo_cfg["alpha"] = kwargs["alpha"]
+        # Layers excluded from the SVDQuant algorithm (no smoothing, no
         # low-rank branch); they stay quantized with plain max calibration.
         if kwargs.get("skip_layers"):
             algo_cfg["skip_layers"] = kwargs["skip_layers"]

examples/diffusers/quantization/models_utils.py

@@ -43,6 +43,7 @@
     filter_func_flux_dev,
     filter_func_ltx2_vae,
     filter_func_ltx_video,
+    filter_func_minimax_h3,
     filter_func_qwen_image,
     filter_func_wan_vae,
     filter_func_wan_video,
@@ -64,6 +65,7 @@ class ModelType(str, Enum):
     WAN22_T2V_14b = "wan2.2-t2v-14b"
     WAN22_T2V_5b = "wan2.2-t2v-5b"
     QWEN_IMAGE = "qwen-image"
+    MINIMAX_H3 = "minimax-h3"
 
 
 _FILTER_FUNC_MAP: dict[ModelType, Callable[[str], bool]] = {
@@ -74,6 +76,7 @@ class ModelType(str, Enum):
     ModelType.WAN22_T2V_14b: filter_func_wan_video,
     ModelType.WAN22_T2V_5b: filter_func_wan_video,
     ModelType.QWEN_IMAGE: filter_func_qwen_image,
+    ModelType.MINIMAX_H3: filter_func_minimax_h3,
 }
 
 _VAE_FILTER_FUNC_MAP: dict[tuple[ModelType, str], Callable[[str], bool]] = {
@@ -107,6 +110,7 @@ def get_model_filter_func(
     ModelType.WAN22_T2V_14b: "Wan-AI/Wan2.2-T2V-A14B-Diffusers",
     ModelType.WAN22_T2V_5b: "Wan-AI/Wan2.2-TI2V-5B-Diffusers",
     ModelType.QWEN_IMAGE: "Qwen/Qwen-Image",
+    ModelType.MINIMAX_H3: "MiniMaxAI/MiniMax-H3",
 }
 
 MODEL_PIPELINE: dict[ModelType, type[DiffusionPipeline] | None] = {
@@ -122,6 +126,9 @@ def get_model_filter_func(
     ModelType.WAN22_T2V_14b: WanPipeline,
     ModelType.WAN22_T2V_5b: WanPipeline,
     ModelType.QWEN_IMAGE: QwenImagePipeline,
+    # MiniMax-H3 uses ModularPipeline and is created by PipelineManager's
+    # text-to-video-only path. Keep it unavailable to the generic TRT loader.
+    ModelType.MINIMAX_H3: None,
 }
 
 # Shared dataset configurations
@@ -204,6 +211,23 @@ def get_model_filter_func(
     },
     ModelType.WAN22_T2V_14b: {
         **_WAN_BASE_CONFIG,
+        # Wan 2.2 A14B is a two-expert pipeline: the high-noise expert
+        # (``transformer``, t >= boundary) and the low-noise expert
+        # (``transformer_2``). A deployable checkpoint quantizes both;
+        # ``transformer`` goes first so the low-noise expert calibrates against
+        # the already-quantized high-noise expert, matching deployment.
+        "backbone": ["transformer", "transformer_2"],
+        # Pre-calibration form of ``filter_func_wan_video``: quantize only the
+        # linears under ``blocks``, excluding the first/last 3 of the 40 blocks
+        # (everything outside ``blocks`` -- patch_embedding, condition_embedder,
+        # proj_out -- stays in original precision). Applied before calibration
+        # via ``build_block_range_quant_cfg`` so SVDQuant never mutates the
+        # excluded blocks' weights. Not applied to VAE backbones (no ``blocks``).
+        "block_range": {
+            "exclude_first_n": 3,
+            "exclude_last_n": 3,
+            "block_module": "blocks",
+        },
         "from_pretrained_extra_args": {
             "boundary_ratio": 0.875,
         },
@@ -271,6 +295,29 @@ def get_model_filter_func(
             "*.txt_mod.1",
         ],
     },
+    ModelType.MINIMAX_H3: {
+        "backbone": "transformer",
+        "dataset": _OPENVID_DATASET,
+        "svdquant_alpha": 0.8,
+        # MiniMax-H3's guidance-distilled T2V workflow consumes only a prompt;
+        # there is no negative prompt or guidance scale. These are the trained
+        # 16:9 canvas and minimum valid frame count. Calibration-only overrides
+        # (for example 544x960) can be supplied through --extra-param.
+        "inference_extra_args": {
+            "height": 768,
+            "width": 1344,
+            "num_frames": 124,
+        },
+        # Apply this before calibration, especially for SVDQuant: quantize only
+        # transformer blocks 2..49. This keeps the first two blocks and every
+        # projection/embedder/final-layer module outside the block list bit-
+        # identical to the source weights.
+        "block_range": {
+            "exclude_first_n": 2,
+            "exclude_last_n": 0,
+            "block_module": "transformer_blocks",
+        },
+    },
 }
 
 

examples/diffusers/quantization/pipeline_manager.py

@@ -16,6 +16,7 @@
 import logging
 import warnings
 from collections.abc import Iterator
+from pathlib import Path
 from typing import Any
 
 import torch
@@ -43,6 +44,7 @@ def __init__(self, config: ModelConfig, logger: logging.Logger):
         self.pipe_upsample: LTXLatentUpsamplePipeline | None = None  # For LTX-Video upsampling
         self._transformer: torch.nn.Module | None = None
         self._video_decoder: torch.nn.Module | None = None
+        self._components_manager: Any | None = None
 
     @staticmethod
     def create_pipeline_from(
@@ -100,6 +102,11 @@ def create_pipeline(self) -> Any:
                 self.logger.info("LTX-2 pipeline created successfully")
                 return self.pipe
 
+            if self.config.model_type == ModelType.MINIMAX_H3:
+                self.pipe = self._create_minimax_h3_pipeline()
+                self.logger.info("MiniMax-H3 T2V modular pipeline created successfully")
+                return self.pipe
+
             pipeline_cls = MODEL_PIPELINE[self.config.model_type]
             if pipeline_cls is None:
                 raise ValueError(
@@ -143,6 +150,14 @@ def setup_device(self) -> None:
             self.logger.info("Skipping device setup for LTX-2 pipeline (handled internally)")
             return
 
+        if self.config.model_type == ModelType.MINIMAX_H3:
+            if self.config.cpu_offloading:
+                self.logger.info("MiniMax-H3 ComponentsManager auto CPU offloading is enabled")
+            else:
+                self.logger.info("Moving MiniMax-H3 T2V components to CUDA")
+                self.pipe.to("cuda")
+            return
+
         if self.config.cpu_offloading:
             self.logger.info("Enabling CPU offloading for memory efficiency")
             self.pipe.enable_model_cpu_offload()
@@ -192,6 +207,68 @@ def iter_backbones(self) -> Iterator[tuple[str, torch.nn.Module]]:
                 raise RuntimeError(f"Pipeline missing backbone module '{name}'.")
             yield name, module
 
+    def _create_minimax_h3_pipeline(self) -> Any:
+        """Load only MiniMax-H3's prompt-only T2V workflow."""
+        try:
+            from diffusers import ComponentsManager, ModularPipeline
+        except ImportError as exc:
+            raise ImportError(
+                "MiniMax-H3 is not part of a Diffusers release yet. Install Diffusers "
+                "from source with: pip install git+https://github.com/huggingface/diffusers.git"
+            ) from exc
+
+        forbidden_conditioning = {
+            key for key in ("image", "last_image", "references") if key in self.config.extra_params
+        }
+        if forbidden_conditioning:
+            raise ValueError(
+                "MiniMax-H3 calibration supports text-only T2V. Remove conditioning "
+                f"parameters: {sorted(forbidden_conditioning)}"
+            )
+
+        self._components_manager = ComponentsManager()
+        local_files_only = Path(self.config.model_path).exists()
+        pipe = ModularPipeline.from_pretrained(
+            self.config.model_path,
+            workflow="t2va",
+            components_manager=self._components_manager,
+            local_files_only=local_files_only,
+        )
+        # The downloaded modular index records the Hub repository in every
+        # component spec. Override it so a local --override-model-path remains
+        # completely local and cannot silently fetch transformer_ref.
+        pipe.load_components(
+            dtype=self.config.model_dtype,
+            pretrained_model_name_or_path=self.config.model_path,
+            local_files_only=local_files_only,
+        )
+        # Enable offload only after every component has been registered. Enabling
+        # it before load_components() makes ComponentsManager rebuild hooks on
+        # each add and resets the requested reserve margin to its default.
+        if self.config.cpu_offloading:
+            memory_reserve_margin = str(
+                self.config.extra_params.get("memory_reserve_margin", "12GB")
+            )
+            self.logger.info(
+                "Enabling MiniMax-H3 automatic CPU offload with memory reserve margin %s",
+                memory_reserve_margin,
+            )
+            self._components_manager.enable_auto_cpu_offload(
+                device="cuda", memory_reserve_margin=memory_reserve_margin
+            )
+        pipe.set_progress_bar_config(disable=True)
+
+        missing_components = pipe.null_component_names
+        if missing_components:
+            raise RuntimeError(
+                f"MiniMax-H3 T2V failed to load required components: {sorted(missing_components)}"
+            )
+        if getattr(pipe, "transformer_ref", None) is not None:
+            raise RuntimeError("MiniMax-H3 T2V workflow unexpectedly loaded transformer_ref.")
+        loaded_components = sorted(pipe.components)
+        self.logger.info("Loaded MiniMax-H3 T2V components only: %s", loaded_components)
+        return pipe
+
     def _ensure_ltx2_transformer_cached(self) -> None:
         if not self.pipe:
             raise RuntimeError("Pipeline not created. Call create_pipeline() first.")