Skip to content

Update MagpieTTS' Inference Parameter Configuration#15254

Merged
blisc merged 11 commits intomainfrom
magpietts_2601_inferfix
Jan 8, 2026
Merged

Update MagpieTTS' Inference Parameter Configuration#15254
blisc merged 11 commits intomainfrom
magpietts_2601_inferfix

Conversation

@blisc
Copy link
Copy Markdown
Collaborator

@blisc blisc commented Jan 5, 2026

What does this PR do ?

This PR separates model specific and run specific inference parameters. Model specific parameters are not stored inside a ModelInferenceParameters dataclass within the model itself. This parameters should be stored in the model yaml, currently this needs to be done manually. In particular, the latest checkpoint: https://huggingface.co/nvidia/magpie_tts_multilingual_357m has been updated to include a inference_parameters field in the top level yaml.
Run specific inference parameters, such as applying local transformer or using CFG, remain as arguments to the inference functions.
Updates inference defaults, and merges the nemo/collections/tts/modules/magpietts_inference/inference.py configuration with this new setup.

Collection: tts

Changelog

  • Update MagpieTTS Inference Configurations and enables LT and prior in do_tts()

PR Type:

  • New Feature
  • Bugfix
  • Documentation

Signed-off-by: blisc <blisc@users.noreply.github.com>
Signed-off-by: Jason <jasoli@nvidia.com>
XuesongYang
XuesongYang previously approved these changes Jan 5, 2026
Copy link
Copy Markdown
Collaborator

@XuesongYang XuesongYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an InferenceParameters dataclass to centralize inference configuration for MagpieTTS models and enables attention prior by default in the do_tts() method. The main purpose is to make attention prior active by default for improved inference quality while simplifying parameter management across the codebase.

Key changes:

  • Added InferenceParameters dataclass with apply_attention_prior=True by default (previously False)
  • Refactored infer_batch() to read parameters from self.inference_parameters instead of method arguments
  • Simplified do_tts() API by removing explicit temperature, topk, max_decoder_steps, and cfg_scale parameters

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
nemo/collections/tts/models/magpietts.py Introduced InferenceParameters dataclass with attention prior enabled by default; refactored infer_batch() to use instance parameters; simplified do_tts() signature
nemo/collections/tts/models/magpietts_preference_optimization.py Updated test_step() and generate_and_reward() to set inference_parameters attributes instead of passing them as arguments
nemo/collections/tts/modules/magpietts_inference/inference.py Modified standard inference to set model's inference_parameters before calling infer_batch()

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread nemo/collections/tts/models/magpietts.py Outdated
Comment thread nemo/collections/tts/models/magpietts.py
Comment thread nemo/collections/tts/modules/magpietts_inference/inference.py Outdated
Comment thread nemo/collections/tts/modules/magpietts_inference/inference.py Outdated
Comment thread nemo/collections/tts/models/magpietts.py Outdated
Comment thread nemo/collections/tts/models/magpietts.py Outdated
topk=topk,
use_cfg=use_cfg,
cfg_scale=cfg_scale,
use_local_transformer_for_inference=True,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use_local_transformer_for_inference be in InferenceParameters?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the intension of InferenceParameters is to save model specific hyperparameters so we can save it as part of the yaml config and subsequently, part of the nemo file. Since some parameters like estimate_alignment_from_layers are model-specific, I want to keep this new class to contain only those parameters. Options like use_cfg or use_LT are run-specific and should lie elsewhere.

Comment thread nemo/collections/tts/models/magpietts.py Outdated
@github-actions github-actions Bot removed the Run CICD label Jan 5, 2026
Signed-off-by: Jason <jasoli@nvidia.com>
@blisc blisc changed the title Quick fix to enable prior by default in magpie Update MagpieTTS' Inference Parameter Configuration Jan 6, 2026
Signed-off-by: blisc <blisc@users.noreply.github.com>
@subhankar-ghosh
Copy link
Copy Markdown
Collaborator

subhankar-ghosh commented Jan 6, 2026

Left one minor comment, otherwise LGTM given standard and long-form inference has been tested and nothing breaks.

XuesongYang
XuesongYang previously approved these changes Jan 6, 2026
blisc added 2 commits January 6, 2026 14:31
Signed-off-by: Jason <jasoli@nvidia.com>
rfejgin
rfejgin previously approved these changes Jan 6, 2026
Copy link
Copy Markdown
Collaborator

@rfejgin rfejgin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me (some internal discussions ongoing but not needed for this PR)

@github-actions github-actions Bot removed the Run CICD label Jan 7, 2026
blisc and others added 2 commits January 7, 2026 16:22
Signed-off-by: blisc <blisc@users.noreply.github.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Jason <jasoli@nvidia.com>
@paarthneekhara
Copy link
Copy Markdown
Collaborator

LGTM.

@github-actions github-actions Bot removed the Run CICD label Jan 8, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 8, 2026

[🤖]: Hi @blisc 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully.

So it might be time to merge this PR or get some approvals.

//cc @chtruong814 @ko3n1g @pablo-garay @thomasdhc

@blisc blisc merged commit 72e2bb0 into main Jan 8, 2026
136 checks passed
@blisc blisc deleted the magpietts_2601_inferfix branch January 8, 2026 14:38
AkCodes23 pushed a commit to AkCodes23/NeMo that referenced this pull request Jan 28, 2026
* move inference params to checkpoint and make do_tts apply prior

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

* Enable LT in do_tts; add docstrings

Signed-off-by: Jason <jasoli@nvidia.com>

* update defaults; merge inference dataclasses

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

* update epsilon value

Signed-off-by: Jason <jasoli@nvidia.com>

* add defaults for inference; fix longform mode; fix bug introduced in NVIDIA-NeMo#15212

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

* fix field key usage

Signed-off-by: Jason <jasoli@nvidia.com>

---------

Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: blisc <blisc@users.noreply.github.com>
Co-authored-by: blisc <blisc@users.noreply.github.com>
Signed-off-by: Akhil Varanasi <akhilvaranasi23@gmail.com>
nune-tadevosyan pushed a commit to nune-tadevosyan/NeMo that referenced this pull request Mar 13, 2026
* move inference params to checkpoint and make do_tts apply prior

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

* Enable LT in do_tts; add docstrings

Signed-off-by: Jason <jasoli@nvidia.com>

* update defaults; merge inference dataclasses

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

* update epsilon value

Signed-off-by: Jason <jasoli@nvidia.com>

* add defaults for inference; fix longform mode; fix bug introduced in NVIDIA-NeMo#15212

Signed-off-by: Jason <jasoli@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: blisc <blisc@users.noreply.github.com>

* fix field key usage

Signed-off-by: Jason <jasoli@nvidia.com>

---------

Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: blisc <blisc@users.noreply.github.com>
Co-authored-by: blisc <blisc@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants