Skip to content

fix(diarization): scale known-speaker clustering for long audio - #3516

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
tandede:fix/large-known-speaker-clustering
Aug 20, 2026
Merged

fix(diarization): scale known-speaker clustering for long audio#3516
LauraGPT merged 1 commit into
modelscope:mainfrom
tandede:fix/large-known-speaker-clustering

Conversation

@tandede

@tandede tandede commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #3514.

When preset_spk_num is set, ClusterBackend currently sends every input size through spectral clustering. For long recordings this bypasses the existing 2,048-embedding safeguard and builds an O(N²) dense affinity/Laplacian matrix before an O(N³) eigendecomposition.

This change keeps the existing dispatch for small inputs and for large inputs with an unknown speaker count, while adding a bounded-memory fixed-K path for large inputs with a known count:

  • L2-normalize the speaker embeddings so Euclidean K-means follows cosine geometry;
  • use deterministic K-means with the requested speaker count;
  • retain spectral clustering below 2,048 embeddings;
  • retain UMAP/HDBSCAN for large inputs without oracle_num.

Type of change

  • Bug fix
  • Documentation
  • Example or demo
  • Runtime or deployment
  • Benchmark or evaluation
  • Model/training change

Validation

  • python -m compileall funasr examples tests
  • Docs or links checked
  • Runtime/deployment command tested

Commands and results:

  • pytest -q tests/test_cluster_backend.py tests/test_top_level_import.py — 3 passed
  • python -m build — sdist and wheel built successfully
  • funasr --help — CLI started successfully
  • Replayed the reported scale with 10,639×192 synthetic speaker embeddings and oracle_num=2: the full ClusterBackend path completed in 0.098 seconds on CPU, returned exactly two clusters, and separated the two cosine clusters with 100% purity.

The dispatch regression fails on the previous implementation because it observes a call to dense spectral clustering at the 2,048 boundary.

User impact

Users can provide preset_spk_num for long recordings without turning a scalable clustering path into an effectively unbounded dense eigendecomposition. Small-recording behavior and automatic speaker-count behavior remain unchanged.

Notes for reviewers

The 2,048 threshold is not changed. The new tests cover both dispatch at the threshold and the actual fixed-K clustering result. The implementation uses the existing scikit-learn dependency and does not add a package or public API.

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 9dbf3e1c0858706b98baebeca89b4b428855cc2f against main@bff70399427fe5a06ea5244d1410dd729efe462e.

  • The base regression fails because N=2048 with a known speaker count still enters dense spectral clustering.
  • The two new focused tests pass on the PR head.
  • The related server, realtime, and clustering suite passes: 82/82.
  • python -m compileall -q funasr examples tests passes.
  • A synthetic 10639 x 192, K=2 case completes in 0.171 seconds and returns the expected two clusters.
  • Production callers pass CPU tensors, and the backend still returns NumPy labels.

The change preserves small-input spectral clustering and large-input unknown-count UMAP/HDBSCAN routing. Approved.

@LauraGPT
LauraGPT merged commit 0d96c57 into modelscope:main Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants