On September 23, 2026, Nvidia open-sourced the weights of the Nemotron 3 Diarization model — which labels 'who spoke when' in audio — under the OpenMDW-1.1 license. The compact model, with approximately 100 million parameters, works with up to 8 speakers, which is twice the limit of the previous Streaming Sortformer, and ranks first among 12 systems on the VoiceArena Diarization-Bench. Speaker labeling is no longer a paid component of individual vendors: the weights are fully published, and the 'who said what' attribution layer can be independently integrated into transcripts, meeting bots, and voice agents.

image
image

What Happened

Nvidia published the weights on Hugging Face in the nvidia/Nemotron-3-Diarization repository: the F32 checkpoint contains 99.2 million parameters. Inside is a 31-layer Transformer encoder with RoPE: mono audio at a 16 kHz frequency is converted into a mel-spectrogram, 10 ms frames are packed into 80 ms windows, and a Conv1D at the output returns the activity probability of each of the anonymous channels speaker_1…speaker_8 every 10 ms. Labeling is done frame-by-frame by channels, not through post-clustering, so simultaneous speech does not break attribution; in streaming mode, memory is maintained via an Arrival-Order Speaker Cache and a FIFO queue. On the VoiceArena Diarization-Bench, the model showed a DER of 14.72% (diarization error rate) compared to 19.3% for the nearest competing system — a relative gap of about 24%.

Context

Diarization covers what standard transcription does not provide: ASR converts sound into words but does not answer who spoke them, so a meeting protocol or call analysis without attribution remains a semi-finished product. The model continues the Sortformer line: according to Nvidia, the average reduction in DER compared to its predecessor is 41% with a latency of 1.04 seconds — on DIHARD III, the metric improves from 19.60% to 13.18%, and on NOTSOFAR1 from 22.12% to 7.70%. A single checkpoint covers a latency range from 0.32 to 30.4 seconds, and at a batch of 32 on an RTX PRO 5000, it achieves RTFx 865× compared to 136× for the previous version — this is the profile of a production component, not a laboratory prototype. Channels are numbered in the order voices appear in the recording, so linking an anonymous channel to a specific name is resolved by application-side logic; the quality increase at the same model size is explained by the architecture and training recipe, which makes reproduction and fine-tuning on a single GPU cheaper.

Why This Matters for the Industry

For the industry, this is a shift in the economics of the attribution layer: the OpenMDW-1.1 license explicitly permits commercial use, so 'who said what' transforms from a paid feature of individual vendors into a free embeddable component, and the moat of startups for which diarization was the core of their product is zeroed out. The 'ASR + diarization' prototype can now be assembled in days: attributed transcripts, live meeting protocols with names, call analytics, and voice agent memory are built from ready-made parts. The expected wave in the coming months — speaker-aware features in notes, meeting bots, and call analytics, independent re-evaluations on DIHARD III and NOTSOFAR1 outside vendor configurations, and community fine-tunes for other languages and domains. Competition in the open stack shifts from word quality to attribution quality, and value moves to the UX of matching channels with names and handling labeling errors.

Why This Matters for Users

Everything is available immediately: the weights can be downloaded from Hugging Face, run with a single line via SortformerEncLabelModel.from_pretrained in a NeMo 3.0 environment on Ampere, Hopper, or Blackwell generation GPUs, and for lightweight scenarios, there is the C++ runtime NeMo-Speech.cpp. The model can be tried without installation in Nvidia's demo Space with live streaming ASR. The model is anonymous by design: it does not determine identity and returns only speaker_Х labels and intervals, so recordings can be labeled on your own hardware without sending audio to external services, and chunked processing removes the limit on recording length. The barrier to entry is low, and the adjustable latency allows the response speed to be tailored to the scenario — from a near-live meeting protocol to a careful offline transcription.

What Is Still Unknown / Limitations

The key figures are still vendor-reported: the VoiceArena Diarization-Bench consists of 139 English dialogues and 12 systems, and the ~24% gap is shown in Nvidia's configuration, so until independent verification, it is more correct to say 'best claimed open system' rather than 'current SOTA'. The claimed 41% reduction in DER compared to the predecessor is also provided in vendor configurations, and on the two-speaker CALLHOME, the new model is slightly worse than the previous one: 5.98% vs. 5.68%. Quality on languages other than English and in heavy domains like call centers has not been independently confirmed — community re-evaluations and fine-tunes should show this. Finally, the model's anonymity shifts the linking of channels to real names and responsibility for attribution errors to the application side.

Sources

Author

Look at AI, editorial team