Nvidia Releases Nemotron 3.5 Transcribe Open-Weight Speech Model

On October 28, 2026, Nvidia published the weights, data curation manifests, and training recipes for Nemotron 3.5 Transcribe, an open-weight speech-augmented language model that processes up to 60 minutes of continuous audio in a single forward pass. Running on Nvidia’s NVFP4 quantization format with DFlash2 speculative decoding, the model delivers a throughput of 2,238 seconds of audio processed per second of compute on a single RTX PRO 6000 Blackwell Server Edition GPU—transcribing a 30-second audio sample in 0.13 seconds.

The release directly targets enterprise speech pipelines that have long relied on multi-stage cascades of automatic speech recognition (ASR), separate speaker diarization networks, punctuation restorers, and text normalization scripts. The checkpoint consolidates these tasks into a single 34-billion-parameter Mixture-of-Experts (MoE) architecture that routes execution through just 4.3 billion active parameters per token.

Benchmark results released alongside the model show a mean Word Error Rate (WER) of 4.71% across the nine public test sets of the Hugging Face English Open ASR Leaderboard, and 4.62% on the multilingual Open ASR Leaderboard spanning seven core languages. On independent testing by Artificial Analysis, the model posted a 2.80% score on the AA-WER index, matching the accuracy of 24-billion-parameter dense models while running on approximately one-sixth of the active compute.

Accompanying the model is Granary v2, an open data curation manifest and pipeline built on NeMo Curator containing 463,062 hours of multilingual speech across 48 languages. Licensed under OpenMDW-1.1 for commercial and non-commercial deployment, the release gives developers unrestricted access to run the model on local workstations, enterprise private clouds, or air-gapped data centers.

========================================================================================
NEMOTRON 3.5 TRANSCRIBE: AT A GLANCE
========================================================================================
Total Parameters:                 34.0 Billion (31.6B LLM, 0.8B Audio/CTC, 1.3B MTP)
Active Parameters Per Token:      4.3 Billion (~3.5B LLM active + 0.8B front-end)
Context Window Budget:            65,536 tokens (shared between audio and text)
Maximum Audio Length:             60 minutes (batch/offline mode)
Supported Languages:              40 (ASR), 31 (Translation), 13 (Diarization)
Max Concurrent Speakers:          8 speakers
English Open ASR Mean WER:        4.71% (5.33% average across extended evaluation)
Multilingual Open ASR Mean WER:   4.62% (7-language evaluation suite)
AA-AgentTalk WER:                 1.80% (Held-out conversational voice-agent audio)
Peak Throughput (NVFP4 + DFlash): 2,238 seconds of audio / second of compute
Hardware Footprint (VRAM):        ~68 GB (BF16), ~34 GB (FP8), ~17 GB (NVFP4)
License:                          OpenMDW-1.1 (Commercial and research permitted)
========================================================================================

The Active Compute Equation: 34 Billion Total Parameters, 4.3 Billion Active

The primary computational barrier in deploying large language models for speech processing has been inference economics. Audio signals produce dense representations: 16 kHz audio sampled at standard intervals yields hundreds of embeddings per minute of speech. In a traditional dense model, passing long audio sequences through tens of billions of parameters quickly exhausts GPU memory bandwidth and inflates serving costs.

Nvidia structured Nemotron 3.5 Transcribe around a sparse Mixture-of-Experts backbone derived from its Nvidia Nemotron 3.5 Lightning 30B-A3B architecture. While the total model footprint spans approximately 34 billion parameters, only 4.3 billion parameters engage during any individual forward step.

+---------------------------------------------------------------------------------------+
|                      NEMOTRON 3.5 TRANSCRIBE PARAMETER ALLOCATION                     |
+---------------------------------------------------------------------------------------+
| Component                               | Parameters  | Active Per Token | Status     |
+-----------------------------------------+-------------+------------------+------------+
| Transformer ASR Acoustic Encoder        | ~0.60 B     | 0.60 B           | Trainable  |
| Nemotron 3 Diarization (Sortformer)     | ~0.10 B     | 0.10 B           | Frozen     |
| Modality Projection Adapter             | ~0.05 B     | 0.05 B           | Trainable  |
| CTC Alignment Adapter                   | ~0.05 B     | 0.05 B           | Post-Pass  |
| Nemotron 3.5 Lightning MoE Backbone     | ~31.6 B     | ~3.50 B          | Trainable  |
| Multi-Token Prediction (MTP) Draft Head | ~1.30 B     | Speculative      | Shared     |
+-----------------------------------------+-------------+------------------+------------+
| TOTAL CHECKPOINT                        | ~34.0 B     | ~4.30 B          | OpenMDW-1.1|
+-----------------------------------------+-------------+------------------+------------+

The language model backbone features 52 distinct layers organized to limit quadratic attention overhead over long sequence lengths:

23 Mamba-2 State-Space Layers: Handle linear-complexity recurrent sequence modeling, maintaining state across long temporal contexts without blowing out KV-cache limits.
23 Mixture-of-Experts Layers: Each MoE layer contains 128 discrete routed experts alongside 1 shared expert. The routing gate routes each token to the shared expert plus the top 6 routed experts.
6 Multi-Head Attention Layers: Periodically inject global cross-token context across the entire sequence budget.

Because only 6 of the 128 routed experts fire per token, the language model backbone activates roughly 3.5 billion parameters per step. Adding the 0.6-billion-parameter acoustic encoder, the 100-million-parameter diarization front-end, and the projection layers yields an operational runtime cost equivalent to a 4.3-billion-parameter dense model.

On the Artificial Analysis accuracy-versus-compute Pareto frontier, this configuration demonstrates measurable mechanical efficiency. Competing open-weights speech LLMs, such as Mistral's Voxtral Small, achieve comparable transcription accuracy (2.77% AA-WER versus 2.80% for Transcribe) but run as 24-billion-parameter dense architectures. Generating a 1,000-word transcript with Nemotron 3.5 Transcribe requires evaluating roughly 4.3 billion parameter operations per generated token, compared to 24 billion operations per token on dense equivalents—a 5.58-fold reduction in mathematical floating-point operations (FLOPs) per token.

Benchmark Audits: Transcription, Diarization, and Context Recovery

Nvidia’s technical report and independent evaluations from third-party testing platforms quantify the model’s performance across standard transcription, overlapping speech, conversational phone calls, and domain-biased vocabularies.

========================================================================================
SPEECH RECOGNITION BENCHMARKS: OPEN-WEIGHT MODEL COMPARISON
========================================================================================
Benchmark / Metric                 Nemotron 3.5 Transcribe  Whisper Large-v3  Qwen3-ASR-1.7B
----------------------------------------------------------------------------------------
English Open ASR Mean WER (9 Sets) 4.71%                    7.16%             4.94%
Open ASR Leaderboard English Avg   5.33%                    7.16%             5.40%
Multilingual Open ASR (7 Langs)    4.62%                    6.85%             6.49%
Long-Form Open ASR Mean WER        10.98%                   14.20%            13.37%
Artificial Analysis AA-WER Index   2.80%                    4.10% - 10.10%    N/A
AA-AgentTalk (Voice Agent Audio)   1.80%                    2.90%             2.30%
Converse-STT Real Calls (Ocular)   3.03%                    4.82%             3.91%
ContextASR-Bench (Entity Errors)   1.14%                    6.84%             N/A
========================================================================================
*All error rates expressed as percentages; lower is better.

English and Multilingual Accuracy

On the Hugging Face English Open ASR Leaderboard, which evaluates models across diverse corpora including LibriSpeech, TedLium, VoxPopuli, and Common Voice, Nemotron 3.5 Transcribe recorded a 5.33% average WER. On Nvidia’s standardized nine-dataset public test split, the model averaged 4.71% WER. By comparison, OpenAI’s Whisper Large-v3 averages 7.16% across identical sets, representing a 34.2% relative error reduction.

On the Multilingual Open ASR Leaderboard across seven major evaluation languages, the model posted a mean WER of 4.62%, outperforming Qwen3-ASR-1.7B (6.49%) and Whisper Large-v3 (6.85%). On long-form recordings—defined as unsegmented audio files ranging from 15 to 60 minutes—the model logged a 10.98% WER, compared to 13.37% for the closest open-weight baseline.

On real-world telephony, the Converse-STT evaluation benchmark tested the model against 21 speech systems using live phone calls from Ocular AI. Nemotron 3.5 Transcribe posted an overall WER of 2.28% across mixed clips and calls, scoring 1.89% on Pipecat clean conversational audio and 3.03% on compressed, bandwidth-limited phone audio.

========================================================================================
SPEAKER DIARIZATION AND OVERLAPPING SPEECH LEADERBOARDS
========================================================================================
Benchmark / Metric                 Nemotron 3.5 Transcribe  Nemotron 3 Diar.  Multitalker Parakeet
----------------------------------------------------------------------------------------
David AI DAI-ASR-I18N (21 Langs)   18.26% DER               18.38% DER        21.40% DER
VoiceArena Diarization-Bench v1    32.40% cpWER             N/A               34.20% cpWER
Multi-Speaker cpWER (AMI/Mixer)    15.80% cpWER             N/A               17.90% cpWER
Converse-STT Turn Accuracy         96.20%                   N/A               91.80%
========================================================================================
*DER = Diarization Error Rate. cpWER = Concatenated minimum-permutation Word Error Rate.

Speaker Attribution and Diarization

Conventional speech processing cascades transcribe audio first and assign speaker labels later through an external clustering algorithm (such as spectral clustering or cosine-distance scoring on x-vectors). This creates alignment errors, particularly during conversational overlap.

Nemotron 3.5 Transcribe evaluates audio and speaker tags simultaneously. On David AI’s DAI-ASR-I18N benchmark—which assesses 147 hours of unscripted, two-speaker dialogues across 21 languages—the model achieved a Diarization Error Rate (DER) of 18.26%. This edged out Nvidia’s dedicated standalone Nemotron 3 Diarization model (18.38% DER) and took first place among 14 evaluated production systems.

On VoiceArena’s Diarization-Bench v1, which tests meeting recordings containing two to eight overlapping speakers, the model posted a concatenated minimum-permutation Word Error Rate (cpWER) of 32.4%. Across a multi-corpus benchmark suite encompassing AMI, CALLHOME, Mixer 6, and NOTSOFAR-1, its cpWER settled at 15.80%.

Contextual Biasing on Domain Vocabularies

Speech models frequently fail when transcribing proper nouns, pharmaceutical brand names, technical jargon, or alphanumeric product codes. Because Nemotron 3.5 Transcribe accepts text instructions alongside audio, users can supply an explicit context list of expected entities within the prompt.

On ContextASR-Bench, when supplied with a target list of 50 to 200 out-of-vocabulary domain terms, Nemotron 3.5 Transcribe reduced named-entity word errors from 6.84% down to 1.14%—an 83.3% relative reduction (six-fold decrease) in entity hallucination or phonetic corruption.

Architectural Breakdown: Parallel Expert Encoders and Modality Fusion

Nemotron 3.5 Transcribe departs from standard encoder-decoder architectures by treating automatic speech recognition as instruction-guided conditional language generation. The model architecture couples three specialized systems:

                                  +------------------------------------+
                                  | 16 kHz Mono Audio Input (<= 1 hr)  |
                                  +-----------------+------------------+
                                                    |
                               +--------------------+--------------------+
                               |                                         |
                               v                                         v
                +------------------------------+         +-------------------------------+
                | Transformer ASR Encoder      |         | Nemotron 3 Diarization        |
                | (0.6B Parameters, 32 Layers) |         | (Sortformer Streaming, 100M)  |
                +--------------+---------------+         +---------------+---------------+
                               |                                         |
                               | [128-bin log-mel, 8x subsampled]       | [Per-frame speaker activity,
                               | 80 ms audio frames                      |  up to 8 speakers]
                               | (Dim: 1280)                             | (Sinusoidal kernel: 8 x 1280)
                               |                                         |
                               +--------------------+--------------------+
                                                    |
                                                    v
                               +-----------------------------------------+
                               | Element-Wise Fusion & Projection        |
                               | (Acoustic + Speaker Vectors -> Dim 2688)|
                               +--------------------+--------------------+
                                                    |
                                                    v
                               +-----------------------------------------+
                               | Text Instruction / Prompt Context       |
                               | (e.g., "Verbatim", "Clean ITN", "Biased")
                               +--------------------+--------------------+
                                                    |
                                                    v
                               +-----------------------------------------+
                               | Nemotron 3.5 Lightning MoE Backbone     |
                               | (52 Layers: 23 Mamba-2, 23 MoE, 6 Attn) |
                               | Routes to 6 of 128 Experts + 1 Shared   |
                               +--------------------+--------------------+
                                                    |
                                                    v
                               +-----------------------------------------+
                               | Autoregressive Text Generation          |
                               | (Speaker Tags + Transcribed Words)      |
                               +--------------------+--------------------+
                                                    |
                                                    v
                               +-----------------------------------------+
                               | CTC Alignment Adapter (50M Parameters)  |
                               | Aligns tokens back to 80 ms audio frame |
                               +--------------------+--------------------+
                                                    |
                                                    v
                               +-----------------------------------------+
                               | Output: Speaker-Attributed Transcript   |
                               | with Millisecond Word-Level Timestamps  |
                               +-----------------------------------------+

The Acoustic and Diarization Branches

The front-end perception stack takes 16 kHz single-channel audio and converts it into 128-bin log-mel filterbank features computed with a 10-millisecond hop size. An 8x frame-stacking subsampling layer compresses these features, outputting one vector for every 80 milliseconds of real audio.

The ASR acoustic branch consists of a 32-layer Transformer encoder with a hidden dimension of 1,280, 10 attention heads, and a feed-forward network expansion factor of 4. It uses Rotary Position Embeddings (RoPE) and Query-Key normalization (QK-norm) to maintain numerical stability across long sequences. The encoder was initialized from a Token and Duration Transducer (TDT) checkpoint pretrained on the Granary v2 dataset.

Operating in parallel is a frozen Nemotron 3 Diarization model—a 100-million-parameter Sortformer streaming network that computes continuous speaker presence probabilities for up to eight individuals simultaneously. These probabilities are binarized at a 0.5 threshold, normalized, and mapped through a fixed sinusoidal projection matrix of size $8 \times 1280$. The resulting speaker representation is added directly to the normalized ASR acoustic embeddings every 80 milliseconds.

Because speaker identity is merged directly into the acoustic frames before entering the language model, the decoder does not require specialized multi-head classification layers to predict speaker changes. The language model emits speaker tags—such as or —as standard vocabulary tokens within the output sequence.

Modality Projection and Context Budget

The fused 1,280-dimensional acoustic-diarization representations pass through an identity connector followed by a learned linear projection layer that maps the features to the language model’s 2,688-dimensional hidden space.

The combined model shares a unified 65,536-token context budget. Each second of audio consumes 12.5 tokens after the 8x subsampling process (one token per 80 ms). Consequently, a complete 60-minute audio recording consumes:

$$\text{Audio Tokens} = 60 \times 60 \times 12.5 = 45,000 \text{ tokens}$$

This leaves roughly 20,536 tokens within the context budget for system instructions, terminology biasing lists, and the generated text transcript.

+---------------------------------------------------------------------------------------+
|                      65,536-TOKEN CONTEXT BUDGET ALLOCATION (60-MINUTE RUN)           |
+---------------------------------------------------------------------------------------+
| Component                           | Tokens Consumed | Percentage of Budget          |
+-------------------------------------+-----------------+-------------------------------+
| Audio Representations (3,600s @ 80ms)| 45,000 tokens   | 68.66%                        |
| System Prompt & Entity Hints        | 1,536 tokens    | 2.34%                         |
| Generated Output Transcript Headroom| 19,000 tokens   | 29.00%                        |
+-------------------------------------+-----------------+-------------------------------+
| TOTAL CONTEXT WINDOW                | 65,536 tokens   | 100.00%                       |
+-------------------------------------+-----------------+-------------------------------+

Connectionist Temporal Classification (CTC) Timestamping

Autoregressive language models predict tokens sequentially and lack inherent mechanisms to associate output words with exact audio timestamps. Other speech LLM implementations rely on coarse cross-attention weight inspection or force the model to generate explicit timestamp tokens, which doubles output sequence lengths and slows inference.

Nvidia resolved this by decoupling text generation from temporal alignment. Nemotron 3.5 Transcribe employs a lightweight, 50-million-parameter Connectionist Temporal Classification (CTC) adapter. During inference, the language model generates transcripts without emitting timestamp strings. Once text generation finishes, the CTC adapter computes a fast post-hoc Viterbi alignment between the generated token IDs and the acoustic encoder's 80 ms frame representations. This yields word-level timestamps without extending decoding latency.

Hardware Economics and Serving Throughput

Deploying speech recognition at enterprise scale requires balancing throughput, VRAM occupancy, and inference latency. The release of Nvidia Nemotron 3.5 Transcribe includes three primary precision configurations: BF16, FP8, and an NVFP4 build tailored for Blackwell tensor cores.

========================================================================================
VRAM OCCUPANCY AND HARDWARE REQUIREMENTS ACROSS PRECISIONS
========================================================================================
Format     Weight Footprint  Operational VRAM  Minimum GPU Profile    Primary Target
----------------------------------------------------------------------------------------
BF16       ~68.0 GB          ~76.5 - 80.0 GB   1x H100 / A100 (80GB)  Reference Evaluation
FP8        ~34.0 GB          ~38.0 - 42.0 GB   1x L40S / A100 (40GB)  Enterprise Scale-Up
NVFP4      ~17.0 GB          ~19.5 - 22.0 GB   1x RTX 4090 / 6000 Ada Edge Workstations
GGUF (Q4)  ~18.5 GB          ~21.0 - 24.0 GB   1x RTX Spark / Jetson  Embedded / Desktop
========================================================================================

In BF16 precision, the 34-billion-parameter checkpoint consumes roughly 68 GB of disk space and requires just over 76 GB of VRAM during execution, making an 80GB GPU mandatory. Quantizing the weights to FP8 cuts memory usage to roughly 38 GB, allowing deployment on mid-tier data-center hardware like the Nvidia L40S.

The NVFP4 checkpoint compresses the model footprint to roughly 17 GB. This enables local execution on a single consumer or workstation graphics card equipped with 24 GB of VRAM, such as an Nvidia GeForce RTX 4090, RTX 6000 Ada, or edge modules like the Jetson AGX Thor.

========================================================================================
MEASURED SERVING THROUGHPUT: REAL-TIME FACTOR MULTIPLIERS (RTFx)
========================================================================================
Engine / Hardware               Precision   Draft Model   Throughput (Audio Sec / Sec)
----------------------------------------------------------------------------------------
Nemotron NIM / RTX PRO 6000 Bl. NVFP4       DFlash2       2,238 RTFx (100 hrs in 2.68 min)
Nemotron NIM / H100 SXM5        FP8         MTP-2         840 RTFx   (100 hrs in 7.14 min)
vLLM Reference / H100 SXM5      BF16        MTP-2         410 RTFx   (100 hrs in 14.6 min)
vLLM Standard / H100 SXM5       BF16        None          215 RTFx   (100 hrs in 27.9 min)
Whisper Large-v3 / A100 (vLLM)  FP16        N/A           180 RTFx   (100 hrs in 33.3 min)
========================================================================================
*RTFx = Real-Time Factor Multiplier (e.g., 2,238 RTFx means 2,238 seconds of audio are 
 processed in 1 second of clock time).

Speculative Decoding via Native Multi-Token Prediction and DFlash2

A primary driver of the model's serving throughput is speculative decoding. Autoregressive language generation is memory-bandwidth bound: emitting one token requires loading the active parameters from VRAM into the compute cores.

Nvidia integrated two speculative decoding pathways:

Native Multi-Token Prediction (MTP): The model was jointly trained with an MTP objective. An integrated 1.3-billion-parameter drafting head proposes two tokens ahead per step. The main backbone verifies these proposed tokens in a single parallel verification pass, raising baseline decoding speed by 1.9x.
DFlash2 Speculative Engine: An external, non-autoregressive draft model (NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-DFlash2) predicts token blocks simultaneously based on the acoustic representations.

When served via Nvidia’s optimized Nemotron Transcribe NIM (Inference Microservice) using NVFP4 and DFlash2 on an RTX PRO 6000 Blackwell GPU, throughput reaches 2,238 seconds of audio processed per second of compute. At this rate, an enterprise can ingest and transcribe 100 hours of recorded audio in 160.8 seconds (2.68 minutes).

When tested in standard vLLM serving environments on an H100 using BF16 without NIM-specific kernel optimizations, the model achieves approximately 410 RTFx. A standard 30-minute conference call completes in 4.39 seconds.

Granary v2: A 463,062-Hour Data Curation Engine

State-of-the-art speech models often struggle with formatting consistency: numbers may be transcribed phonetically ("one hundred twenty dollars") or as formatted digits ("$120"). Conventional ASR models pick one convention at training time and bake it into the weights.

Alongside the model, Nvidia open-sourced Granary v2, an audio curation manifest and pipeline containing 463,062 hours of speech across 48 languages and 16 source corpora. Built on NeMo Curator, Granary v2 applies automated pipelines that generate three distinct reference transcripts for every audio recording:

+---------------------------------------------------------------------------------------+
|                    GRANARY v2 MULTI-TARGET TRANSCRIPTION TARGETS                      |
+---------------------------------------------------------------------------------------+
| Target Format | Target Label | Sample Pipeline Output                                 |
+---------------+--------------+--------------------------------------------------------+
| Spoken Form   | tn_raw       | "um we received twenty five euros on monday morning"   |
| Raw Written   | itn_raw      | "um we received €25 on Monday morning"                 |
| Clean Written | itn_clean    | "We received €25 on Monday morning."                   |
+---------------+--------------+--------------------------------------------------------+

Spoken Form (tn_raw): Captures verbatim speech, maintaining disfluencies ("um", "uh"), verbal hesitations, and fully spelled-out phonetic numbers.
Raw Written Form (itn_raw): Applies inverse text normalization (converting spoken numbers into formatted currencies, dates, and symbols) while retaining conversational fillers.
Clean Written Form (itn_clean): Strips conversational disfluencies and applies punctuation, sentence casing, and standardized numerical formatting.

========================================================================================
GRANARY v2 DATASET METRICS AND REJECTION FILTERING
========================================================================================
Language / Corpus               Raw Hours     Filtered Hours   Rejection Rate
----------------------------------------------------------------------------------------
VoxPopuli (22 European Langs)   354,000 hrs   111,000 hrs      68.64% (Strict Cleanup)
Cross-Lingual Audio-to-English  N/A           142,800 hrs      Curated Alignment
English-to-Multilingual Speech  N/A           55,000 hrs       Curated Alignment
Multilingual General Corpora    N/A           154,262 hrs      High-Fidelity Target
----------------------------------------------------------------------------------------
TOTAL CURATED MANIFEST          N/A           463,062 hrs      48 Languages
========================================================================================

The data pipeline enforces strict signal-to-noise and alignment checks. In the VoxPopuli corpus, for example, the NeMo Curator pipeline filtered 354,000 hours of raw parliamentary recordings down to 111,000 hours across 22 languages—a 68.64% data rejection rate designed to eliminate misaligned text, background noise, and overlapping translation tracks.

The training mixture balances supervised examples by token mass rather than duration. This allows secondary capabilities—such as cross-lingual speech translation (covering 142,800 hours into English and 55,000 hours across 31 target languages) and spoken question answering—to be injected using sub-one-percent sampling rates without degrading core transcription accuracy.

Domain Fine-Tuning Economics: The Medical ASR Case Study

While base models provide broad conversational coverage, specialized industries require vocabulary adaptation. To measure how easily teams can adapt the model, Nvidia published an empirical case study evaluating accented medical dictation.

The study evaluated the model against the Eka Care benchmark, an open testing set of 3,619 audio clips featuring Indian doctors and medical students dictating clinical evaluations in accented English. The vocabulary features complex pharmacological compounds, clinical abbreviations, and distinct regional accents.

========================================================================================
EKA CARE MEDICAL ASR BENCHMARK ADAPTATION RESULTS
========================================================================================
Model State / Training Setup            Word Error Rate (WER)   Relative Reduction
----------------------------------------------------------------------------------------
Zero-Shot Base Prompting                22.44%                  Baseline
Contextual Entity Prompting (Biased)    17.12%                  23.71%
NeMo Speech Fine-Tuned (LoRA + Enc)     12.17%                  45.77%
----------------------------------------------------------------------------------------
Total Compute Expended:                 14 GPU-Hours (1x Nvidia GB300 Accelerator)
Dataset Mixture Used:                   153 hrs Synthetic + 29 hrs Accented + 63 hrs MultiMed
========================================================================================

Using the base model without adaptation, standard prompting yielded an initial WER of 22.44% on held-out speakers. Supplying an entity biasing list containing clinical terms (such as metformin, HbA1c, and amlodipine) reduced the error rate to 17.12%.

Nvidia then applied a parameter-efficient fine-tuning workflow using NeMo Speech:

Dataset Assembly: The training split combined 153 hours of synthetic medical speech, 29 hours of accented Indian English extracted from Mozilla Common Voice, and 63 hours of open clinical audio from MultiMed.
Layer Targeting: The acoustic Transformer encoder and modality projection layers were fully trained, while the language model backbone was adapted using Low-Rank Adaptation (LoRA).
Compute Investment: The full training run, data generation, and evaluation completed in 14 GPU-hours on a single Nvidia GB300 accelerator.

On the held-out test split, the fine-tuned checkpoint recorded a WER of 12.17%—a 45.77% relative error reduction over the base model. Across 17 other downstream tasks evaluated in the technical report, domain adaptation runs routinely converged within 0.5 to 2.3 GPU-hours on a single accelerator, underscoring that fine-tuning an open-weight speech LLM does not require hyperscale infrastructure.

Enterprise TCO Modeling: Self-Hosting vs. Commercial Speech APIs

For high-volume enterprise operations—such as contact centers, compliance recording, intelligence analysis, and automated media production—speech transcription costs scale directly with audio volume.

Consider an enterprise processing 1,000,000 hours of customer support call recordings per month (equivalent to 60,000,000 minutes of audio). The economic comparison below evaluates commercial cloud APIs against self-hosting Nvidia Nemotron 3.5 Transcribe on dedicated hardware:

========================================================================================
MONTHLY INFERENCE COST COMPARISON: 1,000,000 HOURS OF AUDIO / MONTH
========================================================================================
Service / Platform               Billing Rate / Minute     Total Monthly Expenditure
----------------------------------------------------------------------------------------
Amazon Transcribe (Standard)     $0.0240 / minute          $1,440,000
Google Cloud Speech-to-Text v2   $0.0160 / minute          $960,000
AssemblyAI (Universal-3 Pro)     $0.0065 / minute          $390,000
Deepgram Nova-3                  $0.0043 / minute          $258,000
----------------------------------------------------------------------------------------
Self-Hosted: Nemotron 3.5 Transcribe (NVFP4 on Blackwell Infrastructure)
- Total Processing Seconds:      3,600,000,000 seconds of audio / month
- Hardware Footprint Required:   ~2x Servers (8x RTX PRO 6000 Blackwell GPUs each)
- Server Amortization (3-Year):  ~$18,500 / month
- Power & Colocation (12 kW):    ~$3,800 / month
- Bandwidth & Networking:        ~$2,200 / month
----------------------------------------------------------------------------------------
TOTAL SELF-HOSTED EXPENDITURE:   ~$24,500 / month ($0.00041 / minute effective)
========================================================================================
*Assumes 2,238 RTFx sustained throughput on NVFP4 with an average 70% hardware load factor.

At a sustained throughput of 2,238 RTFx, a single RTX PRO 6000 Blackwell GPU transcribes 2,238 hours of audio in one hour of clock time. Over a 720-hour operational month, one GPU operating at a realistic 70% load factor processes:

$$\text{Capacity per GPU} = 2,238 \times 720 \times 0.70 = 1,127,952 \text{ hours of audio / month}$$

Processing 1,000,000 hours of monthly audio requires just two dedicated server nodes hosting a total of 16 Blackwell GPUs, providing surplus capacity for burst demand and redundancy.

Amortizing the server acquisition cost over 36 months ($18,500/month) and adding power, colocation, and networking expenses ($6,000/month) brings total self-hosted operational expenditure to roughly $24,500 monthly—or $0.00041 per audio minute. Compared to standard managed APIs billing at $0.0043 to $0.0240 per minute, self-hosting yields an operational cost reduction between 90.5% and 98.3% at enterprise scale.

Operational Constraints and Architectural Boundaries

Despite its benchmark gains, Nemotron 3.5 Transcribe introduces operational limitations that teams must evaluate before replacing legacy voice stacks.

+---------------------------------------------------------------------------------------+
|                    CAPABILITY AND OPERATIONAL CONSTRAINT MATRIX                       |
+---------------------------------------------------------------------------------------+
| Capability / Feature          | Supported Status    | Operational Constraints         |
+-------------------------------------+---------------+---------------------------------+
| Batch Audio Transcription           | Full (40 Langs) | Up to 60 minutes per file       |
| Real-Time Sub-Second Streaming      | Not Supported   | Requires Nemotron 3.5 ASR 0.6B  |
| Speaker Diarization / Attribution   | Partial (13)    | Max 8 concurrent speakers       |
| Cross-Lingual Translation           | Partial (31)    | Must translate to/from English  |
| Audio Question Answering            | English Only    | Unreliable in non-English audio |
| Post-Decoded Word Timestamps        | Full            | Incompatible with DFlash2 engine|
+-------------------------------------+---------------+---------------------------------+

1. Batch Execution vs. Live Streaming

Nemotron 3.5 Transcribe is an offline, batch-processing system. While its internal architecture divides input audio into 80 ms acoustic tokens, the language model processes the audio sequence as a unified block. It cannot operate as a sub-second, real-time streaming engine for interactive voice agents.

For live conversational AI applications requiring sub-300-millisecond response latency, developers must use streaming-specific models—such as the 600-million-parameter Nemotron 3.5 ASR FastConformer-RNNT checkpoint—rather than this 34-billion-parameter MoE transcription model.

2. Language Asymmetry Across Tasks

While the base ASR encoder transcribes speech across 40 languages, secondary capabilities exhibit uneven linguistic coverage:

Speech Translation: Supports 31 languages, but translation pairs are restricted to translating to or from English; direct non-English to non-English translation (such as French to Japanese) is unsupported.
Speaker Attribution: Speaker diarization is trained and validated across 13 primary languages: Arabic, Mandarin, English, French, German, Hindi, Italian, Japanese, Korean, Portuguese, Russian, Spanish, and Thai. Outside these 13 languages, diarization accuracy drops noticeably.
Audio Question Answering: Audio-grounded analytical reasoning and spoken instruction following were trained almost exclusively on English corpora. Testing on non-English audio question answering yields frequent linguistic drift and factual hallucinations.

3. Inference Serving Trade-Offs

To achieve maximum serving throughput (2,238 RTFx), deployments require Blackwell-generation GPUs capable of running native NVFP4 tensor math, alongside the DFlash2 speculative decoding draft model.

In the initial release, the CTC timestamping adapter cannot run concurrently with the DFlash2 speculative decoding engine. Teams requiring both word-level timestamps and speculative decoding must execute two passes or rely on the native MTP layer, which drops peak throughput from 2,238 RTFx to roughly 840 RTFx.

The Evolving Voice Stack

The release of Nvidia Nemotron 3.5 Transcribe signals an architectural shift in production voice engineering: the replacement of fragmented speech processing pipelines with instruction-guided, open-weight foundation models.

========================================================================================
LEGACY CASCADED SPEECH STACK vs. CONSOLIDATED SPEECH-LLM STACK
========================================================================================
Feature / Stage            Legacy Pipeline (2022 - 2025)   Consolidated SALM (2026+)
----------------------------------------------------------------------------------------
Acoustic Modeling          Conformer / Whisper Base        Transformer Encoder (0.6B)
Speaker Labeling           PyAnnote / External Diarizer    Parallel Sortformer Embedding
Punctuation & ITN          NeMo Inverse Text Norm Scripts  Instruction-Prompted LLM
Entity Vocabulary Biasing  Wand/WFST Language Model Rescore Context Window Text Prompts
Word Timestamping          Forced Alignment Pass (Kaldi)   Integrated CTC Viterbi Adapter
Operational Pipeline       4-6 Discrete Software Services  1 Unified MoE Checkpoint
Deployment Flexibility     CPU/GPU Hybrid Cluster          Single Workstation GPU (NVFP4)
========================================================================================

By packaging transcription, multi-speaker diarization, translation, formatting normalization, and alignment into a single 4.3-billion active parameter checkpoint, the model eliminates the integration overhead that has long complicated enterprise speech infrastructure.

Developers no longer need to manage complex pipeline glue code between Whisper inference servers, PyAnnote clustering workers, and rule-based text normalizers. A single natural language prompt now dictates whether the system outputs a word-for-word legal transcript, clean business meeting minutes with disfluencies removed, or a translation into English with speaker turns attributed.

Supported by open dataset curation manifests in Granary v2 and reproducible fine-tuning recipes in NeMo Speech, the barrier to training high-accuracy, domain-specific speech engines has shifted from massive infrastructure investments to accessible single-node workflows.

For enterprise engineering teams, the critical milestones to monitor over the coming quarters are the community's adaptation of the model into quantized GGUF runtimes for consumer hardware, the optimization of parallel CTC timestamping within high-throughput speculative decoding engines, and the eventual convergence of offline batch models with low-latency streaming architectures.