NVIDIA NeMo Speech is the open-source framework behind some of the most accurate automatic speech recognition models available today. Released under Apache-2.0 with more than 17,000 GitHub stars, it is the toolkit that produces NVIDIA's Parakeet and Canary ASR models, several of which top the English Open ASR Leaderboard. In 2026 the project was split out of the larger NeMo repository into a dedicated home focused on audio, speech, and multimodal speech LLMs, signalling how central speech has become to NVIDIA's open model strategy. ## What It Is NeMo Speech is built for researchers and PyTorch developers working on speech models, spanning automatic speech recognition (ASR), text-to-speech (TTS), and speech LLMs. It provides the training recipes, model architectures, and inference tooling that turn raw audio into text and back again, and it publishes ready-to-use open-weight checkpoints on Hugging Face. Rather than being a single fixed model, it is the engine that a family of production-grade speech models is built and served from. ## State-of-the-Art Recognition Models The recognition lineup is the headline. Parakeet V3 and Canary V2 deliver speech recognition and translation across 25 European languages, while Canary-Qwen-2.5B reached a record 5.63% word error rate on the English Open ASR Leaderboard, an unusually strong result for an openly available model. This is the practical payoff of the framework: teams can run transcription that is competitive with, and in some benchmarks ahead of, closed commercial APIs, on their own hardware and with open weights. ## Streaming and Latency Control NeMo Speech puts real effort into the latency-accuracy trade-off that matters for live applications. Nemotron-3.5-ASR-Streaming-0.6B supports 40 languages with controllable latency from roughly 80 milliseconds to one second, and can handle hundreds to thousands of concurrent streams on a single H100 thanks to a cache-aware FastConformer architecture. Parakeet-unified-en-0.6b goes a step further by combining high-quality offline and streaming inference in a single English model, with a minimum latency around 160 milliseconds and built-in punctuation and capitalization. The recurring theme is letting developers pick their own point on the latency-accuracy curve instead of forcing a single default. ## Beyond Recognition Though its recognition models draw the most attention, the framework is broader. It also covers TTS through models like MagpieTTS for multilingual synthesis, and full-duplex conversational speech through Nemotron VoiceChat, which pairs a Nemotron LLM backbone with NeMo speech and TTS decoders for natural, interruptible dialogue. That end-to-end coverage lets a team build an entire voice stack, from transcription to response to synthesis, within one consistent framework. ## Usability in Practice For teams already invested in NVIDIA hardware, NeMo Speech slots in cleanly, with open checkpoints on Hugging Face, packaged NIM microservices for deployment, and NGC containers for reproducible environments. The main considerations are that it is GPU-centric and genuinely a research-grade framework, so there is a learning curve compared with a hosted transcription API, and the project is actively transforming after the mid-2026 repository split, with the first standalone release scheduled around that time. Users who want stability today are pointed to the tagged container releases while the new repo matures. For quick one-off transcription, a lighter tool may be simpler; for building a serious speech product, the depth pays off. ## Pros and Cons The strengths are compelling: benchmark-leading ASR accuracy, genuine streaming with controllable latency, multilingual coverage, open Apache-2.0 weights, production tooling through NIM and NGC, and the credibility of NVIDIA's research behind it. The trade-offs are that it is optimized for NVIDIA GPUs, the framework carries real complexity and a learning curve, its broad scope can be more than a simple transcription task needs, and the ongoing repository transition means some churn and potential breaking changes in the near term. ## Outlook NeMo Speech reflects a broader shift in which open speech models have caught up to, and in places surpassed, proprietary services on accuracy while remaining self-hostable. As voice interfaces spread across agents, devices, and real-time applications, a framework that spans accurate recognition, low-latency streaming, and full-duplex conversation is well positioned. The dedicated repository and steady cadence of new Parakeet, Canary, and Nemotron speech checkpoints suggest sustained investment rather than a plateau. ## Conclusion NVIDIA NeMo Speech is a powerful, production-oriented framework for teams that need best-in-class open speech recognition and a path to full voice pipelines. It is best suited to developers with NVIDIA hardware and real speech-AI ambitions, and less convenient for those wanting a lightweight, hardware-agnostic transcription tool. For anyone building serious speech-to-text or voice applications on open models in 2026, it is among the strongest foundations available.