Open ASR Leaderboard: Trends and Insights with New Multilingual & Long-Form Tracks

The Open ASR Leaderboard has introduced new tracks for multilingual and long-form transcription, highlighting a shift toward LLM-based decoders for accuracy and CTC/TDT for high-speed processing.

150 Audio-Text-to-Text and 27K ASR models on the Hub 🤯

Most benchmarks focus on short-form English transcription (<30s), and overlook other important tasks, such as (1) multilingual performance and (2) model throughput, which can be a deciding factor for long-form audio like meetings and podcasts.

Over the past two years, the Open ASR Leaderboard has become a standard for comparing open and closed-source models on both accuracy and efficiency. Recently, multilingual and long-form transcription tracks have been added to the leaderboard 🎉

TL;DR - Open ASR Leaderboard

Best accuracy: Conformer encoder + LLM decoders (open-source ftw 🥳)
Fastest: CTC / TDT decoders
Multilingual: Comes at the cost of single-language performance
Long-form: Closed-source systems still lead (for now 😉)
Fine-tuning guides: (Parakeet, Voxtral, Whisper) to continue pushing performance

As of 21 Nov 2025, the Open ASR Leaderboard compares 60+ open and closed-source models from 18 organizations, across 11 datasets.

Models combining Conformer encoders with large language model (LLM) decoders currently lead in English transcription accuracy. For example, NVIDIA’s Canary-Qwen-2.5B, IBM’s Granite-Speech-3.3-8B, and Microsoft’s Phi-4-Multimodal-Instruct achieve the lowest word error rates (WER), showing that integrating LLM reasoning can significantly boost ASR accuracy.

While highly accurate, these LLM decoders tend to be slower than simpler approaches. For even faster inference, CTC and TDT decoders deliver 10–100× faster throughput, making them ideal for real-time or batch transcription tasks.

OpenAI’s Whisper Large v3 remains a strong multilingual baseline, supporting 99 languages. However, focusing on English tends to reduce multilingual coverage — a classic case of the tradeoff between specialization and generalization.

For long-form audio, closed-source systems still edge out open ones. Among open models, OpenAI’s Whisper Large v3 performs the best. But for throughput, CTC-based Conformers shine: NVIDIA’s Parakeet CTC 1.1B achieves an RTFx of 2793.75, compared to 68.56 for Whisper Large v3.

We’ll keep expanding the Open ASR LeaderBoard with more models, more languages, and more datasets so stay tuned 👀

Source: Hugging Face Blog