Beyond Word Error Rate：流式听写 ASR 的失败分类研究
Beyond Word Error Rate: A Failure Taxonomy of Streaming Dictation ASR
这篇论文把 11 个 ASR 系统的错误按四个层级逐条分类，发现 WER 相近的产品在耳语场景下表现天差地别。如果你在做语音输入或听写功能，Silero VAD 会把 Whisper 的耳语 WER 从 10.1% 推到 51.4% 这个结论直接关系到你的技术选型，全部数据集也开源了。
arXiv 论文提出超越词错误率（WER）的 ASR 评估方法，将 106 条英文听写音频（含安静环境、开风扇、耳语三种条件）流式送入 11 个 ASR 系统，并把每个错误按识别、输出格式、语言处理、音频丢弃四个层级分类。结果显示 WER 相近的系统失败方式差异很大：Zoom Scribe 与 GPT-Live-Transcribe 在安静英语上的 WER 分别为 7.4% 和 7.3%，但耳语英语下急剧恶化到 34.7% 和 8.8%。
技术细节上，Zoom 与两个 AssemblyAI 模型在耳语音频中漏掉 26-28% 的词，即使关闭端点检测仍漏 21-26%，而其他七个系统漏词不超过 3%；在 107 位 EARS 说话人上验证，同样的四个系统耳语漏词 7-31%。语言处理方面，Gemini、Deepgram 与自托管 Whisper 在仅选英语时会丢弃 86-99% 双语录音中的非英语部分。另外，Whisper 前置 Silero 语音活动检测器后，耳语 WER 从 10.1% 升至 51.4%。
对开发者而言，该研究说明部署流式听写时不能只看 WER 排名，VAD 前置与端点检测策略对低音量语音的影响可能超过模型本身的选择。代码、录音、参考文本与全部转写已公开。
Beyond Word Error Rate: A Failure Taxonomy of Streaming Dictation ASR
Speech recognition (ASR) systems are usually compared by word error rate (WER) on recorded files, after punctuation, capital letters and number format are removed. In dictation, the audio is streamed live and the text goes into the user's message as written, so what matters is which mistakes the user has to fix. The test set, streamed to eleven ASR systems, comprises 106 scripted English dictations from one speaker (read in a quiet room, with a fan on, and whispered), 20 clips in each of eleven other language and dialect sets, and 400 recordings that join clips in two languages. Every error was sorted by a fixed rule into four levels: recognition, output formatting, language handling and, for the self-hosted models, audio dropped before the model. Systems with similar WER failed in different ways. Zoom Scribe and OpenAI's GPT-Live-Transcribe have almost the same WER on quiet English (7.4% and 7.3%), but 34.7% and 8.8% on whispered English. Zoom and both AssemblyAI models left out 26-28% of whispered words, and still 21-26% with their end-of-turn detection relaxed or off; seven other systems left out at most 3%. Qwen3-ASR's 32% came from the speech detection in our own deployment (6% when relaxed). Across 107 EARS speakers, the same four systems lost 7-31% of whispered words, while six others lost under 2%. With only English selected, Gemini, Deepgram and our self-hosted Whisper lost the non-English part of a two-language recording in 86-99% of recordings, against 33-46% with both languages selected, while GPT-Live-Transcribe, OpenAI's realtime Whisper and Meta lost it in at most 20%. Running Whisper behind a Silero voice activity detector that drops audio judged to be silence raised whispered WER from 10.1% to 51.4%. Most English results come from one read speaker. The code, the recordings, both references and every transcript are released.
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：ASR、Whisper、Silero VAD、语音识别，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始信息发布自一手权威信源「arXiv: OpenAI」（Tier-1 一手发布），具备高确定性参考价值。截至目前，全网已有 4 家独立信源对该事件进行交叉印证与跟进报道（含 ElevenLabs、Gorden Sun、AWS Machine Learning Blog 等），属于近期高关注度行业信号。
