Voice Arena 发布 Dictation Bench：首个 AI 听写应用开源基准
.@chamath recently argued that we should evaluate the model and the harness together, not make decis...
Voice Arena 在 Mac 上用 761 段录音实测了 10 款听写应用，5 个 LLM 评审只判草稿能不能直接发。结果 Wispr Flow 79%、Apple 35%，差距很直观。经常用语音输入或语音写 prompt 的人可以照着这份白paper选工具。
Voice Arena 发布 Dictation Bench，号称首个针对 AI 听写应用（而非 API）的开源基准。测试团队在 Mac 上向 10 款应用播放 761 段录音（来自 87 位说话者），再由 5 个 LLM 评审判断每份草稿能否直接发送。结果显示 Wispr Flow 达到 79%，Apple 自带听写仅 35%。
该基准的核心方法是只看最终成品：评审判读的是用户真正会发送出去的文本，而不是底层模型的转写准确率。它进一步拆分了"需要修几次"和"要改多少字"两个维度，并展示失败案例的具体内容。这一思路呼应了 Chamath 近期的观点——应把模型和 harness 作为一个整体来评估，而不是只看模型排行榜。
对在 macOS 上选择英文听写工具的用户来说，这份附带白 paper 的测试提供了可检验的依据。它把评测焦点从 API 层面拉回真实产品，为工作流选型提供了具体参考。
.@chamath recently argued that we should evaluate the model and the harness together, not make decis...
. @chamath recently argued that we should evaluate the model and the harness together, not make decisions from model-only leaderboards. @voicearena_ai 's Dictation Bench is a practical example of that broader shift: judge the product people actually use. The question is immediately useful. If I am choosing a dictation tool, I want to know what ends up in my message or coding prompt, not just how accurately the underlying model transcribes speech. This benchmark puts ten products through the same 761 recordings from 87 speakers and examines the drafts they deliver. It separates how often something needs fixing from how much text needs fixing, then shows the failures behind those numbers. That gives me a concrete basis for deciding which tools to test for my own workflow. Congratulations @voicearena_ai on an incredibly thorough benchmark. The detailed whitepaper makes the comparison inspectable; the focus on finished output makes it useful. For US-English dictation on macOS, this is the kind of evidence I would want in front of me before choosing a product, not just another accuracy claim. Shobhit Banga @shobhitbanga Drop 4/15: Announcing the Voice Arena Dictation Bench - the first open benchmark of AI dictation apps. Not their APIs: the real apps. We played 761 recordings from 87 speakers into each of 10 apps on a Mac, then 5 LLM judges asked if each draft could be sent as-is. Wispr Flow: 79%. Apple: 35% 🧵
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Dictation Bench、Wispr Flow、voicearena、听写工具，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「AI Breakfast」（@AiBreakfast），经 AITOP 多维评分管线结构化提炼。
