FETTUCCINE:首个可在手机上实时运行的脑机接口文字解码框架
FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devices
失语患者的脑机接口系统以前得靠服务器级算力,这篇论文用 ASR 架构把解码搬到了手机上,还能跑到 370 倍实时。4.39% 的词错误率比以前的端到端方法更低,做脑机接口或语音模型的开发者都值得看看技术细节。
arXiv 论文 FETTUCCINE 提出端到端脑机接口文字解码框架,借用 ASR 模型的速度和可移植性,替代传统依赖重型语言模型的神经假体解码方案。在使用皮层内语音神经假体用户的公开数据评测中,词错误率低至 4.39%,优于此前端到端方法。
技术上,模型可直接在市售移动设备上运行,吞吐量达 370 倍实时,是首个在移动设备上完成脑转文字解码的演示;微调后性能可延续到后续使用日,还能通过梯度显著性图定位对解码最关键的神经特征,结果与已知生理先验吻合。
此前系统受限于计算密集的语言建模栈,推理延迟波动大,难以支撑自然对话;FETTUCCINE 将 WER 控制在 10% 以内的可用水平,并解决了部署和跨日稳定性问题,让失语患者通过便携设备交流成为可能。
FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devices
Recent advances in speech neuroprostheses have enabled new communication avenues for people who have lost the ability to speak due to neurological illness or injury. These systems can decode neural activity during attempted speech into text by relying on language priors to achieve high decoding accuracy. This performance currently comes at the cost of using a compute-intensive language modeling stack, presenting a major bottleneck for real-world use. Further, current language models incur highly variable and disruptive inference latencies, limiting communication throughput and preventing users from reliably participating in natural conversation. We introduce FETTUCCINE, an end-to-end brain-to-text framework that leverages the speed and portability of automatic speech recognition (ASR) models to address the aforementioned practical challenges, while achieving usable (<10%) word error rates (WER), efficient adaptation to neural non-stationarities, and reliable performance on mobile devices. When evaluated using publicly available data from an intracortical speech neuroprosthesis user, FETTUCCINE outperforms previous end-to-end methods, achieving as low as 4.39% WER. Most notably, our models successfully run on a commercially-available mobile device with throughput reaching up to 370x real-time, providing the first demonstration of brain-to-text decoding on a mobile device. We also show that our models can be successfully finetuned to extend usable performance to future days. Beyond performance, our approach enables localization of the neural features most relevant for decoding via gradient-based saliency maps, which we show align with well-established physiological priors across different ASR model families. Taken together, our results show that FETTUCCINE overcomes core infrastructural and computational barriers, yielding a new class of portable brain-to-text communication.
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:FETTUCCINE、脑机接口、语音识别、神经假体,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Jonathan D. McCart, Pranav I. Deevi, Mehdi Azabou, Nanda H. Krishna, Caleb A. McKinney, Nicholas S. Card, Sergey D. Stavisky, Chethan Pandarinath),经 AITOP 多维评分管线结构化提炼。