论文

UbiHearo 论文提出面向听障用户的场景感知声音辅助移动智能体

UbiHearo: Bringing Scenario-Aware Sound Guidance to DHH Users with Mobile Agents

精选理由

一篇把声音语义讲清楚的听障辅助论文,端侧音频语言模型靠结构化中间表示推理,不用写提示词,省电还能个性化。做端侧多模态、无障碍应用的研究者和工程师值得细看,车载场景的实验也有参考价值。

arXiv 论文 UbiHearo 针对听障(DHH)用户的声音感知问题,提出一个能闭环完成感知—推理—行动的移动端听觉辅助智能体。现有移动助手大多把声音简化为孤立标签或笼统提醒,无法区分同一声音在不同场景下的含义与紧急程度。

正文

技术上 UbiHearo 采用神经符号(neuro-symbolic)架构,把连续的声学与物理上下文流转化为结构化中间表示(IR)。三个核心机制包括:感知阈值触发以实现省电的常开监听并保留安全关键提示的确定性覆盖;分阶段神经符号推理框架让端侧音频语言模型基于结构化 IR 完成免提示的场景理解;以及有界人机协同个性化机制,通过轻量端侧决策边界更新适配用户偏好。

团队在智能手机和车载系统上跨多种场景评测,结果显示在精度—延迟—能耗上取得较优折衷,并支持个性化引导。对端侧多模态推理和无障碍应用开发者而言,这篇论文展示了结构化 IR 替代提示词工程的可行路径。

原文 · arXiv cs.AI

UbiHearo: Bringing Scenario-Aware Sound Guidance to DHH Users with Mobile Agents

For Deaf and Hard-of-Hearing (DHH) users, effective sound awareness goes beyond detecting \emph{what} sound occurs; it requires understanding \emph{what it means}, \emph{how urgent it is}, and \emph{what to do}. However, existing mobile assistants largely reduce acoustic scenes to isolated labels or generic alerts, failing to provide contextual guidance when the same sound implies different risks across scenarios. We present \sysname, a mobile hearing assistant agent that closes the sensing--reasoning--action loop for \textit{scenario-aware} and \textit{personalized} DHH assistance. \sysname adopts a \textit{neuro-symbolic} design that transforms continuous acoustic and physical context streams into structured intermediate representations (IRs) for controllable reasoning and guidance. It introduces three key mechanisms. \textit{First}, a perception-thresholded sensing mechanism enables energy-efficient always-on awareness by activating processing only for perceptually meaningful events while preserving safety-critical cues through deterministic overrides. \textit{Second}, a staged neuro-symbolic reasoning framework performs promptless scenario understanding by conditioning an on-device audio-language model on structured IRs instead of user-authored context descriptions. \textit{Third}, a bounded human-in-the-loop personalization mechanism adapts user-specific guidance through lightweight on-device decision-boundary updates while preserving population-aligned reasoning. We evaluate \sysname on smartphones and in-vehicle systems across diverse scenarios. It achieves favorable accuracy--latency--energy tradeoffs and personalized guidance.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:音频理解、神经符号、无障碍、端侧推理,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Fengmin Wu, Sicong Liu, Zimu Zhou, Chenren Xu, Wanbing Zhao, Peiwen Sun, Zhiwen Yu),经 AITOP 多维评分管线结构化提炼。