音频语言模型评测新方法：同类声音替换对照实验
Beyond Answer Changes: Matched Same-Class Controls for Local Audio Evidence
音频模型评测有了新招：把声音换成同类还是异类，答案变不变一试便知，600 个合成场景验证了方法。
一篇 arXiv 论文提出针对音频语言模型的新评测协议，比较查询事件被替换为同类声音与不同类声音后模型答案的变化。实验覆盖 600 个合成声景和来自两个架构家族的三个模型，结果显示不同类替换导致的答案变化比同类替换多 49.7 到 60.9 个百分点。同类替换下模型准确率与原始准确率的差距保持在 5.2 个百分点以内。在原始和同类答案均正确的试验中，针对插入类别的加权选择性准确率为 73.7 到 81.9%。
Beyond Answer Changes: Matched Same-Class Controls for Local Audio Evidence
For audio-language models, an answer change after a local sound-event edit can reflect a response to the event's class, sensitivity to waveform rewriting, or both. We compare matched same-class and different-class replacements of a queried event. Both use the same interval, level estimate, donor fitting, and crossfade, while the same-class replacement preserves the correct answer. Across 600 synthetic soundscapes and three models from two architecture families, different-class replacements produce 49.7 to 60.9 percentage points more answer changes than same-class replacements. Same-class accuracy remains within 5.2 percentage points of original accuracy. Among trials with correct original and same-class answers, soundscape-weighted selective accuracy for the inserted class is 73.7 to 81.9%, averaging eligible trials within each soundscape and then weighting eligible soundscapes equally. This protocol tests how model answers respond to local changes in event class.