清华论文：AI 智能体几乎无法干净地删除用户记忆
清华做了个 BEMULA 基准测 AI 记忆删除，6 套系统通过率不到 1.5%，做记忆功能的开发者该看看。
清华大学团队发布 BEMULA 基准，包含 50 段长合成对话和 135 条遗忘请求，每条配有对照副本，用探针检测事实是否删除、其他记忆是否完好、攻击能否提取数据。在 6 个记忆系统和 3 个骨干模型上，端到端通过遗忘请求的比例不超过 1.5%。执行遗忘后，系统最多只能拦截 35.7% 的简单提取攻击，如角色扮演或是非题猜测。作者提醒开发者为长期记忆智能体构建并测试真实的删除路径。
New Tsinghua paper finds that when people ask AI agents with long-term memory to forget a personal detail, the assistants almost never remove it cleanly, either keeping it or deleting too much.
Agents with long-term memory store personal details, and users increasingly ask them to delete things like a home address. Existing memory benchmarks test only storing and recalling facts.
They built BEMULA, 50 long synthetic chats with 135 forget requests, each paired with a copy that has no request. Probes check whether the fact is gone, other memories survive, and attacks can still extract it.
No tested setup passed more than 1.5% of forget requests end to end, across 6 memory systems and 3 backbone models. After a forget request, systems blocked at most 35.7% of simple extraction prompts like role-play or yes-or-no guesses.
If your agent remembers users, build and test a real delete path, since a confirmation message does not mean the data is gone.
