论文多源确认

研究复现:RL 训练使 LLM 红队攻击多样性收窄

Attacker Diversity Reduction in Adaptive LLM Red Teaming

精选理由

对做 LLM 安全的人来说这篇挺有用:它用 Qwen2.5-14B 攻 Phi-4-mini 的实战设置复现了一个此前只有 GRPO 报告过的现象,还给出了 ADR 这个可量化的指标。如果你在搭自动化红队流程,这里提供了具体训练配置,可以直接借鉴。

一项复现实验验证了强化学习在提升自动化红队攻击成功率的同时会收窄攻击多样性的现象。实验使用 Qwen2.5-14B 作为攻击方、Phi-4-mini 作为防御方,通过 DPO 训练,在 26 个 AdvBench 行为上评估,三轮独立运行的 Attack Diversity Retention(ADR)分别为 0.63、0.61 和 0.63,均低于 0.75 的多样性收窄判定阈值。

正文

技术细节上,攻击方采用仅成功奖励、长度无偏损失、有界更新、warm-up 与 ratio clipping 等手段,共执行 24 次攻击方和 12 次防御方基于 LORA 的更新,训练在 12 轮后依据开发集 checkpoint 决策停止。第 4 轮和第 8 轮的中间检查未达标,第 12 轮三轮全部达标。

值得注意的是,尽管攻击向量收窄了约 25% 以上,攻击成功率仍随每轮更新持续提升,说明多样性与有效性在对抗训练中并非同步变化。

原文 · arXiv cs.LG

Attacker Diversity Reduction in Adaptive LLM Red Teaming

Reinforcement learning can make automated red-teaming more successful while narrowing the attacks it produces. Co-training an attacker and a defender with group-relative policy optimization (GRPO) has been reported to reduce attacker diversity. This experiment reproduces attack diversity reduction using three runs with a Qwen2.5-14B attacker and a Phi-4-mini defender trained by direct preference optimization (DPO). The attacker receives a success-only reward and uses a length-unbiased loss, bounded updates, warm-up and ratio clipping. The attacker is evaluated on 26 AdvBench behaviors. Training ended after 12 rounds, following a decision based on development checkpoints, so the reported intervals are nominal. Attack Diversity Retention (ADR) compares the within-behavior of the trained attacker spread with the untrained attacker. In order for an attack diversity to be consider as having been reduced, a criterion requires the ADR to be below a 0.75 target with a 95% confidence interval together with continued attacker effectiveness. Therefore, if an attacker narrows its attack vector by 25% consistently its considered less diverse. After 24 attacker and 12 defender LORA-based updates, ADR for the three runs were 0.63, 0.61 and 0.63, respectively. Each run met the criterion at round 12; the earlier checks at rounds 4 and 8 fell short. Despite attacks being less diverse, attack success continued to improve as the attacker was updated after each training run.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:LLM红队测试、Qwen2.5、DPO、AI安全,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Chris Tava),经 AITOP 多维评分管线结构化提炼。截至目前,全网已有 3 家独立信源对该事件进行交叉印证与跟进报道(含 elvis、rohanpaul_ai 等),属于近期高关注度行业信号。