CATCH 方法通过持续对抗训练提升 LLM 诚实度
CAT-alyzing Honesty: Continuous Adversarial Training for Transferable LLM Honesty
一篇把对抗训练用在诚实度上的论文，几百个样本就能让 Llama-3.3-70B 在三个诚实度基准上变靠谱，还能不掉能力。
论文提出 Continuous Adversarial Training for Consistent Honesty（CATCH），借鉴对抗性拒绝训练的思路，让模型朝诚实答案训练、远离不诚实答案。在 Llama-3.3-70B-Instruct 和 Qwen3-3.6-27B 两个模型上实验，仅用数百个自我报告场景构建的目标即可训练。CATCH 在 MASK 基准上降低压力下的不诚实行为，在 Among Us 多智能体游戏中提高坦白率，在 AuditBench 中提升模型隐藏行为的发现率。效果上超过提示词和监督微调，与 SOTA 诚实度激活引导相当，且因采用新的效用数据选择，在保持模型能力上优于激活引导。
CAT-alyzing Honesty: Continuous Adversarial Training for Transferable LLM Honesty
Large language models (LLMs) are becoming prevalent in scenarios where honesty is critical. However, despite alignment training, these models still exhibit dishonest behaviour, such as outputting claims that contradict some knowledge or are not supported by what is accessible in context. We introduce Continuous Adversarial Training for Consistent Honesty (CATCH), inspired by a method originally developed to robustly refuse harmful queries. CATCH trains the model towards an honest answer and away from a dishonest one while maintaining the model's capabilities. We evaluate two architectures (Llama-3.3-70B-Instruct and Qwen3-3.6-27B). We show that trained with only hundreds of targets built from self-reporting scenarios, CATCH learns an honest behaviour that robustly generalizes and transfers across distinct benchmarks. It consistently reduces dishonesty under pressure on MASK, increases confessions in the multi-agent game of Among Us, and improves the discovery rates of the model's hidden behaviours in AuditBench. We show that CATCH outperforms prompting and supervised fine-tuning in improving honesty, and it is on par with state-of-the-art honesty activation steering. However, CATCH surpasses honesty steering in maintaining utility, thanks to a novel choice of utility data.
