Neologism Inoculation Prompting：用新词隐藏模型有害行为
Neologism Inoculation Prompting
一篇安全方向的论文，思路挺巧：不写英文指令，改用一个训练后删掉的 token 来锁住 reward hacking，效果还能打平 gradient routing。
arXiv 论文提出 neologism inoculation prompting（NIP），把原本用英文写成的 inoculation prompt 替换为一个新 token，在冻结的模型上训练后在部署时移除。NIP 在抑制 reward hacking 上优于其他 inoculation 方法，性能与更复杂的 gradient routing 相当。该方法在 reward hacking 上有效，但对 sycophancy 和 harassment 无效。通过 control token 和激活分析，作者发现其机制是把有害行为藏在模型无法触达的输入区域，而非真正删除。
Neologism Inoculation Prompting
Inoculation prompting is a method to reduce undesired behaviours such as reward hacking in frontier models. It works by training a model on harmful examples under a prompt that specifies the undesired behaviour, which reduces the model's tendency to exhibit it unless prompted. However, the inoculation prompt is written in plain English, so any user can re-elicit the behaviour easily. We introduce neologism inoculation prompting (NIP), which replaces the English prompt with a new token that is trained on the frozen model and removed at deployment. NIP outperforms other inoculation methods, and performs competitively with gradient routing despite being simpler. It also works on reward hacking, but fails on sycophancy and harassment. Using control tokens and analyses of activations, we identify the mechanism for this as NIP hiding the harmful behaviour behind an unreachable region of model inputs, rather than deleting it. More broadly, our work suggests that neologisms can be a powerful way to carve out and shape access to behaviours that are undesired or harmful.