研究：线性探针可在蒸馏与指令微调后的模型上直接复用
Linear Probes Transfer from Base to Distilled Language Models despite Rotation
训练好的安全探针换模型不用重训了，Qwen 和 Llama 系列实测迁移后 AUC 几乎不掉，不过误报率有点高，做安全审计的可以看看细节。
研究测试了安全探测分类器能否跨模型复用，涉及 Qwen2.5-Math-7B、Llama-3.1-8B、Qwen2.5-14B、Qwen2.5-32B 四个基座模型。在基座上用 XSTest 训练的线性探针，不改参数直接迁移到 DeepSeek-R1 蒸馏推理模型和指令微调聊天模型上，AUC 变化仅在 -0.011 到 +0.053 之间。虽然后继模型将unsafe/safe分离方向旋转了 17° 到 35°，探测效果基本保持。但在 OR-Bench 上，探针会把 29% 到 42% 听起来有害实则无害的提示误判，说明部分学到的是文本表层特征而非真实危害。
Linear Probes Transfer from Base to Distilled Language Models despite Rotation
Linear probes, classifiers that read a language model's internal activations, are widely used to detect harmful requests. They are cheap to train and run, and an auditor who has trained one would like to reuse it when the model is replaced by a successor rather than retrain it for every release. We test how well probes can be reused. For four base models (Qwen2.5-Math-7B, Llama-3.1-8B, Qwen2.5-14B, Qwen2.5-32B) we train a probe on the base model with the XSTest prompt set and apply it, unchanged, to two successors of that base: the DeepSeek-R1 distilled reasoning model and the developer's instruction-tuned chat model. We find that the successors rotate the direction separating unsafe from safe prompts by 17$^\circ$ to 35$^\circ$, yet on XSTest the transferred probe's AUC changes by only $-0.011$ to $+0.027$, and on an independent OR-Bench evaluation, never used for training, by $-0.015$ to $+0.053$. On XSTest, whose safe prompts deliberately contain alarming words, the probes beat a classifier that sees only the prompt text by 0.15 to 0.20 AUC. On the OR-Bench prompts that margin shrinks to 0.03 to 0.20, and the probes flag 29\% to 42\% of prompts that sound harmful but are benign, so part of what they read is the appearance of harm rather than harm itself. A probe trained on a base model can thus be reused, in these families, after distillation and after instruction tuning; how much it reads beyond textual cues depends on the test set.
