论文提出按输入检索目标状态改进 LLM 激活转向方法
When Should an LLM Steering Edit Depend on the Input?
arXiv 上这篇论文把 CAA 固定方向转向升级成按查询检索目标状态，词级和从句级编辑还给出了不同的施加深度，做可解释性研究的可以看看。
arXiv 论文研究了 Contrastive Activation Addition（CAA）这类固定方向激活转向的局限，认为转向编辑应在目标在请求类别内部发生变化时依赖输入。方法上从类均值差逐步过渡到按查询检索最近的 k 个存储状态均值，两个基线构成端点。在两个模型、六个替换语料和一个外部基准上，检索式转向在每个语料的每一层都优于固定方向转向。实验还显示词级编辑适合在网络前十分之一深度用少量近邻完成，从句级编辑则可在任意深度用类均值完成，并由四个仅从状态计算的量判别适用情形。
When Should an LLM Steering Edit Depend on the Input?
An activation steering edit should depend on the input exactly when the destination varies \emph{inside} the class a query requests, and that is measurable before any rule is fitted or applied. Contrastive Activation Addition (CAA) adds one fixed direction to every input, the difference of the class means. Conditioning it on the requested class accounts for the between-class variation in the required displacement and leaves the within-class variation untouched, which no rule that reads only the source state can express. We study the impact of increasing input-dependence in the steering edit, moving from a difference of means to a destination retrieved per query --- the mean of the $k$ stored states nearest the query, inside the requested class --- in which the two baselines are endpoints: at $k$ equal to the class size the retrieved rule \emph{is} the class-conditional difference of means. We measure input-dependent steering on six substitution corpora that vary in whether the edit changes a word or a clause, and on an external benchmark, and find that input-dependent steering outperforms fixed-direction steering when the destination varies within the class. Retrieval wins at every layer of every substitution corpus in two models, and the level of input-dependence needed governs how to apply it: word-level edits are best made in the first tenth of the network from a few neighbours, clause-level edits at any depth from the class mean. Where the class already determines the answer the class mean suffices and retrieval only matches it, though a fixed direction still fails there. Four quantities computed from the states alone identify the regime in which the edit should be applied.