研究揭示 LLM 如何在注意力头中编码目标并指导后续行动
Goal Encoding in LLMs
这篇论文回答了一个很有意思的问题：模型明明没有记忆机制，怎么在长任务里一直记住目标。作者找到了具体负责的注意力头，还提出了 NRGS 解释，做模型可解释性或 agent 研究的值得看原始论文。
一项新研究考察 LLM 是否以及如何内部编码目标。研究团队构建了一个包含多个子任务的编程环境，用来评估模型的目标追求行为与子目标选择，结果在一组稀疏的注意力头输出中定位到了"下一个子目标"的编码，同一组注意力头还能用自然语言解码出模型的未来动作，例如"我正在检查一个词是否是回文"。
通过因果干预实验，研究证实这一编码对下一步动作具有因果作用，并且具备可塑性：把它移植到另一个环境（比如另一种编程语言）后，模型会用不同的 token 序列实现同一操作。令人意外的是，模型在整个动作执行过程中持续维持这一编码，而普通 transformer 没有循环结构，理论上无法把状态向前传递超过剩余层数的 token。
为此研究提出非循环目标寻求（NRGS）机制：模型不需要真正携带状态，而是频繁从任务描述中重新读取目标，与上下文中可见的进展对比，然后继续追求剩余部分。研究者用定向因果干预验证了这一解释的多项预测。
Goal Encoding in LLMs
We investigate whether and how LLMs encode goals: internal neural representations that precede and guide action, and that are flexible enough to be realized through many different execution paths. We construct a coding setting with multiple sub-tasks that allows us to evaluate goal-seeking behavior and the selection of sub-goals. In this setting, we localize an encoding of the next sub-goal in the outputs of a sparse set of attention heads. The same heads can be used to decode the LM's future actions in natural language (e.g., "I'm checking if a word is a palindrome"). We verify that this encoding is causal for next action and is plastic: transplanting it into a different environment, e.g., another programming language, triggers the same operation realized with a different sequence of tokens. The LM also sustains the encoding throughout the action's execution, which is puzzling because a transformer LM without recurrence cannot carry a state forward for more tokens than it has layers left. We propose that it does not need to. Under a non-recurrent goal-seeking (NRGS) account, the LM frequently re-reads the goal from the task specification, compares it with the progress visible in the growing context, and pursues what remains. We test the predictions of this account with targeted causal interventions.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：LLM、注意力头、可解释性、NRGS，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Arnab Sen Sharma, Eric Todd, Sheridan Feucht, Andy Arditi, Rohit Gandikota, Natalie Shapira, Gabriele Sarti, Constanza Fierro, Lisa Schut, David Bau），经 AITOP 多维评分管线结构化提炼。