Harness Learning:让智能体在测试时自动改写自身执行程序
Harness Learning Enables Generalizable Test-Time Adaptation
这篇讲了个挺新的思路:不微调模型,而是让一个 proposer 用强化学习学会改智能体的 harness,测试时边跑边改。做 Agent 框架的可以看看。
论文提出 harness learning,把智能体的 harness(组织模型调用、工具使用和信息流的执行程序)作为可学习对象。方法是用强化学习训练一个 proposer 模型,以改写后 harness 的任务表现作为奖励。测试时 proposer 依据连续执行的反馈迭代优化 harness,不做任何参数更新。在推理和多跳问答任务上,改写质量提升且适配能力可迁移到未见任务。
Harness Learning Enables Generalizable Test-Time Adaptation
A language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Because different tasks call for different ways of organizing these operations, the harness needs to be adapted using feedback from the task at hand. We introduce harness learning, which trains a proposer model to revise a solver's harness using execution feedback. We formulate this process as meta-learning over executable programs, with harness revisions playing the role of weight updates in gradient-based adaptation. We train the proposer with reinforcement learning, using the task performance of revised harnesses as the reward. At test time, the proposer uses feedback from successive executions on a new task to refine the harness, without performing any parameter-space update. Experiments on reasoning and multi-hop question answering show that harness learning improves revision quality and that the ability to adapt at test time transfers to unseen tasks. Policies trained on individual revisions can continue improving harnesses over multiple rounds, while the benefits of training on revision sequences vary across settings. These findings suggest a path towards continually learning agents that turn accumulated experience into generalizable improvements.