论文

Environment Duels:评估 LLM 出题与解题双重能力的动态基准

Environment Duels: A Microcosm of LLM Co-Evolution

精选理由

让 9 个模型互相出题、互相解题,结果最强的解题者反而不是最好的出题者,小模型出题更狠,挺有意思的视角。

arXiv 论文提出 Environment Duels,一个动态智能体基准,让模型同时担任环境的作者和解题者。每个模型编写带特权提示的交互环境,目标是设计出解题者没有提示时失败、有提示时通过的任务。在九个前沿 LLM 上的测试显示,解题与出题是两种不同能力,最强的解题者并非最强的出题者。较小的模型从其他模型的对局记录中学习更多,并写出最难的环境和最有用的提示。论文还用 Qwen3.8-27B 做强化学习实验,发现提示带来的提升与在相应环境上训练后的held-out基准改善相关。

原文 · arXiv cs.AI

Environment Duels: A Microcosm of LLM Co-Evolution

Through a tangle of real-world interactions, from synthetic training pipelines to data shared on the open web, frontier LLMs absorb what their predecessors and competitors create. How well does each model play its role in this ecosystem, as a creator and as a solver? We introduce Environment Duels, a dynamic agentic benchmark that evaluates both capabilities on interactive environments. Each model writes environments, each with a privileged hint, aiming for tasks that capable solvers fail without the hint and pass with it. Across nine frontier LLMs, solving and authoring prove to be distinct skills: the strongest solvers are not the strongest authors, while smaller models learn more from how others played and write the hardest environments and the most useful hints. Beyond the scores, the pairwise interactions form a microcosm of how models affect one another. Head-to-head outcomes are transitive and so yield a stable ranking; independent authors converge on the same designs; and in a reinforcement-learning study with Qwen3.8-27B, the learner's hint lift on an author's environments tracks how much training on those environments improves it on held-out reasoning and agentic benchmarks, whether the environments are its own or another model's. Each of these patterns has a counterpart in how models shape one another outside the duel. By staging these interactions under controlled conditions, Environment Duels offers a window onto how LLMs teach, challenge and learn from one another. Code is available at https://github.com/LLM-Duels/EnvDuels.

  • TestingCatalog10-20 22:02原文