Re-Search：用二十问游戏拆解 LLM 的科研构思能力
Re-Search: Dissecting LLM Ideation Through a Two-Player Questioning Game
ICML 2026 的 200 篇论文被变成一个二十问游戏，让 GPT 等六个模型倒推论文核心想法，结果发现它们识货但不擅长自己想出来。做自动科研或智能体搜索的人，这篇给的“外部注入候选”思路值得直接搬进你的 pipeline。
一篇 arXiv 论文提出 Re-Search，把二十问游戏改造成科研构思评测：模型只拿到已发表论文的研究问题，通过提出是/否问题来还原论文核心想法。基准由 200 篇 ICML 2026 论文构建，把“构思”从“执行”中剥离出来，追踪模型想法随二值反馈逐步演化的过程。
在六个近期 LLM 上的结果显示：100 个问题的配额里，几乎全部进展发生在前 30 个问题内，之后多数模型进入平台期。通过受控干预候选问题和假设空间，作者发现瓶颈在生成而非判断——模型面对好想法或好问题时能识别，但自己往往提不出来。轨迹分析还揭示两类失败模式：过度深耕单一方向，或同时探索过多方向。
这一结论对自动科研系统设计有直接指导意义：既然模型能评判自己生成不了的候选，引入外部来源的想法和问题来丰富搜索，可能比单纯提升单个模型的生成能力更有效。
Re-Search: Dissecting LLM Ideation Through a Two-Player Questioning Game
Research ideation is iterative: researchers refine their ideas step by step based on feedback. Yet existing evaluations obscure this search process, making it difficult to understand how LLMs explore the space of research ideas and what limits their progress. We introduce Re-Search, which adapts the format of Twenty Questions to research ideation: given only the research problem of a published paper, a model must rediscover the paper's core idea by asking yes-or-no questions. Built from 200 ICML 2026 papers, Re-Search tracks how a model's idea evolves as it asks questions and receives binary oracle feedback, isolating ideation from execution. Across six recent LLMs, nearly all progress happens within the first 30 of 100 questions, which largely decide how the game ends; after that, most models plateau. Through controlled interventions on the candidate questions and the hypothesis space, we find that the bottleneck is generation, not judgment: models recognize a good idea or a good question when one is offered, but often cannot produce it on their own. We further analyze the ideation traces and find that models fail by either over-exploiting one direction or over-exploring too many. These findings point to concrete design principles for autoresearch systems: since models can judge candidates they cannot produce, enriching the search with externally sourced ideas and questions may be more effective than improving any single model's generation ability.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Re-Search、LLM、ICML、科研智能体，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Muhan Gao, Hangxiao Zhu, Xinqi Su, Yu Zhang, Daniel Khashabi, Kuan-Hao Huang），经 AITOP 多维评分管线结构化提炼。