腾讯发布探索基准测试ExplorationBench
腾讯团队推出探索基准测试,评估AI系统发现规则的能力,10个前沿模型测试结果揭示重要发现。
腾讯、复旦大学和清华大学联合推出探索基准测试ExplorationBench,包含两个沙盒环境AlienCode和AlienLogic,共101个任务。
测试显示,获取反馈比单独思考更有效,最佳系统在四轮反馈后准确率从15.7%提升至89.
0%。同一系统在不同沙盒中的表现差异巨大,准确率从5.
7%到79.0%不等。
New Research: We are releasing ExplorationBench, a benchmark for measuring how AI systems explore.
Scientific discovery begins where known problems end: a system has to frame hypotheses, design experiments, and learn from the results. Evaluating this is hard. Genuinely new answers cannot be checked quickly, and in familiar domains a model can simply recall what it has seen.
Addressing this challenge, researchers from Tencent Hy, Fudan University, and Tsinghua University built verifiable Alien Worlds. Their rules are executable, so every answer is checked exactly, and they conflict with familiar knowledge, so recall alone cannot solve the tasks.
🔹 Two sandboxes: AlienCode (31 hidden rule changes, 70 tasks) and AlienLogic (24 patched inference rules, 70 theorems) 🔹 A flawed manual, four rounds of self-designed probes, and closed-book tests after every round 🔹 Every answer graded by an interpreter or a proof checker, with no LLM judge
What we found across 10 frontier AI systems: 1️⃣ Getting feedback is more effective than thinking alone. No AlienCode run starts above 15.7%; after four rounds the best reaches 89.0%, while the same turns without feedback stay at 0.5–11.0%. 2️⃣ Designing the experiments matters. Replaying a system's own best probes gives it exactly the same evidence, yet in AlienCode 9 of 10 systems do worse than when they chose the probes themselves. 3️⃣ Knowing a rule is not using it. Even when every required rule is stated correctly, tasks are solved only 73.4% of the time. 4️⃣ One score hides a lot. The same system under the same budget ended anywhere from 5.7% to 79.0%, and rankings barely transfer between the two worlds.
CL-bench asked whether models can learn from context. ExplorationBench asks whether they can discover the rules themselves.
📄 Paper: https://t.co/eamwrsJyxQ 🌐 Website & leaderboard: https://t.co/gxSeoCfc2s 📝 Blog: https://t.co/jyrwH7ZnQH 💻 Code (coming soon): https://t.co/i7VlgErkcl
本条情报归属 AITOP「论文」核心栏目,重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:ExplorationBench、腾讯、AlienCode、AlienLogic,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「腾讯混元」(@TencentHunyuan),经 AITOP 多维评分管线结构化提炼。截至目前,全网已有 2 家独立信源对该事件进行交叉印证与跟进报道(含 Hunyuan 等),属于近期高关注度行业信号。
- Hunyuan09-30 13:29原文