Science's Last Exam 用可复现验证给科研智能体打分
给科研智能体出开放题，用隐藏检查打分，还能防背题，DeepSeek V4.1 Flash 猜中了两成多后续研究。
AItonomy Foundation 推出基准 Science's Last Exam（SLE），包含 113 个任务，覆盖 9 个学科。它不是让 AI 答有标准答案的题，而是给智能体一个研究目标、数据和实验软件，再用智能体看不到的隐藏检查来评分。任务的验证标准由领域专家在构建时封存，成果可以被打开重跑。SLE 用训练截止日期之后发表的论文来核对 AI 假设，防止记忆作弊。结果显示 DeepSeek V4.1 Flash 领先，其 21.28% 的假设预测了后续研究。
Lean gave AI math a grader nobody can argue with, and empirical science is now getting its own version of that idea.
Science’s Last Exam (SLE), a benchmark, grades research agents on artifacts that can be opened and re-run, with checks that domain experts seal when each task is built.
Led by the US nonprofit AItonomy Foundation, the benchmark spans 113 tasks in 9 disciplines and grades submissions with checks hidden from agents.
Most AI science tests ask questions with known answers; SLE hands an AI agent a research goal, data and lab software, then checks its finished work with tests the agent never sees.
It scores AI hypotheses against studies published after the model’s training cutoff, so memory cannot help. DeepSeek V4.1 Flash led, with 21.28% of its hypotheses anticipating later research.
