研究提出 SEER-Repro 基准，用 LLM 智能体复现医学论文结论
AI-Driven Reproducibility and Analytical Robustness in Scientific Discovery
研究者做的实验挺有意思：让 AI 智能体去复现 20 篇癌症研究论文，77.4% 的论断能复现出来，还能自动检查结论稳不稳。
arXiv 论文介绍 SEER-Repro 基准，收录 20 篇基于美国癌症登记数据库 SEER 的已发表论文及其复现资源。论文构建的 AI 智能体成功复现了 2,190 条原子论断中的 77.4%，并提供可执行的基线分析。团队还设计了智能体分析流水线，对每项研究发现背后的分析选择提出替代方案并重新运行分析。在完全复现的发现中，流水线在 9.5% 中识别出可能影响结论的分析选择，标记供专家复核。
AI-Driven Reproducibility and Analytical Robustness in Scientific Discovery
Reproducible and analytically robust research findings are essential to scientific progress, but evaluating these properties typically requires costly manual effort. We investigate whether LLM agents can help automate reproduction and robustness analysis, using a medical case study grounded in the Surveillance, Epidemiology, and End Results (SEER) database, a widely used U.S. cancer registry underlying tens of thousands of published studies. First, we introduce SEER-Repro, a benchmark of 20 published SEER-based papers together with the data and resources required to reproduce their findings. Second, we create an AI agent that successfully reproduces 77.4% of 2,190 atomic claims, while providing executable baselines for further analysis. Third, we develop an agentic analysis pipeline to test the robustness of the medical findings by identifying analytical choices underlying each finding, proposing defensible alternatives, and rerunning the analysis to determine whether those choices affect the conclusion. Across published papers in SEER-Repro, our pipeline identifies potentially consequential analytical choices in 9.5% of fully reproduced findings and flags them for expert review. These results show how LLM agents can support scientific review by reproducing findings, systematically testing their robustness, and directing expert attention toward potentially consequential analytical choices.