七款模型端到端做数据科学项目：反馈循环平均分从59%提升到76%
Iterated creation, grading, and revision of data science projects by language models
七个模型轮流写、轮流评数据科学项目，谁提升大、谁自评虚高，这篇都量出来了，做评测的朋友值得细看。
一项研究让 7 个模型（来自 Anthropic、OpenAI、Google、DeepSeek、Z.AI）互为写手和评分者，完成 49 组配对、各跑 3 次的端到端数据科学项目，包括下载数据、清洗、分析和写论文。147 份初始提交中 107 份可运行渲染，平均分从 59% 升至 76%。单个模型的提升幅度从 Anthropic Opus 5.5 的 3 个百分点到 Haiku 4.5 的 46 个百分点不等。模型给自己的作品打分平均比其他模型高约 9 个百分点，且约一半的轮次间提升来自评分者自身的噪声。
Iterated creation, grading, and revision of data science projects by language models
We evaluate the extent to which agentic AI models can complete an end-to-end data science project, and how this improves with feedback. A 'writer' model with specific tools creates a complete data science project including downloading, cleaning, and analyzing a dataset, and writing a short paper about it. The project is executed and a 'grader' model evaluates it. The writer can then revise its submission, and this repeats for a total of three rounds. We consider seven models from five developers: Anthropic, OpenAI, Google, DeepSeek, and Z.AI. Each model is both a writer and a grader, so there are 49 writer-grader pairs and we run each pair three times. Of the 147 initial submissions, 107 run and render, and the average score increases from 59 per cent to 76 per cent after the final round of feedback. By writer the extent of the increase ranges from 3 percentage points for Anthropic's Opus 5.5 to 46 percentage points for Anthropic's Haiku 4.5, and by grader it ranges from 13 percentage points for Google's Gemini 3.8 Flash to 32 percentage points for Anthropic's Haiku 4.5. When a model grades its own writing the score tends to be about 9 percentage points higher than the average of what the other models give it. About half of the difference that we see from round to round is within the grader's own noise. Feedback generally results in the highest marks, but even just having the chance to revise, or having the additional budget, in terms of time or tool calls, without any feedback, results in considerable increases for some of the models but not for others. We find that how much such a loop helps depends on which model plays which role, but weaker models generally do not improve stronger models.
