StudentBench 研究显示 AI 辅导 GRE 学习效果与人类导师相当
StudentBench: AI and human tutoring yield equivalent GRE learning gains
斯坦福团队做了个 StudentBench，拿 2383 人测 AI 教 GRE，效果跟人类导师打平，价格便宜 918 倍，数学部分回得越快学生学得越好。
arXiv 论文 StudentBench 基于 175,000 条学生-AI 对话消息，对 2,383 名参与者进行 GRE 学习效果测试。结果显示 AI 辅导与专家人类导师的学习增益在统计上等效（p = .015），七个 GRE 领域中五个领域表现最好的 AI 导师超过人类导师。其中一款 AI 导师达到与人类等效的学习增益（p = .044），而成本低 918 倍——每提升一个百分点，AI 花费 0.0052 美元，人类为 4.81 美元。第二项研究让专家导师通过 2,028 次成对评分对比 LLM 生成的教案和练习题。
StudentBench: AI and human tutoring yield equivalent GRE learning gains
Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We introduce StudentBench, a suite of AI teaching evaluations and a public platform that enables large-scale data collection with over 175,000 student-AI messages to study whether large language models (LLMs) produce learning gains equivalent to human tutoring. Using StudentBench, we measured learning gains on Quantitative and Verbal GRE questions across 2,383 human participants receiving AI tutoring, human tutoring, or no tutoring. We establish that AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and in five of the seven GRE domains, the best performing AI tutor surpassed the human tutor, on average. In a second study, expert human tutors compared LLM-generated lesson plans and practice problems through 2,028 pairwise rubric evaluations. Together, the two studies clearly separate AI tutors across: (1) lesson planning, (2) practice-problem creation, (3) conversational pedagogy, (4) cost, and (5) engagement. Surprisingly, one AI tutor achieved learning gains equivalent to human tutoring (p = .044) at 918 times lower cost (USD 0.0052 for AI versus USD 4.81 for human, per percentage point gained). For Quantitative GRE sessions, faster AI replies correlated with more student messages, more messages with more correct practice, and more correct practice with larger learning gains (all p < .002). The StudentBench platform is freely available at https://studentbench.org.
