Stanford AI Lab 提出 Self-Play 预训练：零真实数据也能训练语言模型
斯坦福团队做了个疯狂实验：不用任何真实数据，靠两个模型自博弈生成训练数据，结果还真学会了上下文学习。
Stanford AI Lab 的 Michael Y.Li 等人提出 Self-Play Pretraining with Zero Data 方法。
两个模型均从随机初始化开始：generator 为通用图灵机生成程序，learner 在其输出上训练，全程不使用任何真实数据。结果显示零样本验证损失在图像、文本、音频和旋律等自然数据上随自博弈计算量可预测地下降，且 learner 发展出 in-context learning 能力。
作者称这是一项概念验证工作。
What would pretraining with zero real data look like? Can a randomly initialized model learn to generate all of its training data entirely through self-play? Find out below! :) Michael Y. Li @michaelyli_ Can an LM, starting from random init (!!), learn to generate all of its pretraining data? Introducing Self-Play Pretraining with Zero Data. Two models start from random initialization: a generator proposes programs for a universal Turing machine and a learner trains on their outputs. We never train on any real data, but see predictable scaling on natural datasets: zero-shot val loss on images, text, audio, and melodies decreases predictably with self-play compute. And the learner develops in-context learning capabilities. A fun proof-of-concept, co-led with @AdityaCowsik and @KfirDolev and co-authors @gbruno_dl , @ANourya @noahdgoodman , and @YoavLevine . 🔗 View Quoted Tweet📊 35 ⚡
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Stanford AI Lab、Self-Play、预训练、开源模型，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「Stanford AI Lab」（@StanfordAILab），经 AITOP 多维评分管线结构化提炼。
