AI代币价值研究:网络文本中的缩放定律
新研究揭示AI代币在模型训练中的价值变化,提出能预测不同规模模型效果的缩放定律。
研究人员分析了2026年6月至8月网络数据,发现经过FineWeb质量过滤后,AI生成文本占比从27.5%上升至31.
1%。研究团队预训练了800个语言模型,通过调整AI代币与人类代币的比例,发现数据匮乏模型添加AI代币初期会降低人类文本损失,但添加过多会转为有害。
对于预算充足的人类文本训练模型,AI代币几乎立即增加损失,而同等数量的人类代币则持续降低损失。
How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text
"After applying FineWeb quality filtering, we find that 27.5% of tokens from June 2026 web data are labeled as AI-generated by Pangram, rising to 31.1% by August."
"How does AI text in the wild affect language model pretraining? To answer this question, we pretrain 800 language models, varying the ratio of added AI tokens to human tokens, and fit scaling laws to held-out losses on both human and AI-generated text. For data-starved models, adding AI tokens to pretraining data initially lowers loss on human text, but the benefit saturates as more are added and quickly *reverses* into harm. For models trained on high budgets of human text, AI tokens raise loss almost immediately, whil"e the same number of fresh human tokens keeps lowering it."
"We propose a new scaling law with separate benefit and harm terms that allows the value of an AI token to change sign while also reducing to Chinchilla in the absence of AI text. When fit on smaller models, our scaling law predicts the effect of AI text on held-out human-text loss for models up to 3.6x larger with 41% lower error than the best existing law over all AI ratios."
link: https://t.co/HqVuEu3DEw
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:AI代币、缩放定律、语言模型、预训练,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「Tanishq Abraham (论文推介)」(@iScienceLuvr),经 AITOP 多维评分管线结构化提炼。