Vals AI发布递归自改进指数评估模型
Vals AI CEO详解如何评估模型自我改进能力,解决了行业缺乏统一标准的问题。
Vals AI推出了递归自改进指数(RSI),用于评估模型构建自身后续版本的能力。该指数通过预训练、后训练、工程级别等多个维度的代理指标进行评估。
目前大模型实验室开始在其模型卡片中报告RSI潜力,但缺乏统一标准。这一评估方法旨在提供跨模型的公平比较基准。
Vals AI CEO Rayan Krishnan on how they score a model's ability to build its own successor: "One benchmark we released recently is our Recursive Self-Improvement Index. It's a topic which a lot of the big labs have been talking about and starting to report on in their model cards." "There isn't a shared language to talk about the RSI potential of models. So we created this as an apples-to-apples way to actually benchmark across the models." "In an ideal world, what you want to do is take a frontier model and have it train the next version of itself and see where the delta comes from. But obviously that's very expensive and slow." "What we're doing is forming a set of proxies for every part of the process it takes to build the next version of the model." "There's work around pre-training, post-training, harness-level engineering, and then seeing in which mechanisms and behaviors the models are able to do very good research work and build something new, and where they're struggling." @RayanKrishnan @JenniferHli
本条情报归属 AITOP「模型」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Vals AI、递归自改进指数、模型评估、RSI,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「a16z」(@a16z),经 AITOP 多维评分管线结构化提炼。