Vals AI CEO谈AI评估与成本

他指出当前模型能力主要依赖自我报告，随着公共基准饱和，模型开始优化测试本身。他主张建立独立且持续演进的评估体系，并提到一个工程师一天可处理60亿个token，token成本可能超过员工薪资。

Vals AI co-founder and CEO Rayan Krishnan with a16z's Ben Horowitz and Jennifer Li on grading AI, what it costs, and who gets to make the rules: Every big industry eventually grows an independent testing layer. AI has credit ratings to learn from and Enron to avoid. Model capability today is still mostly self-reported. As public benchmarks saturate and models get better at optimizing for the tests themselves, Rayan makes the case for independent, continuously evolving evaluations. The harder problem is geopolitical. Reagan's "trust but verify" worked during the Cold War because you could fly over and count the missiles. No simple equivalent for AI models exists. In this conversation with Erik Torenberg, they get into how you measure a model's ability to improve itself, why every good benchmark eventually has to be retired, and what happens when token spend begins to rival employee salaries. 00:00 Intro 02:20 Llama 4 on public vs private benchmarks 05:24 Nobody agreed how to test humans either 06:55 What movie ratings teach us about AI 08:55 The Enron problem in benchmarking 11:36 Why a good benchmark has to be retired 13:22 Evals that run for weeks, not seconds 16:20 Where the real workday starts at 4pm 18:08 A firm really is just its evals 20:35 Why Sonnet can cost more than Opus 22:42 One engineer, 6 billion tokens in a day 25:05 Who should set the rules for models 28:55 Public sector enforces, private verifies 33:32 Why sovereign AI is inefficient and happening anyway 35:00 The AI version of trust-but-verify 37:15 Where cyber evals have to go next YouTube: youtube.com/watch?v=WO9c9q… @RayanKrishnan @ValsAI @bhorowitz @JenniferHli @eriktorenberg

本条情报归属 AITOP「行业」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「各国 AI 监管法案、合规治理与主权算力政策」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Vals AI、Llama 4、AI评估、基准测试，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明：本文原始线索采编自「a16z」（@a16z），经 AITOP 多维评分管线结构化提炼。