论文精选

Meta 研究发现预训练模型加轻量脚手架在智能体任务上可超越 RL 后训练版本

精选理由

Meta 这篇论文挺反直觉的:给基座模型配个轻量脚手架,多采样几次,居然比 RL 后训练版本更能解智能体任务,做训练的人可以看看。

Meta Superintelligence Labs 的论文提出 Sharpening Tax 概念:RL 后训练让模型每题趋于必对或必错,一致性上升但覆盖面下降。

正文

在 BFCL v4 多轮、ACEBench 和 WebShop 上,42 对基座与后训练模型对比显示,采样数足够大时基座模型常能解出后训练模型从未解开的任务。团队据此提出 PTGS 方法,在 RL 阶段按每条提示词的估计难度设置采样温度,既缩小 Sharpening Tax 又提升 pass@1。

原文 · elvis

Great paper for all you pre-training and post-training nerds. The surprising finding is that pre-trained LLMs, equipped with a light inference harness, can serve as capable agents and can even surpass post-trained counterparts with a sufficient test-time budget. DAIR.AI @dair_ai Banger paper from Meta Superintelligence Labs. They find something super interesting and unexpected. (bookmark it) Base models with a light harness often solve more agentic tasks than their RL post-trained versions when both get enough samples. Post-trained models win on pass @1 . At large K, base models frequently solve tasks the post-trained ones never solve on BFCL v4 multi-turn, ACEBench, and WebShop. This is because post-training pushes each task toward always solved or never solved. Consistency goes up, and coverage goes down. The authors call the lost test-time scalability the Sharpening Tax. Across 42 base and post-trained pairs, it shows up in most settings, grows with model size, and can be estimated from a few rollouts. Their fix, PTGS, sets the sampling temperature per prompt from its estimated difficulty during RL. It pays a smaller tax and also raises pass @1 . Paper: academy.dair.ai/papers/sharpen… 🔗 View Quoted Tweet📊 11 ⚡

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Sharpening Tax、RL后训练、Meta Superintelligence Labs、BFCL v4,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「elvis」(@omarsar0),经 AITOP 多维评分管线结构化提炼。