论文精选

Meta 论文:ML 工程智能体在生产搜索排序器上完成 1097 次实验

精选理由

Meta 把智能体直接跑在生产搜索排序器上,1097 次实验自动完成、325 次被采纳,还有 0.23% 的真实线上提升。想在自己训练管线上接智能体的工程师,先看看他们怎么校准目标和设计 harness,比换更强的模型更管用。

Meta 发表一篇关于 ML 工程智能体的论文,智能体负责提出模型改动、实现、训练并评估,工程师则掌控优化目标、搜索方向和最终审查。该系统在大型 A/B 测试中应用于生产搜索排序器。

正文

在一次实验活动中,系统共完成 1097 次实验,其中 325 次被接受,接受率为 29.6%,相比早前采用不同目标时 15.6% 的接受率有明显提升。表现最好的模型在线上测试中将搜索量提升了 0.23%。多个被接受的改动是团队此前从未尝试过的架构。

团队指出三大难点:每次实验耗时数小时、离线指标与线上效果经常不一致、早期收益后增长放缓。对应方案是并行训练、将目标校准到线上影响,以及精细的上下文工程。论文强调仅靠更强的模型并不够,围绕模型的流程设计同样关键。

原文 · DAIR.AI

Great paper from Meta on ML engineering agents, tested with a large-scale A/B test on production search rankers.

(bookmark it)

They also show how important the harness design is.

The agents propose a model change, implement it, train it, and evaluate it. Engineers keep control of the objective, the search direction, and the final review.

In one campaign, the system completed 1,097 experiments and accepted 325 of them. That is a 29.6% acceptance rate, against 15.6% in an earlier campaign with a different objective. The best model raised search volume by 0.23% in the live test.

They share that three problems made this hard. Each experiment takes hours, offline metrics often disagree with online impact, and gains flatten after the early wins. The team answered with parallel training runs, an objective calibrated to online impact, and careful context engineering.

Several accepted changes were architectures the team had never tried.

The authors also report that a stronger model was not enough on its own. The loop around it mattered just as much. If you plan to put agents on your training pipeline, calibrate the objective first.

https://t.co/OeZdpd9xTQ

Chat with Paper: https://t.co/DpjKciVwe6

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Meta、ML工程、智能体、A/B测试,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「DAIR.AI」(@dair_ai),经 AITOP 多维评分管线结构化提炼。

Meta 论文:ML 工程智能体在生产搜索排序器上完成 1097 次实验|AI热报