arXiv 论文:任务委派式生成搜索效果不佳,直接工具调用更稳
Lost in Delegation: Studying the Effectiveness of Task Orchestration in Generative Search
做 multi-agent 系统的人应该看看这篇。它用 pre-/post-cutoff 数据集实测发现:委派别的模型干活有时还不如让同一个 LLM 直接调工具,还总结了查询被扭曲、信息传递丢失等失败模式,能帮你判断什么时候别乱委派。
arXiv 论文《Lost in Delegation》用带 pre-/post-cutoff 切分的自建数据集,系统比较了生成搜索中不同委派模式的效果,结论是委派相比不委派并未带来一致的准确率提升,效果因模型和任务而异。
当任务涉及编排 LLM 知识截止日期之后的信息时,委派会明显降低准确率;同样场景下,让同一个 LLM 直接调用工具的表现好得多。论文归纳出三类反复出现的失败模式:委派产生的查询被扭曲、工具使用低效、以及多个 agent 之间信息未能保留和传递。
论文指出,有效的编排依赖两点:委派决策本身的校准,以及 agent 之间信息交换的可靠性。对构建 multi-agent 系统的开发者来说,这意味着“能委派就委派”不是默认最优解。
Lost in Delegation: Studying the Effectiveness of Task Orchestration in Generative Search
LLMs have quickly evolved from models used in isolation to agentic systems that delegate tasks to external tools and other LLMs to solve complex assignments. This agentic augmentation has led to impressive capability gains. However, when and how delegation improves performance remains poorly understood. For example, it is unclear whether delegation helps when tasks involve post-cutoff information that the orchestrating LLM cannot reliably resolve from its own knowledge. In this work, we use purpose-built datasets with pre- and post-cutoff splits to compare various delegation modes. We find that delegation provides no consistent accuracy improvement over non-delegated setups, with effects varying across models and tasks. When tasks involve knowledge unavailable to the orchestrator, delegation can substantially degrade accuracy, while direct tool use with the same LLM performs much better. Our analysis identifies distorted delegated queries, inefficient tool use, and failures to preserve and use information between agents as recurring failure modes. These findings suggest that effective orchestration requires both well-calibrated delegation decisions and effective exchange of information between agents.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:多智能体、RAG、LLM、工具调用,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Tamara Stojanovska, Elisabeth Kirsten, Leon Swazinna, Muhammad Bilal Zafar),经 AITOP 多维评分管线结构化提炼。