论文

研究提出 DCPS:简单生成-选择即可匹敌复杂提示词优化器

Simplicity Goes Far: Auditing Prompt Optimizers with Demonstration-Conditioned Prompt Search

精选理由

很多人以为提示词优化必须上 PPO 或进化搜索,这篇论文用实测数据说明:简单生成-选择就能在低 32% 成本下打平 GEPA 和 PPO。做 Agent 或 DSPy 管线的人应该先跑这个免费对照再决定要不要上复杂优化器。

一篇新论文提出 Demonstration-Conditioned Prompt Search(DCPS),将 Automatic Prompt Engineer(APE)流程改造成机制归因控制:冻结生成器、基于随机采样的示例生成候选提示词、在验证集上打分并保留最优,不涉及任何学习更新、反思或进化搜索。

正文

作者用 DCPS 审计了两类主流优化器。在 StablePrompt 中,DCPS 替换 Proximal Policy Optimization(PPO)后,分类准确率差距在 0.

5 个百分点以内,成本约为原方法的三分之一;在基于 DSPy 的复合 AI 系统中,DCPS 在 GPT-4.

1-mini 上以 55% 更少的 rollout、平均 32% 更低的 API 成本,匹配了 GEPA 的 macro accuracy。审计揭示出任务依赖的模式:生成-选择机制在结构化与指令跟随任务上能复现大部分复杂优化器的性能,而在部分开放式与重推理任务上,额外机制仍保有优势。

作者建议基准测试应将提示词优化器与生成-选择对照比较,并报告 rollout 数量、token 用量与美元成本。

原文 · arXiv cs.AI

Simplicity Goes Far: Auditing Prompt Optimizers with Demonstration-Conditioned Prompt Search

Automatic prompt optimization is growing more elaborate. Recent methods stack policy-gradient updates, trace reflection, and evolutionary search. These methods share a generate-and-select step; without controlling for it, reported gains may conflate the contributions of optimizer mechanisms and larger sampling budgets. We isolate this step as Demonstration-Conditioned Prompt Search (DCPS), repurposing the Automatic Prompt Engineer (APE) recipe as a mechanism-attribution control: it freezes the generator, conditions it on randomly sampled demonstrations, scores candidates on validation data, and keeps the best---no learned update, reflection, or evolution. Across two optimizer families, DCPS replaces Proximal Policy Optimization (PPO) in StablePrompt, matching classification accuracy within 0.5 pp at roughly one-third the measured cost; in DSPy-based compound AI systems, under the reported budgets, it matches GEPA's macro accuracy on GPT-4.1-mini while using 55% fewer rollouts at 32% lower measured API cost on average. The audit reveals a task-dependent pattern: proposal and selection recover much of heavier optimizers' performance on structured and instruction-following tasks, while additional mechanisms retain advantages in some open-ended and reasoning-heavy settings. We recommend benchmarking prompt optimizers against generate-and-select controls and reporting rollout counts, token usage, and dollar costs alongside accuracy. Code and artifacts: https://github.com/bhc6/DCPS.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:GEPA、DSPy、APE、提示词工程,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Bohe Chen),经 AITOP 多维评分管线结构化提炼。