论文

arXiv 论文提出带风险认证的 LLM 守栏提示词优化框架 CPS 与 CRISP

Risk-certified prompt optimization for LLM-as-a-judge guardrails

精选理由

安全团队常用系统提示词做 LLM 审查守栏,但普通提示词优化只追平均分。这篇论文的 CPS 和 CRISP 给漏检攻击率加了统计认证上限,四个数据集验证有效,做内容安全或 agent 守栏的值得看看。

arXiv 论文将 LLM-as-a-judge 守栏的系统提示词选择建模为约束优化问题:在将漏检攻击率控制在用户指定水平 α 之内的前提下最大化良性通过率,并提供有限样本统计认证。论文给出两种实现:CPS 从训练数据生成固定提示词池,用同时上置信界在留出评估集上认证候选,选出效用最高的合规提示词;CRISP 则把风险反馈融入自适应搜索,通过难例引导改写和可复用留出集接口向可行效用边界推进。

正文

实验覆盖提示词注入、系统提示词泄露和敏感信息披露四类数据集,结果显示经风险认证选出的提示词在留出测试上稳定不超目标漏检水平,放宽风险额度后效用随之提升;当固定池搜索找不到可认证的高效用候选时,CRISP 更为有效。对依赖系统提示词做守栏控制的团队,这提供了区别于平均性能优化的安全约束路径。

原文 · arXiv cs.LG

Risk-certified prompt optimization for LLM-as-a-judge guardrails

LLM-as-a-judge guardrails are often controlled primarily through system prompts, yet standard prompt optimization methods target average performance rather than explicit safety constraints. This paper studies system-prompt selection as a constrained optimization problem: maximize benign pass rate while keeping the missed-attack rate below a user-specified level α with finite-sample statistical control. The framework combines LLM-based judging, automatic prompt generation, and distribution-free risk certification. This work instantiates it in two procedures: CPS and CRISP. CPS generates a fixed pool of prompts from training data, then certifies candidates on a held-out evaluation set using simultaneous upper confidence bounds and selects the certified prompt with the highest benign utility. CRISP instead integrates risk feedback into an adaptive search, using hard example-guided rewriting and a reusable-holdout interface to steer generation toward the feasible utility frontier. Experiments on four datasets covering prompt injection, system-prompt leakage, and sensitive-information disclosure show that risk-certified selection consistently respects the target missed-attack level on heldout tests, while utility improves as the permitted risk is relaxed. CRISP is especially useful when fixed-pool search fails to produce certifiable high-utility candidates.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:LLM-as-a-judge、提示词工程、AI安全、prompt injection,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Hussein Jawad, Alexandre Landeau, Guy Fleury Iradukunda, Paul Beaumont, Théodore Rezel, Sofiane Schaack, Nicolas J-B. Brunel),经 AITOP 多维评分管线结构化提炼。