DeFiFlowBench 测试并改进自然语言 DeFi 工作流合成中的安全执行
DeFiFlowBench: Benchmarking and Improving Safe Executability in Natural-Language DeFi Workflow Synthesis
这个研究团队做了个 DeFiFlowBench 基准测试，用 207 个提示测试了自然语言 DeFi 工作流合成，发现直接、受限和少样本提示在 5% 价格影响下会有 14-19 次不安全执行。他们还提出了 Koan-Safe 模型，在 75 个测试上比最佳基线好很多，能避免不安全执行。
我们介绍了 DeFiFlowBench，一个包含 207 个团队编写的提示的基准测试，用于自然语言 DeFi 工作流合成。它测量图覆盖率、配置完整性和声明安全谓词，然后在本地 EVM 上测试受支持的交易配置。直接、受限和少样本提示在固定 5% 价格影响上限下，每个配置会产生 14-19 次不安全的外部执行。我们提出了 Koan-Safe，它结合了提示式意图解析器、可替换生成器和结构修复。在 75 个外部工作流提示上，其混合变体在静态安全代理上得分 0.67，而最佳基线为 0.33。Koan-Safe 在保存的基准输出中记录了 0 次不安全执行。当强制执行禁用时，匹配候选的消融测试会产生 14-17 次不安全执行。这些结果支持明确的交易保护和基于执行的评估，同时区分声明安全与一般保证。
DeFiFlowBench: Benchmarking and Improving Safe Executability in Natural-Language DeFi Workflow Synthesis
A structurally valid DeFi workflow can still authorize a costly trade. We introduce DeFiFlowBench, a benchmark of 207 team-authored prompts for natural-language DeFi workflow synthesis. It measures graph coverage, configuration completeness, and declared safety predicates, then tests supported trade configurations on a local EVM. Direct, constrained, and few-shot prompting produce 14-19 unsafe held-out executions per configuration under a fixed 5% price-impact cap. A slippage bound derived from a quote does not prevent the price impact of the order itself. We propose Koan-Safe, which combines a prompt-only intent parser, a replaceable generator, and structural repair with default safety parameters. On 75 held-out workflow prompts, its hybrid variant scores 0.67 on the static safety proxy, compared with 0.33 for the best baseline. Koan-Safe records no unsafe executions on the saved benchmark outputs. A matched-candidate ablation produces 14-17 unsafe executions when enforcement is disabled. Additional tests expose the limits of default injection: permissive existing thresholds can still authorize unsafe trades. A separately evaluated policy cap addresses this failure on a 36-case diagnostic grid. These results support explicit trade protections and execution-based evaluation, while distinguishing declared safety from a general guarantee.