论文

CG2框架微调LLM生成行为契约,提升代码生成正确率

Better Specifications, Better Code: Fine-Tuning LLMs for Behavioral Contract Generation

精选理由

一组研究者发了 CG2,让 LLM 先写行为契约再写代码,在 Qwen2.5-Coder 等 5 个模型上 pass@1 最多提升 22%,写代码总出 bug 的可以看看思路。

arXiv 论文提出 CG2,一个契约引导的代码生成框架,把自然语言任务转成包含接口、前置条件、后置条件和不变量的结构化行为契约。CG2 微调 LLM 生成契约,并通过迭代式语义验证与修复流程在代码生成前修正不完整或不一致的规格。在 1,007 个 held-out 任务上测试 DeepSeek-Coder-V2-Lite-Instruct、Qwen2.5-Coder-14B/32B-Instruct 等 5 个代码模型,pass@1 比直接生成提升 1.49-3.27 个百分点(相对提升 7.58%-22.45%),比未微调的契约引导多提升 1.19-2.38 个百分点。人工分析显示语义正确的契约与更高的代码生成准确率相关,且无需修改下游代码生成模型本身。

原文 · arXiv: DeepSeek

Better Specifications, Better Code: Fine-Tuning LLMs for Behavioral Contract Generation

Large language models (LLMs) demonstrate strong code-generation capabilities but often struggle with functional correctness due to implicit or ambiguous behavioral requirements. We present CG2, a contract-guided code generation framework that translates natural-language tasks into structured behavioral contracts specifying interfaces, preconditions, postconditions, and invariants. CG2 fine-tunes an LLM to generate these contracts and introduces an iterative semantic validation and repair process to identify and correct incomplete, inconsistent, or incorrect specifications before guiding an unchanged downstream code-generation model. We evaluate CG2 on 1,007 held-out tasks using five code LLMs: DeepSeek-Coder-V2-Lite-Instruct, Ministral-3-14B-Instruct-2512, Qwen2.5-Coder-14B-Instruct, Qwen2.5-Coder-32B-Instruct, and Qwen3.5-27B. Results show that CG2 consistently improves pass@1 over direct code generation by 1.49-3.27 percentage points (7.58%-22.45% relative improvement) and over contract guidance without fine-tuning by 1.19-2.38 percentage points (7.04%-14.65% relative improvement). Manual analysis further reveals that semantically correct contracts are associated with higher code-generation accuracy. These findings demonstrate that explicitly specifying and validating behavioral requirements can improve functional correctness without modifying downstream code-generation models.