SWE-Critix：评估编码代理轨迹的通用验证器
SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories
SWE-Critix能评估编码代理的长期任务执行，准确率提升21%，还开源了模型和数据集。
研究人员推出SWE-Critix，一种用于评估编码代理轨迹的通用验证器。该模型基于Qwen3-30B-A3B推理模型，通过监督微调(SFT)和强化学习(RL)训练，在分类准确率上从57%提升至78%。SWE-Critix能够预测完成标签并生成自然语言分析来解释其判断。
SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories
Coding agents are increasingly deployed for end-to-end software engineering tasks, where a readily available oracle evaluator that defines the success criteria is often absent. This motivates verifications for coding agents that can reliably judge and explain their behavior. However, verification is non-trivial as coding agents are increasingly capable of solving long-horizon software engineering tasks. Their execution trajectories can be extremely long, containing hundreds of iterative interactions with code repositories and external tools. Existing verifiers either rely on general-purpose LLMs without specialized training or train dedicated critics that predominantly focus on producing scalar rewards, highlighting the relatively limited exploration of training-based generalist verification. We introduce SWE-Critix, a post-training pipeline for training generalist verifiers of coding-agent trajectories. Given a task and its execution trajectory, the learned verifier predicts the completion label and generates a natural-language analysis that explains its judgment. Our pipeline combines supervised fine-tuning (SFT) with reinforcement learning (RL) and trains a series of 30B models on trajectories from CodeForge-Preview. On a held-out test set, SWE-Critix improves classification accuracy from 57% for the base Qwen3-30B-A3B reasoning model to 66% after SFT and 78% after RL. We release our models, datasets, and training code to facilitate further research.
