HERMES harness提升代码迁移性能
HERMES harness让代码迁移效率提升5倍，每个组件有专属LLM，比基线平均高12.4分，还省26.2%成本。
HERMES harness在代码迁移任务中将GPT-5.6 Sol的性能从6.5%提升至31.0%。该系统为每个代码组件配备专用LLM，了解其代码和依赖关系。在四个软件工程基准测试中，HERMES平均比基线harness高12.4分。使用Qwen3-8B组件时，性能接近全GPT-5.6 Sol设置，并降低26.2%的推理成本。
Build your own harness, folks. Reading papers like this makes me realize how underexplored harness engineering really is. The authors find that on whole-repository migration, GPT-5.6 Sol goes from 6.5% to 31.0% when Codex is replaced with the HERMES harness, with the same model and effort setting. The gain comes from the harness. HERMES pairs each repository component with a resident LLM that knows its own code and dependencies. A dependency-aware step decides which components to activate, and a diagnosis step maps test failures back to the components that need changes. Across four software engineering benchmarks, it beats matched baseline harnesses by 12.4 points on average. With strong activation and diagnosis models, Qwen3-8B components come within 4.5 points of an all-GPT-5.6 Sol setup and cut Terminal-Bench 4.0 inference cost by 26.2%. Paper: arxiv.org/abs/2610.07832 Chat with Paper: academy.dair.ai/papers/harness… 💬 10 🔄 8 ❤️ 50 👀 3257 📊 25 ⚡
