RefCraft 基准发布：测试 AI 智能体参照截图开发网页的能力
RefCraft: Benchmarking Visual-Reference-Conditioned Agentic Generation through Web Development
新基准 RefCraft 拿 300 个真实网页任务考 AI 智能体，要求照截图开发网页，9 个顶级模型最高才拿 52 分，差距很直观。
arXiv 论文提出 RefCraft 基准，包含 300 个多模态网页开发任务，每项任务将自然语言指令与 1 到 5 个参考网页截图结合。智能体需要选择性解读这些视觉参考，并完成渲染、视觉检查和迭代修正。评估采用可独立验证的 keypoints，辅以程序化评测和 VLM 评审。对 9 个前沿（M）LLM 的测试显示，最强系统总得分仅 52.12/100，说明参照视觉示例进行智能体生成仍是当前模型的明显短板。
RefCraft: Benchmarking Visual-Reference-Conditioned Agentic Generation through Web Development
Beyond basic tool use and coding skills, effectively incorporating visual references into agentic generation is equally essential for applying AI agents to real-world productivity tasks. We study this crucial capability through webpage development, a practical and readily evaluable setting in which existing benchmarks remain largely idealized: they typically focus on reconstructing visual targets or generating webpages from textual instructions alone, overlooking the more common workflow of developing from visual examples. Specifically, we introduce RefCraft, a benchmark comprising 300 realistic multimodal webpage-development tasks that combine natural-language instructions with screenshots from one to five reference webpages. Agents must selectively interpret and utilize these references while rendering, visually inspecting, and iteratively refining their outputs. We evaluate the resulting open-ended artifacts using independently verifiable keypoints and complementary programmatic and VLM judges. Across nine frontier (M)LLMs, the strongest system achieves an overall score of only 52.12 out of 100, revealing that visual-reference-conditioned agentic generation remains a substantial challenge for current models in practical webpage-development settings.