别用前沿 LLM 做视觉任务:$6 训练专用检测器更划算
HF 的 Merven 把自己选视觉模型的经验做成了一套工具:检测、OCR 打包成 Agent 可调用的工具,还有从标注到训练到评估的完整训练管线,$6 就能训出一个检测器。做 Agent 视觉相关项目的开发者可以直接抄这套流程。
Merven (HF 工程师) 在 AI Engineer WF 演讲中分享:直接调用前沿 LLM 做物体检测成本高、效果差,训练一个专用检测器只需 $6,运行成本也极低。
她把 Keypoint 检测、OCR 等首选视觉模型封装成 Agent 可调用的工具,并做了一个端到端目标检测训练管线:从数据标注、VLM 当裁判评估,到训练和 eval 全流程打通。她还为 Agent 补上了决策模型支持,并更新了 LFM3 和 Qwen3.
6/8 等 VLM。
it's a skill issue to use frontier LLMs to detect objects or do vision
instead, if you use LLMs as vision interns you can train a detector for $6 that runs at little cost. but your agent is a bad vision intern, it only knows old models that are proprietary and makes rookie mistakes
so I built something for it, I basically distilled myself for you to use🤝 I put my favorite models from keypoint detection to OCR as a tool for your agent to call and I also made an e2e object detection trainer (from labelling to VLM as a judge to training and eval)
if you're interested in it and the results, my talk at @aiDotEngineer WF is finally out, you can watch it here https://t.co/2cdSRc3bOa many thanks @swyx
on the next post you can find the github repository, I also shipped decision model support and updated VLMs (LFM3, Qwen3.6/8)
本条情报归属 AITOP「技巧」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:目标检测、视觉模型、智能体、OCR,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「Merve Noyan」(@mervenoyann),经 AITOP 多维评分管线结构化提炼。