ViGeo:用时空类比把视觉上下文学习扩展到视频任务
Video, Ergo Genero: Unifying Video Tasks via Spatiotemporal Analogy
训练免了!ViGeo 用时空类比让视频模型靠示例学新任务,还能零样本处理 event cameras,顺手拆了个提示词捷径的坑。
研究团队提出 ViGeo 框架,通过时空画布补全(spatiotemporal canvas completion)把视觉类比式上下文学习从图像扩展到视频。
该方法免训练即可适配多种视频任务,并能零样本泛化到未见过的视频操作和 event cameras 等新模态。论文还发现任务内化现象:预训练任务的查询格式会覆盖示例,加入少量任务无关数据即可消除这一捷径。
Video, Ergo Genero: Unifying Video Tasks via Spatiotemporal Analogy
Adapting video models to new tasks typically requires dedicated data curation and fine-tuning. While visual analogy provides a training-free alternative by specifying tasks in-context, it remains restricted to the image domain. To explore whether analogy-based methods can unify diverse video tasks and generalize to out-of-distribution scenarios, we introduce ViGeo, a framework that extends visual in-context learning to the video domain via spatiotemporal canvas completion. Evaluated on a diverse task taxonomy with a strict train-test split, ViGeo generalizes to unseen video manipulations and zero-shot modalities (e.g., event cameras). Finally, we identify task internalization, where a query format associated with a pretrained task overrides the demonstration, and show that this shortcut can be removed with a small amount of task-unrelated data, highlighting the need to decorrelate prompt format from task identity.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:ViGeo、视频生成、上下文学习、零样本学习,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Chia-Hsiang Kao, Belinda Zeng, Bharath Hariharan, Menglin Jia),经 AITOP 多维评分管线结构化提炼。截至目前,全网已有 2 家独立信源对该事件进行交叉印证与跟进报道(含 宝玉 等),属于近期高关注度行业信号。
- 宝玉09-27 07:30原文