论文

CounterFactual Foreground Retrieval:动物个体识别评估新协议

CounterFactual Retrieval for Evidence-Aware Individual Animal Identification

精选理由

一篇评估方法的论文,专治动物识别模型靠背景“作弊”的问题,还开源了 34 个数据集的 50 万掩码,做野生动物识别的可以拿来测自己的模型。

arXiv 论文提出 Counterfactual Foreground Retrieval(CFR),一种证据感知的动物个体识别评估协议。CFR 在三种视图下评估同一查询-图库任务:完整图像、仅前景、仅捷径(背景或轮廓),将单一的检索得分拆解为证据画像。该方法能判断模型究竟依赖动物本体特征,还是身份相关的背景捷径。作者发布了覆盖 34 个数据集约 50 万个掩码,以及掩码生成与 CFR 评估的开源代码。

原文 · arXiv cs.LG

CounterFactual Retrieval for Evidence-Aware Individual Animal Identification

Automated identification of individual animals is moving from small curated studies toward shared benchmarks and reusable foundation models. This progress is useful, but current evaluation leaves the evidence behind a match unresolved, i.e., high retrieval accuracy does not show whether a model used animal features or identity-correlated shortcuts. We address this with Counterfactual Foreground Retrieval (CFR), an evidence-aware evaluation protocol. CFR evaluates the same query-gallery task under different views: full image, foreground-only, and shortcut-only. The full-image measures standard retrieval performance; the foreground-only measures foreground-supported retrieval; and shortcut-only view measure how much identity signal is recoverable from background, or silhouette. This turns a retrieval score into an evidence profile: how well the model performs on the original images, how much of that performance remains when only the animal is visible, and how much identity signal remains recoverable when the animal is removed. We release around 0.5M masks for 34 datasets and code for mask generation and CFR evaluation.