研究发现 VGGT 冻结特征可预测自身 3D 位姿失败
Does VGGT Know What It Does Not Know?
有团队发现 VGGT 的冻结特征里藏着它自己位姿对不对的信号，线性探针就能以 93.5% 的 AUC 挑出失败案例，还能直接迁移到 π³ 等模型上。
VGGT 等前馈式 3D 重建模型在单次前向传播中回归相机位姿和几何，但不输出任何不确定性，错误的相对位姿与正确结果看起来一样。在包含 1000 个困难图像对、VGGT 失败率达 35% 的 HardMatch 基准上，对冻结特征做线性探针能以 93.5% 的 ROC-AUC 预测位姿失败，高于 VGGT 自身 point-map 置信度的 79.7%。该探针无需重训练即可迁移到 PhotoTourism 和 ScanNet-1500 上的旋转失败检测，同样的规律在 π³ 模型上也成立。研究还发现这种自知识出现在双视图融合的中间层，而不在单图编码器中。
Does VGGT Know What It Does Not Know?
Feed-forward 3D reconstruction models such as VGGT regress cameras and geometry in a single forward pass, but they report no uncertainty for the camera poses they predict: an incorrect relative pose looks no different from a correct one. We ask whether the frozen model nonetheless knows when its pose is wrong. On HardMatch, a benchmark of 1,000 deliberately difficult image pairs on which VGGT fails on 35% of pairs, a linear probe on VGGT's frozen features predicts pose failure on held-out categories with ROC-AUC 93.5%, compared with 79.7% for VGGT's own point-map confidence and 77.7% for a classifier trained on all of VGGT's outputs. The probe catches failures VGGT is confident about, lets a downstream system abstain on the pairs it distrusts, and transfers without retraining to rotation failures on PhotoTourism and ScanNet-1500, beyond what co-visibility explains. This knowledge is absent from the per-image encoder: it is computed partway through the layers that fuse the two views, and the same holds for a second model, $π^3$. In essence, we find that VGGT knows more about its own failures than it reports.