论文

WildMAE:面向野生动物相机陷阱图像的结构感知掩码自编码器

WildMAE: Structure-Aware Masked Autoencoding for Wildlife Image Reconstruction

精选理由

有人专门为野外相机拍的动物照片改进了 MAE,重建指标比原版 MAE 高出好几 dB,做生态监测或自监督的可以看看架构细节。

WildMAE 是一个基于 MAE 的解码器侧扩展,针对野生动物相机陷阱图像重建任务设计。它在 Snapshot Serengeti 数据集 4,744 张测试图像、75% 掩码率下取得 27.90 dB PSNR 和 0.840 SSIM,优于 vanilla MAE 的 24.33 dB 和 0.780,LPIPS 为 0.160 对 0.320。方法上引入零门控卷积残差路径提供二维邻域先验,并通过门控交叉注意力让掩码位置从可见位置特征中检索上下文。训练配方逐步引入归一化像素、Sobel 边缘和感知目标。作者说明这些指标是历史协议下的结果,并非重新统一复现的对比。

原文 · arXiv cs.LG

WildMAE: Structure-Aware Masked Autoencoding for Wildlife Image Reconstruction

Camera-trap imagery presents substantial variation in animal appearance, illumination, background clutter, and visual degradation. Masked autoencoders (MAEs) offer a label-free framework for modeling such data through reconstruction of randomly hidden image patches. We present WildMAE, a decoder-side extension of MAE that combines local structure-aware refinement with masked-to-visible cross-attention. A zero-gated convolutional residual path introduces a two-dimensional neighborhood prior, while gated cross-attention allows masked positions to retrieve contextual information from visible-position features without replacing the original Transformer decoder. The associated training recipe progressively introduces normalized-pixel, Sobel-edge, and perceptual objectives. In the original Snapshot Serengeti study, covering 4,744 test images at 75% masking, the reported reconstruction scores were 27.90 versus 24.33 dB PSNR, 0.840 versus 0.780 SSIM, and 0.160 versus 0.320 LPIPS for WildMAE and vanilla MAE, respectively. Additional historical records document favorable average differences under four synthetic degradation conditions and describe component variants. We present the method, experimental observations, and their scope together: the retained metrics are historical protocol-dependent results, rather than a newly reproduced, harmonized comparison or a validation of restoration from genuinely unavailable pixels. WildMAE provides a practical architectural case study for combining global token mixing with spatial and targeted contextual refinement in wildlife masked-image modeling.