论文

研究:相机噪声残差对人脸换脸检测是冗余而非互补

Camera-Noise Residuals for Face-Swap Detection: Redundant, Not Complementary, and Why

精选理由

做深fake检测的同学看下,这篇用 FaceForensics++ 和 Noiseprint++ 实测:噪声指纹融合 RGB 不涨点,还指出了 InstanceNorm 抹掉信号的具体原因,避坑参考。

一项在 FaceForensics++ 上的研究测试了将 Noiseprint++ 噪声残差通道与 RGB Xception 主干融合用于换脸检测的效果。三模型消融显示,融合并未超越仅用 RGB 的方案,残差分支单独使用时接近随机水平。七级瓶颈诊断发现,噪声图确实携带判别信号,但该信号是统计性的,由残差的均值、方差、能量等一二阶矩承载,而沿袭 TruFor 模板设在噪声分支输入处的 InstanceNorm 层恰好把这些矩标准化掉了(五折交叉验证 AUC 从 0.747 降至 0.554)。即使移除瓶颈、用两种固定融合变体恢复统计信号,仍无法在任一数据集上胜过 RGB。结论是:在该操纵分布下,噪声残差与 RGB 冗余而非互补。

原文 · arXiv cs.LG

Camera-Noise Residuals for Face-Swap Detection: Redundant, Not Complementary, and Why

Fusing a learned camera-noise fingerprint with an RGB appearance backbone is an appealing route to generator-independent deepfake detection, because the noise residual is grounded in image-formation physics rather than in the texture statistics of a particular generator. We test, on FaceForensics++, whether a Noiseprint++ residual channel carries information \emph{complementary} to an RGB Xception backbone for face-swap detection. A three-model ablation (RGB-only, residual-only, late-fusion) shows that fusion does not improve over RGB alone and that the residual branch alone is near chance. A seven-level bottleneck diagnostic localizes the cause: the noise maps do carry a discriminative signal, but it is statistical---carried by the per-sample first and second moments (mean, variance, energy) of the residual---and the per-sample \texttt{InstanceNorm} layer placed at the noise-branch input, following the TruFor template, standardizes exactly those moments away (five-fold cross-validated AUC drops from $0.747$ to $0.554$). A context-crop control rules out cropping geometry, and two fixed-fusion variants that remove the bottleneck recover the statistical signal yet still fail to beat RGB on every dataset. We conclude that, on this manipulation distribution, the noise residual is redundant with RGB rather than complementary, and we give concrete guidance for practitioners adopting noise-residual fusion for face-swap detection.