SIFR:频谱引导双流框架提升域泛化语义分割
Spectral Invariant Fusion and Refinement for Domain Generalized Semantic Segmentation
一篇域泛化分割的新论文,用频谱建模统一处理 CLIP 加 DINOv2 的特征融合和解码精修,恶劣天气等场景成绩超 SOTA。
arXiv 论文提出 SIFR,一个面向域泛化语义分割(DGSS)的频谱感知双流框架。它不再直接融合冻结的 CLIP 和 DINOv2 特征,而是通过频谱引导的不变融合过滤域敏感响应,保留语义与结构信息。解码端采用基于低频统计的归一化和自适应频率选择来稳定多尺度特征,减少残余域偏差传播。在合成到真实、真实到真实和恶劣天气等多种 DGSS 基准上,成绩超过此前 SOTA 方法。
Spectral Invariant Fusion and Refinement for Domain Generalized Semantic Segmentation
Domain generalized semantic segmentation is challenging because it requires accurate dense prediction on unseen domains without any access to target-domain data during training. Existing methods improve transferability through data diversification, invariant learning, or direct use of foundation models, but they often struggle to simultaneously suppress domain-sensitive interference and preserve fine-grained spatial cues. In this work, we propose SIFR, a frequency-aware dual-stream framework that tackles domain shift at both the fusion and decoding stages. SIFR uses spectral modeling as a unified principle to regulate both heterogeneous prior interaction and dense decoder refinement. Instead of directly fusing frozen CLIP and DINOv2 features, SIFR performs spectral-guided invariant fusion to filter out domain-sensitive responses while preserving complementary semantic and structural information. In addition, SIFR incorporates a spectral decoder refinement module that stabilizes multi-scale decoder features through low-frequency statistics-based normalization and adaptive frequency selection, thereby reducing the propagation of residual domain bias in dense prediction. This design enables SIFR to jointly improve domain robustness and segmentation precision under severe distribution shifts. Extensive experiments on multiple DGSS benchmarks, including synthetic-to-real, real-to-real, and adverse-weather settings, demonstrate consistent improvements over previous state-of-the-art methods.