CSC:单项目标函数统一自监督学习中的对齐与多样性
Centroid Spectral Coupling: Implicit Alignment and Representation Diversity in Self-Supervised Learning
一个损失函数搞定对齐和防坍缩,不用拼多个加权项。在 CIFAR-10、CIFAR-100、Imagenette 上线性探针准确率超过 SimSiam、VICReg 这类非对比基线。做自监督表征研究的同学可以直接看公式和坍缩分析部分。
论文提出 Centroid Spectral Coupling(CSC),将视图对齐、表征多样性和防坍缩三项目标用一个谱目标函数隐式耦合,替代传统非对比方法的多加权损失项。方法上,CSC 从独立归一化的增广视图对构建质心,对其未中心化二阶矩矩阵施加基于 ridge 的谱目标:保留质心幅度编码视图一致性,同时在各谱方向上均衡表征能量。
作者给出对齐项与谱不均衡项的精确非负分解,刻画了全局最优的几何结构,推导出对齐与多样性的有限损失保证,并证明常数输出的完全坍缩在归一化表征空间中是严格鞍点。在 CIFAR-10、CIFAR-100 和 Imagenette 三个基准上,CSC 在所有被评估的非对比方法中取得最高线性 Top-1 准确率,kNN Top-1 准确率在两个 CIFAR 数据集上也居首。ridge 参数敏感性实验揭示了对齐与多样性的权衡:增大 κ 增强视图对齐但降低表征有效秩。
Centroid Spectral Coupling: Implicit Alignment and Representation Diversity in Self-Supervised Learning
Self-Supervised Learning (SSL) aims to align augmented views while preserving representation diversity and preventing collapse. Existing non-contrastive methods often address these goals using multiple weighted loss terms. We introduce Centroid Spectral Coupling (CSC), a single-term SSL objective that implicitly couples view alignment, representation diversity, and collapse resistance. CSC constructs centroids from independently normalized augmented-view pairs and applies a ridge-based spectral objective to their uncentered second-moment matrix. Preserving centroid magnitudes encodes view agreement while encouraging balanced representation energy across spectral directions. We establish an exact decomposition into nonnegative alignment and spectral-imbalance components, characterize the geometry of global optima, and derive finite-loss guarantees for alignment and diversity. We also show that complete constant-output collapse is a strict saddle in normalized representation space. Experiments on CIFAR-10, CIFAR-100, and Imagenette show that CSC achieves the highest linear Top-1 accuracy among the evaluated non-contrastive methods on all three benchmarks, and the highest kNN Top-1 accuracy on both CIFAR datasets. A ridge-parameter sensitivity study reveals an empirical alignment--diversity trade-off: increasing $κ$ strengthens view alignment but reduces representation effective rank. These results show that a unified spectral objective yields competitive visual representations without explicit contrastive negatives or separately weighted alignment and diversity penalties.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:自监督学习、CSC、CIFAR-10、对比学习,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.LG」(@M. Hadi Sepanj, Benyamin Ghojogh, Saed Moradi, Paul Fieguth),经 AITOP 多维评分管线结构化提炼。