ConSDL 提出用稀疏编码替代 SAE 线性编码器，提升跨种子一致性
Beyond SAEs: Consistent Concept Discovery with Sparse Coding
做可解释性研究的同学可以看看，SAE 跨种子结果不稳定的老问题，这篇用逐样本稀疏编码给出了解法，一致性提升挺明显。
论文提出 Consistent Sparse Dictionary Learning（ConSDL），针对 SAE 跨训练运行学到的词典不一致问题，用逐样本稀疏编码替代标准线性编码器。在 16 个合成设置中，ConSDL 实现了近乎完美的词典与系数恢复。在四个视觉和语言设置上，相比六个 SAE 基线的平均值，ConSDL 将词典一致性和系数一致性分别提升 17.6% 和 35.9%。
Beyond SAEs: Consistent Concept Discovery with Sparse Coding
Sparse autoencoders (SAEs) explain foundation model representations by decomposing activations into sparse combinations of learned dictionary atoms. However, the learned dictionaries can vary across training runs, limiting the reliability and reproducibility of SAE-based analyses. We propose Consistent Sparse Dictionary Learning (ConSDL), which replaces the standard linear encoder with per-sample sparse coding. This is motivated by our synthetic analysis showing that coefficients from a single linear encoder pass are suboptimal compared with those obtained by per-sample optimization, suggesting that sparse coding is a reliable mechanism for dictionary learning. Across 16 synthetic settings, ConSDL achieves near-perfect dictionary and coefficient recovery. These gains translate to state-of-the-art cross-seed consistency across four vision and language settings, where ConSDL improves dictionary and coefficient consistency by 17.6% and 35.9%, respectively, over the average of six SAE baselines.