研究：语言子空间干预对多语言推理的准确率与语言保真度难以兼得
Beyond Accuracy: What Language-Subspace Interventions Capture, Remove, and Improve
Qwen2.5-7B 上的实验挺扎心：想让模型用目标语言答对题，最好的干预也只提升 1.60 点，做多语言的可以看看。
该研究在 MATH 和 MGSM 两个基准上测试了四种开源权重 LLM，评估语言子空间干预对多语言推理的影响。研究用 Mean-SVD 和 DiffPCA 两种方法构建子空间，两者都能捕捉大量与翻译相关的变化。但线性探针实验显示，投影去除子空间后仍能近乎完美地识别语言，说明子空间并未穷尽语言信息。在 Qwen2.5-7B 上的实验中，只有一种干预配置让联合准确率（答案正确且语言符合目标）提升了 1.60 点，而提升 MGSM 准确率最多的干预牺牲了响应语言保真度。
Beyond Accuracy: What Language-Subspace Interventions Capture, Remove, and Improve
Language-subspace interventions aim to improve multilingual reasoning by suppressing language-related components in mid-layer activations. Using MATH and MGSM across four open-weight LLMs, we ask three questions: do estimated language subspaces capture translation-associated variation? Does projecting them out remove language information? And do they improve accuracy without compromising response language? For the first question, we build subspaces using the existing Mean-SVD construction and our DiffPCA. Both capture substantial translation-associated variation and show similar patterns across model depth. For the second, we project out each subspace and train linear probes to classify language from the projected activations. The probes still identify language almost perfectly, showing that neither subspace exhaustively captures language information. For the third, we evaluate several intervention configurations on Qwen2.5-7B, by varying subspace creation methods and intervention operators. We find that the intervention that most improves MGSM accuracy sacrifices response-language fidelity, with only one intervention improving joint accuracy--answers that are correct and in the target language--slightly, by 1.60 points. Interestingly, subtle changes in the prompt alone can reproduce the accuracy gains albeit with significant damage to language fidelity. We argue that causal interventions must be judged on more than accuracy, to fully account for language substitution trade-offs.