CTTA：测试时反事实增强方法去除语言模型捷径学习
Counterfactual Test-Time Augmentation for Shortcut Removal in Language Models
训练好的模型爱抄捷径？这篇论文给出测试时就能用的修正办法，不用重新训练，效果还接近专门微调过的模型。
论文提出反事实测试时增强（CTTA）方法家族：用辅助模型对输入生成改变捷径值的反事实编辑，再对 LLM 在各编辑上的预测加权平均。论文证明最优捷径不变预测器属于 CTTA 家族，并刻画了其权重；权重依赖训练分布，在一般情形下无法从训练后模型和无标注测试数据中识别。作者给出可用 token 概率获得权重的假设条件，提出检测假设违背的测试，并据此构建 Select-CTTA 聚合方法。实验显示 Select-CTTA 优于测试时基线，接近在捷径平衡数据上微调的模型表现。
Counterfactual Test-Time Augmentation for Shortcut Removal in Language Models
Large language models (LLMs) can exploit shortcuts, i.e., correlations present in training data that may not hold at test time. This can lead to poor predictions when correlations shift. Most existing solutions for removing shortcuts operate at training time, requiring labeled training data, model weight access, and significant computational resources. Here, we study the problem of test-time shortcut removal. We propose a family of counterfactual test-time augmentation (CTTA) methods: given an example, each method uses an auxiliary model to generate counterfactual edits in which the shortcut value is changed, and then takes a weighted average of the LLM's predictions across these edits. We prove that the optimal shortcut-invariant predictor is a member of this CTTA family, and we characterize its weights. The weights depend on the training distribution, and therefore are, in general, not identifiable from the trained model and unlabeled test data. However, we present assumptions under which the weights can be obtained using the token probabilities of the trained LLM, and propose tests to detect violations of these assumptions. We use these tests to develop Select-CTTA, an aggregate method that selects among CTTA methods. Empirically, we find that Select-CTTA outperforms test-time baselines and approaches the performance of models fine-tuned on shortcut-balanced data.
