研究：抗遗忘方法保不住微调模型的翻译指令遵循能力
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
这篇论文很扎心：EWC 保住了通用能力，翻译里的正式度、性别控制这些指令照样丢。做翻译微调的人建议看看，选抗遗忘方法前先想清楚。
arXiv 上一项研究检验了灾难性遗忘缓解方法在机器翻译微调场景是否有效。实验先在 Llama 3.2 1B Instruct 上筛选，再用 Llama 3.1 8B Instruct 在阿拉伯语-英语和西班牙语-英语双向数据上微调。Elastic Weight Consolidation 在通用基准上表现最好，8B 西语模型的通用分数只下降 1.7 分，而标准微调下降 11.0 分，但它对正式度和语法性别控制指令的效果与标准微调接近。只有混入控制任务示例的数据混合方法能保留这些指令控制能力，且收益无法迁移到同一任务的未见提示上。
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Fine-tuning large language models on parallel data improves translation quality but can cause catastrophic forgetting. Mitigation methods are generally evaluated by retention on general benchmarks. We ask whether these findings transfer to machine translation (MT) fine-tuning and to MT-specific instruction following (MT-IF): instructions that modify a translation, such as formality, grammatical gender, and length control. We compare methods anchored to auxiliary data, to model outputs, and to the base model parameters, first in a screening study with Llama 3.2 1B Instruct, then on Llama 3.1 8B Instruct fine-tuned on bidirectional Arabic-English or Spanish-English data. Elastic Weight Consolidation preserves general capabilities best in both stages; on the 8B Spanish model the average score on general benchmarks drops 1.7 points versus 11.0 for standard fine-tuning, yet its scores for formality and grammatical gender control remain close to standard fine-tuning. Only data mixing with control-task examples preserves these controls, but its gains do not transfer to unseen prompts for the same task.
