模型合并对涌现能力的影响：保留、无法创造、更快稀释
On Emergent Capabilities and Model Merging
想把两个模型权重合并凑出新能力？这篇论文用三个模型家族证明涌现能力合不出来，错位反而会跟着保留。
一篇 arXiv 论文研究了模型合并（对微调检查点和适配器权重做算术运算来拼接能力）对涌现能力的影响，即那些从未被显式训练目标覆盖的行为。研究在激活预言机（activation oracles）和涌现错位模型两个独立测试床上进行，覆盖三个模型家族。结果显示：当两个父模型都携带某涌现能力时，合并能保留它，例如两个错位检查点合并后在全混合比例范围内保留大部分错位。但合并无法创造超出父模型叠加的涌现能力，两个单任务预言机的任意加权合并都达不到联合训练预言机的审计能力。当只有一方携带该能力时，合并对其稀释速度快于伴随的已训练能力，多数设置下差距显著。
On Emergent Capabilities and Model Merging
Fine-tuned checkpoints and adapters now fill public repositories, and the most common operation applied to these artifacts is model merging: arithmetic on their weights that assembles capabilities cheaply. We ask what this operation does to emergent capabilities: behaviors an artifact carries that were never an explicit training target. Studying two independent testbeds (activation oracles and emergent-misaligned models) across three model families, we find that the answer is threefold. First, merging preserves an emergent capability that both parents carry: merging two misaligned checkpoints retains most of their broad misalignment across the whole mixing range. Second, merging cannot create an emergent capability that is superadditive in its parents: no weighted merge of two single-task oracles reaches the jointly-trained oracle's auditing ability. Third, when only one parent carries the capability, merging dilutes it faster than the trained capability that accompanies it: the gap is significant in most settings. In short, emergent behaviors of an artifact do not compose the way its trained capability does.