论文

Microfitting:为每篇文档缓存专用适配器实现持续学习

Microfitting: Continual Learning via Cached Test-Time Training

精选理由

给每篇文档单独训一个小适配器,查到哪篇就挂哪个,遗忘问题直接绕开了。相比 RAG,它在 EDO 上对基座模型有 38.2% 的相对提升,本地硬件就能跑。要给私有文档库做持续定制的企业和开发者可以看看这个思路。

研究者提出名为 microfitting 的持续学习方法,为每篇文档训练并缓存一个小型适配器,从机制上规避灾难性遗忘问题。每个适配器只离线训练一次,训练数据是针对基座模型已暴露短板生成的合成问题,文档被再次查询时直接复用对应适配器。该方法借鉴 test-time training 思路,通过参数隔离让冻结的基座模型在不改动旧适配器的前提下持续吸收新语料。

正文

在 EDO(Enterprise Document Operations)这一基于真实企业文档的开卷问答环境中,microfitting 在 DeepSeek V4 Flash 上相比基座模型取得 38.2% 的相对准确率提升。生成训练数据的流程还能自我改进,依据此前结果和目标工作负载的新样本不断修订配方。对于需要持续私有化定制的企业,该方法可在本地硬件上让模型隔夜适配不断更新的私有语料,无需修改基座模型。

原文 · arXiv: DeepSeek

Microfitting: Continual Learning via Cached Test-Time Training

Continual learning is a promising approach to specialize language models to an organization's private data. However, current methods exhibit a number of drawbacks including catastrophic forgetting, where new training updates overwrite prior knowledge. We present microfitting, which trains and caches a small adapter for each document. Each adapter is trained once, offline, on synthetic questions targeted at demonstrated base-model shortcomings and reused whenever that document is queried. This method borrows from test-time learning literature and sidesteps the problem of continual learning through parameter isolation, allowing the system to specialize on new data without overwriting prior knowledge. The process that generates adapter training data also improves over time, revising its recipe from previous outcomes and fresh examples of the target workload. On DeepSeek V4 Flash, microfitting substantially improves accuracy over the base model. In Enterprise Document Operations (EDO), an open-book Q&A environment over real-world enterprise documents, our method achieves a 38.2% relative improvement over base-model performance. Microfitting makes continual specialization practical and cost-efficient: a frozen model can adapt to an evolving private corpus overnight on local hardware, without modifying the base model or previously cached adapters.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:DeepSeek、微调、持续学习、RAG,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始信息发布自一手权威信源「arXiv: DeepSeek」(Tier-1 一手发布),具备高确定性参考价值。