得克萨斯大学论文：用快速打分模型替代 LLM 决策，记忆构建速度提升 6.6 倍
做记忆管线的朋友看看这篇：把路由、打分这类小决策换成快速分类器，LLM 只留着答问题，构建速度直接快 6.6 倍。
得克萨斯大学发布的 Jev-Mem 论文提出，把记忆系统中链接选择、搜索停止等小决策交给快速打分模型 Jev，以批量返回概率代替生成文本。LLM 只在最后读取选定记忆并生成回答时被调用。在 LoCoMo 长对话基准上，Jev-Mem 得分 0.777，高于最佳基线的 0.700。记忆构建耗时 158 秒，而最快对手为 1044 秒，速度提升 6.6 倍；回答耗时 0.93 秒对 1.47 秒。
University of Texas paper finds that handing small memory decisions to a fast scoring model let an AI memory system build its memory of long conversations 6.6 times faster
Many memory systems ask an LLM to make small choices, like what to link or when to stop searching.
Generating text for each choice is slow, and those calls pile up during memory building and search.
Jev-Mem gives those choices to a fast scorer, Jev, that returns probabilities in batches instead of text. The LLM is called only at the end, to read the chosen memories and answer.
On the LoCoMo long-conversation benchmark, it got a 0.777 answer score versus 0.700 for the best baseline. It built memory in 158 seconds versus 1,044 for the fastest rival and answered in 0.93 seconds versus 1.47.
If your memory pipeline calls an LLM for routing, scoring, or stopping, try a fast classifier there and save the LLM for answers.
