PEARL 算法：按预测误差选择性遗忘，应对用户偏好漂移
PEARL: Prediction Error-Adaptive Regularized Learning for Selective Forgetting under Partial Preference Drift
一篇处理偏好漂移的 bandit 新算法，核心是只遗忘出错的方向，比滑动窗口这类一刀切方法高 3 到 10 分，还能直接套在任何 API 模型上。
论文提出 PEARL（Prediction Error-Adaptive Regularized Learning），针对偏好部分漂移的场景，在预测出错的特征方向上逆转 Gram 矩阵更新，其余维度置信度保持不变。理论上证明了非自适应策略在偏好漂移后会累积线性遗憾，且固定探索率无法同时适配平稳与非平稳场景。在 PRISM 基准（1,500 名用户、21 个 LLM）上，PEARL 比 greedy 高 4.0 分，比最佳神经 bandit 基线高 2.5 分（p<0.001），比滑动窗口和折扣类方法高 3 到 10 分。选择策略是一个约 100K 参数的 MLP，可在 CPU 上毫秒级运行，LLM 权重保持冻结，兼容任意 API 模型。
PEARL: Prediction Error-Adaptive Regularized Learning for Selective Forgetting under Partial Preference Drift
User preferences shift as needs, context, and tastes evolve. This matters for personalized recommendation, LLM response generation, and RLHF-trained reward models, which can become miscalibrated as preferences drift. Existing non-stationary bandit methods forget uniformly across all dimensions, but real preference shifts are typically partial: some dimensions go stale while others remain valid. Neural contextual bandits accumulate confidence monotonically via the Gram matrix, with no mechanism to direct uncertainty where predictions have gone wrong. We propose PEARL (Prediction Error-Adaptive Regularized Learning), which reverses the Gram update along feature directions where predictions are surprisingly wrong, while leaving confidence intact elsewhere. We prove that (i) any non-adaptive policy suffers linear regret after a preference shift, (ii) no fixed exploration rate achieves low regret in both stationary and non-stationary regimes, and (iii) PEARL's cumulative regret scales with the number of shifted users N_s, not the total population N. On PRISM, a personalized LLM response selection benchmark (1,500 users, 21 LLMs), PEARL beats greedy by +4.0 points and the best neural bandit baseline by +2.5 points (p<0.001), while performing on par with greedy when preferences are stable. It beats sliding-window and discounted approaches by 3 to 10 points, showing that selective forgetting outperforms uniform forgetting. The selection policy is a ~100K-parameter MLP that runs in milliseconds on CPU, leaves LLM weights frozen, and works with any API-based model.
