将估计生成形式化为推荐过程，提出正则化感知推荐者遗憾
Recommender regret for improving regularised RL estimation with bandits and MCTS
这篇论文把 RL 里的估计问题换了个推荐视角，还证明了均匀采样就能达到遗憾下界，做 MCTS 或 bandit 的可以看看。
一篇 arXiv 论文将正则化强化学习中的估计生成形式化为推荐过程，把 bandit 采样视为改进算子。作者定义了正则化感知的推荐者遗憾，分析了其下界，并证明均匀采样等简单算法可以达到该下界。论文还把 bandit 和搜索式目标改进统一到 recommenders MCTS 框架下。实验覆盖合成树、街机游戏和科学发现任务，结果显示推荐者方法带来更快的目标收敛和更好的下游性能，并首次刻画了正则化 MCTS 在科学应用中的潜力。
Recommender regret for improving regularised RL estimation with bandits and MCTS
Regularised reinforcement learning (RL) algorithms rely on estimators, like Monte Carlo tree search or one-step look-ahead, to provide policy, value, or Q-value estimates in training and inference. However, existing bandit-based estimation frameworks evaluate sampling efficiency, not the quality of estimates. We address this by formalising estimate generation as a recommendation process, treating bandit-based sampling as an improvement operator. To evaluate the quality of a recommendation (estimate), we define regulariser-aware recommender regrets, analysing their lower bounds, and proving that simple algorithms like uniform sampling can achieve these bounds. Finally, we unify bandit- and search-based target improvement under recommender MCTS. Across synthetic trees, arcade games, and scientific discovery tasks, recommenders often yield faster target convergence and superior downstream performance. Our experiments characterise for the first time the potential of regularised MCTS in scientific applications.
