NPCB 算法:用前景理论替代期望奖励的非参数 Bandit 研究
Risk-Sensitive Non-Parametric and Kernel Bandits
把行为经济学的前景理论塞进 Bandit 决策里,遗憾界还和风险中性一样,理论党会喜欢这篇。
论文研究以 Cumulative Prospect Theory(CPT)而非期望奖励来评估臂的非参数 Bandit 问题,覆盖连续动作空间和臂相关噪声。提出的 NPCB 算法将均值估计与 CPT 决策解耦,证明了量级为 Õ(√(T·dim_E·log N)) 的 CPT 遗憾界,其中 dim_E 是函数类的 Eluder 维度、N 是覆盖数。该结果与经典非参数 Bandit 的遗憾阶一致,说明 CPT 风险敏感不带来额外的阶级代价。论文还扩展到未知噪声 CDF 的情形,并对 Kernel-CPT Bandit 给出基于信息增益的遗憾界,在多种核函数下与风险中性版本只差对数因子。
Risk-Sensitive Non-Parametric and Kernel Bandits
We study non-parametric bandits in which arms are evaluated using Cumulative Prospect Theory (CPT) rather than expected reward. We consider continuous action spaces with arm-dependent noise and propose Non-Parametric CPT Bandits ($\mathtt{NPCB}$), an optimistic algorithm that decouples mean estimation from CPT-based decision-making. We prove CPT regret of order $\widetilde{\mathcal O}\bigl(\sqrt{T\,\dim_E(\mathcal F,T^{-2})\,\log N(\mathcal F,T^{-2},\|\cdot\|_\infty)}\bigr)$, where $\dim_E$ is the Eluder dimension of the function class $\mathcal F$, $T$ is the time horizon, and $N$ is its covering number in the sup norm. This matches the order-wise regret of classical non-parametric bandits, showing that CPT risk sensitivity incurs no additional order-wise regret cost. We extend $\mathtt{NPCB}$ to unknown noise CDFs, retaining the known-CDF mean-learning term and adding a CDF-learning cost governed by a distributional Eluder dimension. We also study Kernel-CPT bandits and obtain information-gain-based regret bounds for a broad range of kernels, matching the corresponding risk-neutral kernel rates up to logarithmic factors.