Defensive Policy Gradient：去掉重要性权重方差假设的 O(ε⁻³) 采样复杂度
Sample complexity of variance-reduced policy gradient: weaker assumptions and lower bounds
优化策略梯度的人可以看看：新算法 Defensive Policy Gradient 不再依赖重要性权重方差假设，就能拿到 O(ε⁻³) 采样复杂度，还配了下界证明。
arXiv 论文 2610.03165 提出 Defensive Policy Gradient 算法，基于防御性重要性采样，在不对普通重要性权重方差做任何假设的情况下，达到与既有方差缩减 REINFORCE 变体相同的 O(ε⁻³) 采样复杂度。论文还在广义黑箱策略优化模型中建立了下界：单策略有界方差反馈为 Θ(ε⁻⁴)，双策略耦合反馈为 Θ(ε⁻³)。REINFORCE 与 Defensive Policy Gradient 分别匹配这两个速率，为后者 O(ε⁻³) 的速率最优性提供了 oracle 层面证据。
Sample complexity of variance-reduced policy gradient: weaker assumptions and lower bounds
Several variance-reduced versions of REINFORCE based on importance sampling achieve an improved $O(ε^{-3})$ sample complexity to find an $ε$-stationary point, under an unrealistic assumption on the variance of the importance weights. In this paper, we propose the \algo (Defensive Policy Gradient) algorithm, based on defensive importance sampling, which achieves the same rate without any assumption on the variance of ordinary importance weights. We also establish lower bounds in a generalized black-box policy-optimization model that hides states and actions and permits parameter-dependent rewards. In this model, the optimal rates are $Θ(ε^{-4})$ with bounded-variance one-policy feedback and $Θ(ε^{-3})$ with mean-square-smooth coupled two-policy feedback. Under standard policy-regularity conditions, REINFORCE and \algo realize the corresponding oracle conditions and attain the $O(ε^{-4})$ and $O(ε^{-3})$ upper bounds, respectively. Although the lower bounds do not apply directly to the classical MDP interaction model in which these algorithms operate, this correspondence provides oracle-level evidence that the faster rate of \algo is optimal and genuinely separated from that of vanilla policy gradient.