连续时间带约束随机线性二次控制的强化学习研究
Reinforcement learning for constrained stochastic linear--quadratic control in continuous time
控制论方向的同学可以看看，这篇把带非负控制约束的 LQ 问题做了完整的 RL 求解方案，理论推导和数值实验都齐了。
论文研究连续时间随机线性二次控制问题，考虑标量状态与非负控制约束。作者引入一个不含熵的辅助问题，通过矩约化和状态分离得到两个区域 Riccati 方程，刻画了有限时域值函数与最优条件均值。论文给出控制-成本相容条件，使该均值与原约束问题的最优反馈一致，并对未知动力学提出有限与无限时域的 model-free actor-critic 算法。数值实验将学习到的策略与解析目标进行对比，并分析探索对最优均值的影响。
Reinforcement learning for constrained stochastic linear--quadratic control in continuous time
We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.