从 PID 到强化学习：暖通空调控制架构综述
From PID to Reinforcement Learning in Building HVAC Control: A Control-Architecture Review of Tuning Strategies, Formulations, and Deployment Trade-offs
搞楼宇控制的朋友可以看，这篇综述把 PID 和强化学习怎么结合分了四类讲得很清楚，比零散看论文省时间。
arXiv 论文（编号 2611.03152）综述了建筑暖通空调（HVAC）系统的控制架构，涵盖双位控制、比例、PI 与 PID 调节及其整定方法，并将强化学习（RL）形式化为马尔可夫决策过程。论文的核心贡献是一套 RL 与 PID 结合方式的分类法：RL 作为现有回路的增益整定器、直接替换调节控制器、与模型预测控制混合、以及在监督层生成设定点与调度方案。综述指出物理控制回路保持不变，变化的是干预点，各方案在执行权限、状态信息、建模、调试和约束处理上的要求不同。结论认为这些架构并非竞争关系，而是同一设备上的嵌套接入点，混合实现可行。
From PID to Reinforcement Learning in Building HVAC Control: A Control-Architecture Review of Tuning Strategies, Formulations, and Deployment Trade-offs
This paper reviews control architectures for building heating, ventilation and air-conditioning (HVAC) systems, addressing the tension among energy demand, occupant comfort and equipment operability, and situating proportional-integral-derivative (PID) regulation within a layered control stack that also includes rule-based and model-based supervisory control and data-driven reinforcement learning (RL). It examines the thermostat-PID layer from two-position and scheduled setpoint control through proportional, PI and PID regulation and their tuning, then formulates RL for building control as a Markov decision process. Its central contribution is a taxonomy of where RL meets PID: RL as a gain tuner for an existing loop, as a direct replacement for the regulatory controller, as a hybrid with model predictive control, and as supervisory setpoint and schedule generation. Across these classes, the review establishes that the physical loop remains invariant while the intervention point changes, and that the classes differ in their requirements for actuation authority, state information, modelling, commissioning and constraint handling. It further establishes that PID is inexpensive and well codified but limited by nonlinearity, time variation, discrete mode switching, loop interaction and slow plant dynamics, while RL's admissible position must be argued rather than assumed. The paper concludes that these architectures are not competing philosophies of control but nested points of attachment on one HVAC plant, with hybrid implementations possible, and that deployment trade-offs must be assessed according to the specific control interface and scaffolding required.