论文

ENA 架构用进化搜索与策略复用缓解持续强化学习的灾难性遗忘

Mitigating Catastrophic Forgetting with Physics-Indexed Specialist Memory: the Evolutionary Neuroplastic Agent

精选理由

arXiv 上这篇论文把进化搜索、环境检测和贝叶斯策略复用拼成一个持续 RL 记忆架构,CartPole 平均成功率 83.2%。它比 PPO 适应慢、不如 SAC 回报高,但策略归档复用的思路对做持续学习和智能体记忆系统的开发者有参考价值。

论文提出 Evolutionary Neuroplastic Agent(ENA),一种无需梯度的持续强化学习架构,核心思路是用进化搜索演化小型神经网络,并把选出的策略按检测到的环境动态归档为可复用记忆,检索阶段采用贝叶斯策略复用(Bayesian policy reuse)。

正文

在非平稳 CartPole 与 HalfCheetah 环境上的评估中,30 次重复实验的 CartPole 消融显示完整架构在五个测试环境上的平均成功率为 83.2%,在较难的 CartPole 环境变体中保持率最为一致。检索规则的消融实验表明其选择机制对性能有明显影响。

不过论文也明确指出局限:ENA 在 CartPole 上适应速度慢于 PPO,在 HalfCheetah 上回报低于 SAC,且证据未分离出无梯度优化本身是保持率的成因,也未证明优于其他持续学习方法。

原文 · arXiv cs.LG

Mitigating Catastrophic Forgetting with Physics-Indexed Specialist Memory: the Evolutionary Neuroplastic Agent

Continual reinforcement learning requires adaptation to changing conditions while preserving performance under previously encountered dynamics. It was investigated whether specialists generated by evolutionary search can provide a reusable memory when combined with environment detection and failure-driven search. The Evolutionary Neuroplastic Agent (ENA) evolves small neural networks without gradients, archives selected policies by detected dynamic regime, and retrieves them through Bayesian policy reuse. The architecture was evaluated in non-stationary CartPole and HalfCheetah environments. In the 30-replicate CartPole ablation, the complete architecture achieved a mean success rate of 83.2% across the five test environments. Among variants that learned the harder CartPole environments, the complete architecture showed the most consistent retention. A selection ablation also showed that the retrieval rule affected performance. In the evaluated sequences, ENA adapted more slowly than PPO on CartPole and achieved lower returns than SAC on HalfCheetah. The evidence concerns the complete architecture and does not isolate gradient-free optimization as the cause of retention or establish superiority over other continual-learning methods.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:持续学习、强化学习、进化算法、Bayesian policy reuse,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Blas Eugenio Vicco Zarate, Miguel Hernán Vicco),经 AITOP 多维评分管线结构化提炼。