研究探讨 Transformer 对非线性系统的上下文控制能力
How do Transformers Perform Low-Level Control In-Context?
有人拿 GPT-2 架构去控制 cartpole 和 acrobot，不给参数只给轨迹，居然真能稳定住没见过的系统，挺有意思。
arXiv 论文研究了 GPT-2 风格的多头 Transformer 能否在未知系统参数的情况下控制不稳定非线性系统。实验选用 cartpole 和 two-link acrobot 两个经典非线性系统，上下文仅包含无外力的系统演化轨迹。训练时使用专家控制器轨迹、推理时不再提供，论文据此找出了训练 Transformer 执行低层控制的必要技巧。按 checkpoint 分析显示，上下文越长策略性能一般越好，模型还能泛化稳定分布外系统。
How do Transformers Perform Low-Level Control In-Context?
One of the remarkable abilities of Transformer-based neural networks is in-context learning. This motivates us to think about "in-context" control. Can we control a system knowing only (1) that it belongs to a particular family of systems and (2) some data on system evolution (provided as the context trajectory) but no specific parameter information? Specifically, we investigate the ability of Transformer-models to perform "in-context" control of unstable, nonlinear dynamical systems, where context consists only of the unforced system dynamics. We leverage a multi-head, GPT-2-style architecture, and we evaluate performance for two classical nonlinear systems -- the cartpole system and the two-link acrobot system, where the system parameters are unknown to the Transformer. We consider a setting where trajectories from an expert controller are available at training time but not at inference time. Here, we identify training techniques that are essential to train Transformers to perform low-level control. Through an analysis over checkpoints, we show that in-context control is possible, and the learned policy performance generally improves as context increases. Furthermore, the Transformer models can generalize to stabilize systems that are out-of-distribution.
