结构优于非线性:用于动态学习的显式交互架构

Structure Over Nonlinearity: Explicit Interaction Architectures for Dynamical Learning

精选理由

这篇论文提出了全新的动态学习范式:用结构化单元替代黑箱非线性,实验证明深度和交互结构能带来更好的表示和泛化,值得做时序建模的人关注。

AI 摘要

该论文提出一种基于波启发交互结构(wave-inspired interaction structures)的显式动态单元,摒弃传统黑箱非线性逼近方法。这些单元采用严格因果组织(causal organization),消除代数循环,无需隐式求解器即可直接评估。在非线性系统识别任务(nonlinear system identification)中,堆叠该单元形成的分层架构在参数优化有限的情况下,深度提升了表示质量和泛化能力。即使仅用readout-only拟合,架构也能产生有用的内部表示,说明交互结构本身即可提供模型表现力。

原文 · arXiv cs.LG

Structure Over Nonlinearity: Explicit Interaction Architectures for Dynamical Learning

Most learning architectures for dynamical systems rely on generic nonlinear function approximation, often requiring high model complexity to capture structured behaviors. In this work, we propose an alternative paradigm in which modeling capability arises primarily from structure rather than from expressive nonlinearities. We introduce a class of explicit structured dynamical units based on wave-inspired interaction structures with internal state. Inspired by wave-based computational principles, the proposed units adopt a strictly causal organization that eliminates algebraic loops, yielding fully explicit models that can be evaluated without implicit solvers. Stacking such units produces layered dynamical architectures with emergent hierarchical behavior. Through experiments on a nonlinear system identification task, we show that depth improves both representation quality and generalization, even under limited parameter optimization. In particular, the proposed architectures produce informative internal representations even under readout-only fitting, indicating that useful dynamical structure emerges from the organization of interactions prior to substantial parameter optimization. These results suggest that structure-first design provides a viable and effective alternative to conventional black-box approaches for learning dynamical systems, highlighting the role of interaction structure as a primary source of model expressivity.