截断BPTT的梯度边界效应被证明呈指数衰减
Truncated backpropagation through time induces a two-sided gradient turnpike property
搞RNN或循环模型训练的可以看看，这篇把截断BPTT到底丢多少梯度信息给出了数学刻画，结论挺实用。
这篇arXiv论文研究截断backpropagation through time（TBPTT）在训练循环模型时的理论性质。作者指出截断会带来两侧边界效应：丢失过去信息影响前向状态演化，截断反向递推则丢失未来信息。在状态转移Jacobian指数衰减等温和假设下，截断梯度与全上下文梯度的误差随距两侧边界的距离呈指数衰减，称为two-sided gradient turnpike property。论文还推导了平均更新梯度的界，并在Polyak-Lojasiewicz条件下得到最优性gap的界，数值实验用Wiener-Hammerstein基准验证了理论与算力开销的权衡。
Truncated backpropagation through time induces a two-sided gradient turnpike property
Truncated backpropagation through time reduces the computational burden of training recurrent models by restricting gradient computations to shorter temporal contexts. This induces two distinct boundary effects: missing past information perturbs the forward state evolution, while truncating the backward recursion omits future information. This motivates decoupling the interval over which gradient contributions are averaged for the parameter update from the surrounding temporal context over which they are computed. Under exponential decay of the state-transition Jacobians and mild regularity conditions, this leads to a two-sided gradient turnpike property: the error between truncated- and full-context gradient contributions decays exponentially with the distance from both context boundaries. We derive bounds for the averaged update gradient, yielding guarantees on gradient alignment and approximate full-context stationarity; under unbiased interval sampling and a Polyak-Lojasiewicz condition, these further imply a bound on the resulting optimality gap. Numerical experiments on the Wiener-Hammerstein benchmark illustrate the role of past and future context and the trade-off between predictive performance and computational cost.