研究提出基于循环的通用计算记忆机制，长序列泛化远超 Transformer
How Finite Brains Generalize Without Bound: Spatial Memory as a Universal Computer
用大脑网格细胞做记忆架构，训练只看 40 个符号，测试能反转一万五千个符号的字符串，Transformer 做不到的 5 个任务它都解了。
一篇 arXiv 论文提出，能泛化的学习器只能读取与输入规模无关的不变量，而读取位置、计数等坐标值的 LSTM 和 Transformer 会在长序列上失效。作者由此推出一种由循环构成的记忆结构，对应大脑中的 grid cells、place cells 与 hippocampal index。在 15 个基准任务上，该机器在 9 个任务达到平均 99% 正确率（测试长度 41 到 500），其中 5 个任务 LSTM 和 Transformer 均无法解决。训练仅用 40 个符号以内的字符串，测试时加入新循环后可无错反转 15015 个符号的字符串，错误出现的位置与理论预测曲线一致。
How Finite Brains Generalize Without Bound: Spatial Memory as a Universal Computer
A child who learns to add small numbers adds numbers of any length. LSTMs and Transformers trained on short sequences fail on longer ones. We trace the difference to what a learner reads when it decides. A position, a count or a time keeps taking new values as the input grows: it is a coordinate, and a learner that reads one meets in every long input values its experience never covered. Whether two things are the same has the same two answers at every size: it is a coincidence, and a learner that reads only coincidences finds in a long input nothing that short inputs lacked, so finite experience fixes its rule for every size. The memory carries the size of the input and the rule stays finite, and this determines the memory a generalizing brain can have: a memory that grows while what the learner reads of it stays finite has to look the same from every place, so it is made of cycles, and the past is reached by asking what happened when the cycles last stood where they stand now. The cycles are the grid modules, the cells that detect their return the place cells, the records the hippocampal index; a prefrontal controller moves the cycles and never sees them. Reading only coincidences costs no power: what finite experience fixes in such a learner is what Turing machines compute in step with their input, which is why a circuit that evolved to find the way home can learn arithmetic and grammar. In a network of neurons this machine of cycles is the dynamics seen through its symmetry, and symbols are its invariants. Trained on strings up to 40 symbols, the machine is at least 99% correct, on average over lengths 41 to 500, on nine of fifteen benchmark tasks, five of them tasks that no LSTM or Transformer solves; given new cycles after training it reverses strings of 15,015 symbols without error and fails beyond that on the curve the theory predicts; and its controllers run unchanged on neurons.