Yann LeCun 的团队提出目标传播算法,优化反向传播效率
Yann LeCun 团队提出的这个算法,能优化反向传播的效率,在稀疏自编码器场景下表现更好。
Yann LeCun 团队提出目标传播算法,将每一层的输入视为自由变量,作为前一层的目标。该算法通过将层间约束转化为惩罚项(输入与输出的差异)来优化,与反向传播优化相同目标但梯度计算方式不同。
在稀疏自编码器中,目标传播在 L1 正则化下比反向传播更高效。
Nice! This ends up being a version of what some of us have called "target prop": every layer's input is a free latent variable that serves as a target for the previous layer. As this paper points out, this can be derived from an "augmented Lagrangian" formulation of backprop in which the constraints (input of layer k+1 = output of layer k) are turned into penalties (divergence between input of layer k+1 and output of layer k). I've always hoped more people would pick up on this idea. I'm happy this is happening! I must say though that target prop, in the end, optimizes the same criterion as backprop and does the same thing as backprop while evaluating the gradient in a different way, perhaps more biologically plausible. My lab did some work on this idea in the context of "sparse auto-encoders" in the late 2000s. It turns out when the code in an auto-encoder is regularized (e.g. with L1 to make it sparse) target prop seems more efficient than backprop. scholar.google.com/citations?view…
本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:target prop、反向传播、稀疏自编码器,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「Yann LeCun」(@ylecun),经 AITOP 多维评分管线结构化提炼。