论文

WeightCode 提出权重空间编程:不训练直接向网络写入新能力

WeightCode: Installing New Capabilities into Neural Networks via Weight-Space Programming

精选理由

不用再跑微调了:一个冻结的语言模型直接写程序改权重,把新技能装进训好的网络。ImageNet 上 8.84-17.62 点的增益超过 8 种微调方法,程序还能跨 5 个独立训练的 ViT 复用。做模型定制和持续学习的工程师值得认真评估这条路子。

一篇发表于 arXiv 的论文(编号 2611.25476)提出 Weight-Space Programming(WSP),用可执行程序直接读取、构造或修改神经网络权重,从而在已训练模型中安装新能力。其框架 WeightCode 由一个冻结的语言模型生成权重程序,无需梯度训练,且安装后新能力可独立于该语言模型运行。

正文

实验数据方面:给定语法或 8 个演示样本后,权重程序将未见过的命令组合准确率从 0% 提升到 99.8%,新 pickup 动作成功率从 0% 升至 98.4%,同时保留原有指令与导航能力。在 ImageNet 上,4 个权重程序取得 8.84-17.62 点的净增益且不损失 clean accuracy,在同等样本数、检查次数和保留约束下超过 8 种微调方法中的最优者。

可迁移性是关键卖点:同一组权重程序在 5 个独立训练的 Vision Transformer 上重跑,平均保留 83-110% 的源网络增益,而直接复制固定权重编辑几乎无增益;6 个程序写入已编辑网络后额外带来 5.23-14.50 点对比鲁棒性,并保留此前安装的噪声增益。论文将 WSP 定位为在训练后网络中积累、复用能力的另一条路线,与 LoRA 等参数高效微调形成互补。

原文 · arXiv cs.AI

WeightCode: Installing New Capabilities into Neural Networks via Weight-Space Programming

Neural networks encode learned capabilities in their weights, yet extending those capabilities with explicit task knowledge remains a fundamental challenge. Here we show that Weight-Space Programming (WSP), in which executable programs read, construct or modify a network's weights directly, can install new capabilities within trained neural networks. WeightCode, a WSP framework, uses a frozen language model to write executable weight programs that install new computations into trained neural networks using task knowledge and their existing weights, without gradient-based training. Once installed, the new capabilities run independently of the language model. Given a grammar or eight demonstrations, weight programs enable previously unseen command compositions (0% to 99.8% exact match) and a new pickup action (0% to 98.4% success), while preserving the original command and navigation capabilities. On ImageNet, four weight programs achieve net gains of 8.84-17.62 points while preserving clean accuracy, outperforming the best of eight fine-tuning methods under matched examples, checks, and retention constraints. In a Vision Transformer, causal interventions show that installed noise-correction gates drive robustness gains, while their selective activation preserves clean accuracy. Rerun on five independently trained Vision Transformers without further language model calls, ten weight programs retain 83-110% of their source-network gains on average, whereas copying their fixed weight edits yields negligible gains. Six programs written for an already edited network add 5.23-14.50 points of contrast robustness while retaining the previously installed noise gain. Together, these results establish WSP as a new route to acquiring and accumulating capabilities in trained neural networks through reusable weight programs.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:WeightCode、Vision Transformer、ImageNet、微调,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Bo Li, Taoyong Cui, Bob Zhang, Mengran Li, Chengyang Zhang, Yang Zhang, Pheng Ann Heng, Wanli Ouyang, Zhenfei Yin, Yingcheng Wu, Ling Yang),经 AITOP 多维评分管线结构化提炼。