论文

PBit1:每个参数用一个可编程 p-bit 表示预训练网络

PBit1: One Programmable P-Bit per Parameter - A Finite-Control Representation of Pretrained Networks

精选理由

这篇论文把每个权重换成单个概率型 p-bit,用时间观测重建原网络,GPT-2 和 LFM3 困惑度只涨 0.66% 和 0.26%。对研究概率计算、新型存算硬件的人来说是个很具体的构造方案,做 THRML 硬件实验的团队尤其该看。

arXiv 论文提出 PBit1,一种预训练神经网络的有限控制表示方法:每个学习到的参数对应一个可编程 p-bit,通过 12-bit 命令设定概率,共享仿射范围将概率映射到权重域,再用时间观测重建权重做推理。方法还引入算子感知的范围分组和基于敏感度的采样分配,使表示适配网络结构。

正文

实验中在 32,768 次观测的条件下,选定的配置在 GPT-2、DistilBERT 和 LFM3-1.7B 三个模型上保留了任务质量,全部学习标量都被转换且无需微调。GPT-2 与 LFM3 的困惑度分别上升约 0.66% 和 0.26%,DistilBERT 准确率从 87.51% 变为 87.49%。基于敏感度的策略把 GPT-2 的重建观测数减少 41.7%,同时保持质量标准。完整模型评估在 BF16 推理中复用重建权重,并在 THRML 算子实验中检验原生采样和时空权衡。

原文 · arXiv cs.AI

PBit1: One Programmable P-Bit per Parameter - A Finite-Control Representation of Pretrained Networks

We introduce PBit1, a finite-control representation of pretrained neural networks that assigns one programmable p-bit to each learned parameter. A 12-bit command sets its probability, a shared affine range maps that probability to the weight domain, and temporal observations reconstruct the weight for inference. Operator-aware range grouping and sensitivity-based sampling allocation adapt the representation to the network. Across five independent weight draws at 32,768 observations per scalar, selected configurations preserve task quality in GPT-2, DistilBERT, and LFM3-1.7B with every learned scalar converted and no fine-tuning. GPT-2 and LFM3 perplexity increase by approximately 0.66% and 0.26%, respectively, while DistilBERT accuracy changes from 87.51% to 87.49%. A sensitivity-based policy reduces GPT-2 reconstruction observations by 41.7% while retaining the quality criterion. Complete-model evaluations reuse reconstructed weights in BF16 inference, and THRML operator experiments examine native sampling and spatial-temporal tradeoffs. Together, these results establish PBit1 as a concrete construction for studying finite-control weight representation and temporal reconstruction in pretrained networks.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:p-bit、GPT-2、DistilBERT、THRML,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Omar Ramadan),经 AITOP 多维评分管线结构化提炼。