ActMap:生成时激活图单次量化不确定性
ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps
朋友推荐:新方法叫 ActMap,能从单个生成里快速判断答案是否可靠,比传统方法更高效。
ActMap 是一种新方法,它能从单个生成过程中提取隐藏状态轨迹,压缩成一个 12x32x128 的张量来量化不确定性。这个方法在短问答、直接回答数学和摘要事实性任务上,对 7-8B 指令微调模型的效果优于采样和概率基线,且与 ACT-ViT 相当。
ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps
Practical uncertainty quantification (UQ) for large language models must decide, from a single generation, whether a specific answer should be trusted. Existing methods either sample multiple generations, read only output-token probabilities, or reduce the model's internal computation to a single hidden state. We introduce ActMap, a white-box representation that compresses the generation-time hidden- state trajectory (every layer, every generated token) into a fixed $12 \times 32 \times 128$ tensor of temporal-statistic channels that preserves structure across transformer depth and pooled hidden coordinates. The map is captured during the generation pass with no measurable overhead, has a fixed shape across model depths and hidden sizes, and occupies 96 KiB: a compact artifact that can be retained for audit-relevant generations and probed directly, with occlusion analysis localizing the classifier's signal to mid-depth regions of the map. A lightweight classifier, instantiated as a compact Vision Transformer, reads an estimated correctness probability from each map in a fraction of a millisecond; capacity-matched MLPs perform comparably, indicating the representation itself carries the result. Trained and evaluated in-domain on short-answer QA, direct- answer math, and summarization factuality with three instruction-tuned 7-8B models, ActMap consistently outperforms sampling, token-probability, attention, and embedding baselines, and matches ACT-ViT, a detector trained on dense activation tensors $67 \times$ larger, at essentially the same mean AUROC with lower calibration error on ten of twelve pairs. The resulting score supports abstention, routing, and selective verification from a single generation, making it a practical primitive for scalable oversight of deployed models.