Token-Energy Law：用活跃字节和解码时间预测 LLM 每 token 能耗
The Token-Energy Law of LLM Decode: Active Bytes and Realized Time Account for Joules per Token
这篇论文把每 token 的电耗拆成活跃字节加解码时间两个量，误差 3.73% 以内，做推理部署省电的可以照着算。
论文提出 Token-Energy Law，把 GPU 上稠密模型每生成一个 token 的能耗归结为两个变量：活跃字节数（区分权重共享读取与每请求私有的 KV 缓存读取）和实际解码时间。两者以仿射形式组合，系数按模型和硬件/运行时配置标定。实验覆盖三个模型在 NVIDIA Hopper 和 Blackwell GPU 上的 BF16 解码，包括 Hopper 上双卡张量并行，留出条件下能量估计误差不超过 3.73%。该公式可解释批处理如何摊薄权重读取成本、更长的注意力历史如何增加 KV 需求。
The Token-Energy Law of LLM Decode: Active Bytes and Realized Time Account for Joules per Token
Energy-efficient deployment of large language models requires understanding how model architecture, hardware, and serving settings shape energy per generated token. We introduce the Token-Energy Law, a compact, interpretable model of GPU-board energy per output token for dense, full-history decoding. It relates token energy to two core dimensions: active bytes and realized decode time. Active bytes describe logical byte demand per output, separating dense weights shared within a decode iteration from private key--value (KV) histories read and updated for each request. Realized decode time is the elapsed decode duration divided by the number of output tokens produced. We combine the two byte coordinates and time in a common affine form, with coefficients calibrated for each model and hardware/runtime configuration. This formulation explains how batching amortizes shared-weight work, longer attended histories increase private KV demand, and throughput determines the time-associated cost per output. Our experiments cover three models under BF16 decoding on NVIDIA Hopper and Blackwell GPUs, including two-GPU tensor parallelism on Hopper. Using fixed coefficients and observed decode time, held-out condition-mean energy estimates differ from measurements by at most 3.73%.
