几何感知双曲残差量化:改进离散表示的层级编码
Geometry-Aware Hyperbolic Residual Quantization
一篇量化方法论文:把残差量化搬到双曲空间,前向反向都做了几何修正,适合做层级离散表示的人看看。
论文提出几何感知的双曲残差量化方法,解决双曲扩展中的几何不一致问题。前向传播中,Hyperbolic Residual Aggregation 在 Poincare 球上恢复残差量化的 telescoping 行为;
反向传播中,discounted Hyperbolic Straight-Through Estimator 将梯度作为单一几何块传递,避免跨残差阶段的不稳定递归梯度传输。在层级预测、推荐、图像 tokenization 和神经音频编码任务上,该方法比朴素双曲基线更稳定。
作者同时指出结构-压缩权衡:纯压缩任务仍适合欧氏残差量化,双曲方法更适合层级组织的离散潜空间。
Geometry-Aware Hyperbolic Residual Quantization
Residual Vector Quantization turns continuous representations into discrete, multi-level token sequences. Yet most methods operate in Euclidean space, despite the coarse-to-fine structure of the resulting codes and the latent hierarchies present in many data domains. Hyperbolic geometry offers a natural alternative for hierarchical representations, but naive hyperbolic extensions introduce geometric inconsistencies: non-associative hyperbolic addition prevents consistent residual aggregation, while standard straight-through gradient estimation ignores the geometry of the latent space. We propose a geometry-aware hyperbolic residual quantization that addresses these issues in both the forward and backward passes. In the forward pass, Hyperbolic Residual Aggregation restores the telescoping behavior of residual quantization on the Poincare ball. In the backward pass, a discounted Hyperbolic Straight-Through Estimator routes the reconstruction gradient through the quantizer as a single geometric block, avoiding unstable recursive gradient transport across residual stages. Evaluations on hierarchical prediction, recommendation, image tokenization, and neural audio coding tasks show that our method improves the stability and structural organization of hyperbolic residual codes over naive hyperbolic baselines. At the same time, we observe a clear structure-compression trade-off: Euclidean residual quantization remains preferable for pure compression, while geometry-aware hyperbolic quantization is most useful for hierarchically organized discrete latent spaces.
本条情报归属 AITOP「论文」核心追踪栏目。涉及核心实体与技术议题包括:残差量化、双曲几何、向量量化、图像tokenization,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Alessio Colombo, Melika Ayoughi),经 AITOP 多维评分管线结构化提炼。