论文

有限智能如何应对无限环境:一项数学理论

Why Finite Intelligence Works in an Infinite World: A Mathematical Theory

精选理由

一篇硬核理论论文,把"模型不用记住所有数据也能应付未知任务"这件事写成了严格的数学框架,还给出了 coordinate 和 parity 任务族率失真曲线不同的具体例子。做表示学习、信息瓶颈或泛化理论研究的值得花时间读。

arXiv 论文 2611.18538 提出 future-task information 概念,定义为一个智能系统在未知未来任务下需要保留的最小信息量,以维持相对现有证据的决策价值。论文构造了 joint decision-regret state,记录每个允许任务下各动作的后验比较,并证明在紧性假设下,任何保留的表示都可替换为基于该状态的校准表示,在不增加信息量的前提下保住所有任务 regret,对应的信息-regret 曲线等于操作编码率。

正文

理论给出两条精确边界:动作数固定时由 weak-star closure 决定,允许所有有限动作菜单时由 source-evidence 律的 singular mass 决定。论文还展示集体结构超出单题难度决定信息需求:协方差相同的 coordinate 与 parity 任务族具有不同的率失真曲线,parity 族的最优保留由 Fourier-entropy 不等式刻画;对平衡嵌套任务族,refinement 尺寸与 weighted reach 在加性常数内确定率,并由显式衰减条件下的前缀编码达到。

这项工作把"为什么有限表示能泛化到无限多任务"形式化为可计算的信息论问题,与 Learned Representations、rate-distortion theory 及 grokking 相关研究同属一类基础理论方向。对研究表示学习的信息瓶颈、压缩与泛化关系的学者而言,提供了新的度量与精确边界工具。

原文 · arXiv cs.LG

Why Finite Intelligence Works in an Infinite World: A Mathematical Theory

An information system often has to learn before it knows what it will be asked. A finite representation can succeed without recording an infinite environment in full when those asks share collective structure, rather than demand unrelated facts scattered arbitrarily across it. We formalize this phenomenon through future-task information, the minimum retained information needed to preserve decision value relative to the available evidence. A joint decision-regret state records posterior comparisons among actions for every permitted task. Under compactness assumptions, every retained representation admits a calibrated replacement based on this state, preserving all task regrets without increasing information; the resulting information-regret curve equals an operational coding rate. Two exact boundaries govern finite preservation: weak-star closure for a fixed action count, and the singular mass of the source-evidence law when all finite action menus are admitted. Collective structure determines information requirements beyond individual-question difficulty. Families with identical source laws, pairwise task metrics, and individual-task rates can have different high-resolution exponents. Covariance-identical coordinate and parity families have different exact rate-distortion curves, with optimal parity retention characterized by a sharp Fourier-entropy inequality. For balanced nested task families, refinement sizes and weighted reach determine the rate within an additive constant, attained by prefix coding under explicit decay conditions. Nonlinear learning and computational separations distinguish preservation from efficient discovery and use. The proposed mathematical theory identifies which shared decision requirements allow finite information to remain useful for infinitely many possible purposes.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:信息论、rate-distortion、泛化理论、表示学习,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Mojtaba Soltanalian),经 AITOP 多维评分管线结构化提炼。