论文

基于 Hellinger 风险的决策树密度比估计方法及有限样本收敛性保证

Hellinger Decision Trees for Density Ratio Estimation: Finite-Sample Guarantees

精选理由

做密度比估计的人可以看看这篇:直接用决策树集成加 Newton boosting 估对数密度比,还给出了能对上 L² minimax 最优速率的有限样本保证,合成和公开数据实验都验证了。做理论或评估生成模型的值得翻一翻。

一篇 arXiv 论文提出绕开分别估计两个密度函数的思路,直接用坐标轴对齐决策树的加性集成来估计对数密度比,训练方式为 Newton boosting,目标是极小化平衡 Hellinger 风险。

正文

作者为该算法建立了有限样本理论保证并给出显式收敛速率:在尾部条件、Hölder 光滑性以及每次 boosting 迭代有足够进展的假设下,速率在对数因子范围内匹配经典的 L² minimax 最优函数估计速率。技术上的关键是对 Hellinger 风险的指数结构做优化分析,证明经验风险的二阶 Taylor 近似下降量能控制实际风险下降量,从而无需依赖理想化的经验风险最小器即可推出算法输出的统计收敛保证。

密度比估计广泛应用于双样本比较、分布偏移适配和生成模型评估。论文在合成数据上验证了理论预测的收敛行为,并在公开数据集上展示了估计器的实用效果。

原文 · arXiv cs.LG

Hellinger Decision Trees for Density Ratio Estimation: Finite-Sample Guarantees

Density ratios play a central role in two-sample comparison, distribution-shift adaptation, and generative-model evaluation. However, estimating a density ratio through separate estimation of the underlying densities can be unnecessarily challenging. In this paper, we develop a direct approach to the log density ratio estimation by minimizing a balanced Hellinger risk over additive ensembles of axis-aligned decision trees, trained using Newton boosting. We establish finite-sample guarantees for the estimator produced by the algorithm and derive an explicit convergence rate. Under suitable tail and Hölder smoothness conditions, together with an additional assumption on the progress made at each boosting iteration, our rate matches, up to logarithmic factors, the classical minimax-optimal $ \mathscr{L}^{2} $ rate for function estimation. A key technical contribution is an optimization analysis that exploits the special exponential structure of the Hellinger risk. In particular, we show that the decrease in the second-order Taylor approximation (which is computable) of the empirical risk controls the actual decrease in that risk, yielding a direct optimization guarantee for the boosting algorithm. This allows us to establish statistical convergence guarantees for the algorithm's output without relying on an idealized empirical-risk minimizer. Experiments on synthetic data corroborate the predicted convergence behavior, while experiments on public datasets demonstrate the practical effectiveness of the proposed estimator.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:密度比估计、决策树、boosting、密度比,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Ren-Rui Liu, Zheng-Chu Guo),经 AITOP 多维评分管线结构化提炼。