大规模生存分析基准:49 个数据集、10 种方法的模型选择研究
What Survival Benchmarks Don't Tell You: Impact of Model Selection and Dataset Regimes
做生存分析的朋友值得看看,9 种方法跑 49 个数据集的结论很实在:选超参数时用 IBS 替代 C-index,校准度能明显提升而排序能力基本不掉。还提醒了删失严重时校准指标会虚高的问题。
一篇 arXiv 论文构建了一个大规模生存分析基准,用 10 种经典、树模型和深度学习方法在 49 个数据集上对比。研究的关键设计是把模型选择准则本身当作实验变量,分别用 C-index、Integrated Brier Score(IBS)和 MAE-PO 三种指标来选超参数。
通过贝叶斯混合效应建模,作者发现评估设计直接决定结论:对深度学习和树模型而言,用 IBS 而非 C-index 来选超参数,能在概率精度和校准度上获得显著提升,而判别排序能力几乎不受损失。方法表现还受数据集特征支配——DL 模型在大样本下扩展性很好,但在高维特征空间出现明显性能下滑。
研究还指出一个结构性缺陷:重度删失会人为推高绝对校准指标,使检验变得过于保守。论文主张放弃单一全局数字的性能总结,要求透明报告评估设置,代码已在 GitHub 开源。
What Survival Benchmarks Don't Tell You: Impact of Model Selection and Dataset Regimes
Despite the proliferation of survival analysis methods, current benchmarks often operate under narrow conditions: they heavily rely on the concordance index (C-index) for evaluation and model selection, and frequently either exclude deep learning (DL) methods or underrepresent challenging dataset regimes. To address these limitations, we conduct a large-scale, neutral benchmark comparing 10 classical, tree-based, and DL methods across 49 diverse datasets. Crucially, we treat the model selection criterion (optimizing for C-index, Integrated Brier Score (IBS), or Mean Absolute Error (MAE-PO)) as an independent experimental variable. Using Bayesian mixed-effects modeling, we demonstrate that evaluation design heavily dictates conclusions. For complex DL and tree-based architectures, selecting hyperparameters to optimize for IBS instead of C-index yields substantial improvements in probabilistic accuracy and calibration at a negligible cost to discriminative ranking. Furthermore, method performance (C-index, IBS, MAE-PO, and X-CAL) is fundamentally governed by dataset characteristics. While DL models scale exceptionally well with larger sample sizes, they suffer notable performance drops in high-dimensional feature spaces. Additionally, we expose a structural vulnerability where heavy censoring artificially inflates absolute calibration metrics, creating overly conservative tests. Ultimately, our findings challenge the utility of global single-number performance summaries and advocate for survival analysis results that transparently report the evaluation setup. Our code is available at: https://github.com/sdening/surv-bench-study
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:生存分析、C-index、IBS、基准测试,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Sandra Dening, Emre Kavak, Christian Wachinger, George H. Chen),经 AITOP 多维评分管线结构化提炼。