论文

3D 医学视觉语言模型经 CT-报告对比学习实现跨机构 HCC 分类

Efficient Self-Supervised Adaptation of 3D Abdominal Vision-Language Model for Institution-Specific HCC Classification via Full Fine-Tuning and PEFT

精选理由

医疗影像 VLM 微调的实操对比来了:同一模型做 HCC 分类,全量微调 AUC 0.82 但几个 epoch 就过拟合,ConvLoRA+LoRA 只训 1% 参数拿到 AUC 0.79。做医学 AI 落地或医院数据适配的团队可以对照自己的场景看取舍。

一项新研究将预训练的 3D 医学视觉语言模型(VLM)微调用于肝细胞癌(HCC)分类,训练数据来自与预训练语料获取特征不同的另一家机构的腹部 CT 与放射报告配对数据。论文对比了两种适配策略:全量微调取得 AUC 0.82、F1 0.71,PEFT 取得 AUC 0.79、F1 0.70。

正文

技术上,方法采用 CT-报告对比学习进行自监督适配,不需要病灶级标注或 HCC 分类标签作为训练目标。PEFT 方案在图像编码器使用 ConvLoRA、文本编码器使用 LoRA,仅更新约 1% 的模型参数即可获得接近全量微调的性能,但需要更多训练步数;全量微调收敛快,却在几个 epoch 内出现过拟合。研究还发现对比学习的温度初始化对分类性能影响显著。

结果显示该方法优于预训练基线,但仍低于有监督 HCC 分类方法的报告水平。对医疗 AI 落地而言,这项工作给出了跨机构域偏移场景下全量微调与 PEFT 的实用取舍参考,适合数据有限、标注稀缺的医院场景评估。

原文 · arXiv cs.LG

Efficient Self-Supervised Adaptation of 3D Abdominal Vision-Language Model for Institution-Specific HCC Classification via Full Fine-Tuning and PEFT

Medical vision-language models (VLMs) capture cross-modal relationships between image and text, yet their adaptation to institution-specific clinical tasks remains underexplored. In this study, we fine-tuned a pretrained 3D medical VLM for hepatocellular carcinoma (HCC) classification using paired abdominal CT scans and radiology reports from a different institution with acquisition characteristics that differ from the model's original pretraining corpus. We compared two adaptation strategies: full fine-tuning and parameter-efficient fine-tuning (PEFT), motivated by the common use of PEFT to reduce computational cost and enable adaptation under limited-data constraints. Both approaches achieve strong downstream HCC classification performance despite the cross-institutional domain shift, with PEFT reaching an AUC of 0.79 and F1 of 0.70, and full fine-tuning achieving an AUC of 0.82 and F1 of 0.71. Although these results remain below the performance reported by supervised HCC classification approaches, our method achieves improvements over the pretrained model through CT-report contrastive learning without requiring lesion-level annotations or HCC classification labels as training targets. Full fine-tuning converges rapidly but overfits within a few epochs, whereas PEFT (ConvLoRA for the image encoder and LoRA for the text encoder) attains comparable performance while updating approximately 1% of the model parameters, although requiring more training steps. We also examine the role of contrastive temperature, observing that temperature initialization significantly affects classification performance. This study demonstrates that a 3D medical VLM can be efficiently adapted to institution-specific HCC classification using self-supervised CT-report contrastive learning, while highlighting the practical trade-offs between full fine-tuning and parameter-efficient fine-tuning.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:医疗影像、VLM、LoRA、微调,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Febryan Putra Kartika, Cheng-Yu Ma, Ying-Jia Lin, Chi-Tung Cheng, Kuan-Fu Chen),经 AITOP 多维评分管线结构化提炼。