MMAP：面向阿尔茨海默病纵向预测的多模态缺失感知预训练方法
MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction
这篇论文做了个挺实用的模型 MMAP，专门解决医疗数据缺图或缺表格的情况，在阿尔茨海默病转化预测上跑赢了基线。
论文提出 MMAP，一种针对不完整多模态医疗数据的预训练方法，结合图像与表格编码器。图像编码器采用 sigmoid 对比学习加生成式重建进行预训练，表格编码器构建在表格基础模型之上，并引入 missing token generator 处理缺失模态。在阿尔茨海默病的两个纵向临床任务上评估：疾病阶段转化预测和 amyloid 状态预测。结果显示该方法优于强多模态与单模态基线。
MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction
Clinical decision making heavily relies on predicting the disease progression trajectory by seeking to understand patient's health status which is characterised by multimodal medical data. AI holds great potential for learning useful representations from multimodal medical data to predict disease progression and aid clinical decision making. However, development of predictive AI models is constrained by missing modalities and incomplete tabular data frequently occurring in medical datasets. In addition, disease labels alone may only provide limited supervisory signals for learning representations from high-dimensional multimodal data. Here, we present MMAP, a novel Multimodal Missing-aware Alignment Pretraining method for learning image-tabular representations from incomplete data. An image encoder is pretrained with efficient sigmoid contrastive learning combined with generative reconstruction. A tabular encoder is built upon a tabular foundation model. A missing token generator enables the two encoders to take incomplete data as input, enabling the model to be robust against missing modalities, either with missing images or missing tabular data. We evaluate the clinical usefulness of the learnt multimodal representations on two challenging longitudinal clinical tasks for Alzheimer's disease: predicting disease stage conversion and predicting amyloid status. The proposed method outperforms strong multimodal and unimodal baselines.
