研究：JEPA 在纵向 EHR 表征学习上不敌 MLM
Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?
JEPA 在图像上很能打，但用 MIMIC-IV 一测，做病历表征还是干不过老老实实的 MLM，结论挺反直觉的。
一项发布在 arXiv 的研究考察 JEPA 是否适用于纵向电子健康记录（EHR）的序列表征学习。基于 MIMIC-IV 数据集的实验显示，三种 JEPA 变体虽能稳定训练且不发生表征坍塌，但下游性能一致低于标准 MLM。研究还发现目标大小（而非目标异质性）会增加预测误差，缩小目标尺寸也无法挽回下游表现。结论是仅靠稳定的潜空间预测不足以支撑有竞争力的 EHR 表征学习。
Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?
Masked language modelling (MLM) is the dominant pretraining objective for transformer encoders applied to longitudinal electronic health records (EHRs). However, reconstructing individual events may overemphasise noisy or weakly informative observations and does not explicitly encourage learning higher-level patient states. Joint-Embedding Predictive Architectures (JEPAs) instead learn by predicting representations in latent space and have shown strong performance in a variety of domains. In this paper, we investigate to what extent JEPA can be adapted to EHR sequence representation. Our empirical investigation using MIMIC-IV shows that although all three of our JEPA formulations train stably without representational collapse, they consistently underperform standard MLM. We find that target size, not target heterogeneity, increases prediction error, yet reducing the target size does not recover downstream performance. This suggests that stable latent prediction alone is insufficient for competitive EHR representation learning and highlights the challenge of defining clinically meaningful latent targets for EHR.
