Anatomical-Clinical Vision Fusion for Multimodal Clinical Prediction with Frozen Brain MRI Encoders
Aligning frozen 3D brain MRI representations with AAL regional volumes and clinical information for lightweight multimodal enhancement and leakage-aware evaluation.
- My role
- Multimodal method design, experiments, and evaluation
- Methods & tools
- 3D MRI · Contrastive learning · BioClinicalBERT · Leakage-aware evaluation
01
Problem & challenge
A pretrained MRI encoder captures imaging patterns but does not directly encode regional atrophy or clinical semantics. Full retraining is costly, while evaluation must guard against shortcut leakage from cognitive variables.
02
Key contributions
Designed a SigLIP-style contrastive objective across imaging, structured volume, and clinical-text representations.
Used frozen encoders and lightweight projection layers to control compute and preserve transferability.
Developed leakage-aware evaluation and cross-modality ablations with systematic failure analysis.
03
Outcome & disclosure boundary
The paper is under review at IEEE BIBM 2026. This public page omits internal metrics and controlled data.
Research in progress; code and manuscript are not public.