Interpretable Integration of Molecular Omics and Medical Imaging for Disease Analysis
Abstract
Purpose
Medical imaging and molecular omics data provide complementary information about disease. Imaging describes tissue and organ structure, while omics measurements, e.g., gene expression, capture cellular and molecular processes. Integrating these data types is important for studying disease mechanisms, but existing deep learning–based multimodal approaches often rely on complex model designs that limit biological interpretation. In particular, it remains difficult to determine how molecular signals are associated with specific regions or features in medical images. Interpretable multimodal frameworks are needed to directly relate molecular omics data to spatial image features across biological scales.
Methods
We propose a unified multimodal framework that integrates molecular omics data with medical images using standard convolutional neural networks. High-dimensional tabular omics data are transformed into two-dimensional image-like representations using Optimal Transport–based mapping. These molecular representations are treated as additional input channels and combined with images, enabling joint learning of imaging and molecular features and supporting analysis of how molecular signals relate to specific image regions. The framework was evaluated on publicly available cancer datasets, whole-slide pathology images with spatial transcriptomics, and neuroimaging MRI with molecular data.
Results
Integrating molecular omics data with medical images improved classification performance by up to 5% compared with image-only models. Beyond performance gains, preliminary analyses indicate that the model focuses on biologically relevant image regions. In pan-cancer classification experiments, the framework further identified omics types contributing to predictions for different cancer classes, with findings consistent with existing biological studies.
Conclusion
This work presents an interpretable multimodal framework that integrates molecular data with medical images using Optimal Transport. By transforming omics data into image-compatible representations, the approach enables joint analysis within a single network and supports investigation of how molecular information relates to spatial image features. The framework provides a practical tool for studying disease biology and advancing integrative imaging-based research.