Uri Kartoun, Akl C. Fahed, et al.
European Heart Journal Open
A fundamental challenge in diagnostics is integrating multiple modalities to develop a joint characterization of physiological state. Using the heart as a model system, we develop a cross-modal autoencoder framework for integrating distinct data modalities and constructing a holistic representation of cardiovascular state. In particular, we use our framework to construct such cross-modal representations from cardiac magnetic resonance images (MRIs), containing structural information, and electrocardiograms (ECGs), containing myoelectric information. We leverage the learned cross-modal representation to (1) improve phenotype prediction from a single, accessible phenotype such as ECGs; (2) enable imputation of hard-to-acquire cardiac MRIs from easy-to-acquire ECGs; and (3) develop a framework for performing genome-wide association studies in an unsupervised manner. Our results systematically integrate distinct diagnostic modalities into a common representation that better characterizes physiologic state.
Uri Kartoun, Akl C. Fahed, et al.
European Heart Journal Open
Uri Kartoun, Shaan Khurshid, et al.
Scientific Reports
James P. Pirruccello, Paolo Di Achille, et al.
Nature Communications
James P. Pirruccello, Paolo Di Achille, et al.
Nature Genetics