Deep Learning-Based Multi-Modal Framework for Early Diagnosis of Cardiovascular Disorders

Abstract

Cardiovascular disorders are still a great challenge in clinical practice, as their symptoms can include electrical, anatomical, physiological, and patient-specific aspects that be encompassed in a single diagnostic method. The present study introduces a deep learning-based multi-modal framework for early diagnosis of cardiovascular disorders, which combines electrocardiogram (ECG) signals, cardiac imaging, and structured clinical information in a single computational architecture. For ECG recordings, the temporal features are extracted by modality-specific deep-learning encoders; for cardiac images, the spatial and structural features; and for clinical variables, the nonlinear relationships. Cross-modal attention was then used to learn interactions between electrical and anatomical and clinical information in the extracted representations in a common latent space. Modality representations are then passed through an adaptive fusion mechanism for the classification of cardiovascular diseases. The framework introduces explainable AI (xAI) techniques for ECG interpretation (Integrated Gradients), cardiac-image localization (Grad-CAM), and clinical-feature attribution (SHAP). The experimental evaluation focuses on the training convergence, the distribution of the prediction error, the ROC characteristics of the classes, the behavior of the multiclass classification, and the comparisons with the ECG-only, image-only, and clinical-only configurations. The reported results show excellent diagnostic discrimination with the AUROC values of the classes ranging from 0.928 to 0.965; the multi-modal configuration has an accuracy of ~ 93%, precision of ~92%, a sensitivity of ~ 91%, a specificity of ~95%, an F1 score of ~91% and an AUROC of ~96%. Results show that fusion of heterogeneous cardiovascular information can yield more complete diagnostic representation than when using any single modality and can be useful in reliable disease identification in the early stages. The explainability feature enhances transparency by linking predictions to specific signal segments, image regions, and clinical features.