A DEEP SPATIO-TEMPORAL FUSION NETWORK FOR HEART FAILURE PROBABILITY ESTIMATION USING DYNAMIC CARDIAC IMAGING

Abstract

Heart failure (HF) is among the most consequential syndromic diagnoses in modern cardiovascular medicine, affecting an estimated 64 million individuals globally and accounting for approximately 11% of all cardiovascular-related hospital admissions in high-income countries. Early and accurate probability estimation of HF onset and progression is indispensable for timely therapeutic intervention, patient risk stratification, and healthcare resource allocation. Contemporary clinical HF prediction relies predominantly on single-modality assessments—echocardiographic ejection fraction, plasma biomarkers such as BNP and NT-proBNP, or clinical symptom scoring—each of which captures only a partial manifestation of the complex three-dimensional haemodynamic and myocardial tissue pathophysiology that underlies cardiac decompensation. The advent of high-

esolution 3D cardiac magnetic resonance imaging (CMR), 4D flow MRI, and three-dimensional echocardiography has created unprecedented opportunities for comprehensive spatiotemporal characterisation of cardiac structure, function, and fluid dynamics, yet the computational tools for integrating these data streams into unified prognostic models remain nascent. Four interconnected challenges impede the development of robust multi-modal deep learning frameworks for cardiac prognostics: (i) the inherent high dimensionality of 3D and 4D volumetric image data imposes severe memory and computational constraints on conventional deep learning architectures; (ii) temporal dependencies across cardiac phases and longitudinal follow-up scans must be jointly modelled with spatial structural features in a unified representation; (iii) systematic heterogeneity in imaging protocols, scanner manufacturers, and acquisition parameters across clinical sites introduces domain shift that degrades model generalisation; and (iv) quantifying and communicating predictive uncertainty to clinical users—a prerequisite for safe deployment in high-stakes medical decision-making—remains inadequately addressed. This dissertation proposes the Spatio-Temporal 3D-4D (ST-3D4D) framework, a novel five-layer deep learning architecture integrating 3D CMR cine sequences, 4D flow MRI velocity fields, three-dimensional echocardiography, PET/CT myocardial perfusion imaging, and longitudinal ECG and biomarker data. The framework incorporates three core algorithms: a Spatio-Temporal Residual Encoder (STRE) with physics-guided myocardial strain extraction, a Cross-Modal Attention Fusion (CMAF) mechanism for heterogeneous imaging data integration, and a Bayesian Prognostic Sequence Model (BPSM) providing calibrated 36-month survival probability estimates with uncertainty quantification. The system was developed and evaluated on 1,440 patients from five international cardiac imaging centres. The ST-3D4D framework achieved an AUC-ROC of 0.961 for HF probability estimation, surpassing the best single-modality baseline (3D CMR alone: AUC 0.891) by 7.0 percentage points. Cardiac structure segmentation yielded a mean DICE coefficient of 0.961 at 5,000 training volumes. Tri-modal fusion (CMR+Echo+PET) achieved balanced accuracy 2.6 percentage points above bi-modal (CMR+Echo) and 7.0 points above CMR alone. Expected Calibration Error was 0.018—significantly superior to 3D-CNN (0.041) and SVM-RBF (0.092). Ten-year MACE prediction C-statistic reached 0.887. The ST-3D4D framework constitutes a significant methodological advance in computational cardiac imaging, demonstrating that principled integration of spatio-temporal deep learning with multi-modal cardiac data yields clinically actionable heart failure prognostics superior to any existing single-modality or ensemble approach.