An Optimized Attention-Based Multimodal Deep Learning Framework for Real-Time Brain Tumor Detection with Enhanced Accuracy and Computational Efficiency

Abstract

Deep-learning provides effective approaches for clinical diagnosis and treatment planning, but traditional deep-learning models are usually complex, and the inability to perform real-time inference in the clinical environment limits the real-world use of these models. In order to achieve high diagnostic accuracy with little computing complexity, we present a hybrid multi-modal deep learning framework in this work that combines a CNN with a feature-level fusion with an attention mechanism. The proposed approach combines complementary information of MRI and CT imaging modalities to improve the representation of features and the accuracy in tumour classification.
To learn the modality-specific features, a staged training strategy is used while attention-based fusion is used to highlight diagnostically relevant regions. Furthermore, optimization techniques such as efficient architecture design, training strategies and more are integrated to keep inference latency low while preserving accuracy.
The suggested attention-based fusion model achieves 97.10% accuracy, 97.85% sensitivity, 96.25% specificity, and 97.26% dice score in the test set, according to the experimental assessment. Furthermore, real-time performance is achieved with an average detection time of less than 1 ms per scan. The suggested approach outperforms the baseline model, according to comparative analysis, and its accuracy has increased by about 0.65% when compared to the conventional fusion model. The findings demonstrate the significance of multimodal learning, which can greatly increase the precision and efficiency of diagnostics and improve the computational capabilities of brain tumour detection systems, while also confirming that the suggested framework is a useful and scalable solution for real-time brain tumour detection in clinical applications.