A Dual-Stage CNN–Transformer Hybrid Network for Robust Feature Representation in Mammography and Ultrasound-Based Cancer Detection

Abstract

Breast cancer is difficult to detect early and accurately with mammography and ultrasound images that are susceptible to high anatomical variability, low contrast, and noise. To overcome these issues, a novel Dual-Stage CNN–Transformer Hybrid Network is proposed in this study, which leverages the advantages of CNN to extract local spatial features and use transformer encoders to reason globally. In the first stage, a multi-scale CNN backbone is used to extract fine-grained morphological features like microcalcifications, lesions, and boundary textures. To capture long-range dependencies and cross-region interactions, a transformer-based encoder models long-range dependencies and cross-region interactions during the second stage, resulting in richer semantic understanding across heterogeneous tissue structures. A fusion module combines both spatial and context-specific representations, resulting in greater modality-invariant robustness against modality-specific artifacts. Experimental results conducted on the combined mammography and ultrasound datasets, show that the proposed hybrid network outperforms the state-of-the-art CNN-only and transformer-only models in terms of the sensitivity, specificity, and overall diagnostic accuracy. The findings suggest the promise of dual-stage hybrid systems for improving computer-aided diagnosis systems and assisting clinicians in accurate breast cancer screening.