A Hybrid CNN–Transformer Cross-Attention Fusion Framework for Diabetic Retinopathy Grading in Retinal Fundus Images

Abstract

Diabetic retinopathy (DR) remains a leading cause of preventable blindness, and automated grading from fundus photographs can substantially reduce screening delays in resource-constrained settings. Convolutional neural networks (CNNs) capture fine-grained local texture such as microaneurysms and hemorrhages, whereas vision transformers (ViTs) are better suited to modeling long-range spatial dependencies between lesions distributed across the retina, but the two families of models are rarely combined in a way that preserves both strengths without a substantial increase in computational cost. This paper proposes HCF-Net, a lightweight hybrid architecture that pairs an EfficientNetV2-S convolutional backbone with a windowed Swin Transformer branch and merges their representations through a learnable cross-attention fusion module, rather than simple concatenation or late-stage averaging. The fused representation is passed to a focal-loss classification head that is explicitly tuned to counteract the severe class imbalance typical of DR grading datasets, where referable stages are underrepresented relative to healthy images. The framework is evaluated on three independent public benchmarks — APTOS 2019, DDR, and RFMiD — using five-fold cross-validation. In our experiments HCF-Net attains a mean grading accuracy of 96.4%, a quadratic-weighted kappa of 0.947, precision of 95.1%, recall of 94.9%, and an AUC of 97.9%, while requiring roughly 40% fewer floating-point operations than a comparable dual-branch fusion baseline built from two full-sized CNNs. An ablation study confirms that the cross-attention module, rather than backbone capacity alone, accounts for the majority of the accuracy gain over single-branch models. We further report a cross-dataset generalization test and a Grad-CAM-based qualitative analysis showing that the fused model attends to clinically relevant lesion regions. These results suggest that hybrid CNN–transformer fusion, guided by cross-attention rather than naive feature stacking, offers a favourable accuracy–efficiency trade-off for deployable DR screening tools.