Vision Transformer Outperforms CNN Architectures in Binary Skin Lesion Classification: An Eight-Model Controlled Deep Learning Study

Abstract

Skin cancer is one of the life-threatening malignancies in the world, timely detection of which is directly proportional to the survival of patients. Traditional dermoscopic diagnosis suffers inter-observer variability, lack of specialists and is not scalable in resource limited healthcare environments. The end-to-end hierarchical feature learning provided by deep learning is a transformative solution to the learned dermoscopic image corpora. The paper empirically comparatively examines eight binary skin lesion classifiers benign versus malignant of a dataset of 2,637 training images, and 661 held out test images. The assessed architectures are located on a wide design range a custom CNN trained using fresh data, two ResNet18 transfer learning pipelines, DenseNet121, MobileNetV3-Large, ViT-Small/16 (ImageNet-21K), ConvNeXt-Tiny (ImageNet-12K) and EfficientNetV2-S (ImageNet-21K). All the models are trained with the same conditions involving stratified splitting, Weighted Random Sampler, two-stage fine tuning with discriminative learning rates, Automatic Mixed Precision, and early stopping. It is evaluated using six metrics accuracy, per-class precision, recall, F1-score, ROC-AUC, and PR-AUC. The highest test accuracy (91.53) and macro-F1 (0.907) is attained with EfficientNetV2-S. ViT-Small/16 has the best ROC-AUC (0.9723) and PR-AUC (0.9699), which proves the effectiveness of Vision Transformer in threshold-free probabilistic discrimination the clinically decisive measure in screening applications. The three contemporary timm-based models have consistently reached ROC-AUC 0.95 and above, but the legacy CNN models are at 0.56 even though the legacy CNN models are at competitive accuracy of 89-90%. MobileNetV3-Large yields a false negative rate of 61 (20% miss rate), which highlights the clinical risk of aggressive model compression. The findings simplify the selection of the model in clinical studies in implementing dermoscopy, suggesting that ViT-Small/16 should be used as a probabilistic-ranked malignancy screening model, and EfficientNetV2-S should be used as a fixed threshold binary triage model in a telemedicine system.