Feature-Decoupled CNN-Boosting Ensemble for Robust Handwritten Digit Recognition: Accuracy, Generalization and Deployment Trade-offs

Abstract

 Handwritten digit recognition is also relevant to the digitization of documents, data capture in education, postal automation and financial record processing, but models that excel on normalized benchmarks might not be resilient to domain shift and computational constraints. The present study suggests a hybrid approach that combines a compact three-block CNN with exports of a 256-dimensional feature vector Dense1 to classifiers (AdaBoost, XGBoost and LightGBM) to obtain spatial representations from handwritten digit images. They make predictions using hard majority voting guided by the validation. The model was trained on the EMNIST Digits split which has 240,000 training images and 40,000 test images, using controlled augmentation with stratified validation and leakage-prevention rules. Evaluation was performed across classes, across the single classifiers, with component ablations, 10-fold cross validation, addition of synthetic noise, external-domain test sets (USPS and CEDAR), a perturbation test with the FGSM and computational profiling. The overall performance with the accuracy of 98.45% and macro precision, recall and F1 score of 0.984 is 1.60% better than the standalone CNN. The accuracy drops to 98.20%, 97.95% and 97.80% when removing AdaBoost, XGBoost or LightGBM, respectively. The accuracy was kept at 97.82% when noise was Gaussian and 97.53% when noise was salt-and-pepper, but dropped to 94.80% on USPS and 90.20% on CEDAR. The ensemble achieved 85.30% accuracy while the CNN achieved 82.10% accuracy for the FGSM with ε = 0.1. The gains were obtained with 45 minutes of training, 150 ms/image CPU inference and 3.5 GB of memory. The contribution is then a characterization, guided by evidence, of when the CNN-to-boosting decoupling gives an edge to recognition and when it is hindered by costs of generalization and deployment.