Implementing Facial Emotion Recognition with Attention-Enhanced Feature Learning and Grad-CAM Explainability

Abstract

Facial Emotion Recognition (FER) is a major study area in computer vision and affective computing because to its many applications in human-computer interaction, healthcare monitoring, intelligent surveillance, education, and behavioural analysis. Existing FER systems struggle with feature discrimination, model interpretability, class imbalance, and real-time deployment despite deep learning breakthroughs. This paper offers an explainable facial emotion detection system using EfficientNet-B0, an attention mechanism, and Gradient-weighted Class Activation Mapping to overcome these restrictions. The system uses transfer learning to extract discriminative facial features and an attention module to highlight emotion-relevant facial areas and suppress extraneous information. Grad-CAM also visualizes model predictions, improving transparency and user trust. Model robustness and generalization are improved by horizontal flipping, brightness modification, Gaussian blur, and coarse dropout. An experimental dataset included eight emotion categories: Angry, Contempt, Disgust, Fear, Happy, Neutral, Sad, and Surprise. The proposed model had 67% classification accuracy, 67% precision, 67% recall, and 65% F1-score. Happy emotion recognition performed best with an F1-score of 0.92. Streamlit-based online applications for real-time emotion prediction, confidence estimate, probability visualization, and explainability analysis were also created. Grad-CAM showed that the model prioritizes mouth, nose, cheeks, and eye regions during categorization. Experimental results show that the proposed framework balances recognition performance, computational efficiency, and interpretability, making it ideal for actual emotion-aware intelligent systems.