Enhanced Facial Expression Recognition Using Multi-Scale Attention Fusion and Deep Learning

Abstract

Facial Expression Recognition (FER) is an important aspect of human-computer interaction, where machines can read through the facial expression to understand how an individual feels. Although the deep learning field has made great progress, it is difficult to detect delicate and contextual emotions in real life since the lighting, pose, occlusions, and low-resolution images are not constant. To overcome these limitations, this paper suggests a Multi-Scale Attention Fusion Network (MSAFN) with an EfficientNet-B1 backbone. This is because we are using three parallel convolutional branches with different kernel sizes of 1x1, 3x3, and 5x5 to capture fine-grained local information, mid-level structural information, and broader contextual information. The attention-based fusion of the extracted multi-scale features is followed by the classification of the features by fully connected layers with the help of dropout regularization. We tested our model on three additional well-known standard benchmark datasets: FER2013, RAF-DB, and AffectNet. We equalized emotion types to the seven universal emotions that Ekman proposed: anger, disgust, fear, happiness, sadness, surprise, and neutral. With accuracies of 72.18% on FER2013, 86.96% on RAF-DB, and 78.93% on AffectNet, the MSAFN outperformed cutting-edge models like ConvNeXt and EfficientNet baselines. Our findings show that multi-scale attention can be used to capture global facial geometry and local action units, which can be used to make them robust in varying visual conditions. The proposed architecture shows a significant improvement in detecting fine-grained emotions in challenging real-world situations with high intra-class variation and high noise.