A Deep Learning Framework for Facial Attribute Recognition: Implementation and Performance Analysis Using Augmented Data

Abstract

Facial Attribute Recognition (FAR) applications, powered by deep learning models, reach impressive benchmark results and target semantic characterisation of faces in images. The facial features semantic characterisation helps identify features such as gender and age, as well as other attributes (e.g. cosmetics, glasses, facial hair). Distinct from face identification, FAR applications intend to pose distinct characterizing questions of face features. Unfortunately, FAR deep learning models cannot be effectively deployed and evaluated in real-world settings due to representativeness issues in the training and testing data. In this paper, we build a deep learning-based FAR framework which attempts to mitigate issues presented in current models. Within the framework, we use a multi-task ResNet-based deep learning model, trained on the CelebA data set, which includes augmentation with simulated occlusion and illumination variations, and combined class-weighted loss functions to address imbalances in class attributes. A robust evaluation paradigm, with per-attribute metrics, cross-dataset testing on LFWA+ and challenging subset evaluations, and fairness and demographic analyses are also incorporated. Results indicate a substantial improvement in robustness to occlusion and low-illumination conditions. Per-attribute metrics also revealed performance issues that were not seen using overall accuracy. LFWA+ benchmark data confirmed the generalisation potential of training under data augmentation. Fairness analyses also suggested improvements in demographic representation. The paper presents a practical FAR model and an evaluation methodology aimed at narrowing the performance gap evident in benchmark tests and the practical and scalable use of FAR in real world conditions.