Age Bias in Emotion Detection: An Analysis of Facial Emotion Recognition Performance on Young, Middle-Aged, and Older Adults

The growing potential for facial emotion recognition (FER) technology has encouraged expedited development at the cost of rigorous validation. Many of its use-cases may also impact the diverse global community as FER becomes embedded into domains ranging from education to security to healthcare. Yet, prior work has highlighted that FER can exhibit both gender and racial biases like other facial analysis techniques. As a result, bias-mitigation research efforts have mainly focused on tackling gender and racial disparities, while other demographic related biases, such as age, have seen less progress. This work seeks to examine the performance of state of the art commercial FER technology on expressive images of men and women from three distinct age groups. We utilize four different commercial FER systems in a black box methodology to evaluate how six emotions - anger, disgust, fear, happiness, neutrality, and sadness - are correctly detected by age group. We further investigate how algorithmic changes over the last year have affected system performance. Our results found that all four commercial FER systems most accurately perceived emotion in images of young adults and least accurately in images of older adults. This trend was observed for analyses conducted in 2019 and 2020. However, little to no gender disparities were observed in either year. While older adults may not have been the initial target consumer of FER technology, statistics show the demographic is quickly growing more keen to applications that use such systems. Our results demonstrate the importance of considering various demographic subgroups during FER system validation and the need for inclusive, intersectional algorithmic developmental practices.

[1]  C. Darwin,et al.  The Expression of the Emotions in Man and Animals , 1956 .

[2]  Amazon Rekognition , 2019, Machine Learning in the AWS Cloud.

[3]  A. Chasteen,et al.  Beyond the double-jeopardy hypothesis: Assessing emotion on the faces of multiply-categorizable targets of prejudice , 2009 .

[4]  C. Carbon,et al.  Age-Dependent Face Detection and Face Categorization Performance , 2013, PloS one.

[5]  J.-M. Sun,et al.  Facial emotion recognition in modern distant education system using SVM , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[6]  Florian Vogt,et al.  Towards More Robust Automatic Facial Expression Recognition in Smart Environments , 2017, PETRA.

[7]  Lauren Rhue,et al.  Racial Influence on Automated Perceptions of Emotions , 2018 .

[8]  Shan Li,et al.  Deep Facial Expression Recognition: A Survey , 2018, IEEE Transactions on Affective Computing.

[9]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[10]  Natalie C. Ebner,et al.  FACES—A database of facial expressions in young, middle-aged, and older women and men: Development and validation , 2010, Behavior research methods.

[11]  De'Aira G. Bryant,et al.  A Comparative Analysis of Emotion-Detecting AI Systems with Respect to Algorithm Performance and Dataset Diversity , 2019, AIES.

[12]  M. Fernández-Ardévol,et al.  Ageism in the era of digital platforms , 2020, Convergence.

[13]  Jahna Otterbacher,et al.  Emotion-based Stereotypes in Image Analysis Services , 2020, UMAP.

[14]  Brandon J. Pitts,et al.  Automated Speech Recognition Systems and Older Adults: A Literature Review and Synthesis , 2019, Proceedings of the Human Factors and Ergonomics Society Annual Meeting.

[15]  Damien Dupré,et al.  Accuracy of three commercial automatic emotion recognition systems across different individuals and their facial expressions , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[16]  Stefanie Rukavina,et al.  Expression intensity, gender and facial emotion recognition: Women recognize only subtle facial emotions better than men. , 2010, Acta psychologica.

[17]  Nikolaos M. Avouris,et al.  EVALUATION OF CLASSIFIERS FOR AN UNEVEN CLASS DISTRIBUTION PROBLEM , 2006, Appl. Artif. Intell..

[18]  Anil K. Jain,et al.  Face Recognition Performance: Role of Demographic Information , 2012, IEEE Transactions on Information Forensics and Security.

[19]  Sinan Kalkan,et al.  Investigating Bias and Fairness in Facial Expression Recognition , 2020, ECCV Workshops.

[20]  Jieyu Zhao,et al.  Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Zhihua Wang,et al.  A facial expression based continuous emotional state monitoring system with GPU acceleration , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[22]  Michael B. Lewis,et al.  Rapid communication: The own-age face recognition bias in children and adults , 2011, Quarterly journal of experimental psychology.

[23]  Wioleta Szwoch,et al.  Emotion Recognition for Affect Aware Video Games , 2014, IP&C.

[24]  Oliver G. B. Garrod,et al.  Facial expressions of emotion are not culturally universal , 2012, Proceedings of the National Academy of Sciences.

[25]  Christine L. Lisetti Affective Intelligent Car Interfaces with Emotion Recognition , 2005 .

[26]  Olga Russakovsky,et al.  Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Nicole Martinez-Martin What Are Important Ethical Implications of Using Facial Recognition Technology in Health Care? , 2019, AMA journal of ethics.

[28]  Yi Zeng,et al.  Responsible Facial Recognition and Beyond , 2019, ArXiv.

[29]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[30]  P. Garner,et al.  Racialized emotion recognition accuracy and anger bias of children's faces. , 2020, Emotion.

[31]  C. Darwin The Expression of the Emotions in Man and Animals , .

[32]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.