Changes in Facial Expression as Biometric: A Database and Benchmarks of Identification

Facial dynamics can be considered as unique signatures for discrimination between people. These have started to become important topic since many devices have the possibility of unlocking using face recognition or verification. In this work, we evaluate the efficacy of the transition frames of video in emotion as compared to the peak emotion frames for identification. For experiments with transition frames we extract features from each frame of the video from a fine-tuned VGG-Face Convolutional Neural Network (CNN) and geometric features from facial landmark points. To model the temporal context of the transition frames we train a Long-Short Term Memory (LSTM) on the geometric and the CNN features. Furthermore, we employ two fusion strategies: first, an early fusion, in which the geometric and the CNN features are stacked and fed to the LSTM. Second, a late fusion, in which the prediction of the LSTMs, trained independently on the two features, are stacked and used with a Support Vector Machine (SVM). Experimental results show that the late fusion strategy gives the best results and the transition frames give better identification results as compared to the peak emotion frames.

[1]  Gholamreza Anbarjafari,et al.  Data Fusion Boosted Face Recognition Based on Probability Distribution Functions in Different Colour Channels , 2009, EURASIP J. Adv. Signal Process..

[2]  Thomas B. Moeslund,et al.  Spatio-temporal Pain Recognition in CNN-Based Super-Resolved Facial Images , 2016, VAAM/FFER@ICPR.

[3]  Ralph Gross,et al.  Individual differences in facial expression: stability over time, relation to self-reported emotion, and ability to inform person identification , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[4]  Thomas B. Moeslund,et al.  Aalborg Universitet Pain Expression as a Biometric Why Patients’ Self-Reported Pain doesn’t Match with the Objectively Measured Pain? , 2016 .

[5]  Thomas B. Moeslund,et al.  Heartbeat Signal from Facial Video for Biometric Recognition , 2015, SCIA.

[6]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[7]  Q. M. Jonathan Wu,et al.  A survey of local feature methods for 3D face recognition , 2017, Pattern Recognit..

[8]  Woo Chaw Seng,et al.  A review of biometric technology along with trends and prospects , 2014, Pattern Recognit..

[9]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Venu Govindaraju,et al.  Facial Expression Biometrics Using Tracker Displacement Features , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Khandaker Abir Rahman,et al.  Facial Expression Analysis as a Means for Additional Biometric Security in Recognition Systems , 2015, MCSS.

[12]  Patrick J. Flynn,et al.  Multiple Nose Region Matching for 3D Face Recognition under Varying Facial Expression , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Stefanos Zafeiriou,et al.  Facial behaviometrics: The case of facial deformation in spontaneous smile/laughter , 2011, CVPR 2011 WORKSHOPS.

[14]  Bernard Zenko,et al.  Is Combining Classifiers with Stacking Better than Selecting the Best One? , 2004, Machine Learning.

[15]  Dacheng Tao,et al.  A Comprehensive Survey on Pose-Invariant Face Recognition , 2015, ACM Trans. Intell. Syst. Technol..

[16]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Alice J. O'Toole,et al.  Face Recognition Algorithms Surpass Humans Matching Faces Over Changes in Illumination , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Alex Graves,et al.  Long Short-Term Memory , 2020, Computer Vision.

[19]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Maja Pantic,et al.  Automatic Analysis of Facial Actions: A Survey , 2019, IEEE Transactions on Affective Computing.

[21]  Thomas B. Moeslund,et al.  Distributed Computing and Monitoring Technologies for Older Patients , 2016, SpringerBriefs in Computer Science.

[22]  Thomas B. Moeslund,et al.  Facial Expression Recognition for Traumatic Brain Injured Patients , 2018, VISIGRAPP.

[23]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[24]  Mihai Gavrilescu Study on using individual differences in facial expressions for a face recognition system immune to spoofing attacks , 2016, IET Biom..

[25]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[26]  Thomas B. Moeslund,et al.  Can contact-free measurement of heartbeat signal be used in forensics? , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[27]  Dacheng Tao,et al.  Pose-invariant face recognition with homography-based normalization , 2017, Pattern Recognit..

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Saeid Nahavandi,et al.  Recent Advances on Singlemodal and Multimodal Face Recognition: A Survey , 2014, IEEE Transactions on Human-Machine Systems.

[30]  H. de Wit,et al.  Drug effects on responses to emotional facial expressions: recent findings , 2015, Behavioural pharmacology.