Attended Relation Feature Representation of Facial Dynamics for Facial Authentication

In psychology, it is known that facial dynamics benefit the perception of identity. This paper proposes a novel deep network framework to capture identity information from facial dynamics and their relations. In the proposed method, facial dynamics occurred from a smile expression are analyzed and utilized for facial authentication. Detailed changes in the local regions of a face such as wrinkles and dimples are encoded in the facial dynamic feature representation. The latent relationships of the facial dynamic features are learned by the facial dynamic relational network. In the facial dynamic relational network, the relation features of the facial dynamic are encoded and the relational importance is encoded based on the relation features. As a result, the proposed method has more attention on the important relation features in facial authentication. Through comprehensive and comparative experiments, the effectiveness of the proposed method has been verified in facial authentication.

[1]  Honggang Zhang,et al.  Joint Patch and Multi-label Learning for Facial Action Unit and Holistic Expression Recognition , 2016, IEEE Transactions on Image Processing.

[2]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  A. Johnston,et al.  Categorizing sex and identity from the biological motion of faces , 2001, Current Biology.

[7]  Zhe Wang,et al.  On the facilitative effects of face motion on face recognition and its development , 2014, Front. Psychol..

[8]  Albert Ali Salah,et al.  Combining Facial Dynamics With Appearance for Age Estimation , 2015, IEEE Transactions on Image Processing.

[9]  Antitza Dantcheva,et al.  Gender Estimation Based on Smile-Dynamics , 2017, IEEE Transactions on Information Forensics and Security.

[10]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Albert Ali Salah,et al.  Are You Really Smiling at Me? Spontaneous versus Posed Enjoyment Smiles , 2012, ECCV.

[12]  A. Johnston,et al.  The Role of Movement in Face Recognition , 1997 .

[13]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[14]  Z. Kourtzi,et al.  A Matching Advantage for Dynamic Human Faces , 2002, Perception.

[15]  V. Bruce,et al.  The role of movement in the recognition of famous faces , 1999, Memory & cognition.

[16]  Ming Shao,et al.  A Multi-stream Bi-directional Recurrent Neural Network for Fine-Grained Action Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Gregory D. Hager,et al.  Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation , 2016, ECCV.

[18]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[19]  Yong Man Ro,et al.  Multi-Objective Based Spatio-Temporal Feature Representation Learning Robust to Expression Intensity Variations for Facial Expression Recognition , 2019, IEEE Transactions on Affective Computing.

[20]  L. Fleischer.  Telling Lies Clues To Deceit In The Marketplace Politics And Marriage , 2016 .

[21]  P. Ekman,et al.  Smiles when lying. , 1988, Journal of personality and social psychology.

[22]  K. Lander,et al.  Independence of face identity and expression processing: exploring the role of motion , 2015, Front. Psychol..

[23]  Ning Ye,et al.  Towards general motion-based face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Vicki Bruce,et al.  Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces , 2001 .

[25]  Yong Man Ro,et al.  Facial dynamic modelling using long short-term memory network: Analysis and application to face authentication , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[26]  Albert Ali Salah,et al.  Like Father, Like Son: Facial Expression Dynamics for Kinship Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  Yong Man Ro,et al.  Spatio-temporal representation for face authentication by using multi-task learning with human attributes , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[28]  C. Metz,et al.  "Proper" Binormal ROC Curves: Theory and Maximum-Likelihood Estimation. , 1999, Journal of mathematical psychology.

[29]  K. Lander,et al.  Why are moving faces easier to recognize? , 2005, Visual Cognition.

[30]  Albert Ali Salah,et al.  Recognition of Genuine Smiles , 2015, IEEE Transactions on Multimedia.

[31]  Sridha Sridharan,et al.  Two Stream LSTM: A Deep Fusion Framework for Human Action Recognition , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[32]  Shiqi Wang,et al.  Learning Generalized Deep Feature Representation for Face Anti-Spoofing , 2018, IEEE Transactions on Information Forensics and Security.

[33]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[34]  Dongqing Zhang,et al.  Neural Aggregation Network for Video Face Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  L. Rubin.  The Mechanism of Human Facial Expression , 1992 .

[36]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[37]  Bingbing Ni,et al.  Progressively Parsing Interactional Objects for Fine Grained Action Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Ser-Nam Lim,et al.  Adaptive RNN Tree for Large-Scale Human Action Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[39]  A. O'Toole,et al.  Psychological and neural perspectives on the role of motion in face recognition. , 2003, Behavioral and cognitive neuroscience reviews.

[40]  H. Bülthoff,et al.  A search advantage for faces learned in motion , 2006, Experimental Brain Research.

[41]  Stefanos Zafeiriou,et al.  Incremental Face Alignment in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Kamal Nasrollahi,et al.  Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification , 2017, IEEE Transactions on Cybernetics.

[44]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  G. Boulogne,et al.  The mechanism of human facial expression: Frontmatter , 1990 .

[46]  A. O'Toole,et al.  Recognizing moving faces: a psychological and neural synthesis , 2002, Trends in Cognitive Sciences.

[47]  Qiang Ji,et al.  Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.