Sparse Orthogonal Tucker Decomposition for 2D+3D Facial Expression Recognition

Multimodal facial expression recognition (FER) has been becoming a promising research hotspot in recent years. In this paper, we propose a novel sparse orthogonal Tucker decomposition for 2D+3D facial expression recognition (FER). Firstly, we construct a 4D tensor model to explore the efficient structure and complement information between 2D and 3D data. Secondly, based on orthogonal Tucker decomposition, the tensor optimization model, which impose sparse constraint on the involved core tensor and factor matrices, are utilized to find strong interactions between the core tensor and factor matrices for better classification prediction. Finally, a alternating direction method is employed to effectively solve the proposed optimization problem. Numerical experiments are implemented on the BU-3DFE database to indicate the effectiveness of the proposed approach.

[1]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Xi Zhao,et al.  An efficient multimodal 2D + 3D feature-based approach to automatic facial expression recognition , 2015, Comput. Vis. Image Underst..

[3]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[4]  Zhaoyu Wang,et al.  Analyses of a Multimodal Spontaneous Facial Expression Database , 2013, IEEE Transactions on Affective Computing.

[5]  Mohammad Mahdi Dehshibi,et al.  A new algorithm for age recognition from facial images , 2010, Signal Process..

[6]  Qiuqi Ruan,et al.  Fast nonnegative tensor factorization based on graph-preserving for 3D facial expression recognition , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[7]  Alberto Del Bimbo,et al.  A Set of Selected SIFT Features for 3D Facial Expression Recognition , 2010, 2010 20th International Conference on Pattern Recognition.

[8]  Emmanuel Dellandréa,et al.  Automatic 3D Facial Expression Recognition Based on a Bayesian Belief Net and a Statistical Facial Feature Model , 2010, 2010 20th International Conference on Pattern Recognition.

[9]  Xiaoou Tang,et al.  Automatic facial expression recognition on a single 3D face by exploring shape deformation , 2009, ACM Multimedia.

[10]  Hasan Demirel,et al.  Facial Expression Recognition Using 3D Facial Feature Distances , 2007, ICIAR.

[11]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[12]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[13]  Mohan M. Trivedi,et al.  Face Expression Recognition by Cross Modal Data Association , 2013, IEEE Transactions on Multimedia.

[14]  Jun Wang,et al.  3D Facial Expression Recognition Based on Primitive Surface Feature Distribution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  L. Mirsky.  A trace inequality of John von Neumann , 1975 .

[16]  Qiuqi Ruan,et al.  Multiple strategies to enhance automatic 3D facial expression recognition , 2015, Neurocomputing.

[17]  Thomas S. Huang,et al.  3D facial expression recognition based on properties of line segments connecting facial feature points , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[18]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[19]  Chun Chen,et al.  Feature level analysis for 3D facial expression recognition , 2011, Neurocomputing.

[20]  Liming Chen,et al.  Author manuscript, published in "Workshop 3D Face Biometrics, IEEE Automatic Facial and Gesture Recognition, Shanghai: China (2013)" Fully Automatic 3D Facial Expression Recognition using Differential Mean Curvature Maps and Histograms of Oriented Gradien , 2013 .

[21]  Wei Zeng,et al.  An automatic 3D expression recognition framework based on sparse representation of conformal images , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[22]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[23]  Yangyang Xu,et al.  Alternating proximal gradient method for sparse nonnegative Tucker decomposition , 2013, Mathematical Programming Computation.

[24]  Alberto Del Bimbo,et al.  3D facial expression recognition using SIFT descriptors of automatically detected keypoints , 2011, The Visual Computer.

[25]  Michael G. Strintzis,et al.  Bilinear elastically deformable models with application to 3D face and facial expression recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[26]  Jie Sun,et al.  An alternating direction method for solving convex nonlinear semidefinite programming problems , 2013 .

[27]  Jian Sun,et al.  Multimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network , 2017, IEEE Transactions on Multimedia.

[28]  Jian Yang,et al.  A Two-Phase Test Sample Sparse Representation Method for Use With Face Recognition , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Sergio Escalera,et al.  Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Michael G. Strintzis,et al.  Bilinear Models for 3-D Face and Facial Expression Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[31]  Liming Chen,et al.  3D facial expression recognition via multiple kernel learning of Multi-Scale Local Normal Patterns , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[32]  Liming Chen,et al.  Automatic 3D facial expression recognition using geometric scattering representation , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[33]  Emmanuel Dellandréa,et al.  A 3D Statistical Facial Feature Model and Its Application on Locating Facial Landmarks , 2009, ACIVS.