Sparse and Low-rank Tucker Decomposition with Its Application to 2D+3D Facial Expression Recognition

In this paper, a novel sparse and low-rank Tucker decomposition approach is proposed and is applied into 2D+3D facial expression recognition (FER). Firstly, a 4D tensor model is bulit to explore the efficient structural and complemental information between 2D and 3D data. Secondly, the information is missed partly in the process of the 4D tensor modeling, and high simirarities among samples are then generated, in which the low-rankness of the generated 4D tensor is used to characterize the involved sample simirarities. At the same time, a tensor completion is then embedded to recover the missed information. Thirdly, under orthogonal Tucker decomposition of the generated 4D tensor, the tensor optimization model is utilized by imposing sparse constraint on the involved core tensor and the invoved factor matrices that are employed finally for classification prediction. Finally, the alternating direction method of multipliers is adopted to solve the proposed optimization problem effectively. Numerical experiments verify the effectiveness of the proposed approach on the BU-3DFE database.

[1]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[2]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[4]  Jun Wang,et al.  3D Facial Expression Recognition Based on Primitive Surface Feature Distribution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Hasan Demirel,et al.  Facial Expression Recognition Using 3D Facial Feature Distances , 2007, ICIAR.

[6]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[7]  Thomas S. Huang,et al.  3D facial expression recognition based on properties of line segments connecting facial feature points , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[8]  Xiaoou Tang,et al.  Automatic facial expression recognition on a single 3D face by exploring shape deformation , 2009, ACM Multimedia.

[9]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[10]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[11]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[12]  David P. Wipf,et al.  Iterative Reweighted 1 and 2 Methods for Finding Sparse Solutions , 2010, IEEE J. Sel. Top. Signal Process..

[13]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[14]  Sotiris Malassiotis,et al.  Real-time 2D+3D facial action and expression recognition , 2010, Pattern Recognit..

[15]  Emmanuel Dellandréa,et al.  Automatic 3D Facial Expression Recognition Based on a Bayesian Belief Net and a Statistical Facial Feature Model , 2010, 2010 20th International Conference on Pattern Recognition.

[16]  Arman Savran,et al.  Facial action unit detection: 3D versus 2D modality , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[17]  Jian Yang,et al.  A Two-Phase Test Sample Sparse Representation Method for Use With Face Recognition , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Alberto Del Bimbo,et al.  3D facial expression recognition using SIFT descriptors of automatically detected keypoints , 2011, The Visual Computer.

[19]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[20]  Chun Chen,et al.  Feature level analysis for 3D facial expression recognition , 2011, Neurocomputing.

[21]  Qiuqi Ruan,et al.  A remarkable standard for estimating the performance of 3D facial expression features , 2012, Neurocomputing.

[22]  Genevera I. Allen,et al.  Sparse Higher-Order Principal Components Analysis , 2012, AISTATS.

[23]  Liming Chen,et al.  Author manuscript, published in "Workshop 3D Face Biometrics, IEEE Automatic Facial and Gesture Recognition, Shanghai: China (2013)" Fully Automatic 3D Facial Expression Recognition using Differential Mean Curvature Maps and Histograms of Oriented Gradien , 2013 .

[24]  Wei Zeng,et al.  An automatic 3D expression recognition framework based on sparse representation of conformal images , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[25]  Qiuqi Ruan,et al.  Analysis of range images used in 3D facial expression recognition , 2013, 2013 IEEE International Conference of IEEE Region 10 (TENCON 2013).

[26]  Jun Fang,et al.  Exact Reconstruction Analysis of Log-Sum Minimization for Compressed Sensing , 2013, IEEE Signal Processing Letters.

[27]  Michael Spann,et al.  Surface Normals with Modular Approach and Weighted Voting Scheme in 3D Facial Expression Classification , 2014 .

[28]  Mohammed Bennamoun,et al.  An Automatic Framework for Textured 3D Video-Based Facial Expression Recognition , 2014, IEEE Transactions on Affective Computing.

[29]  Hasan Demirel,et al.  Entropy-based feature selection for improved 3D facial expression recognition , 2013, Signal, Image and Video Processing.

[30]  Hasan Demirel,et al.  Feature selection for improved 3D facial expression recognition , 2014, Pattern Recognit. Lett..

[31]  Hong Cheng,et al.  Sparse Representation, Modeling and Learning in Visual Recognition , 2015, Advances in Computer Vision and Pattern Recognition.

[32]  Liming Chen,et al.  Automatic 3D facial expression recognition using geometric scattering representation , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[33]  Qiuqi Ruan,et al.  Multiple strategies to enhance automatic 3D facial expression recognition , 2015, Neurocomputing.

[34]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[35]  Qiuqi Ruan,et al.  Fast nonnegative tensor factorization based on graph-preserving for 3D facial expression recognition , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[36]  Fernando De la Torre,et al.  Selective Transfer Machine for Personalized Facial Expression Analysis , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Jian Sun,et al.  Multimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network , 2017, IEEE Transactions on Multimedia.

[38]  Qiuqi Ruan,et al.  Sparse Orthogonal Tucker Decomposition for 2D+3D Facial Expression Recognition , 2018, 2018 14th IEEE International Conference on Signal Processing (ICSP).

[39]  Yunhong Wang,et al.  Texture and Geometry Scattering Representation-Based Facial Expression Recognition in 2D+3D Videos , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[40]  Qiuqi Ruan,et al.  FERLrTc: 2D+3D facial expression recognition via low-rank tensor completion , 2019, Signal Process..

[41]  Qiuqi Ruan,et al.  Orthogonal Low Rank Tucker Decomposition for 2D+3D Facial Expression Recognition , 2019, FSDM.