Transfer subspace learning for cross-dataset facial expression recognition

In this paper, we propose a transfer subspace learning approach cross-dataset facial expression recognition. To our best knowledge, this problem has been seldom addressed in the literature. While many facial expression recognition methods have been proposed in recent years, most of them assume that face images in the training and testing sets are collected under the same conditions so that they are independently and identically distributed. In many real applications, this assumption does not hold as the testing data are usually collected online and are generally more uncontrollable than the training data. Hence, the testing samples are likely different from the training samples. In this paper, we define this problem as cross-dataset facial expression recognition as the training and testing data are considered to be collected from different datasets due to different acquisition conditions. To address this, we propose a transfer subspace learning approach to learn a feature subspace which transfers the knowledge gained from the source domain (training samples) to the target domain (testing samples) to improve the recognition performance. To better exploit more complementary information for multiple feature representations of face images, we develop a multi-view transfer subspace learning approach where multiple different yet related subspaces are learned to transfer information from the source domain to the target domain. Experimental results are presented to demonstrate the efficacy of these proposed methods for the cross-dataset facial expression recognition task.

[1]  Nicu Sebe,et al.  Egocentric Daily Activity Recognition via Multitask Clustering , 2015, IEEE Transactions on Image Processing.

[2]  Jiwen Lu,et al.  Regularized Locality Preserving Projections and Its Extensions for Face Recognition , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Matti Pietikäinen,et al.  Boosted multi-resolution spatiotemporal descriptors for facial expression recognition , 2009, Pattern Recognit. Lett..

[4]  Jiwen Lu,et al.  Gait recognition for human identification based on ICA and fuzzy SVM through multiple views fusion , 2007, Pattern Recognit. Lett..

[5]  Marcelo H. Ang,et al.  Cross-dataset facial expression recognition , 2011, 2011 IEEE International Conference on Robotics and Automation.

[6]  Ioannis Pitas,et al.  Discriminant Graph Structures for Facial Expression Recognition , 2008, IEEE Transactions on Multimedia.

[7]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Gang Wang,et al.  Human Identity and Gender Recognition From Gait Sequences With Arbitrary Walking Directions , 2014, IEEE Transactions on Information Forensics and Security.

[9]  M. Pietikäinen,et al.  Facial expression recognition based on local binary patterns , 2007, Pattern Recognition and Image Analysis.

[10]  Subramanian Ramanathan,et al.  No Matter Where You Are: Flexible Graph-Guided Multi-task Learning for Multi-view Head Pose Classification under Target Motion , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[12]  Daijin Kim,et al.  Natural facial expression recognition using differential-AAM and manifold learning , 2009, Pattern Recognit..

[13]  Shaogang Gong,et al.  Appearance Manifold of Facial Expression , 2005, ICCV-HCI.

[14]  Gang Wang,et al.  Image-to-Set Face Recognition Using Locality Repulsion Projections and Sparse Reconstruction-Based Similarity Measure , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Jiwen Lu,et al.  Kinship verification from facial images under uncontrolled conditions , 2011, ACM Multimedia.

[16]  Gang Wang,et al.  Discriminative multi-manifold analysis for face recognition from a single training sample per person , 2011, 2011 International Conference on Computer Vision.

[17]  PietikainenMatti,et al.  Face Description with Local Binary Patterns , 2006 .

[18]  Nicu Sebe,et al.  Event Oriented Dictionary Learning for Complex Event Detection , 2015, IEEE Transactions on Image Processing.

[19]  Jiwen Lu,et al.  Discriminative Multimetric Learning for Kinship Verification , 2014, IEEE Transactions on Information Forensics and Security.

[20]  Bogdan Raducanu,et al.  Dynamic facial expression recognition using Laplacian Eigenmaps-based manifold learning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[21]  Shaogang Gong,et al.  A Comprehensive Empirical Study on Linear Subspace Methods for Facial Expression Analysis , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[22]  Dacheng Tao,et al.  Bregman Divergence-Based Regularization for Transfer Subspace Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[23]  Jiwen Lu,et al.  Nearest Feature Space Analysis for Classification , 2011, IEEE Signal Processing Letters.

[24]  Jiwen Lu,et al.  Cost-Sensitive Local Binary Feature Learning for Facial Age Estimation , 2015, IEEE Transactions on Image Processing.

[25]  PanticMaja,et al.  A Survey of Affect Recognition Methods , 2009 .

[26]  Yan Yan,et al.  $L_{1}$ -Norm Low-Rank Matrix Factorization by Variational Bayesian Method , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Nicu Sebe,et al.  GLocal tells you more: Coupling GLocal structural for feature selection with sparsity for image and video classification , 2014, Comput. Vis. Image Underst..

[28]  Subramanian Ramanathan,et al.  Multitask Linear Discriminant Analysis for View Invariant Action Recognition , 2014, IEEE Transactions on Image Processing.

[29]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Jiwen Lu,et al.  Cost-Sensitive Subspace Analysis and Extensions for Face Recognition , 2013, IEEE Transactions on Information Forensics and Security.

[31]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[32]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[33]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[34]  Jiwen Lu,et al.  Ordinary Preserving Manifold Analysis for Human Age and Head Pose Estimation , 2013, IEEE Transactions on Human-Machine Systems.

[35]  Jiwen Lu,et al.  Learning Compact Binary Face Descriptor for Face Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Maja Pantic Automatic Analysis of Facial Expressions , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[38]  Jiwen Lu,et al.  Prototype-Based Discriminative Feature Learning for Kinship Verification , 2015, IEEE Transactions on Cybernetics.

[39]  Qiuqi Ruan,et al.  Discriminant spectral analysis for facial expression recognition , 2008, 2008 15th IEEE International Conference on Image Processing.

[40]  Yun Fu,et al.  Multiple feature fusion by subspace learning , 2008, CIVR '08.

[41]  Jiwen Lu,et al.  Neighborhood repulsed metric learning for kinship verification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Gang Wang,et al.  Localized Multifeature Metric Learning for Image-Set-Based Face Recognition , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[44]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[45]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[46]  Gang Wang,et al.  Joint Feature Learning for Face Recognition , 2015, IEEE Transactions on Information Forensics and Security.

[47]  Gang Wang,et al.  Reconstruction-Based Metric Learning for Unconstrained Face Verification , 2015, IEEE Transactions on Information Forensics and Security.

[48]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[49]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Nicu Sebe,et al.  Multi-task linear discriminant analysis for multi-view action recognition , 2013, 2013 IEEE International Conference on Image Processing.

[51]  Jiwen Lu,et al.  A Doubly Weighted Approach for Appearance-Based Subspace Learning Methods , 2010, IEEE Transactions on Information Forensics and Security.

[52]  Qi Tian,et al.  Cross-database age estimation based on transfer learning , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[53]  Jiwen Lu,et al.  Cost-Sensitive Semi-Supervised Discriminant Analysis for Face Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[54]  Qijun Zhao,et al.  Facial expression recognition on multiple manifolds , 2011, Pattern Recognit..

[55]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[56]  W. Zheng,et al.  Facial expression recognition using kernel canonical correlation analysis (KCCA) , 2006, IEEE Transactions on Neural Networks.

[57]  Michel Valstar,et al.  Automatic Facial Expression Analysis , 2015 .

[58]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Jiwen Lu,et al.  Uncorrelated Discriminant Nearest Feature Line Analysis for Face Recognition , 2010, IEEE Signal Processing Letters.

[60]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[61]  Yousef Saad,et al.  Orthogonal neighborhood preserving projections , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[62]  Siu Cheung Hui,et al.  Facial expression recognition from line-based caricatures , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[63]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[64]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.