Dual-graph regularized discriminative transfer sparse coding for facial expression recognition

Abstract Facial expression recognition has recently received an increasing attention due to its great potentiality in real world applications. Conventional facial expression recognition is often conducted on the assumption that training data and testing data are obtained from the same dataset. However, in reality, the data are often collected from different devices or environments, which will severely degrade the recognition performance. To tackle this problem, in this paper, we investigate the cross-dataset facial expression recognition problem, and propose a novel dual-graph regularized transfer sparse coding method (DGTSC). Specifically, aiming to reduce the distribution divergence of different databases while preserving the geometrical structures, we construct a dual-graph, by defining the inter-domain and intra-domain similarity, to measure the distance between different databases. Moreover, we further present a dual-graph regularized discriminative transfer sparse coding method (DGDTSC), which exploits the label information, to make our model has more discriminative power. Extensive experimental results and analysis on several facial expression datasets show the feasibility and effectiveness of the proposed methods.

[1]  Randy Goebel,et al.  Facial expression recognition using SVM classification on mic-macro patterns , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[2]  Haibin Yan,et al.  Transfer subspace learning for cross-dataset facial expression recognition , 2016, Neurocomputing.

[3]  Tong Zhang,et al.  A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition , 2016, IEEE Transactions on Multimedia.

[4]  Thomas S. Huang,et al.  Multi-view Facial Expression Recognition Analysis with Generic Sparse Coding Feature , 2012, ECCV Workshops.

[5]  Fernando De la Torre,et al.  Selective Transfer Machine for Personalized Facial Expression Analysis , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ling Shao,et al.  Transfer Learning for Visual Categorization: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Dian Tjondronegoro,et al.  Facial Expression Recognition Using Facial Movement Features , 2011, IEEE Transactions on Affective Computing.

[8]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Yongzhao Zhan,et al.  A neural-AdaBoost based facial expression recognition system , 2014, Expert Syst. Appl..

[10]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[11]  Tao Liu,et al.  Supervised Sparse Coding With Decision Forest , 2019, IEEE Signal Processing Letters.

[12]  Qingshan Liu,et al.  Learning Multiscale Active Facial Patches for Expression Analysis , 2015, IEEE Transactions on Cybernetics.

[13]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[14]  Anima Majumder,et al.  Automatic Facial Expression Recognition System Using Deep Network-Based Data Fusion , 2018, IEEE Transactions on Cybernetics.

[15]  David Zhang,et al.  Guide Subspace Learning for Unsupervised Domain Adaptation , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Alice Caplier,et al.  Spontaneous Facial Expression Recognition using Sparse Representation , 2017, VISIGRAPP 2017.

[17]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[18]  Jean Meunier,et al.  Emotion recognition using dynamic grid-based HoG features , 2011, Face and Gesture 2011.

[19]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[20]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[21]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Cheng Wu,et al.  Domain Invariant and Class Discriminative Feature Learning for Visual Domain Adaptation , 2018, IEEE Transactions on Image Processing.

[23]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[24]  Xiaoyan Zhou,et al.  Cross-Domain Color Facial Expression Recognition Using Transductive Transfer Subspace Learning , 2018, IEEE Transactions on Affective Computing.

[25]  Hui Xiong,et al.  A Unified Framework for Metric Transfer Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[26]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[27]  Philip S. Yu,et al.  Visual Domain Adaptation with Manifold Embedded Distribution Alignment , 2018, ACM Multimedia.

[28]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[29]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[30]  Jianmin Wang,et al.  Unsupervised Domain Adaptation With Distribution Matching Machines , 2018, AAAI.

[31]  Zhifeng Hao,et al.  A Selective Multiple Instance Transfer Learning Method for Text Categorization Problems , 2018, Knowl. Based Syst..

[32]  Shan Li,et al.  Deep Facial Expression Recognition: A Survey , 2018, IEEE Transactions on Affective Computing.

[33]  Jian Sun,et al.  Product Sparse Coding , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Y. V. Venkatesh,et al.  Facial expression recognition using radial encoding of local Gabor features and classifier synthesis , 2012, Pattern Recognit..

[35]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[36]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[37]  Philip S. Yu,et al.  Transfer Joint Matching for Unsupervised Domain Adaptation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  I. Jolliffe Principal Component Analysis , 2002 .

[40]  R. Fletcher Practical Methods of Optimization , 1988 .

[41]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[43]  Ming Shao,et al.  Missing Modality Transfer Learning via Latent Low-Rank Constraint , 2015, IEEE Transactions on Image Processing.

[44]  Jun Huan,et al.  Knowledge Transfer with Low-Quality Data: A Feature Extraction Issue , 2012, IEEE Trans. Knowl. Data Eng..

[45]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  H. Bischof,et al.  Multi-view Facial Expressions Recognition using Local Linear Regression of Sparse Codes , 2015 .

[47]  Dacheng Tao,et al.  Bregman Divergence-Based Regularization for Transfer Subspace Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[48]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[49]  Wenming Zheng,et al.  Feature Selection Based Transfer Subspace Learning for Speech Emotion Recognition , 2020, IEEE Transactions on Affective Computing.

[50]  peng song,et al.  Transfer Linear Subspace Learning for Cross-Corpus Speech Emotion Recognition , 2019, IEEE Transactions on Affective Computing.

[51]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[52]  Aggelos K. Katsaggelos,et al.  Automatic facial expression recognition using facial animation parameters and multistream HMMs , 2006, IEEE Transactions on Information Forensics and Security.

[53]  Philip S. Yu,et al.  Transfer Sparse Coding for Robust Image Representation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[55]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[56]  Shang-Hong Lai,et al.  A Compact Deep Learning Model for Robust Facial Expression Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[57]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[58]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[59]  Aurobinda Routray,et al.  Automatic facial expression recognition using features of salient facial patches , 2015, IEEE Transactions on Affective Computing.

[60]  Jian Yang,et al.  Fisher discrimination dictionary pair learning for image classification , 2017, Neurocomputing.

[61]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[62]  Liang-Tien Chia,et al.  Local features are not lonely – Laplacian sparse coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[63]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[65]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[66]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.