A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition

This paper proposes a facial expression recognition system using evolutionary particle swarm optimization (PSO)-based feature optimization. The system first employs modified local binary patterns, which conduct horizontal and vertical neighborhood pixel comparison, to generate a discriminative initial facial representation. Then, a PSO variant embedded with the concept of a micro genetic algorithm (mGA), called mGA-embedded PSO, is proposed to perform feature optimization. It incorporates a nonreplaceable memory, a small-population secondary swarm, a new velocity updating strategy, a subdimension-based in-depth local facial feature search, and a cooperation of local exploitation and global exploration search mechanism to mitigate the premature convergence problem of conventional PSO. Multiple classifiers are used for recognizing seven facial expressions. Based on a comprehensive study using within- and cross-domain images from the extended Cohn Kanade and MMI benchmark databases, respectively, the empirical results indicate that our proposed system outperforms other state-of-the-art PSO variants, conventional PSO, classical GA, and other related facial expression recognition models reported in the literature by a significant margin.

[1]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[2]  Tardi Tjahjadi,et al.  A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences , 2015, Pattern Recognit..

[3]  Renato A. Krohling,et al.  Bare Bones Particle Swarm Optimization With Scale Matrix Adaptation , 2014, IEEE Transactions on Cybernetics.

[4]  Hui Wang,et al.  Gaussian Bare-Bones Differential Evolution , 2013, IEEE Transactions on Cybernetics.

[5]  Ahmad Bagheri,et al.  HEPSO: High exploration particle swarm optimization , 2014, Inf. Sci..

[6]  Min Chen,et al.  Facial expression recognition in dynamic sequences: An integrated approach , 2014, Pattern Recognit..

[7]  Wei-Der Chang,et al.  A modified particle swarm optimization with multiple subpopulations for multimodal function optimization problems , 2015, Appl. Soft Comput..

[8]  Shu Liao,et al.  Dominant Local Binary Patterns for Texture Classification , 2009, IEEE Transactions on Image Processing.

[9]  Fei Chao,et al.  Feature Selection Inspired Classifier Ensemble Reduction , 2014, IEEE Transactions on Cybernetics.

[10]  Ananth Ramaswamy,et al.  Optimal fuzzy logic control for MDOF structural systems using evolutionary algorithms , 2009, Eng. Appl. Artif. Intell..

[11]  Wei-Yun Yau,et al.  A novel phase congruency based descriptor for dynamic facial expression analysis , 2014, Pattern Recognit. Lett..

[12]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[13]  Kamlesh Mistry,et al.  Intelligent facial emotion recognition using a layered encoding cascade optimization model , 2015, Appl. Soft Comput..

[14]  Yuxiao Hu,et al.  Audio-Visual Spontaneous Emotion Recognition , 2007, Artifical Intelligence for Human Computing.

[15]  Li-Yeh Chuang,et al.  Chaotic maps based on binary particle swarm optimization for feature selection , 2011, Appl. Soft Comput..

[16]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[17]  Kamlesh Mistry,et al.  Intelligent affect regression for bodily expressions using hybrid particle swarm optimization and adaptive ensembles , 2015, Expert Syst. Appl..

[18]  Yun Fu,et al.  Hierarchical facial expression animation by motion capture data , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[19]  Hongying Meng,et al.  Affective State Level Recognition in Naturalistic Facial and Vocal Expressions , 2014, IEEE Transactions on Cybernetics.

[20]  Maja Pantic,et al.  Web-based database for facial expression analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[21]  Dun-Wei Gong,et al.  Feature selection algorithm based on bare bones particle swarm optimization , 2015, Neurocomputing.

[22]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[23]  P. Ekman,et al.  Facial action coding system , 2019 .

[24]  Carlos A. Coello Coello,et al.  A Micro-Genetic Algorithm for Multiobjective Optimization , 2001, EMO.

[25]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[26]  A. Rezaee Jordehi,et al.  Enhanced leader PSO (ELPSO): A new PSO variant for solving global optimisation problems , 2015, Appl. Soft Comput..

[27]  Marko Heikkilä,et al.  Description of interest regions with local binary patterns , 2009, Pattern Recognit..

[28]  Yuxiao Hu,et al.  One-class classification for spontaneous facial expression analysis , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[29]  Qingshan Liu,et al.  Learning Multiscale Active Facial Patches for Expression Analysis , 2015, IEEE Transactions on Cybernetics.

[30]  Li Zhang,et al.  Affect Sensing Using Linguistic, Semantic and Cognitive Cues in Multi-threaded Improvisational Dialogue , 2012, Cognitive Computation.

[31]  Shiguang Shan,et al.  AU-inspired Deep Networks for Facial Expression Feature Learning , 2015, Neurocomputing.

[32]  Luiz Eduardo Soares de Oliveira,et al.  Fusion of feature sets and classifiers for facial expression recognition , 2013, Expert Syst. Appl..

[33]  C. A. Coello Coello,et al.  Multiobjective structural optimization using a microgenetic algorithm , 2005 .

[34]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[35]  S. Ramachandran,et al.  Face recognition using transform domain feature extraction and PSO-based feature selection , 2014, Appl. Soft Comput..

[36]  Aurobinda Routray,et al.  Automatic facial expression recognition using features of salient facial patches , 2015, IEEE Transactions on Affective Computing.

[37]  Maja Pantic,et al.  Discriminative Shared Gaussian Processes for Multiview and View-Invariant Facial Expression Recognition , 2015, IEEE Transactions on Image Processing.

[38]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[39]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[40]  Qijun Zhao,et al.  Facial expression recognition on multiple manifolds , 2011, Pattern Recognit..

[41]  Kamlesh Mistry,et al.  Adaptive facial point detection and emotion recognition for a humanoid robot , 2015, Comput. Vis. Image Underst..

[42]  Baochang Zhang,et al.  Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor , 2010, IEEE Transactions on Image Processing.

[43]  Mohammad Soleymani,et al.  Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection , 2016, IEEE Transactions on Affective Computing.

[44]  Kalmanje Krishnakumar,et al.  Micro-Genetic Algorithms For Stationary And Non-Stationary Function Optimization , 1990, Other Conferences.

[45]  Chen-Chien James Hsu,et al.  Hybrid particle swarm optimization incorporating fuzzy reasoning and weighted particle , 2015, Neurocomputing.

[46]  Li Zhang,et al.  Adaptive 3D facial action intensity estimation and emotion recognition , 2015, Expert Syst. Appl..