Identifying universal facial emotion markers for automatic 3D facial expression recognition

Facial expressions convey human emotions as a simple and effective non-verbal communication method. Motivated by this special characteristic, facial expression recognition rapidly gains attention in social computing fields, especially in Human Computer Interaction (HCI). Identifying the optimal set of facial emotion markers is an important technique that not only reduces the feature vector dimensionality, but also impacts the recognition accuracy. In this paper, we propose a new emotion marker identification algorithm for automatic and person-independent 3D facial expression recognition system. First, we mapped the 3D face images into the 2D plane via conformal geometry to reduce the dimensionality. Then, the identification algorithm is designed to seek the best discriminative markers and the classifier parameters simultaneously by integrating three techniques viz., Differential Evolution (DE), Support Vector Machine (SVM) and Speed Up Robust Feature (SURF). The proposed system yielded an average recognition rate of 79% and outperformed the previous studies using the Bosphorus database.

[1]  Alberto Del Bimbo,et al.  3D facial expression recognition using SIFT descriptors of automatically detected keypoints , 2011, The Visual Computer.

[2]  Václav Hlavác,et al.  Detector of Facial Landmarks Learned by the Structured Output SVM , 2012, VISAPP.

[3]  P. Ekman,et al.  Measuring facial movement , 1976 .

[4]  Thomas S. Huang,et al.  3D facial expression recognition based on automatically selected features , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[5]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[6]  Lijun Yin,et al.  Evaluation of 3D Facial Feature Selection for Individual Facial Model Identification , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[7]  Maja Pantic,et al.  Meta-Analysis of the First Facial Expression Recognition Challenge , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Sang-Heon Lee,et al.  Facial Expression Recognition Using Extended Local Binary Patterns of 3D Curvature , 2013, MUE.

[9]  Liming Chen,et al.  Author manuscript, published in "Workshop 3D Face Biometrics, IEEE Automatic Facial and Gesture Recognition, Shanghai: China (2013)" Fully Automatic 3D Facial Expression Recognition using Differential Mean Curvature Maps and Histograms of Oriented Gradien , 2013 .

[10]  Qiuqi Ruan,et al.  3D facial expression recognition based on variation faces , 2012, 2012 IEEE 11th International Conference on Signal Processing.

[11]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[12]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..

[13]  Peter Schröder,et al.  Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on Geometry processing , 2003 .

[14]  Shing-Tung Yau,et al.  Computational Conformal Geometry , 2016 .

[15]  Ioannis Pitas,et al.  3D facial expression recognition using Zernike moments on depth images , 2011, 2011 18th IEEE International Conference on Image Processing.

[16]  SK Mishra,et al.  Global Optimization by Differential Evolution and Particle Swarm Methods: Evaluation on Some Benchmark Functions , 2006 .

[17]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[18]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[19]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Hasan Demirel,et al.  Feature selection for person-independent 3D facial expression recognition using NSGA-II , 2009, 2009 24th International Symposium on Computer and Information Sciences.

[21]  Lijun Yin,et al.  Automatic Registration of Vertex Correspondences for 3D Facial Expression Analysis , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[22]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[23]  Lijun Yin,et al.  Static and dynamic 3D facial expression recognition: A comprehensive survey , 2012, Image Vis. Comput..

[24]  T. Chan,et al.  Genus zero surface conformal mapping and its application to brain surface mapping. , 2004, IEEE transactions on medical imaging.

[25]  S. Yau,et al.  Global conformal surface parameterization , 2003 .

[26]  Hasan Demirel,et al.  Application of NSGA-II to feature selection for facial expression recognition , 2011, Comput. Electr. Eng..

[27]  Sen Wang,et al.  Conformal Geometry and Its Applications on 3D Shape Matching, Recognition, and Stitching , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[29]  Tony DeRose,et al.  Multiresolution analysis of arbitrary meshes , 1995, SIGGRAPH.

[30]  Yiding Wang,et al.  Learning Encoded Facial Curvature Information for 3D Facial Emotion Recognition , 2013, 2013 Seventh International Conference on Image and Graphics.

[31]  Dimitris Samaras,et al.  Conformal mapping-based 3D face recognition , 2010 .