A Novel Two-Stage Illumination Estimation Framework for Expression Recognition

One of the critical issues for facial expression recognition is to eliminate the negative effect caused by variant poses and illuminations. In this paper a two-stage illumination estimation framework is proposed based on three-dimensional representative face and clustering, which can estimate illumination directions under a series of poses. First, 256 training 3D face models are adaptively categorized into a certain amount of facial structure types by k-means clustering to group people with similar facial appearance into clusters. Then the representative face of each cluster is generated to represent the facial appearance type of that cluster. Our training set is obtained by rotating all representative faces to a certain pose, illuminating them with a series of different illumination conditions, and then projecting them into two-dimensional images. Finally the saltire-over-cross feature is selected to train a group of SVM classifiers and satisfactory performance is achieved when estimating a number of test sets including images generated from 64 3D face models kept for testing, CAS-PEAL face database, CMU PIE database, and a small test set created by ourselves. Compared with other related works, our method is subject independent and has less computational complexity O(C × N) without 3D facial reconstruction.

[1]  Mariette Yvinec,et al.  Conforming Delaunay triangulations in 3D , 2002, SCG '02.

[2]  Xu-Dong Li.  Morphable Linear Fitting Method for Facial Expression Synthesis: Morphable Linear Fitting Method for Facial Expression Synthesis , 2008 .

[3]  Rama Chellappa,et al.  Illuminating light field: image-based face recognition across illuminations and poses , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[4]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[5]  Baining Guo,et al.  Geometry-driven photorealistic facial expression synthesis , 2003, IEEE Transactions on Visualization and Computer Graphics.

[6]  Ralph Gross,et al.  Eigen light-fields and face recognition across pose , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[7]  Stefanos Zafeiriou,et al.  Recognition of 3D facial expression dynamics , 2012, Image Vis. Comput..

[8]  Mohan M. Trivedi,et al.  A two-stage head pose estimation framework and evaluation , 2008, Pattern Recognit..

[9]  Hao Zhang,et al.  Delaunay mesh construction , 2007, Symposium on Geometry Processing.

[10]  Wang Li.  Survey on human eyes detection in images , 2008 .

[11]  Sethuraman Panchanathan,et al.  Biased Manifold Embedding: A Framework for Person-Independent Head Pose Estimation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Yen-Wei Chen,et al.  Pose-Robust Face Recognition Based on 3D Shape Reconstruction , 2009, 2009 Fifth International Conference on Natural Computation.

[13]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[14]  Yun Fu,et al.  Graph embedded analysis for head pose estimation , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[15]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Zheng Zhang,et al.  Expression Recognition Based on Multi-scale Block Local Gabor Binary Patterns with Dichotomy-Dependent Weights , 2009, ISNN.

[17]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[18]  Li Zhao,et al.  An Enhanced LBP Feature Based on Facial Expression Recognition , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[19]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[20]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[21]  Ioan Nafornita,et al.  Facial Expression Synthesis and Animation , 2011 .

[22]  John Shawe-Taylor,et al.  Structural Risk Minimization Over Data-Dependent Hierarchies , 1998, IEEE Trans. Inf. Theory.

[23]  John Shawe-Taylor,et al.  A framework for structural risk minimisation , 1996, COLT '96.

[24]  Lijun Yin,et al.  Static and dynamic 3D facial expression recognition: A comprehensive survey , 2012, Image Vis. Comput..

[25]  Wen Gao,et al.  Efficient 3D reconstruction for face recognition , 2005, Pattern Recognit..

[26]  Dai Jing.  Rapid Eye Localization Based on Projection Peak , 2009 .

[27]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[29]  Marc Levoy,et al.  Fitting smooth surfaces to dense polygon meshes , 1996, SIGGRAPH.

[30]  Zhang Zhuang.  Nonuniform Resampling Based on Method for Pixel-wise Correspondence Between 3D Faces , 2007 .

[31]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[32]  Kai-Tai Song,et al.  Facial expression recognition under illumination variation , 2007, 2007 IEEE Workshop on Advanced Robotics and Its Social Impacts.

[33]  Gao Wen,et al.  Pose and Illumination Invariant Face Recognition Based on 3D Face Reconstruction , 2006 .

[34]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Ruigang Yang,et al.  Estimating pose and illumination direction for frontal face synthesis , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[36]  Xiaodong Li,et al.  Face recognition based on improved PCA reconstruction , 2010, 2010 8th World Congress on Intelligent Control and Automation.

[37]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[38]  Alberto Del Bimbo,et al.  3D Face Reconstruction from Two Orthogonal Images for Face Recognition Applications , 2010, Int. J. Digit. Libr. Syst..

[39]  Jian-Huang Lai,et al.  Face illumination normalization on large and small scale features , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.