Automatic landmarking and alignment for facial expression analysis

In this paper we develop a fully automatic emotional expression recognition system. We propose improvements on a recent facial landmark localization method. This method is then used to bootstrap a Bezie´r volume tracker for tracking facial features in video. The motion of the facial features is used in classifying the expression of the face. We present extensive cross-database results for the landmarking and expression recognition modules, and show that automatic landmarking significantly improves the subsequent expression recognition.

[1]  Fred Nicolls,et al.  Locating Facial Features with an Extended Active Shape Model , 2008, ECCV.

[2]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[3]  Bülent Sankur,et al.  Robust facial landmarking for registration , 2007, Ann. des Télécommunications.

[4]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[5]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[6]  Maja Pantic,et al.  Fully automatic facial feature point detection using Gabor feature based boosted classifiers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[7]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[8]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Albert Ali Salah,et al.  Incremental mixtures of factor analysers , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[10]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[11]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  N. Sebe,et al.  Facial Expression Recognition: A Fully Integrated Approach , 2007, 14th International Conference of Image Analysis and Processing - Workshops (ICIAPW 2007).

[13]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Thomas S. Huang,et al.  Connected vibrations: a modal analysis approach for non-rigid motion tracking , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[16]  Osamu Yamaguchi,et al.  Facial feature localization using weighted vector concentration approach , 2010, Image Vis. Comput..

[17]  Lei Zhang,et al.  3D shape constraint for facial feature localization using probabilistic-like output , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[18]  Saman K. Halgamuge,et al.  Optimised landmark model matching for face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[19]  Paola Campadelli,et al.  A face recognition system based on automatically determined facial fiducial points , 2006, Pattern Recognit..

[20]  David Cristinacce,et al.  Automatic feature localisation with constrained local models , 2008, Pattern Recognit..

[21]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..