Interactive Facial Feature Localization

We address the problem of interactive facial feature localization from a single image. Our goal is to obtain an accurate segmentation of facial features on high-resolution images under a variety of pose, expression, and lighting conditions. Although there has been significant work in facial feature localization, we are addressing a new application area, namely to facilitate intelligent high-quality editing of portraits, that brings requirements not met by existing methods. We propose an improvement to the Active Shape Model that allows for greater independence among the facial components and improves on the appearance fitting step by introducing a Viterbi optimization process that operates along the facial contours. Despite the improvements, we do not expect perfect results in all cases. We therefore introduce an interaction model whereby a user can efficiently guide the algorithm towards a precise solution. We introduce the Helen Facial Feature Dataset consisting of annotated portrait images gathered from Flickr that are more diverse and challenging than currently existing datasets. We present experiments that compare our automatic method to published results, and also a quantitative evaluation of the effectiveness of our interactive method.

[1]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[2]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[3]  Shaogang Gong,et al.  A Multi-View Nonlinear Active Shape Model Using Kernel PCA , 1999, BMVC.

[4]  Timothy F. Cootes,et al.  A mixture model for representing shape variation , 1999, Image Vis. Comput..

[5]  Klaus J. Kirchberg,et al.  Robust Face Detection Using the Hausdorff Distance , 2001, AVBPA.

[6]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Mads Nielsen,et al.  Computer Vision — ECCV 2002 , 2002, Lecture Notes in Computer Science.

[8]  Alejandro F. Frangi,et al.  Active shape model segmentation with optimal features , 2002, IEEE Transactions on Medical Imaging.

[9]  Jim Graham,et al.  Robust Active Shape Model Search , 2002, ECCV.

[10]  Yi Zhou,et al.  Bayesian tangent shape model: estimating shape and pose parameters via Bayesian inference , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[11]  Daniel P. Huttenlocher,et al.  Pictorial Structures for Object Recognition , 2004, International Journal of Computer Vision.

[12]  Josef Kittler,et al.  Audio- and Video-Based Biometric Person Authentication, 5th International Conference, AVBPA 2005, Hilton Rye Town, NY, USA, July 20-22, 2005, Proceedings , 2005, AVBPA.

[13]  Qingshan Liu,et al.  A Component Based Deformable Model for Generalized Face Alignment , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[14]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[15]  Takeo Kanade,et al.  A Generative Shape Regularization Model for Robust Face Alignment , 2008, ECCV.

[16]  Jian Sun,et al.  Face Alignment Via Component-Based Discriminative Search , 2008, ECCV.

[17]  Fred Nicolls,et al.  Locating Facial Features with an Extended Active Shape Model , 2008, ECCV.

[18]  Adam Schmidt,et al.  The put face database , 2008 .

[19]  Timothy F. Cootes,et al.  Combining Local and Global Shape Models for Deformable Object Matching , 2009, BMVC.

[20]  M. S. Ryan,et al.  The Viterbi Algorithm 1 1 The Viterbi Algorithm . , 2009 .

[21]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[22]  Stephen Milborrow.  The MUCT Landmarked Face Database , 2010 .

[23]  David J. Kriegman,et al.  Localizing parts of faces using a consensus of exemplars , 2011, CVPR.

[24]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.