Statistical 3D face shape estimation from occluding contours

A novel 3D face estimation method based on a regression matrix and occluding contours.3D vertices around occluding boundaries and their corresponding 2D pixel projections are highly correlated.The 3D face estimation method resembles dense surface shape recovery from missing data. This paper addresses the problem of 3D face shape approximation from occluding contours, i.e., the boundaries between the facial region and the background. To this end, a linear regression process that models the relationship between a set of 2D occluding contours and a set of 3D vertices is applied onto the corresponding training sets using Partial Least Squares. The result of this step is a regression matrix which is capable of estimating new 3D face point clouds from the out-of-training 2D Cartesian pixel positions of the selected contours. Our approach benefits from the highly correlated spaces spanned by the 3D vertices around the occluding boundaries of a face and their corresponding 2D pixel projections. As a result, the proposed method resembles dense surface shape recovery from missing data. Our technique is evaluated over four scenarios designed to investigate both the influence of the contours included in the training set and the considered number of contours. Qualitative and quantitative experiments demonstrate that using contours outperform the state of the art on the database used in this article. Even using a limited number of contours provides a useful approximation to the 3D face surface.

[1]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[2]  Paul A. Beardsley,et al.  High-quality passive facial performance capture using anchor frames , 2011, SIGGRAPH 2011.

[3]  Thomas Vetter,et al.  3D Reconstruction of Human Faces from Occluding Contours , 2007, MIRAGE.

[4]  Alan Brunton,et al.  Multilinear Wavelets: A Statistical Shape Space for Human Faces , 2014, ECCV.

[5]  Robert J. Woodham,et al.  Photometric Stereo: A Reflectance Map Technique For Determining Surface Orientation From Image Intensity , 1979, Optics & Photonics.

[6]  Bernhard Egger,et al.  A Monte Carlo Strategy to Integrate Detection and Model-Based Face Analysis , 2013, GCPR.

[7]  Ira Kemelmacher-Shlizerman,et al.  Face reconstruction in the wild , 2011, 2011 International Conference on Computer Vision.

[8]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[9]  Mario Castelán,et al.  3D face shape approximation from intensities using Partial Least Squares , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Takeo Kanade,et al.  Dense 3D face alignment from 2D videos in real-time , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[11]  Ira Kemelmacher-Shlizerman,et al.  Internet Based Morphable Model , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Xin Tong,et al.  Automatic acquisition of high-fidelity facial performances using monocular videos , 2014, ACM Trans. Graph..

[13]  Roberto Marcondes Cesar Junior,et al.  3D human face reconstruction using principal components spaces , 2011 .

[14]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[15]  William J. Christmas,et al.  Fitting 3D Morphable Face Models using local features , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[16]  Mario Castelán,et al.  Using Subspace Multiple Linear Regression for 3D Face Shape Prediction from a Single Image , 2009, ISVC.

[17]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[18]  Oswald Aldrian,et al.  Inverse Rendering of Faces with a 3D Morphable Model , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Horst Bischof,et al.  3D and Infrared Face Reconstruction from RGB data using Canonical Correlation Analysis , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[20]  Derek Bradley,et al.  High resolution passive facial performance capture , 2010, SIGGRAPH 2010.

[21]  Ira Kemelmacher-Shlizerman,et al.  Molding Face Shapes by Example , 2006, ECCV.

[22]  Hans-Peter Seidel,et al.  Lightweight binocular facial performance capture under uncontrolled lighting , 2012, ACM Trans. Graph..

[23]  Bernhard Egger,et al.  Background modeling for generative image models , 2015, Comput. Vis. Image Underst..

[24]  Thabo Beeler,et al.  High-quality single-shot capture of facial geometry , 2010, SIGGRAPH 2010.

[25]  Ira Kemelmacher-Shlizerman,et al.  Total Moving Face Reconstruction , 2014, ECCV.

[26]  Thomas Vetter,et al.  Probabilistic Modeling and Visualization of the Flexibility in Morphable Models , 2009, IMA Conference on the Mathematics of Surfaces.

[27]  Raghu Machiraju,et al.  Model-based 3D face capture with shape-from-silhouettes , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[28]  Timo Bolkart,et al.  3D faces in motion: Fully automatic registration and statistical analysis , 2015, Comput. Vis. Image Underst..

[29]  Jihun Yu,et al.  Realtime facial animation with on-the-fly correctives , 2013, ACM Trans. Graph..

[30]  S. Wold,et al.  A PLS kernel algorithm for data sets with many variables and fewer objects. Part 1: Theory and algorithm , 1994 .

[31]  Ran He,et al.  Face shape recovery from a single image using CCA mapping between tensor spaces , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Jongmoo Choi,et al.  Accurate 3D face reconstruction from weakly calibrated wide baseline images with profile contours , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Mario Castelán,et al.  3D face shape prediction from a frontal image using cylindrical coordinates and partial least squares , 2013, Pattern Recognit. Lett..

[34]  Christian Theobalt,et al.  Reconstructing detailed dynamic face geometry from monocular video , 2013, ACM Trans. Graph..

[35]  Ferran Marqués,et al.  Region-Based Particle Filter for Video Object Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[37]  Qiuqi Ruan,et al.  3D face reconstruction using a single 2D face image , 2010, 2010 International Conference on Educational and Information Technology.

[38]  Andrew W. Fitzgibbon,et al.  Reconstructing High Quality Face-Surfaces using Model Based Stereo , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[39]  Rama Chellappa,et al.  Contour-based 3D Face Modeling from a Monocular Video , 2004, BMVC.

[40]  Mario Castelán,et al.  Face synthesis from a frontal pose image using partial least squares and b-splines , 2012, 2012 19th IEEE International Conference on Image Processing.

[41]  Alan Brunton,et al.  Review of statistical shape spaces for 3D data with comparative analysis for human faces , 2012, Comput. Vis. Image Underst..

[42]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Sami Romdhani,et al.  Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  Edwin R. Hancock,et al.  A Coupled Statistical Model for Face Shape Recovery From Brightness Images , 2007, IEEE Transactions on Image Processing.

[45]  Thomas Vetter,et al.  Optimal landmark detection using shape models and branch and bound , 2011, 2011 International Conference on Computer Vision.

[46]  Stefanos Zafeiriou,et al.  300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge , 2013, 2013 IEEE International Conference on Computer Vision Workshops.