Review of statistical shape spaces for 3D data with comparative analysis for human faces

Abstract With systems for acquiring 3D surface data being evermore commonplace, it has become important to reliably extract specific shapes from the acquired data. In the presence of noise and occlusions, this can be done through the use of statistical shape models, which are learned from databases of clean examples of the shape in question. In this paper, we review, analyze and compare different statistical models: from those that analyze the variation in geometry globally to those that analyze the variation in geometry locally. We first review how different types of models have been used in the literature, then proceed to define the models and analyze them theoretically, in terms of both their statistical and computational aspects. We then perform extensive experimental comparison on the task of model fitting, and give intuition about which type of model is better for a few applications. Due to the wide availability of databases of high-quality data, we use the human face as the specific shape we wish to extract from corrupted data.

[1]  Michael J. Black,et al.  The Naked Truth: Estimating Body Shape Under Clothing , 2008, ECCV.

[2]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[3]  Andrew E. Johnson,et al.  Recognizing objects by matching oriented points , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  P. Ellen Grant,et al.  Cortical Folding Development Study based on Over-Complete Spherical Wavelets , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Bernd Hamann,et al.  Generalized B-spline subdivision-surface wavelets for geometry compression , 2004, IEEE Transactions on Visualization and Computer Graphics.

[6]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[7]  Wojciech Matusik,et al.  A statistical model for synthesis of detailed facial geometry , 2006, SIGGRAPH 2006.

[8]  Derek Bradley,et al.  High-quality passive facial performance capture using anchor frames , 2011, ACM Trans. Graph..

[9]  Aaron F. Bobick,et al.  Shape-Driven 3D Segmentation Using Spherical Wavelets , 2006, MICCAI.

[10]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[11]  KwangYun Wohn,et al.  3D Body Reconstruction from Photos Based on Range Scan , 2006, Edutainment.

[12]  P. Thomas Fletcher,et al.  Principal geodesic analysis for the study of nonlinear statistics of shape , 2004, IEEE Transactions on Medical Imaging.

[13]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[14]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[15]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[16]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[17]  Nikos Paragios,et al.  Hierarchical 3D diffusion wavelet shape priors , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  Fei Yang,et al.  Facial expression editing in video using a temporally-smooth factorization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Chang Shu,et al.  Posture-invariant statistical shape analysis using Laplace operator , 2012, Comput. Graph..

[21]  Roberto Cipolla,et al.  Learning shape priors for single view reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[22]  Sunil Arya,et al.  ANN: library for approximate nearest neighbor searching , 1998 .

[23]  Hao Li,et al.  Example-based facial rigging , 2010, ACM Transactions on Graphics.

[24]  Chak-Kuen Wong,et al.  Worst-case analysis for region and partial region searches in multidimensional binary search trees and balanced quad trees , 1977, Acta Informatica.

[25]  Fei Yang,et al.  Expression flow for 3D-aware face component transfer , 2011, SIGGRAPH 2011.

[26]  Marcus A. Magnor,et al.  Sparse localized deformation components , 2013, ACM Trans. Graph..

[27]  D. Louis Collins,et al.  A Statistical Parts-Based Appearance Model of Inter-subject Variability , 2006, MICCAI.

[28]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, SIGGRAPH 2005.

[29]  William A. P. Smith,et al.  3D morphable face models revisited , 2009, CVPR.

[30]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[31]  Michael G. Strintzis,et al.  Bilinear Models for 3-D Face and Facial Expression Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[32]  Michael J. Black,et al.  Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape , 2012, ECCV.

[33]  Luc Van Gool,et al.  Optimal Regions for Linear Model-Based 3D Face Reconstruction , 2010, ACCV.

[34]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[35]  Zicheng Liu,et al.  Tensor-Based Human Body Modeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Hans-Peter Seidel,et al.  Multilinear pose and body shape estimation of dressed subjects from image sets , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Michael J. Black,et al.  Home 3D body scans from noisy image and range data , 2011, 2011 International Conference on Computer Vision.

[38]  Wojciech Matusik,et al.  Video face replacement , 2011, ACM Trans. Graph..

[39]  Flavio Prieto,et al.  Fully automatic expression-invariant face correspondence , 2013, Machine Vision and Applications.

[40]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[41]  Won-Sook Lee,et al.  A Data-driven Approach to Human-body Cloning Using a Segmented Body Database , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[42]  Alessandro Verri,et al.  Fitting 3D morphable models using implicit representations , 2007, J. Virtual Real. Broadcast..

[43]  W. Sweldens.  The Lifting Scheme: A Custom - Design Construction of Biorthogonal Wavelets "Industrial Mathematics , 1996 .

[44]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[45]  William A. P. Smith,et al.  Exploring the Identity Manifold: Constrained Operations in Face Space , 2010, ECCV.

[46]  Yang Li,et al.  Model-Guided Segmentation of 3D Neuroradiological Image Using Statistical Surface Wavelet Model , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[48]  Andrew W. Fitzgibbon,et al.  Reconstructing High Quality Face-Surfaces using Model Based Stereo , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[49]  Chang Shu,et al.  Estimating 3D human shapes from measurements , 2012, Machine Vision and Applications.

[50]  Jim Austin,et al.  3D face landmark labelling , 2010, 3DOR '10.

[51]  Hans-Peter Seidel,et al.  Fast Face Detector Training Using Tailored Views , 2013, 2013 IEEE International Conference on Computer Vision.

[52]  Paul M. Thompson,et al.  Exploration of Shape Variation Using Localized Components Analysis , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Aaron F. Bobick,et al.  Multiscale 3D Shape Analysis Using Spherical Wavelets , 2005, MICCAI.

[54]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[55]  Mohammed Benjelloun,et al.  Fast 3D Spine Reconstruction of Postoperative Patients Using a Multilevel Statistical Model , 2012, MICCAI.

[56]  Hans-Peter Seidel,et al.  A Statistical Model of Human Pose and Body Shape , 2009, Comput. Graph. Forum.

[57]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[58]  Dinggang Shen,et al.  Hierarchical active shape models, using the wavelet transform , 2003, IEEE Transactions on Medical Imaging.

[59]  Chang Shu,et al.  Three-dimensional human shape inference from silhouettes: reconstruction and validation , 2011, Machine Vision and Applications.

[60]  Peter Schröder,et al.  Spherical wavelets: efficiently representing functions on the sphere , 1995, SIGGRAPH.

[61]  Martin Styner,et al.  Evaluation of 3D Correspondence Methods for Model Building , 2003, IPMI.

[62]  D. Cohen-Or,et al.  Parametric reshaping of human bodies in images , 2010, ACM Trans. Graph..

[63]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[64]  Jochen Lang,et al.  Efficient Multi-scale Stereo of High-Resolution Planar and Spherical Images , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[65]  Timothy F. Cootes,et al.  Statistical models of appearance for medical image analysis and computer vision , 2001, SPIE Medical Imaging.

[66]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[68]  Michael J. Black,et al.  Estimating human shape and pose from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[69]  Timo Bolkart,et al.  Statistical Analysis of 3D Faces in Motion , 2013, 2013 International Conference on 3D Vision.

[70]  Jihun Yu,et al.  Realtime facial animation with on-the-fly correctives , 2013, ACM Trans. Graph..

[71]  Andrew W. Fitzgibbon,et al.  What Shape Are Dolphins? Building 3D Morphable Models from 2D Images , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  Jochen Lang,et al.  Wavelet Model-based Stereo for Fast, Robust Face Reconstruction , 2011, 2011 Canadian Conference on Computer and Robot Vision.

[73]  Frank B. ter Haar,et al.  3D Face Model Fitting for Recognition , 2008, ECCV.

[74]  Christian Theobalt,et al.  Reconstructing detailed dynamic face geometry from monocular video , 2013, ACM Trans. Graph..

[75]  Hans-Peter Seidel,et al.  MovieReshape: tracking and reshaping of humans in videos , 2010, ACM Trans. Graph..

[76]  Hans-Peter Seidel,et al.  Estimating body shape of dressed humans , 2009, Comput. Graph..

[77]  Christopher J. Taylor,et al.  Statistical models of shape - optimisation and evaluation , 2008 .