3D Face Modeling

Acquiring, modeling, and synthesizing realistic 3D human faces and their dynamics have emerged as an active research topic in the border area between the computer vision and computer graphics fields of research. This has resulted in a plethora of different acquisition systems and processing pipelines that share many fundamental concepts as well as specific implementation details. The research community has investigated the possibility of targeting either end-to-end consumer-level or professional-level applications, such as facial geometry acquisition for 3D-based biometrics and its dynamics capturing for expression cloning or performance capture and, more recently, for 4D expression analysis and recognition. Despite the rich literature, reproducing realistic human faces remains a distant goal because the challenges that face 3D face modeling are still open. These challenges include the motion speed of the face when conveying expressions, the variabilities in lighting conditions, and pose. In addition, human beings are very sensitive to facial appearance and quickly sense any anomalies in 3D geometry or dynamics of faces. The techniques developed in this field attempt to recover facial 3D shapes from camera(s) and reproduce their actions. Consequently, they seek to answer the following questions:

[1]  Maurício Pamplona Segundo,et al.  Automatic Face Segmentation and Facial Landmark Detection in Range Images , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  L. B. Wolff,et al.  Three-dimensional stereo by photometric ratios , 1994 .

[3]  Lawrence B. Wolff,et al.  Surface Curvature and Shape Reconstruction from Unknown Multiple Illumination and Integrability , 1997, Comput. Vis. Image Underst..

[4]  Tim Weyrich,et al.  Post-processing of Scanned 3D Surface Data , 2004, PBG.

[5]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Thabo Beeler,et al.  High-quality single-shot capture of facial geometry , 2010, SIGGRAPH 2010.

[7]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, SIGGRAPH 2005.

[8]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  Pau Gargallo,et al.  Bayesian 3D modeling from images using multiple depth maps , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[11]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[12]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[13]  Jean Ponce,et al.  Dense 3D motion capture from synchronized video streams , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[15]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[16]  Jean Ponce,et al.  Dense 3D motion capture for human faces , 2009, CVPR.

[17]  Tianli Yu,et al.  Shape and View Independent Reflectance Map from Multiple Views , 2004, International Journal of Computer Vision.

[18]  Li Zhang,et al.  Spacetime faces: high resolution capture for modeling and animation , 2004, SIGGRAPH 2004.

[19]  Szymon Rusinkiewicz,et al.  Spacetime stereo: a unifying framework for depth from triangulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Camillo J. Taylor.  Surface reconstruction from feature based stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  Björn Stenger,et al.  Non-rigid Photometric Stereo with Colored Lights , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Jovan Popović,et al.  Deformation transfer for triangle meshes , 2004, SIGGRAPH 2004.

[23]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Derek Bradley,et al.  High resolution passive facial performance capture , 2010, SIGGRAPH 2010.

[25]  Paul A. Beardsley,et al.  High-quality passive facial performance capture using anchor frames , 2011, SIGGRAPH 2011.

[26]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[27]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[28]  Roberto Cipolla,et al.  Overcoming Shadows in 3-Source Photometric Stereo , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Luc Van Gool,et al.  Face/Off: live facial puppetry , 2009, SCA '09.

[33]  Ioannis A. Kakadiaris,et al.  Accurate Landmarking of Three-Dimensional Facial Data in the Presence of Facial Expressions and Occlusions Using a Three-Dimensional Statistical Facial Feature Model , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[34]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[36]  Caroline Jay,et al.  A perceptually validated model for surface depth hallucination , 2008, SIGGRAPH 2008.

[37]  Shervin Mehryar,et al.  Automatic landmark detection for 3D face image processing , 2010, IEEE Congress on Evolutionary Computation.

[38]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[39]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[40]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[41]  Faouzi Ghorbel,et al.  Multi-camera 3D Scanning with a Non-rigid and Space-Time Depth Super-Resolution Capability , 2011, CAIP.

[42]  Sung Yong Shin,et al.  An example-based approach for facial expression cloning , 2003, SCA '03.

[43]  Song Zhang,et al.  High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm , 2007 .

[44]  N. Amenta,et al.  Defining point-set surfaces , 2004, SIGGRAPH 2004.

[45]  E. McClintock,et al.  THE PAST AND FUTURE OF THE AMERICAN MATHEMATICAL SOCIETY. , 1895, Science.

[46]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[47]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[48]  Lijun Yin,et al.  Facial Expression Recognition Based on 3D Dynamic Range Model Sequences , 2008, ECCV.

[49]  Meenakshisundaram Gopi,et al.  Surface Reconstruction based on Lower Dimensional Localized Delaunay Triangulation , 2000, Comput. Graph. Forum.

[50]  Xiaolin Li,et al.  Multilevel quality-guided phase unwrapping algorithm for real-time three-dimensional shape reconstruction. , 2007, Applied optics.