A Survey on 3D Modeling of Human Faces for Face Recognition

In its quest for more reliability and higher recognition rates the face recognition community has been focusing more and more on 3D based recognition. Depth information adds another dimension to facial features and provides ways to minimize the effects of pose and illumination variations for achieving greater recognition accuracy. This chapter reviews, therefore, the major techniques for 3D face modeling, the first step in any 3D assisted face recognition system. The reviewed techniques are laser range scans, 3D from structured light projection, stereo vision, morphing, shape from motion, shape from space carving, and shape from shading. Concepts, accuracy, feasibility, and limitations of these techniques and their effectiveness for 3D face recognition are discussed.

[1]  Qian Chen,et al.  Building 3-D Human Face Models from Two Photographs , 2001, J. VLSI Signal Process..

[2]  Rama Chellappa,et al.  Face recognition from video: a CONDENSATION approach , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[3]  Joaquim Salvi,et al.  Recent progress in coded structured light as a technique to solve the correspondence problem: a survey , 1998, Pattern Recognit..

[4]  Sami Romdhani,et al.  Face identification across different poses and illuminations with a 3D morphable model , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[5]  Raghu Machiraju,et al.  Silhouette-Based 3D Face Shape Recovery , 2003, Graphics Interface.

[6]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Gaile G. Gordon,et al.  Face recognition based on depth maps and surface curvature , 1991, Optics & Photonics.

[8]  Michael G. Strintzis,et al.  Pose and illumination compensation for 3D face recognition , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[9]  Xiangyang Ju,et al.  A high resolution stereo imaging system , 2003 .

[10]  Zheng Tan,et al.  Automatic 3D facial model reconstruction from single front-view image , 2004, International Conference On Virtual Reality and Its Applications in Industry.

[11]  P. Jonathon Phillips,et al.  Face recognition vendor test 2002 , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[12]  Josef Kittler,et al.  3D Assisted Face Recognition: A Survey of 3D Imaging, Modelling and Recognition Approachest , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[13]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[14]  Joaquim Salvi,et al.  Laser stripe peak detector for 3D scanners. A FIR filter approach , 2004, ICPR 2004.

[15]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[16]  E.P. Kukula,et al.  Effects of illumination changes on the performance of Geometrix FaceVision/spl reg/ 3D FRS , 2004, 38th Annual 2004 International Carnahan Conference on Security Technology, 2004..

[17]  Alexander M. Bronstein,et al.  Expression-Invariant 3D Face Recognition , 2003, AVBPA.

[18]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[19]  Patrick J. Flynn,et al.  A survey of approaches to three-dimensional face recognition , 2004, ICPR 2004.

[20]  Josef Kittler,et al.  3D Assisted Face Recognition: A Survey , 2007 .

[21]  Sridha Sridharan,et al.  Techniques for improving stereo depth maps of faces , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[22]  Michael G. Strintzis,et al.  Face localization and authentication using color and depth images , 2005, IEEE Transactions on Image Processing.

[23]  M. Abidi,et al.  Performance Comparison of Visual and Thermal Signatures for Face Recognition , 2003 .

[24]  Xing Chen,et al.  A laser range scanner designed for minimum calibration complexity , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[25]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[26]  Thomas Malzbender,et al.  A Survey of Methods for Volumetric Scene Reconstruction from Photographs , 2001, VG.

[27]  Alex Pentland,et al.  A simple algorithm for shape from shading , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Rama Chellappa,et al.  3D model enhanced face recognition , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[29]  Jie Yan,et al.  Synthesized virtual view-based eigenspace for face recognition , 2000, Proceedings Fifth IEEE Workshop on Applications of Computer Vision.

[30]  Towards 3-dimensional face recognition , 1999, 1999 IEEE Africon. 5th Africon Conference in Africa (Cat. No.99CH36342).

[31]  Quang-Tuan Luong,et al.  Self-Calibration of a Moving Camera from Point Correspondences and Fundamental Matrices , 1997, International Journal of Computer Vision.

[32]  Augusto Sarti,et al.  Accurate depth-map estimation for 3D face modeling , 2005, 2005 13th European Signal Processing Conference.

[33]  Juan B. Hurtado-Ramos,et al.  Surface shape estimation from photometric images , 2004 .

[34]  Rama Chellappa,et al.  SFS based view synthesis for robust face recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[35]  Zicheng Liu,et al.  Model-based bundle adjustment with application to face modeling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[36]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.

[37]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[38]  Long Quan,et al.  Match Propagation for Image-Based Modeling and Rendering , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  André Oosterlinck,et al.  Range Image Acquisition with a Single Binary-Encoded Light Pattern , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Jean-Christophe Nebel,et al.  3D thermography imaging standardization technique for inflammation diagnosis , 2005, SPIE/COS Photonics Asia.

[41]  Hong Yan,et al.  Comparison of face verification results on the XM2VTFS database , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[42]  Paul J. Besl,et al.  Active, optical range imaging sensors , 1988, Machine Vision and Applications.

[43]  Yuxiao Hu,et al.  Automatic 3D reconstruction for face recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[44]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Patrick J. Flynn,et al.  Face Recognition Using 2D and 3D Facial Data , 2003 .

[46]  W J McKay,et al.  Head movement in normal subjects during simulated brain imaging. , 1998, Journal of nuclear medicine technology.

[47]  Chengjun Liu,et al.  A shape- and texture-based enhanced Fisher classifier for face recognition , 2001, IEEE Trans. Image Process..

[48]  Long Quan,et al.  Invariants of Six Points and Projective Reconstruction From Three Uncalibrated Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[50]  Volker Blanz,et al.  Face Recognition Using Component-Based SVM Classification and Morphable Models , 2002, SVM.

[51]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[52]  Chin-Seng Chua,et al.  Integrated 2D and 3D images for face recognition , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[53]  P. Jonathon Phillips,et al.  Face Recognition Vendor Test 2002 Performance Metrics , 2003, AVBPA.

[54]  Urs E. Ruttimann,et al.  Head motion during positron emission tomography: is it significant? , 1995, Psychiatry Research: Neuroimaging.

[55]  Pascal Fua,et al.  3D face modeling from stereo and differential constraints , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[56]  Jean-Christophe Nebel,et al.  3D stereophotogrammetric image superimposition onto 3D CT scan images: the future of orthognathic surgery. A pilot study. , 2002, The International journal of adult orthodontics and orthognathic surgery.

[57]  S. Lai,et al.  An integrated approach to 3D face model reconstruction from video , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[58]  Roberto Cipolla,et al.  A surface evolution approach of probabilistic space carving , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[59]  Rama Chellappa,et al.  Wide baseline image registration with application to 3-D face modeling , 2004, IEEE Transactions on Multimedia.

[60]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[61]  Anuj Srivastava,et al.  Face recognition using optimal linear components of range images , 2006, Image Vis. Comput..

[62]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[63]  Karl Sammut,et al.  Example-based shape from shading: 3D heads form 2D face images , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[64]  Mongi A. Abidi,et al.  Stereo matching with energy-minimizing snake grid for 3D face modeling , 2004, SPIE Defense + Commercial Sensing.

[65]  Raghu Machiraju,et al.  Estimation of 3D faces and illumination from single photographs using a bilinear illumination model , 2005, EGSR '05.

[66]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[67]  Luiz Velho,et al.  Adaptive space carving , 2004 .

[68]  Fernand S. Cohen,et al.  3-D face structure extraction and recognition from images using 3-D morphing and distance mapping , 2002, IEEE Trans. Image Process..

[69]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Ralph Gross,et al.  Eigen light-fields and face recognition across pose , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[71]  Edwin R. Hancock,et al.  Recovering facial pose with the EM algorithm , 2002, Pattern Recognit..

[72]  Songde Ma,et al.  3D face image acquisition and reconstruction system , 1998, IMTC/98 Conference Proceedings. IEEE Instrumentation and Measurement Technology Conference. Where Instrumentation is Going (Cat. No.98CH36222).

[73]  Rama Chellappa,et al.  Estimation of Illuminant Direction, Albedo, and Shape from Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Raghu Machiraju,et al.  Finding optimal views for 3D face shape modeling , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[75]  Carlo Tomasi,et al.  A Pixel Dissimilarity Measure That Is Insensitive to Image Sampling , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  Lamei Yan,et al.  SFS based neural algorithm for robust 3D face shape recovery [robust read robust] , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[77]  Nicola D'Apuzzo,et al.  Modeling human faces with multi-image photogrammetry , 2002, IS&T/SPIE Electronic Imaging.

[78]  A-Nasser Ansari,et al.  3D face modeling using two orthogonal views and a generic face model , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[79]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[80]  A. Pentland,et al.  Real time tracking and modeling of faces: an EKF-based analysis by synthesis approach , 1999, Proceedings IEEE International Workshop on Modelling People. MPeople'99.

[81]  Chin-Seng Chua,et al.  Facial feature detection and face recognition from 2D and 3D images , 2002, Pattern Recognit. Lett..

[82]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[83]  Ingemar J. Cox,et al.  A multiple-baseline stereo for precise human face acquisition , 1997, Pattern Recognit. Lett..

[84]  Marc Acheroy,et al.  Automatic 3D face authentication , 2000, Image Vis. Comput..

[85]  Nader Bagherzadeh,et al.  CARVING 3D MODELS FROM UNCALIBRATED VIEWS , 2002 .

[86]  A. Pentland.  Finding the illuminant direction , 1982 .

[87]  Seong G. Kong,et al.  Recent advances in visual and infrared face recognition - a review , 2005, Comput. Vis. Image Underst..

[88]  R. Cipolla,et al.  A probabilistic framework for space carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[89]  Shinn-Ying Ho,et al.  Facial modeling from an uncalibrated face image using a coarse-to-fine genetic algorithm , 2001, Pattern Recognit..

[90]  François Blais.  Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[91]  Wen Gao,et al.  Pose Invariant Face Recognition Under Arbitrary Illumination Based on 3D Face Reconstruction , 2005, AVBPA.

[92]  Rama Chellappa,et al.  3D face reconstruction from video using a generic model , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[93]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[94]  Seong-Whan Lee,et al.  Face reconstruction with a morphable face model , 2002, Object recognition supported by user interaction for service robots.

[95]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[96]  Itsuo Kumazawa,et al.  Analysis of shape from shading algorithms for fast and realistic 3D face reconstruction APCCAS2002 , 2002, Asia-Pacific Conference on Circuits and Systems.

[97]  Wen Gao,et al.  Efficient 3D reconstruction for face recognition , 2005, Pattern Recognit..

[98]  Wen Gao,et al.  Deformable model-based generation of realistic 3-D specific human face , 1998, ICSP '98. 1998 Fourth International Conference on Signal Processing (Cat. No.98TH8344).

[99]  Ulrich Neumann,et al.  SYNTHESIS OF 3D FACES , 1999 .

[100]  Patrick J. Flynn,et al.  An evaluation of multimodal 2D+3D face biometrics , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[101]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[102]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[103]  Domenico Vitulano,et al.  Surface recovery by self shading projection , 2004, Signal Process..

[104]  Shihong Lao,et al.  3D template matching for pose invariant face recognition using 3D facial model built with isoluminance line based stereo vision , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[105]  Seong-Whan Lee,et al.  Reconstruction of Partially Damaged Face Images Based on a Morphable Face Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[106]  Edwin R. Hancock,et al.  Face Recognition Using Shape-from-shading , 2002, BMVC.

[107]  Feng Han,et al.  3D human face recognition using point signature , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[108]  Hervé Delingette,et al.  Low cost 3D face acquisition and modeling , 2001, Proceedings International Conference on Information Technology: Coding and Computing.

[109]  Ralph Gross,et al.  Appearance-Based 3-D Face Recognition from Video , 2002, DAGM-Symposium.

[110]  Harry Shum,et al.  Automatic 3D face modeling from video , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[111]  Steven Yi,et al.  3D FaceCam: a fast and accurate 3D facial imaging device for biometrics applications , 2004, SPIE Defense + Commercial Sensing.

[112]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[113]  Alex Pentland,et al.  Mixtures of eigenfeatures for real-time structure from texture , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[114]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[115]  Gérard G. Medioni,et al.  Generation of a 3-D face model from one camera , 2002, Object recognition supported by user interaction for service robots.

[116]  A. Ardeshir Goshtasby,et al.  Approximating Digital 3D Shapes by Rational Gaussian Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[117]  R. Waupotitsch,et al.  Effects of illumination changes on the performance of geometrix FaceVision® 3D FRS , 2004 .

[118]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[119]  Patrick J. Flynn,et al.  Visible-light and Infrared Face Recognition , 2003 .

[120]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[121]  Ioannis A. Kakadiaris,et al.  Multimodal face recognition: combination of geometry with physiological information , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[122]  P. Jonathon Phillips,et al.  Face recognition based on frontal views generated from non-frontal images , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[123]  Anil K. Jain,et al.  Semantic face matching , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[124]  Takashi Nakamura,et al.  High-resolution 3-D shape integration of dentition and face measured by new laser scanner , 2004, IEEE Transactions on Medical Imaging.

[125]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[126]  Tieniu Tan,et al.  Three-dimensional face recognition using geometric model , 2004, SPIE Defense + Commercial Sensing.

[127]  Luc Van Gool,et al.  Euclidean 3D Reconstruction from Image Sequences with Variable Focal Lenghts , 1996, ECCV.

[128]  Luiz Velho,et al.  Space carving with a hand-held camera , 2004, Proceedings. 17th Brazilian Symposium on Computer Graphics and Image Processing.

[129]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[130]  Nikolaos Grammalidis,et al.  Building Three Dimensional Head Models , 2001, Graph. Model..

[131]  Rama Chellappa,et al.  Face reconstruction from monocular video using uncertainty analysis and a generic model , 2003, Comput. Vis. Image Underst..