Matching 2.5D face scans to 3D models

The performance of face recognition systems that use two-dimensional images depends on factors such as lighting and subject's pose. We are developing a face recognition system that utilizes three-dimensional shape information to make the system more robust to arbitrary pose and lighting. For each subject, a 3D face model is constructed by integrating several 2.5D face scans which are captured from different views. 2.5D is a simplified 3D (x,y,z) surface representation that contains at most one depth value (z direction) for every point in the (x, y) plane. Two different modalities provided by the facial scan, namely, shape and texture, are utilized and integrated for face matching. The recognition engine consists of two components, surface matching and appearance-based matching. The surface matching component is based on a modified iterative closest point (ICP) algorithm. The candidate list from the gallery used for appearance matching is dynamically generated based on the output of the surface matching component, which reduces the complexity of the appearance-based matching stage. Three-dimensional models in the gallery are used to synthesize new appearance samples with pose and illumination variations and the synthesized face images are used in discriminant subspace analysis. The weighted sum rule is applied to combine the scores given by the two matching components. Experimental results are given for matching a database of 200 3D face models with 598 2.5D independent test scans acquired under different pose and some lighting and expression changes. These results show the feasibility of the proposed matching scheme.

[1]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[2]  J. Cartoux,et al.  Face authentification or recognition by profile extraction from range images , 1989, [1989] Proceedings. Workshop on Interpretation of 3D Scenes.

[3]  Anil K. Jain,et al.  Three-dimensional model based face recognition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[4]  David M. Weinstein.  The Analytic 3-D Transform for the Least-Squared Fit of Three Pairs of Corresponding Points , 1998 .

[5]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[6]  Horst Bunke,et al.  Face recognition using range images , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[7]  Chitra Dorai,et al.  COSMOS - A Representation Scheme for 3D Free-Form Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Samy Bengio,et al.  Evaluation of Biometric Technology on XM2VTS , 2001 .

[9]  Arun Ross,et al.  Learning user-specific parameters in a multibiometric system , 2002, Proceedings. International Conference on Image Processing.

[10]  Pascal Fua,et al.  Accurate face models from uncalibrated and ill-lit video sequences , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[11]  James C. Miller,et al.  Computer graphics principles and practice, second edition , 1992, Comput. Graph..

[12]  Gang Wang,et al.  Registration and Integration of Multiple Object Views for 3D Model Construction , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[15]  Zhengyou Zhang,et al.  Image-based modeling of objects and human faces , 2000, IS&T/SPIE Electronic Imaging.

[16]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[17]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[18]  Qian Chen,et al.  Building 3-D Human Face Models from Two Photographs , 2001, J. VLSI Signal Process..

[19]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Pascal Fua,et al.  Accurate face models from uncalibrated and ill-lit video sequences , 2004, CVPR 2004.

[21]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[22]  Zhaohui Wu,et al.  Automatic 3D face verification from range data , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[23]  Patrick J. Flynn,et al.  Multi-Modal 2D and 3D Biometrics for Face Recognition , 2003, AMFG.

[24]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[25]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[26]  Marc Acheroy,et al.  Automatic 3D face authentication , 2000, Image Vis. Comput..

[27]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Alexander M. Bronstein,et al.  Expression-Invariant 3D Face Recognition , 2003, AVBPA.

[29]  Gaile G. Gordon,et al.  Face recognition based on depth and curvature features , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Anil K. Jain,et al.  Deformation Analysis for 3D Face Matching , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[31]  Peter Liepa,et al.  Filling Holes in Meshes , 2003, Symposium on Geometry Processing.

[32]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[33]  J. E. Porter,et al.  Normalizations and selection of speech segments for speaker recognition scoring , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[34]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[35]  Wen Gao,et al.  Curse of mis-alignment in face recognition: problem and a novel mis-alignment learning solution , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[36]  Marian Stewart Bartlett,et al.  Independent component representations for face recognition , 1998, Electronic Imaging.

[37]  Hiromi T. Tanaka,et al.  Curvature-based face surface recognition using spherical correlation-principal directions for curved object recognition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[38]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[39]  Ross T. Whitaker,et al.  Geometric surface smoothing via anisotropic diffusion of normals , 2002, IEEE Visualization, 2002. VIS 2002..

[40]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[41]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Marc Levoy,et al.  Geometrically stable sampling for the ICP algorithm , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[44]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[45]  Raghu Machiraju,et al.  Model-based 3D face capture with shape-from-silhouettes , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[46]  Feng Han,et al.  3D human face recognition using point signature , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[47]  Evangelos E. Milios,et al.  Matching range images of human faces , 1990, [1990] Proceedings Third International Conference on Computer Vision.