Surface representations for 3D face recognition

For long, face recognition has been a 2D discipline. However, 2D face recognition has shown to be extremely difficult to be robust against a.o. lighting conditions and pose variations (Phillips et al., 2003). At the same time, technological improvements are making 3D surface capturing devices affordable for security purposes. As a result of these recent developments face recognition shifts from 2D to 3D. This means that in the current state-of-the-art face recognition systems the problem is no longer the comparison of 2D color photos, but the comparison of (textured) 3D surface shapes. With the advent of the third dimension in face recognition, we think it is necessary to investigate the known surface representations from this point of view. Throughout recent decades, a lot of research focused on finding an appropriate digital representation for three dimensional real-world objects, mostly for use in computer graphics (Hubeli & Gross, 2000; Sigg, 2006). However, the needs for a surface representation in computer graphics, where the primary concerns are visualization and the ability to process it on dedicated computer graphics hardware (GPUs), are quite different from the needs of a surface representation for face recognition. Another motivation for this work is the non-existence of an overview of 3D surface representations, altough the problem of object representation is studied since the birth of computer vision (Marr, 1982). With this in mind, we will, in this chapter, try to give an overview of surface representations for use in biometric face recognition. Also surface representations that are not yet reported in current face recognition literature, but we consider to be promising for future research – based on publications in related fields such as 3D object retrieval, computer vision, computer graphics and 3D medical imaging – will be discussed. What are the desiderata for a surface representation in 3D face recognition? It is certainly useful for a surface representation in biometric applications, to be accurate, usable for all sorts of 3D surfaces in face recognition (open, closed. . . ), concise (efficient in memory usage), easy to acquire/construct, intuitive to work with, have a good formulation, be suitable for computations, convertible in other surface representations, ready to be efficiently displayed and useful for statistical modelling. It is nevertheless also certainly necessary to look further than a list of desiderata. Herefore, our approach will be the following: we make a taxonomy of all surface representations within the scope of 3D face recognition. For each of the of the representations in this taxonomy, we will shortly describe the mathematical theory behind it. Advantages and disadvantages of the surface representation will be stated. Related research using these representations will be discussed and directions for future research will be indicated.

[1]  Jun-Hai Yong,et al.  Optimal Parameterizations of Bézier Surfaces , 2006, ISVC.

[2]  P. Suetens,et al.  3D Face Recognition using Point Cloud Kernel Correlation , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[3]  Shigeru Muraki,et al.  Volumetric shape description of range data using “Blobby Model” , 1991, SIGGRAPH.

[4]  Angel R. Martinez,et al.  Computational Statistics Handbook with MATLAB, Second Edition (Chapman & Hall/Crc Computer Science & Data Analysis) , 2007 .

[5]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[6]  Paul Suetens,et al.  Isometric deformation modeling using singular value decomposition for 3D expression-invariant face recognition , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[7]  Anand Rangarajan,et al.  Simultaneous Nonrigid Registration of Multiple Point Sets and Atlas Construction , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Tomoyuki Nishita,et al.  A Method for Displaying Metaballs by using Bézier Clipping , 1994, Comput. Graph. Forum.

[9]  Michael G. Strintzis,et al.  Bilinear Models for 3-D Face and Facial Expression Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[10]  U. von Toussaint,et al.  Bayesian inference and maximum entropy methods in science and engineering , 2004 .

[11]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[12]  Huafeng Liu,et al.  Meshfree particle method , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[14]  N. Sukumar,et al.  Maximum Entropy Approximation , 2005 .

[15]  Charlie C. L. Wang,et al.  Ellipsoidal-blob approximation of 3D models and its applications , 2007, Comput. Graph..

[16]  James F. Blinn,et al.  A Generalization of Algebraic Surface Drawing , 1982, TOGS.

[17]  Bernhard Schölkopf,et al.  Implicit Surface Modelling with a Globally Regularised Basis of Compact Support , 2006, Comput. Graph. Forum.

[18]  Alexander M. Bronstein,et al.  Expression-Invariant 3D Face Recognition , 2003, AVBPA.

[19]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[20]  Steven J. Gortler,et al.  Geometry images , 2002, SIGGRAPH.

[21]  Rasmus Larsen,et al.  Surface-to-Surface Registration Using Level Sets , 2007, SCIA.

[22]  D. Veleba,et al.  Survey of Errors in Surface Representation and their Detection and Correction , 2007 .

[23]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[24]  Marc Levoy,et al.  The Use of Points as a Display Primitive , 2000 .

[25]  Ioannis A. Kakadiaris,et al.  Elastically Adaptive Deformable Models , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Moulay A. Akhloufi,et al.  Infrared Face Recognition Using Distance Transforms , 2008 .

[27]  Nick Pears.  RBF shape histograms and their application to 3D face processing , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[28]  Paul J. Besl,et al.  The Free-Form Surface Matching Problem , 1990 .

[29]  Frank B. ter Haar,et al.  SHREC’08 entry: 3D face recognition using facial contour curves , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[30]  T. Theoharis,et al.  Partial matching of interpose 3D facial data for face recognition , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[31]  Anil K. Jain,et al.  Deformation Analysis for 3D Face Matching , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[32]  Faisal R. Al-Osaimi,et al.  An Expression Deformation Approach to Non-rigid 3D Face Recognition , 2009, International Journal of Computer Vision.

[33]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[34]  Takeo Kanade,et al.  A Correlation-Based Approach to Robust Point Set Registration , 2004, ECCV.

[35]  Herbert Edelsbrunner,et al.  Three-dimensional alpha shapes , 1992, VVS.

[36]  Irene Y. H. Gu,et al.  3D Face Recognition Using Affine Integral Invariants , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[37]  L. Akarun,et al.  A 3D Face Recognition System for Expression and Occlusion Invariance , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[38]  Andrew Nealen,et al.  Physically Based Deformable Models in Computer Graphics , 2006, Comput. Graph. Forum.

[39]  Patrick J. Flynn,et al.  Multiple Nose Region Matching for 3D Face Recognition under Varying Facial Expression , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Alexander M. Bronstein,et al.  Three-Dimensional Face Recognition , 2005, International Journal of Computer Vision.

[41]  Tamal K. Dey,et al.  Shape Segmentation and Matching with Flow Discretization , 2003, WADS.

[42]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[43]  Jakob Andreas Bærentzen,et al.  3D distance fields: a survey of techniques and applications , 2006, IEEE Transactions on Visualization and Computer Graphics.

[44]  R. Basri,et al.  Shape representation and classification using the Poisson equation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[45]  Jean-Louis Dillenseger,et al.  Spherical Harmonics Based Intrasubject 3-D Kidney Modeling/Registration Technique Applied on Partial Information , 2006, IEEE Transactions on Biomedical Engineering.

[46]  Alberto Del Bimbo,et al.  Description and retrieval of 3D face models using iso-geodesic stripes , 2006, MIR '06.

[47]  Li Li,et al.  3D face recognition by constructing deformation invariant image , 2008, Pattern Recognit. Lett..

[48]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  R. Beatson,et al.  A short course on fast multipole methods , 1997 .

[50]  Sunghee Choi,et al.  The power crust , 2001, SMA '01.

[51]  Jake K. Aggarwal,et al.  3D Face Recognition Founded on the Structural Diversity of Human Faces , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Brian Wyvill,et al.  Introduction to Implicit Surfaces , 1997 .

[53]  A. Bovik,et al.  Three Dimensional Face Recognition Using Iso-Geodesic and Iso-Depth Curves , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[54]  Markus H. Gross,et al.  Multi‐scale Feature Extraction on Point‐Sampled Surfaces , 2003, Comput. Graph. Forum.

[55]  Zhaohui Wu,et al.  Exploring Facial Expression Effects in 3D Face Recognition Using Partial ICP , 2006, ACCV.

[56]  Berk Gökberk,et al.  3D face recognition for biometric applications , 2005, 2005 13th European Signal Processing Conference.

[57]  Chris Boehnen,et al.  3D Face Recognition Using 3D Alignment for PCA , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[58]  Raphaëlle Chaine,et al.  Efficient Spherical Harmonics Representation of 3D Objects , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[59]  Hao Zhang,et al.  Adapting Geometric Attributes for Expression-Invariant 3D Face Recognition , 2007, IEEE International Conference on Shape Modeling and Applications 2007 (SMI '07).

[60]  R. Fabio.  From point cloud to surface the modeling and visualization problem , 2003 .

[61]  Berk Gökberk,et al.  3D shape-based face recognition using automatically registered facial surfaces , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[62]  Paul Suetens,et al.  Computerized craniofacial reconstruction using CT-derived implicit surface representations. , 2006, Forensic science international.

[63]  Nira Dyn,et al.  Meshfree Thinning of 3D Point Clouds , 2008, Found. Comput. Math..

[64]  Michael G. Strintzis,et al.  3-D Face Recognition With the Geodesic Polar Representation , 2007, IEEE Transactions on Information Forensics and Security.

[65]  Patrick J. Flynn,et al.  A Survey Of Free-Form Object Representation and Recognition Techniques , 2001, Comput. Vis. Image Underst..

[66]  Pascal Frossard,et al.  3D Face Recognition with Sparse Spherical Representations , 2008, Pattern Recognit..

[67]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[68]  Nick Pears,et al.  Isoradius Contours: New Representations and Techniques for 3D Face Registration and Matching , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[69]  George C. Stockman,et al.  Canonical Face Depth Map: A Robust 3D Representation for Face Verification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[70]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[71]  Thomas Bülow,et al.  Surface Representations Using Spherical Harmonics and Gabor Wavelets on the Sphere , 2001 .

[72]  H. Krim,et al.  3D Face Recognition using Euclidean Integral Invariants Signature , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[73]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[74]  Christian Sigg,et al.  Representation and rendering of implicit surfaces , 2006 .

[75]  Anil K. Jain,et al.  Deformation Modeling for Robust 3D Face Matching , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Kin-Man Lam,et al.  Image Synthesis and Face Recognition Based on 3D Face Model and Illumination Model , 2005, ICNC.

[77]  Anuj Srivastava,et al.  Three-Dimensional Face Recognition Using Shapes of Facial Curves , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[78]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[79]  Markus Gross,et al.  A survey of surface representations for geometric modeling , 2000 .

[80]  Fabio Remondino.  From point cloud to surface , 2003 .

[81]  J A Sethian,et al.  Computing geodesic paths on manifolds. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[82]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[83]  Patrick J. Flynn,et al.  A Region Ensemble for 3-D Face Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[84]  Herbert Freeman,et al.  Machine Vision for Three-Dimensional Scenes , 1990 .

[85]  Amitabh Varshney,et al.  Statistical Point Geometry , 2003, Symposium on Geometry Processing.

[86]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[87]  Koichi Harada,et al.  Single-Patch NURBS Face , 2007, 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System.