Automatic Face Recognition Using Stereo Images

Face recognition is an important pattern recognition problem in the study of natural and artificial learning systems. In typical optical image based face recognition systems, the systematic variability that arises from representing the three dimensional (3D) shape of a face by a two dimensional (2D) illumination intensity matrix is treated as a random variable, and it is obtained by collecting examples of faces in different poses with respect to the camera. More sophisticated 3D recognition systems employ specialist equipment (e.g. laser scanners) to measure the shape of the face, and they perform either pattern matching in three dimensions or they use projections from 3D models to match against 2D images. It is shown here that optical images obtained with a pair of stereo cameras may be used to extract depth information in the form of disparity values, and thereby significantly enhance the performance of a face recognition system

[1]  Anil K. Jain,et al.  Deformation Analysis for 3D Face Matching , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[2]  Franz Rottensteiner,et al.  Semi-automatic extraction of buildings based on hybrid adjustment using 3D surface models and management of building data in a TIS , 2001 .

[3]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[4]  P. Jonathon Phillips,et al.  Support Vector Machines Applied to Face Recognition , 1998, NIPS.

[5]  A. A. El-Harby,et al.  Face Recognition: A Literature Review , 2008 .

[6]  Peter F. Sturm.  Self-calibration of a moving zoom-lens camera by pre-calibration , 1997, Image Vis. Comput..

[7]  Gaile G. Gordon,et al.  Face recognition from frontal and profile views , 1995 .

[8]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[9]  Stéphane Mallat,et al.  Singularity detection and processing with wavelets , 1992, IEEE Trans. Inf. Theory.

[10]  Andrew E. Johnson,et al.  Spin-Images: A Representation for 3-D Surface Matching , 1997 .

[11]  K Nakayama,et al.  Binocular visual surface perception. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Narendra Ahuja,et al.  A geometric approach to train support vector machines , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[13]  Guodong Guo,et al.  Face recognition by support vector machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Hyeonjoon Moon,et al.  The FERET verification testing protocol for face recognition algorithms , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[15]  Richard I. Hartley,et al.  Projective Reconstruction and Invariants from Multiple Images , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  I. Daubechies,et al.  Biorthogonal bases of compactly supported wavelets , 1992 .

[17]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[18]  Ying Wu.  Stereo and Multiview Geometry , 2007 .

[19]  Shaogang Gong,et al.  Constructing Facial Identity Surfaces for Recognition , 2003, International Journal of Computer Vision.

[20]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[21]  John Oliensis,et al.  An Experimental Study of Projective Structure From Motion , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  G. Strang,et al.  THE APPLICATION OF MULTIWAVELET FILTER BANKS TO IMAGE PROCESSING ∗ , 1995 .

[23]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[24]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[25]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Julian Magarey,et al.  Motion estimation using complex wavelets , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[27]  Amara Lynn Graps,et al.  An introduction to wavelets , 1995 .

[28]  Gérard G. Medioni,et al.  Face modeling and recognition in 3-D , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[29]  Margaret M. Fleck.  A topological stereo matcher , 1991, International Journal of Computer Vision.

[30]  Ching Y. Suen,et al.  Application of majority voting to pattern recognition: an analysis of its behavior and performance , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[31]  T. Vetter,et al.  Representations of human faces , 1996 .

[32]  Pascal Fua,et al.  3D stereo reconstruction of human faces driven by differential constraints , 2000, Image Vis. Comput..

[33]  C. Liu,et al.  Does Face Recognition Rely on Encoding of 3-D Surface? Examining the Role of Shape-from-Shading and Shape-from-Stereo , 2000, Perception.

[34]  Nicholas Costen,et al.  How Should We RepresentFaces for Automatic Recognition? , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[36]  Rama Chellappa,et al.  Discriminant analysis of principal components for face recognition , 1998 .

[37]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[38]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  M.M. Van Hulle,et al.  View-based 3D object recognition with support vector machines , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[40]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Jean-Marc Lina,et al.  Image Processing with Complex Daubechies Wavelets , 1997, Journal of Mathematical Imaging and Vision.

[42]  Massimiliano Pontil,et al.  Support Vector Machines for 3D Object Recognition , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Robert Laganière,et al.  An Empirical Study of Some Feature Matching Strategies , 2002 .

[44]  Bruce A. Draper,et al.  Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures , 2003 .

[45]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[46]  Martin D. Levine,et al.  Computer determination of depth maps , 1973, Comput. Graph. Image Process..

[47]  Julian Magarey,et al.  Motion estimation using a complex-valued wavelet transform , 1998, IEEE Trans. Signal Process..

[48]  Seong G. Kong,et al.  Recent advances in visual and infrared face recognition - a review , 2005, Comput. Vis. Image Underst..

[49]  Martial Hebert,et al.  Large data sets and confusing scenes in 3-D surface matching and recognition , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[50]  Ronald A. DeVore,et al.  Image compression through wavelet transform coding , 1992, IEEE Trans. Inf. Theory.

[51]  Rama Chellappa,et al.  Discriminant Analysis for Recognition of Human Face Images (Invited Paper) , 1997, AVBPA.

[52]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[53]  N. Kingsbury.  Complex Wavelets for Shift Invariant Analysis and Filtering of Signals , 2001 .

[54]  Luis Pastor,et al.  Wavelets for Object Representation and Recognition in Computer Vision , 1999 .

[55]  Anil K. Jain,et al.  Multimodal Facial Feature Extraction for Automatic 3D Face Recognition , 2005 .

[56]  Jen-Tzung Chien,et al.  Discriminant Waveletfaces and Nearest Feature Classifiers for Face Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[58]  Rolf P. Würtz,et al.  Image Representation by the Magnitude of the Discrete Gabor Wavelet Transform , 2007 .

[59]  Peter Sollich,et al.  Advances in neural information processing systems 11 , 1999 .

[60]  Tomaso A. Poggio,et al.  Face recognition with support vector machines: global versus component-based approach , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[61]  Anil K. Jain,et al.  Integrating Range and Texture Information for 3D Face Recognition , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[62]  T. Valentine.  The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology a Unified Account of the Effects of Distinctiveness, Inversion, and Race in Face Recognition , 2022 .

[63]  T. Aoki,et al.  3D face recognition using passive stereo vision , 2005, IEEE International Conference on Image Processing 2005.

[64]  Jim Austin,et al.  Combining multiple face recognition systems using Fisher's linear discriminant , 2004, SPIE Defense + Commercial Sensing.

[65]  Sun-Yuan Kung,et al.  Face recognition/detection by probabilistic decision-based neural network , 1997, IEEE Trans. Neural Networks.

[66]  Julian Magarey,et al.  An improved motion estimation algorithm using complex wavelets , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[67]  Maurizio Pilu,et al.  A direct method for stereo correspondence based on singular value decomposition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[68]  Reg G. Willson.  Modeling and calibration of automated zoom lenses , 1994, Other Conferences.

[69]  Elaine M. Newton,et al.  Meta-Analysis of Face Recognition Algorithms , 2001 .

[70]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[71]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[72]  Jim Austin,et al.  Three-Dimensional Face Recognition: A Fishersurface Approach , 2004, ICIAR.

[73]  Bülent Sankur,et al.  ARTICLE IN PRESS Image and Vision Computing xx (2005) 1–9 www.elsevier.com/locate/imavis , 2004 .

[74]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[75]  Julian Magarey,et al.  Multiresolution Phase-Based Bidirectional Stereo Matching with Provision for Discontinuity and Occlusion , 1998, Digit. Signal Process..

[76]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[77]  Gordon Erlebacher,et al.  A novel technique for face recognition using range imaging , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[78]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[79]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[80]  Mohan M. Trivedi,et al.  Region-based stereo analysis for robotic applications , 1989, IEEE Trans. Syst. Man Cybern..

[81]  W. Zhao.  Subspace methods in object/face recognition , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[82]  Alpheus W. Burner.  Zoom lens calibration for wind tunnel measurements , 1995, Other Conferences.

[83]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[84]  He-Ping Pan,et al.  Uniform Full-Information Image Matching Using Complex Conjugate Wavelet Pyramids , 1996 .

[85]  Sami Romdhani,et al.  Face identification across different poses and illuminations with a 3D morphable model , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[86]  Alberto Del Bimbo,et al.  Spin images for retrieval of 3D objects by local and global similarity , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[87]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[88]  Andrew Zisserman,et al.  Feature Based Methods for Structure and Motion Estimation , 1999, Workshop on Vision Algorithms.

[89]  M. Hebert,et al.  The Representation, Recognition, and Locating of 3-D Objects , 1986 .

[90]  Javier Ruiz-del-Solar,et al.  Eigenspace-based face recognition: a comparative study of different approaches , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[91]  Wayne Lawton,et al.  Applications of complex valued wavelet transforms to subband decomposition , 1993, IEEE Trans. Signal Process..

[92]  Gita Sukthankar,et al.  Face Recognition: A Critical Look at Biologically-Inspired Approaches , 2000 .

[93]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[94]  P. Schyns,et al.  Information and viewpoint dependence in face recognition , 1997, Cognition.

[95]  David J. Fleet,et al.  Phase-based disparity measurement , 1991, CVGIP Image Underst..

[96]  Xiangyang Ju,et al.  Facial Shape Recovery by Feature Driven Stereo Analysis , 1999, BMVC.

[97]  Jun Zhang,et al.  Pace recognition: eigenface, elastic matching, and neural nets , 1997, Proc. IEEE.

[98]  John K. Tsotsos,et al.  Techniques for disparity measurement , 1991, CVGIP Image Underst..

[99]  Michael A. Gennert,et al.  Stereo vision using Gabor receptive fields , 1992, Other Conferences.

[100]  G. Xu,et al.  A unified approach to image matching and segmentation in stereo, motion and object recognition via recovery of epipolar geometry , 1997 .

[101]  Katsuhiko Sakaue,et al.  Utilization of stereo disparity and optical flow information for the computer analysis of human interactions , 2003, Machine Vision and Applications.

[102]  He-Ping Pan,et al.  General stereo image matching using symmetric complex wavelets , 1996, Optics & Photonics.

[103]  Patrick J. Flynn,et al.  A Survey Of 3D and Multi-Modal 3D+2D Face Recognition , 2004 .

[104]  Anil K. Jain,et al.  Three-dimensional model based face recognition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[105]  Alexander M. Bronstein,et al.  Three-Dimensional Face Recognition , 2005, International Journal of Computer Vision.

[106]  Stéphane Mallat,et al.  Characterization of Signals from Multiscale Edges , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[107]  Michael G. Strintzis,et al.  Use of depth and colour eigenfaces for face recognition , 2003, Pattern Recognit. Lett..

[108]  L. Prasad,et al.  WAVELET ANALYSIS with Applications to IMAGE PROCESSING , 1997 .

[109]  T. Heseltine.  Evaluation of Image Pre-processing Techniques for Eigenface Based Face Recognition , 2002 .

[110]  Radu Horaud,et al.  Self-calibration and Euclidean reconstruction using motions of a stereo rig , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[111]  Shimon Edelman,et al.  Learning to Recognize Faces from Examples , 1992, ECCV.

[112]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[113]  Jenny C. A. Read,et al.  A Bayesian Approach to the Stereo Correspondence Problem , 2002, Neural Computation.

[114]  Jim Austin,et al.  Three-Dimensional Face Recognition Using Surface Space Combinations , 2004, BMVC.

[115]  P. O. Bishop.  NEURAL MECHANISMS FOR BINOCULAR DEPTH DISCRIMINATION , 1981 .

[116]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[117]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[118]  Alireza Khotanzad,et al.  Stereopsis by constraint learning feed-forward neural networks , 1993, IEEE Trans. Neural Networks.

[119]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[120]  Paul F. Whelan,et al.  Epipolar line extraction using feature matching , 2001 .

[121]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[122]  Stéphane Mallat,et al.  Multifrequency channel decompositions of images and wavelet models , 1989, IEEE Trans. Acoust. Speech Signal Process..

[123]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[124]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[125]  Alex Pentland,et al.  Beyond eigenfaces: probabilistic matching for face recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[126]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[127]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[128]  Radu Horaud,et al.  Projective translations and affine stereo calibration , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[129]  Julian Magarey,et al.  Multiresolution stereo image matching using complex wavelets , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[130]  Penio S. Penev,et al.  Local feature analysis: A general statistical theory for object representation , 1996 .

[131]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[132]  Patrick J. Flynn,et al.  An evaluation of multimodal 2D+3D face biometrics , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[133]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[134]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[135]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[136]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[137]  Roland Wilson,et al.  A generalized wavelet transform for Fourier analysis: The multiresolution Fourier transform and its application to image and audio signal analysis , 1992, IEEE Trans. Inf. Theory.

[138]  Greg Turk,et al.  Re-tiling polygonal surfaces , 1992, SIGGRAPH.

[139]  W.A. Barrett,et al.  A survey of face recognition algorithms and testing results , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[140]  Gabriel Taubin,et al.  Curve and surface smoothing without shrinkage , 1995, Proceedings of IEEE International Conference on Computer Vision.

[141]  Chi-Fa Chen,et al.  Combination of PCA and Wavelet Transforms for Face Recognition on 2.5D Images , 2003 .

[142]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[143]  Nasser M. Nasrabadi,et al.  Hopfield network for stereo vision correspondence , 1992, IEEE Trans. Neural Networks.

[144]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[145]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[146]  John L. Barron,et al.  Image Reconstruction Error for Optical Flow , 1995, Research in Computer and Robot Vision.

[147]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[148]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[149]  D Marr,et al.  A computational theory of human stereo vision. , 1979, Proceedings of the Royal Society of London. Series B, Biological sciences.

[150]  John L. Barron,et al.  Computation of time-varying motion and structure parameters from real image sequences , 1996, TFCV.

[151]  Amnon Shashua,et al.  Projective Structure from Uncalibrated Images: Structure From Motion and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[152]  Horst Bunke,et al.  Combination of face classifiers for person identification , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[153]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[154]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .