Face Recognition by Multi-Frame Fusion of Rotating Heads in Videos

This paper presents a face recognition study that implicitly utilizes the 3D information in 2D video sequences through multi-sample fusion. The approach is based on the hypothesis that continuous and coherent intensity variations in video frames caused by a rotating head can provide information similar to that of explicit shapes or range images. The fusion was done on the image level to prevent information loss. Experiments were carried out using a data set of over 100 subjects and promising results have been obtained: (1) under regular indoor lighting conditions, rank one recognition rate increased from 91% using a single frame to 100% using 7-frame fusion; (2) under strong shadow conditions, rank one recognition rate increased from 63% using a single frame to 85% using 7-frame fusion.

[1]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[2]  Alexander M. Bronstein,et al.  Three-Dimensional Face Recognition , 2005, International Journal of Computer Vision.

[3]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Marc Acheroy,et al.  Face verification from 3D and grey level clues , 2001, Pattern Recognit. Lett..

[5]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Xin Chen,et al.  Face Recognition Using 2-D, 3-D, and Infrared: Is Multimodal Better Than Multisample? , 2006, Proceedings of the IEEE.

[7]  Dimitris N. Metaxas,et al.  Optical Flow Constraints on Deformable Models with Applications to Face Tracking , 2000, International Journal of Computer Vision.

[8]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Sudeep Sarkar,et al.  Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Stefan Gehlen,et al.  Evaluation of implicit 3D modeling for pose-invariant face recognition , 2004, SPIE Defense + Commercial Sensing.

[11]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[12]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[13]  Patrick Flynn,et al.  Multi-frame Approaches To Improve Face Recognition , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[14]  David J. Kriegman,et al.  Video-based face recognition using probabilistic appearance manifolds , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Dmitry B. Goldgof,et al.  The scale space aspect graph , 1992, CVPR.

[16]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[17]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..