3D Model-Based Face Recognition in Video

Face recognition in video has gained wide attention due to its role in designing surveillance systems. One of the main advantages of video over still frames is that evidence accumulation over multiple frames can provide better face recognition performance. However, surveillance videos are generally of low resolution containing faces mostly in non-frontal poses. Consequently, face recognition in video poses serious challenges to state-of-the-art face recognition systems. Use of 3D face models has been suggested as a way to compensate for low resolution, poor contrast and non-frontal pose. We propose to overcome the pose problem by automatically (i) reconstructing a 3D face model from multiple non-frontal frames in a video, (ii) generating a frontal view from the derived 3D model, and (iii) using a commercial 2D face recognition engine to recognize the synthesized frontal view. A factorization-based structure from motion algorithm is used for 3D face reconstruction. The proposed scheme has been tested on CMU's Face In Action (FIA) video database with 221 subjects. Experimental results show a 40% improvement in matching performance as a result of using the 3D models.

[1]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[2]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[3]  Thomas S. Huang,et al.  Accurate Head Pose Tracking in Low Resolution Video , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[4]  Jiří Matas,et al.  Computer Vision - ECCV 2004 , 2004, Lecture Notes in Computer Science.

[5]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[6]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Mikkel B. Stegmann,et al.  The AAM-API: An Open Source Active Appearance Model Implementation , 2003, MICCAI.

[8]  Terry M. Peters,et al.  Medical Image Computing and Computer-Assisted Intervention - MICCAI 2003 , 2003, Lecture Notes in Computer Science.

[9]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[10]  Rama Chellappa,et al.  SFS based view synthesis for robust face recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[11]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[12]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[13]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Patrick J. Flynn,et al.  Preliminary Face Recognition Grand Challenge Results , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[15]  Matthew Brand,et al.  A direct method for 3D factorization of nonrigid motion observed in 2D , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Jing Xiao,et al.  A Closed-Form Solution to Non-rigid Shape and Motion Recovery , 2004, ECCV.

[17]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[18]  Wen Gao,et al.  Local Linear Regression (LLR) for Pose Invariant Face Recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[19]  David J. Kriegman,et al.  Video-based face recognition using probabilistic appearance manifolds , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[20]  Shaogang Gong,et al.  Analysis and Modelling of Faces and Gestures , 2008 .

[21]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[22]  Tsuhan Chen,et al.  The CMU Face In Action (FIA) Database , 2005, AMFG.