A manifold approach to face recognition from low quality video across illumination and pose using implicit super-resolution

We consider the problem of matching a face in a low resolution query video sequence against a set of higher quality gallery sequences. This problem is of interest in many applications, such as law enforcement. Our main contribution is an extension of the recently proposed Generic Shape-Illumination Manifold (gSIM) framework. Specifically, (i) we show how super-resolution across pose and scale can be achieved implicitly, by off-line learning of subsampling artefacts; (ii) we use this result to propose an extension to the statistical model of the gSIM by compounding it with a hierarchy of subsampling models at multiple scales; and (iii) we describe an extensive empirical evaluation of the method on over 1300 video sequences – we first measure the degradation in performance of the original gSIM algorithm as query sequence resolution is decreased and then show that the proposed extension produces an error reduction in the mean recognition rate of over 50%.

[1]  Yücel Altunbasak,et al.  Eigenface-domain super-resolution for face recognition , 2003, IEEE Trans. Image Process..

[2]  Tomaso A. Poggio,et al.  People recognition and pose estimation in image sequences , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[3]  Larry S. Davis,et al.  Person identification using automatic height and stride estimation , 2002, Object recognition supported by user interaction for service robots.

[4]  James J. Clark,et al.  A transformation method for the reconstruction of functions from nonuniformly spaced samples , 1985, IEEE Trans. Acoust. Speech Signal Process..

[5]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  Trevor Darrell,et al.  Integrated face and gait recognition from multiple views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Michael I. Jordan,et al.  Mixtures of Probabilistic Principal Component Analyzers , 2001 .

[8]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[9]  M. Nixon,et al.  People Detection and Recognition using Gait for Automated Visual Surveillance , 2006 .

[10]  Alex Pentland,et al.  Human Face Recognition and the Face Image Set's Topology , 1994 .

[11]  Haitao Wang,et al.  Face recognition under varying lighting conditions using self quotient image , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[12]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[13]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[14]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[15]  A. Murat Tekalp,et al.  High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  Shaogang Gong,et al.  Multi-Resolution Patch Tensor for Facial Expression Hallucination , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  V. Chandran,et al.  Investigation into Optical Flow Super-Resolution for Surveillance Applications , 2005 .

[18]  Kiyoharu Aizawa,et al.  Signal-processing based method for acquiring very high resolution images with multiple cameras and its theoretical analysis , 1992 .

[19]  Alex Po Leung,et al.  Coupling Face Registration and Super-Resolution , 2006, BMVC.

[20]  Yongyi Yang,et al.  Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics , 1998 .

[21]  Andrew Zisserman,et al.  Super-resolution from multiple views using learnt image models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  Roberto Cipolla,et al.  Face Recognition from Video Using the Generic Shape-Illumination Manifold , 2006, ECCV.

[23]  M. Ibrahim Sezan,et al.  An overview of convex projections theory and its application to image recovery problems , 1992 .

[24]  Harpreet S. Sawhney,et al.  Is Super-Resolution with Optical Flow Feasible? , 2002, ECCV.

[25]  Yucel Altunbasak,et al.  Face recognition with independent component-based super-resolution , 2006, Electronic Imaging.

[26]  Seong G. Kong,et al.  Recent advances in visual and infrared face recognition - a review , 2005, Comput. Vis. Image Underst..