Multi-Frame Image Restoration for Face Recognition

Face recognition at a distance is a challenging and important law-enforcement surveillance problem, with low image resolution and blur contributing to the difficulties. We present a method for combining a sequence of video frames of a subject in order to create a restored image of the face with reduced blur. A generic Active Appearance Model of face shape and appearance is used for registration. By warping and averaging registered video frames, noise is reduced, allowing a Wiener filter to deblur the face to a greater degree than can be achieved on a single video frame. This process is theoretically justified and tested with real-world outdoor video using a PTZ camera and a commercial face recognition engine. Improvement is demonstrated for both face recognition and authentication.

[1]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[2]  Timothy F. Cootes,et al.  A Trainable Method of Parametric Shape Description , 1991, BMVC.

[3]  Patrick J. Flynn,et al.  Face Recognition Using 2D and 3D Facial Data , 2003 .

[4]  Takeo Kanade,et al.  Super-Resolution Optical Flow , 1999 .

[5]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[6]  Patrick J. Flynn,et al.  Assessment of Time Dependency in Face Recognition: An Initial Study , 2003, AVBPA.

[7]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Henry Schneiderman,et al.  Learning a restricted Bayesian network for object detection , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[10]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[11]  Peter H. Tu,et al.  Face Model Fitting on Low Resolution Images , 2006, BMVC.

[12]  anonymous,et al.  Evaluation Report , 1990, The Aboriginal Child at School.

[13]  Timothy F. Cootes,et al.  Trainable method of parametric shape description , 1992, Image Vis. Comput..

[14]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.