Extracting a Good Quality Frontal Face Image From a Low-Resolution Video Sequence

Feeding low-resolution and low-quality images, from inexpensive surveillance cameras, to systems like, e.g., face recognition, produces erroneous and unstable results. Therefore, there is a need for a mechanism to bridge the gap between on one hand low-resolution and low-quality images and on the other hand facial analysis systems. The proposed system in this paper deals with exactly this problem. Our approach is to apply a reconstruction-based super-resolution algorithm. Such an algorithm, however, has two main problems: first, it requires relatively similar images with not too much noise and second is that its improvement factor is limited by a factor close to two. To deal with the first problem we introduce a three-step approach, which produces a face-log containing images of similar frontal faces of the highest possible quality. To deal with the second problem, limited improvement factor, we use a learning-based super-resolution algorithm applied to the result of the reconstruction-based part to improve the quality by another factor of two. This results in an improvement factor of four for the entire system. The proposed system has been tested on 122 low-resolution sequences from two different databases. The experimental results show that the proposed system can indeed produce a high-resolution and good quality frontal face image from low-resolution video sequences.

[1]  James L. Crowley,et al.  Head Pose Estimation on Low Resolution Images , 2006, CLEAR.

[2]  Thomas B. Moeslund,et al.  Face Quality Assessment System in Video Sequences , 2008, BIOID.

[3]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[5]  Dmitry O. Gorodnichy,et al.  Video-based framework for face recognition in video , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[6]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[7]  Shiguang Shan,et al.  Aligning Coupled Manifolds for Face Hallucination , 2009, IEEE Signal Processing Letters.

[8]  Christopher M. Bishop,et al.  Bayesian Image Super-Resolution , 2002, NIPS.

[9]  Yücel Altunbasak,et al.  Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants , 2001, IEEE Trans. Image Process..

[10]  Shmuel Peleg,et al.  Two motion-blurred images are better than one , 2005, Pattern Recognit. Lett..

[11]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Thomas B. Moeslund,et al.  Complete face logs for video sequences using face quality measures , 2009 .

[13]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[14]  Dattatraya S. Bormane,et al.  Super Resolution Using Neural Network , 2008, 2008 Second Asia International Conference on Modelling & Simulation (AMS).

[15]  Vivek Bannore,et al.  Iterative-Interpolation Super-Resolution Image Reconstruction - A Computationally Efficient Technique , 2009, Studies in Computational Intelligence.

[16]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[17]  Roberto Cipolla,et al.  A manifold approach to face recognition from low quality video across illumination and pose using implicit super-resolution , 2007, ICCV 2007.

[18]  Yizhen Huang,et al.  Super-resolution using neural networks based on the optimal recovery theory , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[19]  Robert Laganière,et al.  Constructing Face Image Logs that are Both Complete and Concise , 2007, Fourth Canadian Conference on Computer and Robot Vision (CRV '07).

[20]  Francisco de Borja Rodríguez Ortiz,et al.  A two-step neural-network based algorithm for fast image super-resolution , 2007, Image Vis. Comput..

[21]  Zhou Wang,et al.  Multi-scale structural similarity for image quality assessment , 2003 .

[22]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[23]  Liangpei Zhang,et al.  A super-resolution reconstruction algorithm for surveillance images , 2010, Signal Process..

[24]  Xuelong Li,et al.  A multi-frame image super-resolution method , 2010, Signal Process..

[25]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[27]  N. K. Bose,et al.  High resolution image formation from low resolution frames using Delaunay triangulation , 2002, IEEE Trans. Image Process..

[28]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).