An Example-Based Face Hallucination Method for Single-Frame, Low-Resolution Facial Images

This paper proposes a face hallucination method for the reconstruction of high-resolution facial images from single-frame, low-resolution facial images. The proposed method has been derived from example-based hallucination methods and morphable face models. First, we propose a recursive error back-projection method to compensate for residual errors, and a region-based reconstruction method to preserve characteristics of local facial regions. Then, we define an extended morphable face model, in which an extended face is composed of the interpolated high-resolution face from a given low-resolution face, and its original high-resolution equivalent. Then, the extended face is separated into an extended shape and an extended texture. We performed various hallucination experiments using the MPI, XM2VTS, and KF databases, compared the reconstruction errors, structural similarity index, and recognition rates, and showed the effects of face detection errors and shape estimation errors. The encouraging results demonstrate that the proposed methods can improve the performance of face recognition systems. Especially the proposed method can enhance the resolution of single-frame, low-resolution facial images.

[1]  Aggelos K. Katsaggelos,et al.  Resolution enhancement of monochrome and color video using motion compensation , 2001, IEEE Trans. Image Process..

[3]  Seong-Whan Lee,et al.  Reconstruction of Partially Damaged Face Images Based on a Morphable Face Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Nikolaos G. Bourbakis,et al.  A survey of skin-color modeling and detection methods , 2007, Pattern Recognit..

[5]  Tsuhan Chen,et al.  Adaptive Markov Random Fields for Example-Based Super-resolution of Faces , 2006, EURASIP J. Adv. Signal Process..

[6]  Dahua Lin,et al.  Neighbor combination and transformation for hallucinating faces , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[7]  Nikolaus F. Troje,et al.  Separation of texture and shape in images of faces for image coding and synthesis , 1997 .

[8]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Dahua Lin,et al.  Hallucinating faces: TensorPatch super-resolution and coupled residue compensation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[13]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Seong-Whan Lee,et al.  Performance Analysis of Face Recognition Algorithms on Korean Face Database , 2007, Int. J. Pattern Recognit. Artif. Intell..

[15]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[16]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[18]  Yücel Altunbasak,et al.  Eigenface-domain super-resolution for face recognition , 2003, IEEE Trans. Image Process..

[19]  Seong-Whan Lee,et al.  Resolution enhancement of facial image using an error back-projection of example-based learning , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[20]  Yang Li,et al.  Face hallucination with pose variation , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[21]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[22]  Patrick Bouthemy,et al.  Restoration of Noisy, Blurred, Undersampled Image Sequences Using a Parametric Motion Model , 2000 .

[23]  Wei Liu,et al.  Bayesian Tensor Inference for Sketch-Based Facial Photo Hallucination , 2007, IJCAI.

[24]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Yang Li,et al.  An improved two-step approach to hallucinating faces , 2004, Third International Conference on Image and Graphics (ICIG'04).

[26]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[27]  Takeo Kanade,et al.  High-zoom video hallucination by exploiting spatio-temporal regularities , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..