From Local Pixel Structure to Global Image Super-Resolution: A New Face Hallucination Framework

We have developed a new face hallucination framework termed from local pixel structure to global image super-resolution (LPS-GIS). Based on the assumption that two similar face images should have similar local pixel structures, the new framework first uses the input low-resolution (LR) face image to search a face database for similar example high-resolution (HR) faces in order to learn the local pixel structures for the target HR face. It then uses the input LR face and the learned pixel structures as priors to estimate the target HR face. We present a three-step implementation procedure for the framework. Step 1 searches the database for example faces that are the most similar to the input, and then warps the example images to the input using optical flow. Step 2 uses the warped HR version of the example faces to learn the local pixel structures for the target HR face. An effective method for learning local pixel structures from an individual face, and an adaptive procedure for fusing the local pixel structures of different example faces to reduce the influence of warping errors, have been developed. Step 3 estimates the target HR face by solving a constrained optimization problem by means of an iterative procedure. Experimental results show that our new method can provide good performances for face hallucination, both in terms of reconstruction error and visual quality; and that it is competitive with existing state-of-the-art methods.

[1]  Guoping Qiu.  Interresolution Look-up Table for Improved Spatial Magnification of Image , 2000, J. Vis. Commun. Image Represent..

[2]  Jian Guan,et al.  Color by linear neighborhood embedding , 2005, IEEE International Conference on Image Processing 2005.

[3]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[4]  Kin-Man Lam,et al.  Example-based image super-resolution with class-specific predictors , 2009, J. Vis. Commun. Image Represent..

[5]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[7]  Bryan S. Morse,et al.  Image magnification using level-set reconstruction , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Long Chen.  INTRODUCTION TO MULTIGRID METHODS , 2005 .

[9]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[10]  Kin-Man Lam,et al.  An efficient algorithm for human face detection and facial feature extraction under different conditions , 2001 .

[11]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[12]  Nikolaus F. Troje,et al.  Separation of texture and shape in images of faces for image coding and synthesis , 1997 .

[13]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[14]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[15]  Kin-Man Lam,et al.  An efficient illumination normalization method for face recognition , 2006, Pattern Recognit. Lett..

[16]  Guoping Qiu.  A progressively predictive image pyramid for efficient lossless coding , 1999, IEEE Trans. Image Process..

[17]  Kwang In Kim,et al.  Example-Based Learning for Single-Image Super-Resolution , 2008, DAGM-Symposium.

[18]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[20]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Takeo Kanade,et al.  Super-Resolution Optical Flow , 1999 .

[22]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[23]  Seong-Whan Lee,et al.  An Example-Based Face Hallucination Method for Single-Frame, Low-Resolution Facial Images , 2008, IEEE Transactions on Image Processing.

[24]  Nanning Zheng,et al.  Image hallucination with primal sketch priors , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Dit-Yan Yeung,et al.  Image Hallucination Using Neighbor Embedding over Visual Primitive Manifolds , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Shiguang Shan,et al.  Locality preserving constraints for super-resolution with neighbor embedding , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[27]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.