Face hallucination based on sparse local-pixel structure

In this paper, we propose a face-hallucination method, namely face hallucination based on sparse local-pixel structure. In our framework, a high resolution (HR) face is estimated from a single frame low resolution (LR) face with the help of the facial dataset. Unlike many existing face-hallucination methods such as the from local-pixel structure to global image super-resolution method (LPS-GIS) and the super-resolution through neighbor embedding, where the prior models are learned by employing the least-square methods, our framework aims to shape the prior model using sparse representation. Then this learned prior model is employed to guide the reconstruction process. Experiments show that our framework is very flexible, and achieves a competitive or even superior performance in terms of both reconstruction error and visual quality. Our method still exhibits an impressive ability to generate plausible HR facial images based on their sparse local structures. Our framework aims to shape the prior model using sparse representation.Global structure and local-pixel structure are incorporated to produce plausible facial details.A method to learn local-pixel structures based on sparse representation is proposed.The proposed method is competitive with other, state-of-the-art face-hallucination methods.

[1]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[2]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[3]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[4]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Jiliu Zhou,et al.  Single Color Image Super-resolution through Neighbor Embedding , 2011 .

[6]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[8]  Kin-Man Lam,et al.  An efficient algorithm for human face detection and facial feature extraction under different conditions , 2001 .

[9]  Guoping Qiu.  Interresolution Look-up Table for Improved Spatial Magnification of Image , 2000, J. Vis. Commun. Image Represent..

[10]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Robert L. Stevenson,et al.  Super-resolution from image sequences-a review , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[12]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[13]  Tara N. Sainath,et al.  Sparse representations for text categorization , 2010, INTERSPEECH.

[14]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[15]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[16]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[17]  Daniel Cremers,et al.  Video Super Resolution Using Duality Based TV-L1 Optical Flow , 2009, DAGM-Symposium.

[18]  Thomas S. Huang,et al.  Face hallucination VIA sparse coding , 2008, 2008 15th IEEE International Conference on Image Processing.

[19]  Seong-Whan Lee,et al.  An Example-Based Face Hallucination Method for Single-Frame, Low-Resolution Facial Images , 2008, IEEE Transactions on Image Processing.

[20]  Kin-Man Lam,et al.  Example-based image super-resolution with class-specific predictors , 2009, J. Vis. Commun. Image Represent..

[21]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[24]  P. Wesseling.  An Introduction to Multigrid Methods , 1992 .

[25]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[26]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[27]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[28]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[29]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[30]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[31]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[32]  Guoping Qiu.  A progressively predictive image pyramid for efficient lossless coding , 1999, IEEE Trans. Image Process..

[33]  Yu Hu,et al.  From Local Pixel Structure to Global Image Super-Resolution: A New Face Hallucination Framework , 2011, IEEE Transactions on Image Processing.

[34]  Kwang In Kim,et al.  Example-Based Learning for Single-Image Super-Resolution , 2008, DAGM-Symposium.

[35]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[36]  Harpreet S. Sawhney,et al.  Is Super-Resolution with Optical Flow Feasible? , 2002, ECCV.

[37]  A. Guitton,et al.  Adaptive subtraction of multiples using the L1‐norm , 2004 .

[38]  Hynek Hermansky,et al.  Sparse auto-associative neural networks: theory and application to speech recognition , 2010, INTERSPEECH.

[39]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[40]  Jian-Huang Lai,et al.  A Survey of Face Hallucination , 2012, CCBR.

[41]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[42]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[43]  Haibin Ling,et al.  Sparse representation of cast shadows via ℓ1-regularized least squares , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[44]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).