Face Hallucination via Gradient Constrained Sparse Representation

Face hallucination is an example of the image super-resolution problem, where the higher resolution face images can be obtained from the lower resolution ones. Many methods based on the sparse representation have been proposed to solve this problem. These methods use the position-patch strategy, which divides the input image into several small patches and represents each patch by the patches at the same position in the training set. An effective image prior is critical to improve the quality of the estimated super-resolution images. Thus, we try to exploit the gradient information during the patch representation to achieve better hallucination result. In this paper, we propose a novel face hallucination model based on the sparse representation, called iterative gradient constrained weighted sparse representation method. Our model incorporates both the gradient information of the images and the $l_{1}$ reweighted constraint into the sparse representation to achieve better performance. An iterative algorithm is proposed to refine these reconstructed high-resolution images. The experiments on several face databases show the better performance of our algorithm compared with other baseline algorithms.

[1]  H Stark,et al.  High-resolution image recovery from image-plane arrays, using convex projections. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[2]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Yihong Gong,et al.  Nonlinear Learning using Local Coordinate Coding , 2009, NIPS.

[4]  Wenhan Yang,et al.  Image Super-Resolution Based on Structure-Modulated Sparse Representation , 2015, IEEE Transactions on Image Processing.

[5]  Harry Shum,et al.  Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement , 2011, IEEE Transactions on Image Processing.

[6]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Truong Q. Nguyen,et al.  Single Image Superresolution Based on Gradient Profile Sharpness , 2015, IEEE Transactions on Image Processing.

[8]  Stéphane Mallat,et al.  Super-Resolution With Sparse Mixing Estimators , 2010, IEEE Transactions on Image Processing.

[9]  Jian Yang,et al.  A novel sparse representation based framework for face image super-resolution , 2014, Neurocomputing.

[10]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[11]  Gaofeng Meng,et al.  Edge-Directed Single-Image Super-Resolution Via Adaptive Gradient Magnitude Self-Interpolation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[13]  Ruimin Hu,et al.  Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation , 2016, Inf. Sci..

[14]  Zhiliang Zhu,et al.  Fast Single Image Super-Resolution via Self-Example Learning and Sparse Representation , 2014, IEEE Transactions on Multimedia.

[15]  Junjun Jiang,et al.  Noise Robust Face Image Super-Resolution Through Smooth Sparse Representation , 2017, IEEE Transactions on Cybernetics.

[16]  Hsieh Hou,et al.  Cubic splines for image interpolation and digital filtering , 1978 .

[17]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[18]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[19]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[21]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[22]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[23]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Nanning Zheng,et al.  Image hallucination with primal sketch priors , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[26]  Michael Elad,et al.  Superresolution restoration of an image sequence: adaptive filtering approach , 1999, IEEE Trans. Image Process..

[27]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[28]  D. Donoho For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[29]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[30]  C. Thomaz,et al.  A new ranking method for principal components analysis and its application to face image analysis , 2010, Image Vis. Comput..

[31]  Maoguo Gong,et al.  Position-Patch Based Face Hallucination Using Convex Optimization , 2011, IEEE Signal Processing Letters.

[32]  Ruimin Hu,et al.  Noise Robust Face Hallucination via Locality-Constrained Representation , 2014, IEEE Transactions on Multimedia.

[33]  Ruimin Hu,et al.  Position-Patch Based Face Hallucination via Locality-Constrained Representation , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[34]  Chun Qi,et al.  Hallucinating face by position-patch , 2010, Pattern Recognit..

[35]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.