SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior

The performance of traditional face recognition systems is sharply reduced when encountered with a low-resolution (LR) probe face image. To obtain much more detailed facial features, some face super-resolution (SR) methods have been proposed in the past decade. The basic idea of a face image SR is to generate a high-resolution (HR) face image from an LR one with the help of a set of training examples. It aims at transcending the limitations of optical imaging systems. In this paper, we regard face image SR as an image interpolation problem for domain-specific images. A missing intensity interpolation method based on smooth regression with a local structure prior (LSP), named SRLSP for short, is presented. In order to interpolate the missing intensities in a target HR image, we assume that face image patches at the same position share similar local structures, and use smooth regression to learn the relationship between LR pixels and missing HR pixels of one position patch. Performance comparison with the state-of-the-art SR algorithms on two public face databases and some real-world images shows the effectiveness of the proposed method for a face image SR in general. In addition, we conduct a face recognition experiment on the extended Yale-B face database based on the super-resolved HR faces. Experimental results clearly validate the advantages of our proposed SR method over the state-of-the-art SR methods in face recognition application.

[1]  Yu-Chiang Frank Wang,et al.  Undersampled Face Recognition via Robust Auxiliary Dictionary Learning , 2015, IEEE Transactions on Image Processing.

[2]  Ke Lu,et al.  Compressed Sensing of a Remote Sensing Image Based on the Priors of the Reference Image , 2015, IEEE Geoscience and Remote Sensing Letters.

[3]  Chun Qi,et al.  Hallucinating face by position-patch , 2010, Pattern Recognit..

[4]  Wenhan Yang,et al.  Image Super-Resolution Based on Structure-Modulated Sparse Representation , 2015, IEEE Transactions on Image Processing.

[5]  Marios Savvides,et al.  Breaking the Limitation of Manifold Analysis for Super-Resolution of Facial Images , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[6]  Bir Bhanu,et al.  Face image super-resolution using 2D CCA , 2014, Signal Process..

[7]  Yuan Yan Tang,et al.  Weighted Joint Sparse Representation for Removing Mixed Noise in Image , 2017, IEEE Transactions on Cybernetics.

[8]  Samee U. Khan,et al.  Fast and Scalable Multiway Analysis of Neural Data , 2013 .

[9]  Chih-Yuan Yang,et al.  Single-Image Super-Resolution: A Benchmark , 2014, ECCV.

[10]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shiguang Shan,et al.  Low-Resolution Face Recognition via Coupled Locality Preserving Mappings , 2010, IEEE Signal Processing Letters.

[12]  Xueming Qian,et al.  Robust Framework of Single-Frame Face Superresolution Across Head Pose, Facial Expression, and Illumination Variations , 2015, IEEE Transactions on Human-Machine Systems.

[13]  Yi Yao,et al.  Improving long range and high magnification face recognition: Database acquisition, evaluation, and enhancement , 2008, Comput. Vis. Image Underst..

[14]  Xiaoli Li,et al.  Brain big data processing with massively parallel computing technology: challenges and opportunities , 2017, Softw. Pract. Exp..

[15]  Jie Xu,et al.  Coupled fisher discrimination dictionary learning for single image super-resolution , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[17]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[18]  Yu Hu,et al.  From Local Pixel Structure to Global Image Super-Resolution: A New Face Hallucination Framework , 2011, IEEE Transactions on Image Processing.

[19]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[20]  Q. M. Jonathan Wu,et al.  Low-resolution face recognition: a review , 2013, The Visual Computer.

[21]  Yueting Zhuang,et al.  Hallucinating faces: LPH super-resolution and neighbor reconstruction for residue compensation , 2007, Pattern Recognit..

[22]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[23]  Ling Li,et al.  Face hallucination: How much it can improve face recognition , 2013, 2013 Australian Control Conference.

[24]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[25]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  C. Thomaz,et al.  A new ranking method for principal components analysis and its application to face image analysis , 2010, Image Vis. Comput..

[27]  Maoguo Gong,et al.  Position-Patch Based Face Hallucination Using Convex Optimization , 2011, IEEE Signal Processing Letters.

[28]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Lizhe Wang,et al.  Fast and Scalable Multi-Way Analysis of Massive Neural Data , 2015, IEEE Transactions on Computers.

[30]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[32]  Ruimin Hu,et al.  Noise Robust Face Hallucination via Locality-Constrained Representation , 2014, IEEE Transactions on Multimedia.

[33]  Junjun Jiang,et al.  Noise Robust Face Image Super-Resolution Through Smooth Sparse Representation , 2017, IEEE Transactions on Cybernetics.

[34]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Kun Li,et al.  Video super-resolution using an adaptive superpixel-guided auto-regressive model , 2016, Pattern Recognit..

[36]  Yu-Chiang Frank Wang,et al.  Recognition at a long distance: Very low resolution face recognition and hallucination , 2015, 2015 International Conference on Biometrics (ICB).

[37]  Ruimin Hu,et al.  Face hallucination with shape parameters projection constraint , 2010, ACM Multimedia.

[38]  Ruimin Hu,et al.  Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation , 2016, Inf. Sci..

[39]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[40]  Zhiliang Zhu,et al.  Fast Single Image Super-Resolution via Self-Example Learning and Sparse Representation , 2014, IEEE Transactions on Multimedia.

[41]  Ching-Ting Tu,et al.  Robust face hallucination using ensemble of feature-based regression functions and classifiers , 2015, Image Vis. Comput..

[42]  Jican Fu,et al.  Rigid Regression for Facial Image Interpolation with Local Structure Prior , 2014, 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics.

[43]  Zhuowen Tu,et al.  Regularized vector field learning with sparse approximation for mismatch removal , 2013, Pattern Recognit..

[44]  Nicu Sebe,et al.  Neighborhood issue in single-frame image super-resolution , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[45]  Pablo H. Hennings-Yeomans,et al.  Simultaneous super-resolution and feature extraction for recognition of low-resolution faces , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[47]  Zheng Wang,et al.  Zero-Shot Person Re-identification via Cross-View Consistency , 2016, IEEE Transactions on Multimedia.

[48]  Seong-Whan Lee,et al.  An Example-Based Face Hallucination Method for Single-Frame, Low-Resolution Facial Images , 2008, IEEE Transactions on Image Processing.

[49]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[50]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[51]  Wai-kuen Cham,et al.  Hallucinating Face in the DCT Domain , 2011, IEEE Transactions on Image Processing.

[52]  Jian Yang,et al.  A novel sparse representation based framework for face image super-resolution , 2014, Neurocomputing.

[53]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[54]  Rama Chellappa,et al.  Super-Resolution of Face Images Using Kernel PCA-Based Prior , 2007, IEEE Transactions on Multimedia.

[55]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Xuesong Zhang,et al.  An adaptive learning method for face hallucination using Locality Preserving Projections , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[57]  Yuan Yan Tang,et al.  Weighted Couple Sparse Representation With Classified Regularization for Impulse Noise Removal , 2015, IEEE Transactions on Image Processing.

[58]  Hayit Greenspan,et al.  Super-Resolution in Medical Imaging , 2009, Comput. J..

[59]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[60]  Ning Wu,et al.  Fast Facial Image Super-Resolution via Local Linear Transformations for Resource-Limited Applications , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[61]  Junping Zhang,et al.  Super-resolution of human face image using canonical correlation analysis , 2010, Pattern Recognit..

[62]  Kin-Man Lam,et al.  Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[63]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[64]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Kun Li,et al.  Video super-resolution based on automatic key-frame selection and feature-guided variational optical flow , 2014, Signal Process. Image Commun..

[66]  Ruimin Hu,et al.  CDMMA: Coupled discriminant multi-manifold analysis for matching low-resolution face images , 2016, Signal Process..

[67]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[68]  Yu-Chiang Frank Wang,et al.  Robust Face Recognition With Structurally Incoherent Low-Rank Matrix Decomposition , 2014, IEEE Transactions on Image Processing.

[69]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[70]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, BTAS.

[71]  Lizhe Wang,et al.  Reference Information Based Remote Sensing Image Reconstruction with Generalized Nonconvex Low-Rank Approximation , 2016, Remote. Sens..

[72]  Alan L. Yuille,et al.  Non-Rigid Point Set Registration by Preserving Global and Local Structures , 2016, IEEE Transactions on Image Processing.

[73]  Bir Bhanu,et al.  Image super-resolution by extreme learning machine , 2012, 2012 19th IEEE International Conference on Image Processing.

[74]  Wen Gao,et al.  The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.