Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation

A novel position-patch face super-resolution method for face super-resolution is developed.We developed a novel patch representation method based on Tikhonov regularized neighbor representation.The proposed patch representation model achieves sparsity and locality simultaneously.The effectiveness of our algorithms was shown on public face databases and some real-world images. In human-machine interaction, human face is one of the core factors. However, due to the limitations of imaging conditions and low-cost imaging sensors, the captured faces are often low-resolution (LR). This will seriously degrade the performance of face detection, expression analysis, and face recognition, which are the basic problems in human-machine interaction applications. Face super-resolution (SR) is the technology of inducing a high-resolution (HR) face from the observed LR one. It has been a hot topic of wide concern recently. In this paper, we present a novel face SR method based on Tikhonov regularized neighbor representation (TRNR). It can overcome the technological bottlenecks (e.g., instable solutionand noise sensitive) of the patch representationscheme in traditional neighbor embedding based image SR methods. Specifically, we introduce the Tikhonov regularization term to regularize the representation of the observation LR patches, leading to a unique and stable solution for the least squares problem. Furthermore, we show a connection of the proposed model to the neighbor embedding model, least squares representation,sparse representation, and locality-constrained representation. Extensive experiments on face SR are carried out to validate the generality, effectiveness, and robustness of the proposed algorithm. Experimental results on the public FEI face database and surveillance images show that the proposed method achieves better performance in terms of reconstruction error and visual quality than existing state-of-the-art methods.

[1]  Zhuowen Tu,et al.  Robust $L_{2}E$ Estimation of Transformation for Non-Rigid Registration , 2015, IEEE Transactions on Signal Processing.

[2]  Yihong Gong,et al.  Nonlinear Learning using Local Coordinate Coding , 2009, NIPS.

[3]  Yuan Yan Tang,et al.  High-Order Distance-Based Multiview Stochastic Learning in Image Classification , 2014, IEEE Transactions on Cybernetics.

[4]  Chun Qi,et al.  Hallucinating face by position-patch , 2010, Pattern Recognit..

[5]  Jianhuang Lai,et al.  Face Hallucination Based on Morphological Component Analysis. Signal Processing , 2012 .

[6]  Kun Li,et al.  Video super-resolution based on automatic key-frame selection and feature-guided variational optical flow , 2014, Signal Process. Image Commun..

[7]  Meng Wang,et al.  Image clustering based on sparse patch alignment framework , 2014, Pattern Recognit..

[8]  Ruimin Hu,et al.  CDMMA: Coupled discriminant multi-manifold analysis for matching low-resolution face images , 2016, Signal Process..

[9]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[10]  C. Thomaz,et al.  A new ranking method for principal components analysis and its application to face image analysis , 2010, Image Vis. Comput..

[11]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[13]  Jun Yu,et al.  Multi-view ensemble manifold regularization for 3D object recognition , 2015, Inf. Sci..

[14]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[15]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[16]  Ruimin Hu,et al.  Noise Robust Face Hallucination via Locality-Constrained Representation , 2014, IEEE Transactions on Multimedia.

[17]  Weiguo Gong,et al.  Combining sparse representation and local rank constraint for single image super resolution , 2015, Inf. Sci..

[18]  Seong-Whan Lee,et al.  An Example-Based Face Hallucination Method for Single-Frame, Low-Resolution Facial Images , 2008, IEEE Transactions on Image Processing.

[19]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[20]  Yu Hu,et al.  From Local Pixel Structure to Global Image Super-Resolution: A New Face Hallucination Framework , 2011, IEEE Transactions on Image Processing.

[21]  Wai-kuen Cham,et al.  Hallucinating Face in the DCT Domain , 2011, IEEE Transactions on Image Processing.

[22]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[23]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Kun Li,et al.  Nonrigid Structure From Motion via Sparse Representation , 2015, IEEE Transactions on Cybernetics.

[25]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[26]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[27]  Kun Li,et al.  Non-rigid structure from motion via sparse representation , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[28]  Bir Bhanu,et al.  Face image super-resolution using 2D CCA , 2014, Signal Process..

[29]  Chen Chen,et al.  Spectral–Spatial Preprocessing Using Multihypothesis Prediction for Noise-Robust Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[30]  Kun Li,et al.  Video super-resolution using an adaptive superpixel-guided auto-regressive model , 2016, Pattern Recognit..

[31]  Xiang Ma,et al.  Sparse Support Regression for Image Super-Resolution , 2015, IEEE Photonics Journal.

[32]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Rama Chellappa,et al.  Super-Resolution of Face Images Using Kernel PCA-Based Prior , 2007, IEEE Transactions on Multimedia.

[34]  YouJane,et al.  Multi-view ensemble manifold regularization for 3D object recognition , 2015 .

[35]  Ashraf A. Kassim,et al.  Gini Index as Sparsity Measure for Signal Reconstruction from Compressive Samples , 2011, IEEE Journal of Selected Topics in Signal Processing.

[36]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[37]  Chen Chen,et al.  Compressed-sensing recovery of images and video using multihypothesis predictions , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[38]  Yu Qiao,et al.  Boosting Optical Character Recognition: A Super-Resolution Approach , 2015, ArXiv.

[39]  Chun Qi,et al.  Global consistency, local sparsity and pixel correlation: A unified framework for face hallucination , 2014, Pattern Recognit..

[40]  Jun Yu,et al.  Coupled Deep Autoencoder for Single Image Super-Resolution , 2017, IEEE Transactions on Cybernetics.

[41]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[42]  Manjunath V. Joshi,et al.  New Learning Based Super-Resolution: Use of DWT and IGMRF Prior , 2010, IEEE Transactions on Image Processing.

[43]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[44]  Tuomas Virtanen,et al.  Exemplar-Based Sparse Representations for Noise Robust Automatic Speech Recognition , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[45]  Jing Li,et al.  A Simplified Method of Making Flexible Blue LEDs on a Plastic Substrate , 2015, IEEE Photonics Journal.

[46]  Junping Zhang,et al.  Super-resolution of human face image using canonical correlation analysis , 2010, Pattern Recognit..

[47]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[48]  Weiguo Gong,et al.  Dual-sparsity regularized sparse representation for single image super-resolution , 2015, Inf. Sci..

[49]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[50]  Jian-Huang Lai,et al.  Face hallucination based on morphological component analysis , 2013, Signal Process..

[51]  Gene H. Golub,et al.  Tikhonov Regularization and Total Least Squares , 1999, SIAM J. Matrix Anal. Appl..

[52]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.