Iterative Relaxed Collaborative Representation With Adaptive Weights Learning for Noise Robust Face Hallucination

In recent years, the collaborative representation (CR)-based techniques have been widely employed for face hallucination. However, the conventional CR model becomes less efficient in handling noisy low-resolution face images. In this paper, an iterative relaxed CR (iRCR) model with adaptive weights learning is presented to enhance the resolution of face images corrupted by noise. The core idea of iRCR is that a diagonal weight matrix is incorporated into the objective function, which helps to debase the influence of noise in representation. Different from existing collaborative methods with reweighting strategy where the weights require manually tuning, the weights in iRCR are adaptively learned to stay more consistent with the model error. Moreover, considering the local manifold structure property and nonlocal prior of small patches, the locality regularization and collaborative regularization are incorporated into a unified framework. This enables the proposed iRCR not only to capture the true topology structure of patch manifold but also to exploit the meaningful patterns among the whole training samples for reconstruction. Experimental results on both face dataset and real-world images demonstrate the superiority of our proposed method over several state-of-the-art face hallucination methods.

[1]  Reuben A. Farrugia,et al.  Face Hallucination Using Linear Models of Coupled Sparse Support , 2015, IEEE Transactions on Image Processing.

[2]  Rama Chellappa,et al.  Super-Resolution of Face Images Using Kernel PCA-Based Prior , 2007, IEEE Transactions on Multimedia.

[3]  Chun Qi,et al.  Position constraint based face image super-resolution by learning multiple local linear projections , 2015, Signal Process. Image Commun..

[4]  Luc Van Gool,et al.  Adaptive and Weighted Collaborative Representations for image classification , 2014, Pattern Recognit. Lett..

[5]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[6]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[7]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Yu Hu,et al.  From Local Pixel Structure to Global Image Super-Resolution: A New Face Hallucination Framework , 2011, IEEE Transactions on Image Processing.

[9]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Kin-Man Lam,et al.  Face hallucination based on sparse local-pixel structure , 2014, Pattern Recognit..

[11]  Meng Wang,et al.  Visual Classification by ℓ1-Hypergraph Modeling , 2015, IEEE Trans. Knowl. Data Eng..

[12]  Qi Wang,et al.  Locality constraint distance metric learning for traffic congestion detection , 2018, Pattern Recognit..

[13]  C. L. Philip Chen,et al.  Quaternion Locality-Constrained Coding for Color Face Hallucination , 2018, IEEE Transactions on Cybernetics.

[14]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Ruimin Hu,et al.  Noise Robust Face Hallucination via Locality-Constrained Representation , 2014, IEEE Transactions on Multimedia.

[16]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[17]  Luming Zhang,et al.  Unified Photo Enhancement by Discovering Aesthetic Communities From Flickr , 2016, IEEE Transactions on Image Processing.

[18]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[19]  Yuan Yan Tang,et al.  Weighted Joint Sparse Representation for Removing Mixed Noise in Image , 2017, IEEE Transactions on Cybernetics.

[20]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[21]  Cao Ming-min Super Resolution Through Neighbor Embedding with Adaptive Neighbor Selection , 2015 .

[22]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Xuesong Zhang,et al.  An adaptive learning method for face hallucination using Locality Preserving Projections , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[24]  Ning Wu,et al.  Fast Facial Image Super-Resolution via Local Linear Transformations for Resource-Limited Applications , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Junping Zhang,et al.  Super-resolution of human face image using canonical correlation analysis , 2010, Pattern Recognit..

[26]  Yan Song,et al.  Local structure based multi-phase collaborative representation for face recognition with single sample per person , 2016, Inf. Sci..

[27]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[28]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[29]  Yuan Yan Tang,et al.  Mixed Noise Removal via Robust Constrained Sparse Representation , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Qi Wang,et al.  Hyperspectral Image Classification via Multitask Joint Sparse Representation and Stepwise MRF Optimization , 2016, IEEE Transactions on Cybernetics.

[31]  Anders P. Eriksson,et al.  Is face recognition really a Compressive Sensing problem? , 2011, CVPR 2011.

[32]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[33]  Kin-Man Lam,et al.  Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Ruimin Hu,et al.  Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation , 2016, Inf. Sci..

[35]  Meng Wang,et al.  Coherent Semantic-Visual Indexing for Large-Scale Image Retrieval in the Cloud , 2017, IEEE Transactions on Image Processing.

[36]  Ruimin Hu,et al.  Noise robust face hallucination employing Gaussian-Laplacian mixture model , 2014, Neurocomputing.

[37]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[38]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[40]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[41]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[42]  Chun Qi,et al.  Hallucinating face by position-patch , 2010, Pattern Recognit..

[43]  Ruimin Hu,et al.  Face hallucination via Cauchy regularized sparse representation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Jian Yang,et al.  A novel sparse representation based framework for face image super-resolution , 2014, Neurocomputing.

[46]  Maoguo Gong,et al.  Position-Patch Based Face Hallucination Using Convex Optimization , 2011, IEEE Signal Processing Letters.

[47]  David Zhang,et al.  Relaxed collaborative representation for pattern classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Seong-Whan Lee,et al.  An Example-Based Face Hallucination Method for Single-Frame, Low-Resolution Facial Images , 2008, IEEE Transactions on Image Processing.

[49]  D. Donoho For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[50]  Meng Wang,et al.  Unified Video Annotation via Multigraph Learning , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[51]  Yao Lu,et al.  Locality-Constrained Collaborative Model for Robust Visual Tracking , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[52]  Yuan Yan Tang,et al.  Robust Face Hallucination via Locality-Constrained Bi-Layer Representation , 2018, IEEE Transactions on Cybernetics.

[53]  Junjun Jiang,et al.  Noise Robust Face Image Super-Resolution Through Smooth Sparse Representation , 2017, IEEE Transactions on Cybernetics.

[54]  Kin-Man Lam,et al.  A novel face-hallucination scheme based on singular value decomposition , 2013, Pattern Recognit..

[55]  Manjunath V. Joshi,et al.  New Learning Based Super-Resolution: Use of DWT and IGMRF Prior , 2010, IEEE Transactions on Image Processing.

[56]  Yihong Gong,et al.  Nonlinear Learning using Local Coordinate Coding , 2009, NIPS.

[57]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[58]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[59]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.