Face Hallucination Using Manifold-Regularized Group Locality-Constrained Representation

Sparsity and locality regularizations are successfully applied to face hallucination algorithms to ameliorate their ill-posed nature. However, most of patch-based face hallucination approaches only consider the manifold structure of single patch, thus resulting in unstable solution for image reconstruction. In this paper, we propose a novel face hallucination, termed manifold-regularized group locality-constrained representation (MGLR), in order to exploit the multiple manifold structures rooted in grouped self-similarly patches. Specifically, we first group similar patches to form a matrix which contains the recurrent non-local patches. Then graph regularization term is formulated to represent the group manifolds for better reconstruction quality. Taking advantages of grouped self-similar patches, MGLR can offer stable sparse solution to take advantage of the the accurate prior for super-resolution reconstruction. Experimental results on LFW database and CMU real-world images demonstrate the superiority of the proposed method over some state-of-the-art face methods both in terms of subjective and objective qualities.

[1]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[2]  Richard H. Bartels,et al.  Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4] , 1972, Commun. ACM.

[3]  Zixiang Xiong,et al.  DLML: Deep linear mappings learning for face super-resolution with nonlocal-patch , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[4]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[5]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  相原 龍,et al.  Alternating Direction Method of Multipliersを用いた声質変換のためのパラレル辞書学習 (音声) -- (第17回音声言語シンポジウム) , 2015 .

[7]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[8]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[10]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[11]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Arindam Banerjee,et al.  Bregman Alternating Direction Method of Multipliers , 2013, NIPS.

[13]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Ruimin Hu,et al.  Locality-constraint iterative neighbor embedding for face hallucination , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[16]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[17]  Tao Lu,et al.  Robust Face Super-Resolution via Locality-Constrained Low-Rank Representation , 2017, IEEE Access.

[18]  Hao Wang,et al.  Face hallucination using region-based deep convolutional networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[19]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[20]  G. Golub,et al.  A Hessenberg-Schur method for the problem AX + XB= C , 1979 .

[21]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Yi Yu,et al.  Context-patch based face hallucination via thresholding locality-constrained representation and reproducing learning , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[23]  Maoguo Gong,et al.  Position-Patch Based Face Hallucination Using Convex Optimization , 2011, IEEE Signal Processing Letters.

[24]  Michael Elad,et al.  Con-Patch: When a Patch Meets Its Context , 2016, IEEE Transactions on Image Processing.

[25]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[26]  Ruimin Hu,et al.  Position-Patch Based Face Hallucination via Locality-Constrained Representation , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[27]  Dong Yue,et al.  Locality-Constrained Double Low-Rank Representation for Effective Face Hallucination , 2016, IEEE Access.

[28]  Chun Qi,et al.  Hallucinating face by position-patch , 2010, Pattern Recognit..