A multi-task approach to face deblurring

Image deblurring is a foundational problem with numerous application, and the face deblurring subject is one of the most interesting branches. We propose a convolutional neural network (CNN)-based architecture that embraces multi-scale deep features. In this paper, we address the deblurring problems with transfer learning via a multi-task embedding network; the proposed method is effective at restoring more implicit and explicit structures from the blur images. In addition, by introducing perceptual features in the deblurring process and adopting a generative adversarial network, we develop a new method to deblur the face images with reservation of more facial features and details. Extensive experiments compared with state-of-the-art deblurring algorithms demonstrate the effectiveness of the proposed approach.

[1]  Michal Irani,et al.  Blind Deblurring Using Internal Patch Recurrence , 2014, ECCV.

[2]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[3]  Ran He,et al.  Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[6]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[7]  Ling Shao,et al.  Blind Image Blur Estimation via Deep Learning. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[8]  Deqing Sun,et al.  Blind Image Deblurring Using Dark Channel Prior , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  James Hays,et al.  Super-resolution Using Constrained Deep Texture Synthesis , 2017, ArXiv.

[10]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  B. Chaib-draa,et al.  Multi-Task Learning by Deep Collaboration and Application in Facial Landmark Detection , 2017, ArXiv.

[12]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[13]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[14]  Sylvain Paris,et al.  Handling Noise in Single Image Deblurring Using Directional Filters , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[16]  Ming-Hsuan Yang,et al.  Deblurring Face Images with Exemplars , 2014, ECCV.

[17]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jan Kautz,et al.  Deep Semantic Face Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[20]  Sunghyun Cho,et al.  Fast motion deblurring , 2009, SIGGRAPH 2009.

[21]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[23]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Thomas S. Huang,et al.  Interactive Facial Feature Localization , 2012, ECCV.

[25]  Sunghyun Cho,et al.  Edge-based blur kernel estimation using patch priors , 2013, IEEE International Conference on Computational Photography (ICCP).

[26]  Martial Hebert,et al.  Cross-Stitch Networks for Multi-task Learning , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Ming-Hsuan Yang,et al.  Deblurring Text Images via L0-Regularized Intensity and Gradient Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[29]  Rynson W. H. Lau,et al.  Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Giacomo Boracchi,et al.  Modeling the Performance of Image Restoration From Motion Blur , 2012, IEEE Transactions on Image Processing.

[31]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[33]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Bernhard Schölkopf,et al.  Learning to Deblur , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[37]  Ayan Chakrabarti,et al.  A Neural Approach to Blind Motion Deblurring , 2016, ECCV.

[38]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[39]  Ling Shao,et al.  Blind Image Blur Estimation via Deep Learning , 2016, IEEE Transactions on Image Processing.