Defocus and Motion Blur Detection with Deep Contextual Features

We propose a novel approach for detecting two kinds of partial blur, defocus and motion blur, by training a deep convolutional neural network. Existing blur detection methods concentrate on designing low‐level features, but those features have difficulty in detecting blur in homogeneous regions without enough textures or edges. To handle such regions, we propose a deep encoder‐decoder network with long residual skip‐connections and multi‐scale reconstruction loss functions to exploit high‐level contextual features as well as low‐level structural features. Another difficulty in partial blur detection is that there are no available datasets with images having both defocus and motion blur together, as most existing approaches concentrate only on either defocus or motion blur. To resolve this issue, we construct a synthetic dataset that consists of complex scenes with both types of blur. Experimental results show that our approach effectively detects and classifies blur, outperforming other state‐of‐the‐art methods. Our method can be used for various applications, such as photo editing, blur magnification, and deblurring.

[1]  Abdul Ghafoor,et al.  A hybrid defocused region segmentation approach using image matting , 2019, Multidimens. Syst. Signal Process..

[2]  Anat Levin,et al.  Accurate Blur Models vs. Image Priors in Single Image Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Guillermo Sapiro,et al.  Deep Video Deblurring for Hand-Held Cameras , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[5]  Seungyong Lee,et al.  Video deblurring for hand-held cameras using patch-based synthesis , 2012, ACM Trans. Graph..

[6]  Jizhou Sun,et al.  Multiscale blur detection by learning discriminative deep features , 2018, Neurocomputing.

[7]  Pichao Wang,et al.  A Spectral and Spatial Approach of Coarse-to-Fine Blurred Image Region Detection , 2016, IEEE Signal Processing Letters.

[8]  Jun Yu,et al.  Coupled Deep Autoencoder for Single Image Super-Resolution , 2017, IEEE Transactions on Cybernetics.

[9]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  William T. Freeman,et al.  Analyzing spatially-varying blur , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  한보형,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015 .

[12]  Nancy Argüelles,et al.  Author ' s , 2008 .

[13]  Terence Sim,et al.  Defocus map estimation from a single image , 2011, Pattern Recognit..

[14]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[15]  Lina J. Karam,et al.  Spatially-Varying Blur Detection Based on Multiscale Fused and Sorted Transform Coefficients of Gradient Magnitudes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Tae Hyun Kim,et al.  Segmentation-Free Dynamic Scene Deblurring , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Shijian Lu,et al.  Blurred image region detection and classification , 2011, ACM Multimedia.

[20]  Andrew Zisserman,et al.  Deblurring shaken and partially saturated images , 2011, ICCV Workshops.

[21]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[22]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[23]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[24]  In-So Kweon,et al.  A Unified Approach of Multi-scale Deep and Hand-Crafted Features for Defocus Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ian D. Reid,et al.  RefineNet : MultiPath Refinement Networks with Identity Mappings for High-Resolution Semantic Segmentation , 2016 .

[26]  Hua Huang,et al.  Bundled Kernels for Nonuniform Blind Video Deblurring , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Jianping Shi,et al.  Just noticeable defocus blur detection and estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ming-Hsuan Yang,et al.  Deblurring Text Images via L0-Regularized Intensity and Gradient Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Jean Ponce,et al.  Learning to Estimate and Remove Non-uniform Image Blur , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[31]  Tien Tsin,et al.  Image Partial Blur Detection and Classification , 2013 .

[32]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Manuel Menezes de Oliveira Neto,et al.  Deep Joint Design of Color Filter Arrays and Demosaicing , 2018, Comput. Graph. Forum.

[34]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Zhou Wang,et al.  Deep Blur Mapping: Exploiting High-Level Semantics by Deep Neural Networks , 2016, IEEE Transactions on Image Processing.

[37]  Peng Jiang,et al.  Salient Region Detection by UFO: Uniqueness, Focusness and Objectness , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Byung Cheol Song,et al.  Video Deblurring Algorithm Using Accurate Blur Kernel Estimation and Residual Deconvolution Based on a Blurred-Unblurred Frame Pair , 2013, IEEE Transactions on Image Processing.

[39]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[40]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[42]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.