A Local Metric for Defocus Blur Detection Based on CNN Feature Learning

Defocus blur detection is an important and challenging task in computer vision and digital imaging fields. Previous work on defocus blur detection has put a lot of effort into designing local sharpness metric maps. This paper presents a simple yet effective method to automatically obtain the local metric map for defocus blur detection, which based on the feature learning of multiple convolutional neural networks (ConvNets). The ConvNets automatically learn the most locally relevant features at the super-pixel level of the image in a supervised manner. By extracting convolution kernels from the trained neural network structures and processing it with principal component analysis, we can automatically obtain the local sharpness metric by reshaping the principal component vector. Meanwhile, an effective iterative updating mechanism is proposed to refine the defocus blur detection result from coarse to fine by exploiting the intrinsic peculiarity of the hyperbolic tangent function. The experimental results demonstrate that our proposed method consistently performed better than the previous state-of-the-art methods.

[1]  Wen Gao,et al.  Spatially variant defocus blur map estimation and deblurring from a single image , 2016, J. Vis. Commun. Image Represent..

[2]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[4]  Stuart Jefferies,et al.  A computational method for the restoration of images with an unknown, spatially-varying blur. , 2006, Optics express.

[5]  Wei Li,et al.  Blur image classification based on deep learning , 2017, 2017 IEEE International Conference on Imaging Systems and Techniques (IST).

[6]  Tien Tsin,et al.  Image Partial Blur Detection and Classification , 2013 .

[7]  Huimin Ma,et al.  Edge Preserving and Multi-Scale Contextual Neural Network for Salient Object Detection , 2016, IEEE Transactions on Image Processing.

[8]  Xiang Zhu,et al.  Estimating Spatially Varying Defocus Blur From A Single Image , 2013, IEEE Transactions on Image Processing.

[9]  William T. Freeman,et al.  Analyzing spatially-varying blur , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[11]  Lina J. Karam,et al.  Spatially-Varying Blur Detection Based on Multiscale Fused and Sorted Transform Coefficients of Gradient Magnitudes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Alex ChiChung Kot,et al.  A novel approach for partial blur detection and segmentation , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[13]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[14]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[15]  Ayan Chakrabarti,et al.  Depth and Deblurring from a Spectrally-Varying Depth-of-Field , 2012, ECCV.

[16]  Shijian Lu,et al.  Blurred image region detection and classification , 2011, ACM Multimedia.

[17]  Jian Yang,et al.  Convolution Neural Networks With Two Pathways for Image Style Recognition , 2017, IEEE Transactions on Image Processing.

[18]  Stefan Winkler,et al.  A no-reference perceptual blur metric , 2002, Proceedings. International Conference on Image Processing.

[19]  Jorge E. Caviedes,et al.  No-reference sharpness metric based on local edge kurtosis , 2002, Proceedings. International Conference on Image Processing.

[20]  Honglak Lee,et al.  Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units , 2016, ICML.

[21]  Changick Kim,et al.  Blurred image region detection and segmentation , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[22]  Huajun Feng,et al.  Automatic blur region segmentation approach using image matting , 2013, Signal Image Video Process..

[23]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[24]  Seungkyu Lee,et al.  Time-of-Flight Depth Camera Motion Blur Detection and Deblurring , 2014, IEEE Signal Processing Letters.

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[27]  In-So Kweon,et al.  A Unified Approach of Multi-scale Deep and Hand-Crafted Features for Defocus Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ying Wu,et al.  Removing partial blur in a single image , 2009, CVPR.

[29]  Michael S. Brown,et al.  Single image defocus map estimation using local contrast prior , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[30]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Sam T. Roweis,et al.  EM Algorithms for PCA and SPCA , 1997, NIPS.

[32]  Jean Ponce,et al.  Learning to Estimate and Remove Non-uniform Image Blur , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Mohammed Bennamoun,et al.  Automatic Shadow Detection and Removal from a Single Image , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Qionghai Dai,et al.  Absolute Depth Estimation From a Single Defocused Image , 2013, IEEE Transactions on Image Processing.

[35]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[36]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Stefano Soatto,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE IEEE TRANSACTION OF PATTERN RECO , 2022 .

[39]  Chunping Hou,et al.  Depth recovery and refinement from a single image using defocus cues , 2015 .

[40]  Chunping Hou,et al.  Defocus map estimation from a single image via spectrum contrast. , 2013, Optics letters.

[41]  Terence Sim,et al.  Defocus map estimation from a single image , 2011, Pattern Recognit..

[42]  Damon M. Chandler,et al.  ${\bf S}_{3}$: A Spectral and Spatial Measure of Local Perceived Sharpness in Natural Images , 2012, IEEE Transactions on Image Processing.

[43]  Yizhou Yu,et al.  Visual Saliency Detection Based on Multiscale Deep CNN Features , 2016, IEEE Transactions on Image Processing.

[44]  Pichao Wang,et al.  A Spectral and Spatial Approach of Coarse-to-Fine Blurred Image Region Detection , 2016, IEEE Signal Processing Letters.

[45]  Jianping Shi,et al.  Just noticeable defocus blur detection and estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).