Defocus Blur Detection via Multi-stream Bottom-Top-Bottom Fully Convolutional Network

Defocus blur detection (DBD) is the separation of in-focus and out-of-focus regions in an image. This process has been paid considerable attention because of its remarkable potential applications. Accurate differentiation of homogeneous regions and detection of low-contrast focal regions, as well as suppression of background clutter, are challenges associated with DBD. To address these issues, we propose a multi-stream bottom-top-bottom fully convolutional network (BTBNet), which is the first attempt to develop an end-to-end deep network for DBD. First, we develop a fully convolutional BTBNet to integrate low-level cues and high-level semantic information. Then, considering that the degree of defocus blur is sensitive to scales, we propose multi-stream BTBNets that handle input images with different scales to improve the performance of DBD. Finally, we design a fusion and recurrent reconstruction network to recurrently refine the preceding blur detection maps. To promote further study and evaluation of the DBD models, we construct a new database of 500 challenging images and their pixel-wise defocus blur annotations. Experimental results on the existing and our new datasets demonstrate that the proposed method achieves significantly better performance than other state-of-the-art algorithms.

[1]  Pichao Wang,et al.  A Spectral and Spatial Approach of Coarse-to-Fine Blurred Image Region Detection , 2016, IEEE Signal Processing Letters.

[2]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[3]  Gerald Schaefer,et al.  UCID: an uncompressed color image database , 2003, IS&T/SPIE Electronic Imaging.

[4]  Yi Yang,et al.  Attention to Scale: Scale-Aware Semantic Image Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Keigo Hirakawa,et al.  Blur Processing Using Double Discrete Wavelet Transform , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Jean Ponce,et al.  Learning to Estimate and Remove Non-uniform Image Blur , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Damon M. Chandler,et al.  ${\bf S}_{3}$: A Spectral and Spatial Measure of Local Perceived Sharpness in Natural Images , 2012, IEEE Transactions on Image Processing.

[9]  Yizhou Yu,et al.  Visual Saliency Detection Based on Multiscale Deep CNN Features , 2016, IEEE Transactions on Image Processing.

[10]  Elhusain Saad,et al.  Defocus Blur-Invariant Scale-Space Feature Extractions , 2016, IEEE Transactions on Image Processing.

[11]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[13]  Peng Jiang,et al.  Salient Region Detection by UFO: Uniqueness, Focusness and Objectness , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Chunping Hou,et al.  Defocus map estimation from a single image via spectrum contrast. , 2013, Optics letters.

[15]  Huajun Feng,et al.  Automatic blur region segmentation approach using image matting , 2013, Signal Image Video Process..

[16]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[17]  Xiaogang Wang,et al.  Object Detection from Video Tubelets with Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Jiaya Jia,et al.  Image partial blur detection and classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Terence Sim,et al.  Defocus map estimation from a single image , 2011, Pattern Recognit..

[20]  Xuelong Li,et al.  Classifying Discriminative Features for Blur Detection , 2016, IEEE Transactions on Cybernetics.

[21]  Michael S. Brown,et al.  Single image defocus map estimation using local contrast prior , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[22]  Jianping Shi,et al.  Just noticeable defocus blur detection and estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[24]  In-So Kweon,et al.  A Unified Approach of Multi-scale Deep and Hand-Crafted Features for Defocus Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Bingbing Ni,et al.  HCP: A Flexible CNN Framework for Multi-Label Image Classification , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Huchuan Lu,et al.  Deep visual tracking: Review and experimental comparison , 2018, Pattern Recognit..

[27]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[28]  Dani Lischinski,et al.  Spectral Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Shijian Lu,et al.  Blurred image region detection and classification , 2011, ACM Multimedia.

[30]  Ming-Hsuan Yang,et al.  Learning Spatial-Aware Regressions for Visual Tracking , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Xiang Zhu,et al.  Estimating Spatially Varying Defocus Blur From A Single Image , 2013, IEEE Transactions on Image Processing.

[33]  Xin Yi,et al.  LBP-Based Segmentation of Defocus Blur , 2016, IEEE Transactions on Image Processing.

[34]  Lina J. Karam,et al.  Spatially-Varying Blur Detection Based on Multiscale Fused and Sorted Transform Coefficients of Gradient Magnitudes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Wen Gao,et al.  Spatially variant defocus blur map estimation and deblurring from a single image , 2016, J. Vis. Commun. Image Represent..

[36]  Steven W. Zucker,et al.  Local Scale Control for Edge Detection and Blur Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Huchuan Lu,et al.  Learning Uncertain Convolutional Features for Accurate Saliency Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).