Defocus Blur Detection via Multi-Stream Bottom-Top-Bottom Network

Defocus blur detection (DBD) is aimed to estimate the probability of each pixel being in-focus or out-of-focus. This process has been paid considerable attention due to its remarkable potential applications. Accurate differentiation of homogeneous regions and detection of low-contrast focal regions, as well as suppression of background clutter, are challenges associated with DBD. To address these issues, we propose a multi-stream bottom-top-bottom fully convolutional network (BTBNet), which is the first attempt to develop an end-to-end deep network to solve the DBD problems. First, we develop a fully convolutional BTBNet to gradually integrate nearby feature levels of bottom to top and top to bottom. Then, considering that the degree of defocus blur is sensitive to scales, we propose multi-stream BTBNets that handle input images with different scales to improve the performance of DBD. Finally, a cascaded DBD map residual learning architecture is designed to gradually restore finer structures from the small scale to the large scale. To promote further study and evaluation of the DBD models, we construct a new database of 1100 challenging images and their pixel-wise defocus blur annotations. Experimental results on the existing and our new datasets demonstrate that the proposed method achieves significantly better performance than other state-of-the-art algorithms.

[1]  Damon M. Chandler,et al.  ${\bf S}_{3}$: A Spectral and Spatial Measure of Local Perceived Sharpness in Natural Images , 2012, IEEE Transactions on Image Processing.

[2]  Wen Gao,et al.  Spatially variant defocus blur map estimation and deblurring from a single image , 2016, J. Vis. Commun. Image Represent..

[3]  Steven W. Zucker,et al.  Local Scale Control for Edge Detection and Blur Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Yizhou Yu,et al.  Visual Saliency Detection Based on Multiscale Deep CNN Features , 2016, IEEE Transactions on Image Processing.

[5]  Xuelong Li,et al.  Classifying Discriminative Features for Blur Detection , 2016, IEEE Transactions on Cybernetics.

[6]  Huchuan Lu,et al.  Learning Uncertain Convolutional Features for Accurate Saliency Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Gerald Schaefer,et al.  UCID: an uncompressed color image database , 2003, IS&T/SPIE Electronic Imaging.

[8]  Xiaogang Wang,et al.  Object Detection from Video Tubelets with Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Ming-Hsuan Yang,et al.  Learning to Deblur Images with Exemplars , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Pichao Wang,et al.  A Spectral and Spatial Approach of Coarse-to-Fine Blurred Image Region Detection , 2016, IEEE Signal Processing Letters.

[12]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[13]  Xin Yi,et al.  LBP-Based Segmentation of Defocus Blur , 2016, IEEE Transactions on Image Processing.

[14]  Lina J. Karam,et al.  Spatially-Varying Blur Detection Based on Multiscale Fused and Sorted Transform Coefficients of Gradient Magnitudes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Keigo Hirakawa,et al.  Blur Processing Using Double Discrete Wavelet Transform , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Xiang Zhu,et al.  Estimating Spatially Varying Defocus Blur From A Single Image , 2013, IEEE Transactions on Image Processing.

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Huchuan Lu,et al.  Deep visual tracking: Review and experimental comparison , 2018, Pattern Recognit..

[19]  Jean Ponce,et al.  Learning to Estimate and Remove Non-uniform Image Blur , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Michael S. Brown,et al.  Single image defocus map estimation using local contrast prior , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[21]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[22]  Dani Lischinski,et al.  Spectral Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Li Xu,et al.  Mutual-Structure for Joint Filtering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[25]  Jiaya Jia,et al.  Image partial blur detection and classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[28]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Terence Sim,et al.  Defocus map estimation from a single image , 2011, Pattern Recognit..

[30]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Huajun Feng,et al.  Automatic blur region segmentation approach using image matting , 2013, Signal Image Video Process..

[32]  Ming-Hsuan Yang,et al.  Learning Spatial-Aware Regressions for Visual Tracking , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[35]  In-So Kweon,et al.  A Unified Approach of Multi-scale Deep and Hand-Crafted Features for Defocus Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Jianping Shi,et al.  Just noticeable defocus blur detection and estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Hui Ji,et al.  Estimating Defocus Blur via Rank of Local Patches , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  Bingbing Ni,et al.  HCP: A Flexible CNN Framework for Multi-Label Image Classification , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[40]  Elhusain Saad,et al.  Defocus Blur-Invariant Scale-Space Feature Extractions , 2016, IEEE Transactions on Image Processing.

[41]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Peng Jiang,et al.  Salient Region Detection by UFO: Uniqueness, Focusness and Objectness , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Chunping Hou,et al.  Defocus map estimation from a single image via spectrum contrast. , 2013, Optics letters.

[44]  Huchuan Lu,et al.  Defocus Blur Detection via Multi-stream Bottom-Top-Bottom Fully Convolutional Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Yi Yang,et al.  Attention to Scale: Scale-Aware Semantic Image Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Qiong Yan,et al.  Cascade Residual Learning: A Two-Stage Convolutional Neural Network for Stereo Matching , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).