LBP-Based Segmentation of Defocus Blur

Defocus blur is extremely common in images captured using optical imaging systems. It may be undesirable, but may also be an intentional artistic effect, thus it can either enhance or inhibit our visual perception of the image scene. For tasks, such as image restoration and object recognition, one might want to segment a partially blurred image into blurred and non-blurred regions. In this paper, we propose a sharpness metric based on local binary patterns and a robust segmentation algorithm to separate in- and out-of-focus image regions. The proposed sharpness metric exploits the observation that most local image patches in blurry regions have significantly fewer of certain local binary patterns compared with those in sharp regions. Using this metric together with image matting and multi-scale inference, we obtained high-quality sharpness maps. Tests on hundreds of partially blurred images were used to evaluate our blur segmentation algorithm and six comparator methods. The results show that our algorithm achieves comparative segmentation results with the state of the art and have big speed advantage over the others.

[1]  Thirapiroon Thongkamwitoon,et al.  An Image Recapture Detection Algorithm Based on Learning Dictionaries of Edge Profiles , 2015, IEEE Transactions on Information Forensics and Security.

[2]  Shijian Lu,et al.  Blurred image region detection and classification , 2011, ACM Multimedia.

[3]  Zhou Wang,et al.  Image Sharpness Assessment Based on Local Phase Coherence , 2013, IEEE Transactions on Image Processing.

[4]  Alex ChiChung Kot,et al.  A novel approach for partial blur detection and segmentation , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[5]  Lina J. Karam,et al.  A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) , 2009, IEEE Transactions on Image Processing.

[6]  D. A. Fish,et al.  Scanning singular-value-decomposition method for restoration of images with space-variant blur , 1996 .

[7]  Stefan Winkler,et al.  A no-reference perceptual blur metric , 2002, Proceedings. International Conference on Image Processing.

[8]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Changick Kim,et al.  Segmenting a low-depth-of-field image using morphological filters and region merging , 2005, IEEE Transactions on Image Processing.

[10]  Xiang Zhu Measuring Spatially Varying Blur and its Application in Digital Image Restoration , 2013 .

[11]  William T. Freeman,et al.  Analyzing spatially-varying blur , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Jean Ponce,et al.  Learning to Estimate and Remove Non-uniform Image Blur , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Deepa Kundur,et al.  Blind image deconvolution , 1996, IEEE Signal Process. Mag..

[14]  Matti Pietikäinen,et al.  Unsupervised texture segmentation using feature distributions , 1997, Pattern Recognit..

[15]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Damon M. Chandler,et al.  ${\bf S}_{3}$: A Spectral and Spatial Measure of Local Perceived Sharpness in Natural Images , 2012, IEEE Transactions on Image Processing.

[19]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Stuart Jefferies,et al.  A computational method for the restoration of images with an unknown, spatially-varying blur. , 2006, Optics express.

[21]  Marko Heikkilä,et al.  A texture-based method for modeling the background and detecting moving objects , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  B. R. Hunt,et al.  Image restoration of space-variant blurs by sectioned methods , 1978 .

[23]  Matti Pietikäinen,et al.  View-based recognition of real-world textures , 2004, Pattern Recognit..

[24]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[25]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[26]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[27]  G. J. Burton,et al.  Color and spatial structure in natural scenes. , 1987, Applied optics.

[28]  Xudong Jiang,et al.  Noise-Resistant Local Binary Pattern With an Embedded Error-Correction Mechanism , 2013, IEEE Transactions on Image Processing.

[29]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Jianping Shi,et al.  Just noticeable defocus blur detection and estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, AMFG.

[32]  Fei-Fei Li,et al.  Object-Centric Spatial Pooling for Image Classification , 2012, ECCV.

[33]  Lina J. Karam,et al.  A No-Reference Image Blur Metric Based on the Cumulative Probability of Blur Detection (CPBD) , 2011, IEEE Transactions on Image Processing.

[34]  Taeg Sang Cho,et al.  Motion blur removal from photographs , 2010 .

[35]  Ying Wu,et al.  Removing partial blur in a single image , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[37]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[38]  H. Joel Trussell,et al.  Identification and restoration of spatially variant motion blurs in sequential images , 1992, IEEE Trans. Image Process..

[39]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[40]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[41]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[42]  Huajun Feng,et al.  Automatic blur region segmentation approach using image matting , 2013, Signal Image Video Process..

[43]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[44]  J. H. van Hateren,et al.  Modelling the Power Spectra of Natural Images: Statistics and Information , 1996, Vision Research.

[45]  Robert M. Gray,et al.  Automatic object segmentation in images with low depth of field , 2002, Proceedings. International Conference on Image Processing.

[46]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[47]  Terence Sim,et al.  Defocus map estimation from a single image , 2011, Pattern Recognit..

[48]  Hans-Peter Kriegel,et al.  Robust Image Segmentation in Low Depth Of Field Images , 2013, ArXiv.

[49]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[50]  Jian Sun,et al.  A global sampling method for alpha matting , 2011, CVPR 2011.

[51]  Tien Tsin,et al.  Image Partial Blur Detection and Classification , 2013 .

[52]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[54]  Xiang Zhu,et al.  Estimating Spatially Varying Defocus Blur From A Single Image , 2013, IEEE Transactions on Image Processing.

[55]  James Ze Wang,et al.  Unsupervised Multiresolution Segmentation for Images with Low Depth of Field , 2001, IEEE Trans. Pattern Anal. Mach. Intell..