Soft-Segmentation Guided Object Motion Deblurring

Object motion blur is a challenging problem as the foreground and the background in the scenes undergo different types of image degradation due to movements in various directions and speed. Most object motion deblurring methods address this problem by segmenting blurred images into regions where different kernels are estimated and applied for restoration. Segmentation on blurred images is difficult due to ambiguous pixels between regions, but it plays an important role for object motion deblurring. To address these problems, we propose a novel model for object motion deblurring. The proposed model is developed based on a maximum a posterior formulation in which soft-segmentation is incorporated for object layer estimation. We propose an efficient algorithm to jointly estimate object segmentation and camera motion where each layer can be deblurred well under the guidance of the soft-segmentation. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art object motion deblurring methods on challenging scenarios.

[1]  Ying Wu,et al.  Removing partial blur in a single image , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[3]  Yanning Zhang,et al.  Multi-image Blind Deblurring Using a Coupled Adaptive Sparse Prior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[5]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Anat Levin,et al.  Blind Motion Deblurring Using Image Statistics , 2006, NIPS.

[7]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[8]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[9]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[10]  Sung Yong Shin,et al.  Coded exposure imaging for projective motion deblurring , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Sunghyun Cho,et al.  Edge-based blur kernel estimation using patch priors , 2013, IEEE International Conference on Computational Photography (ICCP).

[12]  Wei Xiong,et al.  Rotational Motion Deblurring of a Rigid Object from a Single Image , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH '06.

[15]  Jian-Feng Cai,et al.  Framelet-Based Blind Motion Deblurring From a Single Image , 2012, IEEE Transactions on Image Processing.

[16]  Michael S. Brown,et al.  Richardson-Lucy Deblurring for Scenes under a Projective Motion Path , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jiaya Jia,et al.  Single Image Motion Deblurring Using Transparency , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[20]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Stephen Lin,et al.  Correction of Spatially Varying Image and Video Motion Blur Using a Hybrid Camera , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[24]  Bernhard Schölkopf,et al.  Space-Variant Single-Image Blind Deconvolution for Removing Camera Shake , 2010, NIPS.

[25]  Sylvain Paris,et al.  Blur kernel estimation using the radon transform , 2011, CVPR 2011.

[26]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Tae Hyun Kim,et al.  Segmentation-Free Dynamic Scene Deblurring , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Stefano Soatto,et al.  Seeing beyond occlusions (and other marvels of a finite lens aperture) , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Seungyong Lee,et al.  Fast motion deblurring , 2009, ACM Trans. Graph..

[30]  Ying Wu,et al.  Motion from blur , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Bernhard Schölkopf,et al.  Recording and Playback of Camera Shake: Benchmarking Blind Deconvolution with a Real-World Database , 2012, ECCV.

[32]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  L. Lucy An iterative technique for the rectification of observed distributions , 1974 .

[34]  Stefan Roth,et al.  Localized Image Blur Removal through Non-parametric Kernel Estimation , 2014, 2014 22nd International Conference on Pattern Recognition.

[35]  Jiaya Jia,et al.  Mathematical models and practical solvers for uniform motion deblurring , 2014, Motion Deblurring.

[36]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[38]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Thomas S. Huang,et al.  Sparse representation based blind image deblurring , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[40]  Daniele Perrone,et al.  Total Variation Blind Deconvolution: The Devil Is in the Details , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Stefano Soatto,et al.  Direct Sparse Deblurring , 2010, Journal of Mathematical Imaging and Vision.

[42]  Dani Lischinski,et al.  Deblurring by Example Using Dense Correspondence , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Tae Hyun Kim,et al.  Dynamic Scene Deblurring , 2013, 2013 IEEE International Conference on Computer Vision.

[44]  Ming-Hsuan Yang,et al.  Joint Depth Estimation and Camera Shake Removal from Single Blurry Image , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[46]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[47]  Richard Szeliski,et al.  PSF estimation using sharp edge prediction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Seungyong Lee,et al.  Recent advances in image deblurring , 2013, SA '13.

[49]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[50]  William T. Freeman,et al.  Analyzing spatially-varying blur , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Ming-Hsuan Yang,et al.  Deblurring Face Images with Exemplars , 2014, ECCV.

[52]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..