Parametric Object Motion from Blur

Motion blur can adversely affect a number of vision tasks, hence it is generally considered a nuisance. We instead treat motion blur as a useful signal that allows to compute the motion of objects from a single image. Drawing on the success of joint segmentation and parametric motion models in the context of optical flow estimation, we propose a parametric object motion model combined with a segmentation mask to exploit localized, non-uniform motion blur. Our parametric image formation model is differentiable w.r.t. the motion parameters, which enables us to generalize marginal-likelihood techniques from uniform blind deblurring to localized, non-uniform blur. A two-stage pipeline, first in derivative space and then in image space, allows to estimate both parametric object motion as well as a motion segmentation from a single image alone. Our experiments demonstrate its ability to cope with very challenging cases of object motion blur.

[1]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Michael J. Black,et al.  Modeling Blurred Video with Layers , 2014, ECCV.

[3]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[4]  Jiaya Jia,et al.  Image partial blur detection and classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[6]  Shahriar Negahdaripour,et al.  Motion recovery from image sequences using only first order optical flow information , 1992, International Journal of Computer Vision.

[7]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[8]  Jiaolong Yang,et al.  Dense, accurate optical flow estimation with piecewise parametric model , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Stefan Roth,et al.  Localized Image Blur Removal through Non-parametric Kernel Estimation , 2014, 2014 22nd International Conference on Pattern Recognition.

[10]  Ying Wu,et al.  Motion from blur , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  William T. Freeman,et al.  Analyzing spatially-varying blur , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Frédo Durand,et al.  Efficient marginal likelihood optimization in blind deconvolution , 2011, CVPR 2011.

[14]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Tae Hyun Kim,et al.  Segmentation-Free Dynamic Scene Deblurring , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Bernhard Schölkopf,et al.  Seeing the Arrow of Time , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Qionghai Dai,et al.  Absolute Depth Estimation From a Single Defocused Image , 2013, IEEE Transactions on Image Processing.

[18]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Patrick Pérez,et al.  Interactive Image Segmentation Using an Adaptive GMMRF Model , 2004, ECCV.

[20]  Konrad Schindler,et al.  Piecewise Rigid Scene Flow , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[22]  B. Schölkopf,et al.  Blind Motion Deblurring Using Image Statistics , 2007 .

[23]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[24]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[26]  David J. C. MacKay,et al.  Ensemble Learning for Blind Image Separation and Deconvolution , 2000 .

[27]  Anat Levin,et al.  Blind Motion Deblurring Using Image Statistics , 2006, NIPS.

[28]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Li Xu,et al.  Discriminative Blur Detection Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[31]  Tae Hyun Kim,et al.  Dynamic Scene Deblurring , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[33]  Raquel Urtasun,et al.  Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation , 2014, ECCV.

[34]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[35]  Daniele Perrone,et al.  Total Variation Blind Deconvolution: The Devil Is in the Details , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Charles M. Bishop,et al.  Variational Message Passing , 2005, J. Mach. Learn. Res..

[37]  Li Xu,et al.  Forward Motion Deblurring , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Jean Ponce,et al.  Learning to Estimate and Remove Non-uniform Image Blur , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[40]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[41]  Yasuyuki Matsushita,et al.  Removing Non-Uniform Motion Blur from Images , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[42]  Michael J. Black,et al.  Layered image motion with explicit occlusions, temporal consistency, and depth ordering , 2010, NIPS.

[43]  Tae Hyun Kim,et al.  Generalized video deblurring for dynamic scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[45]  L. Armijo Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .