Restoration of Fast Moving Objects

If an object is photographed at motion in front of a static background, the object will be blurred while the background sharp and partially occluded by the object. The goal is to recover the object appearance from such blurred image. We adopt the image formation model for fast moving objects and consider objects undergoing 2D translation and rotation. For this scenario we formulate the estimation of the object shape, appearance, and motion from a single image and known background as a constrained optimization problem with appropriate regularization terms. Both similarities and differences with blind deconvolution are discussed with the latter caused mainly by the coupling of the object appearance and shape in the acquisition model. Necessary conditions for solution uniqueness are derived and a numerical solution based on the alternating direction method of multipliers is presented. The proposed method is evaluated on a new dataset.

[1]  Daniele Perrone,et al.  A Clearer Picture of Total Variation Blind Deconvolution , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Richard Szeliski,et al.  PSF estimation using sharp edge prediction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Haichao Zhang,et al.  Revisiting Bayesian blind deconvolution , 2013, J. Mach. Learn. Res..

[5]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[6]  Jiri Matas,et al.  Intra-Frame Object Tracking by Deblatting , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[7]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  K. Egiazarian,et al.  Blind image deconvolution , 2007 .

[9]  Jiri Matas,et al.  The World of Fast Moving Objects , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Sergio Barbero,et al.  Nonconvex Bayesian Restoration of Blurred Foreground Images , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[12]  Seungyong Lee,et al.  Handling outliers in non-blind image deconvolution , 2011, 2011 International Conference on Computer Vision.

[13]  Yi Wang,et al.  Scale-Recurrent Network for Deep Image Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Deqing Sun,et al.  Deblurring Images via Dark Channel Prior , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Tae Hyun Kim,et al.  Dynamic Scene Deblurring , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Jiaya Jia,et al.  Single Image Motion Deblurring Using Transparency , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Xiaoyong Shen,et al.  Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Aggelos K. Katsaggelos,et al.  Bayesian Blind Deconvolution with General Sparse Image Priors , 2012, ECCV.

[19]  Shmuel Peleg,et al.  Two motion-blurred images are better than one , 2005, Pattern Recognit. Lett..

[20]  Jan Flusser,et al.  Multichannel blind deconvolution of spatially misaligned images , 2005, IEEE Transactions on Image Processing.

[21]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[22]  Bernhard Schölkopf,et al.  Learning to Deblur , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Aggelos K. Katsaggelos,et al.  Blind Deconvolution Using a Variational Approach to Parameter, Image, and Blur Estimation , 2006, IEEE Transactions on Image Processing.

[25]  Ryan J. Tibshirani,et al.  Dykstra's Algorithm, ADMM, and Coordinate Descent: Connections, Insights, and Extensions , 2017, NIPS.

[26]  Zhangyang Wang,et al.  DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Meiguang Jin,et al.  Learning to Extract a Video Sequence from a Single Motion-Blurred Image , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Daniele Perrone,et al.  A Logarithmic Image Prior for Blind Deconvolution , 2016, International Journal of Computer Vision.

[29]  Ming-Hsuan Yang,et al.  Robust Kernel Estimation with Outliers Handling for Image Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[31]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Wei Xiong,et al.  Rotational Motion Deblurring of a Rigid Object from a Single Image , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[33]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[34]  Yanning Zhang,et al.  Multi-Observation Blind Deconvolution with an Adaptive Sparse Prior , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Ming-Hsuan Yang,et al.  Soft-Segmentation Guided Object Motion Deblurring , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Jan Kotera,et al.  Motion Estimation and Deblurring of Fast Moving Objects , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[37]  Xu Zhou,et al.  Variational Bayesian Blind Image Deconvolution: A review , 2015, Digit. Signal Process..

[38]  Ying Wu,et al.  Motion from blur , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[41]  Václav Šmídl,et al.  Blind Deconvolution With Model Discrepancies , 2017, IEEE Transactions on Image Processing.

[42]  Tae Hyun Kim,et al.  Segmentation-Free Dynamic Scene Deblurring , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Yoram Bresler,et al.  Perfect blind restoration of images blurred by multiple filters: theory and efficient algorithms , 1999, IEEE Trans. Image Process..

[45]  Jan Flusser,et al.  Multichannel blind iterative image restoration , 2003, IEEE Trans. Image Process..