Motion-based motion deblurring

Motion blur due to camera motion can significantly degrade the quality of an image. Since the path of the camera motion can be arbitrary, deblurring of motion blurred images is a hard problem. Previous methods to deal with this problem have included blind restoration of motion blurred images, optical correction using stabilized lenses, and special CMOS sensors that limit the exposure time in the presence of motion. In this paper, we exploit the fundamental trade off between spatial resolution and temporal resolution to construct a hybrid camera that can measure its own motion during image integration. The acquired motion information is used to compute a point spread function (PSF) that represents the path of the camera during integration. This PSF is then used to deblur the image. To verify the feasibility of hybrid imaging for motion deblurring, we have implemented a prototype hybrid camera. This prototype system was evaluated in different indoor and outdoor scenes using long exposures and complex camera motion paths. The results show that, with minimal resources, hybrid imaging outperforms previous approaches to the motion blur problem. We conclude with a brief discussion on how our ideas can be extended beyond the case of global camera motion to the case where individual objects in the scene move with different velocities.

[1]  David Pearce MacAdam,et al.  Digital Image Restoration by Constrained Deconvolution , 1970 .

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  Michael Potmesil,et al.  Modeling motion blur in computer-generated images , 1983, SIGGRAPH.

[4]  Nelson L. Max,et al.  A two-and-a-half-D motion-blur algorithm , 1985, SIGGRAPH '85.

[5]  Joel S. Fox.  Range from translational motion blurring , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  David Malah,et al.  Robust identification of motion and out-of-focus blur parameters from blurred and noisy images , 1991, CVGIP Graph. Model. Image Process..

[7]  Satwinderpal S. Makkad,et al.  Range from motion blur , 1993 .

[8]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[9]  Yitzhak Yitzhaky,et al.  Identification of the blur extent from motion-blurred images , 1995, Defense, Security, and Sensing.

[10]  Pat Hanrahan,et al.  A realistic camera model for computer graphics , 1995, SIGGRAPH.

[11]  Worthy N. Martin,et al.  Image Motion Estimation From Motion Smear-A New Computational Model , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  P. Jansson.  Deconvolution of images and spectra (2nd ed.) , 1996 .

[13]  Andrew Blake,et al.  Motion Deblurring and Super-resolution from an Image Sequence , 1996, ECCV.

[14]  Ioannis M. Rekleitis.  Optical flow recognition from the power spectrum of a single blurred image , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[15]  Aggelos K. Katsaggelos,et al.  Regularized blur-assisted displacement field estimation , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[16]  Peter Csillag,et al.  Estimation of accelerated motion for motion-compensated frame interpolation , 1996, Other Conferences.

[17]  N. Kopeika,et al.  Analytical method to calculate optical transfer functions for image motion and vibrations using moments , 1997 .

[18]  S. Hammett.  Motion blur and motion sharpening in the human visual system , 1997, Vision Research.

[19]  Sang Hwa Lee,et al.  Recovery of blurred video signals using iterative image restoration combined with motion estimation , 1997, Proceedings of International Conference on Image Processing.

[20]  N. Kopeika,et al.  Direct method for restoration of motion-blurred images , 1998 .

[21]  M. Georgeson,et al.  Motion blur and motion sharpening: temporal smear and local contrast non-linearity , 1998, Vision Research.

[22]  Yuan-Fang Wang,et al.  3D shape and motion analysis from image blur and smear: a unified approach , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[23]  Yao Jianchao.  Motion blur identification based on phase change experienced after trial restorations , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[24]  Rama Chellappa,et al.  Data-driven multichannel superresolution with application to video sequences , 1999 .

[25]  Til Aach,et al.  Blur identification using a spectral inertia tensor and spectral zeros , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[26]  Ying Zhang,et al.  Estimation of motion parameters from blurred images , 2000, Pattern Recognit. Lett..

[27]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Yitzhak Yitzhaky,et al.  Restoration of an image degraded by vibrations using only a single frame , 2000 .

[29]  Robin R. Murphy,et al.  Motion detection from temporally integrated images , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[30]  Shmuel Peleg,et al.  Restoration of multiple images with motion blur in different directions , 2000, Proceedings Fifth IEEE Workshop on Applications of Computer Vision.

[31]  Keith J. Hanna,et al.  Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences , 2001, SIGGRAPH.

[32]  Xinqiao Liu,et al.  Simultaneous image formation and motion blur restoration via multiple capture , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[33]  Irfan A. Essa,et al.  Image-based motion blur for stop motion animation , 2001, SIGGRAPH.

[34]  Kiyoharu Aizawa,et al.  A computational image sensor with adaptive pixel-based integration time , 2001 .

[35]  M. Akke,et al.  An open and shut case , 2001, Nature Structural Biology.

[36]  Yaron Caspi,et al.  Increasing Space-Time Resolution in Video , 2002, ECCV.

[37]  Adrian Stern,et al.  Recognition of motion-blurred images by use of the method of moments. , 2002, Applied optics.

[38]  Robert C. Zeleznik,et al.  Interactive real-time motion blur , 2005, The Visual Computer.