A variational approach to scene reconstruction and image segmentation from motion-blur cues

In this paper we are interested in the joint reconstruction of geometry and photometry of scenes with multiple moving objects from a collection of motion-blurred images. We make simplifying assumptions on the photometry of the scene (we neglect complex illumination effects) and infer the motion field of the scene, its depth map, and its radiance. In particular, we choose to partition the image into regions where motion is well approximated by a simple planar translation. We model motion-blurred images as the solution of an anisotropic diffusion equation, whose initial conditions depend on the radiance and whose diffusion tensor encodes the depth map of the scene and the motion field. We propose an algorithm to infer the unknowns of the model by minimizing the discrepancy between the measured images and the ones synthesized via diffusion. Since the problem is ill-posed, we also introduce additional Tikhonov regularization terms.

[1]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[2]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[3]  J HeegerDavid,et al.  Subspace methods for recovering rigid motion I , 1992 .

[4]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[5]  Shmuel Peleg,et al.  Restoration of multiple images with motion blur in different directions , 2000, Proceedings Fifth IEEE Workshop on Applications of Computer Vision.

[6]  Worthy N. Martin,et al.  Image Motion Estimation From Motion Smear-A New Computational Model , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Shree K. Nayar,et al.  Motion deblurring using hybrid imaging , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  M. Georgeson,et al.  Motion blur and motion sharpening: temporal smear and local contrast non-linearity , 1998, Vision Research.

[9]  Yitzhak Yitzhaky,et al.  Direct method for restoration of motion-blurred images , 1998 .

[10]  Irfan A. Essa,et al.  Image-based motion blur for stop motion animation , 2001, SIGGRAPH.

[11]  Jun Ma,et al.  Depth from zooming , 1990 .

[12]  Stefano Soatto,et al.  Scene and Motion Reconstruction from Defocused and Motion-Blurred Images via Anisotropic Diffusion , 2004, ECCV.

[13]  Aggelos K. Katsaggelos,et al.  Regularized blur-assisted displacement field estimation , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[14]  Shmuel Peleg,et al.  Robust super-resolution , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[15]  Kiyoharu Aizawa,et al.  Arbitrary view and focus image generation: rendering object-based shifting and focussing effect by linear filtering , 2002, Proceedings. International Conference on Image Processing.

[16]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[17]  Joonki Paik,et al.  Segmentation-based spatially adaptive motion blur removal and its application to surveillance systems , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[18]  Robert L. Stevenson,et al.  Super-resolution from image sequences-a review , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[19]  Joonki Paik,et al.  Segmentation-based image restoration for multiple moving objects with different motions , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[20]  Alan S. Willsky,et al.  A curve evolution-based variational approach to simultaneous image restoration and segmentation , 2002, Proceedings. International Conference on Image Processing.