Noises removal from image sequences acquired with moving camera by estimating camera motion from spatio-temporal information

This paper describes a method for removing adherent noises from image sequences. In outdoor environments, it is often the case that scenes taken by a camera are deteriorated because of adherent noises such as waterdrops on the surface of the lens-protecting glass of the camera. To solve this problem, our method takes advantage of image sequences captured with a moving camera whose motion is unknown. Our method estimates a camera motion only from image sequences, and makes a spatio-temporal image to extract the regions of adherent noises by examining differences of track slopes in cross section images between adherent noises and other objects. Finally, regions of noises are eliminated by replacing with image data corresponding to object regions. Experimental results show the effectiveness of our method.

[1]  Harry Shum,et al.  Full-frame video stabilization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[3]  Anil C. Kokaram,et al.  Interpolation of missing data in image sequences , 1995, IEEE Trans. Image Process..

[4]  Paul R. Cohen,et al.  Camera Calibration with Distortion Models and Accuracy Evaluation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[6]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[7]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Stefano Soatto,et al.  Inpainting from multiple views , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[9]  Olivier Buisson,et al.  Detection and removal of line scratches in motion picture films , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[10]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[11]  Atsushi Yamashita,et al.  Removal of adherent noises from images of dynamic scenes by using a pan-tilt camera , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[12]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[13]  Shree K. Nayar,et al.  Detection and removal of rain from videos , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[14]  Atsushi Yamashita,et al.  Removal of adherent noises from image sequences by spatio-temporal image processing , 2008, 2008 IEEE International Conference on Robotics and Automation.

[15]  Masaaki Yoneda,et al.  Real-time snowfall noise elimination , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[16]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[17]  Atsushi Yamashita,et al.  A virtual wiper-restoration of deteriorated images by using a pan-tilt camera , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[18]  Atsushi Yamashita,et al.  A virtual wiper - restoration of deteriorated images by using multiple cameras , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[19]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Atsushi Yamashita,et al.  Removal of adherent noises from images of dynamic scenes by using a pan-tilt camera , 2004, IROS.

[21]  Atsushi Yamashita,et al.  Removal of Adherent Waterdrops from Images Acquired with a Stereo Camera System , 2006, IEICE Trans. Inf. Syst..

[22]  Atsushi Yamashita,et al.  Removal of adherent waterdrops from images acquired with stereo camera , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Xiaochun Cao,et al.  Video Completion for Perspective Camera Under Constrained Motion , 2006, 18th International Conference on Pattern Recognition (ICPR'06).