Automatic Video Object Extraction with Camera in Motion

Automatic moving object extraction has been explored extensively in the image processing and computer vision community. Generally, moving object extraction schemes rely on either optical flow or frame difference. Optical flow methods can deal with moving cameras, but they are inconsistent at object boundaries and the object segmentation tends to be inaccurate. Although frame difference approaches can detect object boundaries, they cannot detect the uniform intensity interior regions. Additionally, the frame difference approaches cannot deal with moving cameras. We present a novel technique for the automatic extraction of a moving object captured by a moving camera by blending the information from the optical flow, the frame differences, and the spatial segmentation. The optical flow is used to compensate the camera motion and to generate a model for the background. Next, the differences in the compensated frames are compared with the background model to detect the changes in the frame. Finally, the detected changes and the spatial segmentation are combined to identify the moving uniform intensity regions. Experimental results of the proposed moving object extraction method for a variety of videos are presented.

[1]  Guojun Lu,et al.  Segmentation of moving objects in image sequence: A review , 2001 .

[2]  A. Murat Tekalp,et al.  Region-Based Parametric Motion Segmentation Using Color Information , 1998, Graph. Model. Image Process..

[3]  Josef Kittler,et al.  A Gradient-Based Method for General Motion Estimation and Segmentation , 1993, J. Vis. Commun. Image Represent..

[4]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[5]  Marko Heikkilä,et al.  A texture-based method for modeling the background and detecting moving objects , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Daniel P. Huttenlocher,et al.  Scene modeling for wide area surveillance and image synthesis , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  D. Ruppert,et al.  A Note on Computing Robust Regression Estimates via Iteratively Reweighted Least Squares , 1988 .

[8]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[9]  Marcel Worring,et al.  Detection of moving objects in video using a robust motion similarity measure , 2000, IEEE Trans. Image Process..

[10]  Haifeng Xu,et al.  Automatic moving object extraction for content-based applications , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  P. Anandan,et al.  A Unified Approach to Moving Object Detection in 2D and 3D Scenes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Liang-Gee Chen,et al.  Efficient moving object segmentation algorithm using background registration technique , 2002, IEEE Trans. Circuits Syst. Video Technol..

[13]  Christoph Stiller.  A statistical image model for motion estimation , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  Anil K. Jain,et al.  Object contour extraction using color and motion , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Tiziana D'Orazio,et al.  Moving object segmentation by background subtraction and temporal analysis , 2006, Image Vis. Comput..

[16]  Amir Averbuch,et al.  Automatic segmentation of moving objects in video sequences: a region labeling approach , 2002, IEEE Trans. Circuits Syst. Video Technol..

[17]  D. Thirde,et al.  Evaluation of Motion Segmentation Quality for Aircraft Activity Surveillance , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[18]  Eun Yi Kim,et al.  Automatic video segmentation using genetic algorithms , 2006, Pattern Recognit. Lett..

[19]  Takashi Matsuyama,et al.  Appearance sphere: background model for pan-tilt-zoom camera , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[20]  Jenq-Neng Hwang,et al.  Fast and automatic video object segmentation and tracking for content-based applications , 2002, IEEE Trans. Circuits Syst. Video Technol..

[21]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[22]  A. Murat Tekalp,et al.  2-D mesh-based video object segmentation and tracking with occlusion resolution , 2001, Signal Process. Image Commun..

[23]  Jenq-Neng Hwang,et al.  Object-based video abstraction for video surveillance systems , 2002, IEEE Trans. Circuits Syst. Video Technol..

[24]  Chong-Wah Ngo,et al.  Moving-object detection, association, and selection in home videos , 2007, IEEE Transactions on Multimedia.

[25]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Anil K. Jain,et al.  Robust motion-based image segmentation using fusion , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[27]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Murat Kunt,et al.  Spatiotemporal Segmentation Based on Region Merging , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Georgios Tziritas,et al.  Adaptive detection and localization of moving objects in image sequences , 1999, Signal Process. Image Commun..

[30]  A. Murat Tekalp,et al.  Motion segmentation by multistage affine classification , 1997, IEEE Trans. Image Process..

[31]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.