Spatiotemporal Region Enhancement and Merging for Unsupervized Object Segmentation

This paper proposes an unsupervized offline video object segmentation method that introduces a number of improvements to existing work in the area. It consists of the following steps. The initial segmentation utilizes object color and motion variance to more accurately classify image pixels in the first frame. Histogram-based merging is then employed to reduce oversegmentation of the first frame. During object tracking, segmentation quality measures based on object color and motion contrast are taken. These measures are then used to enhance video objects through selective pixel reclassification. After object enhancement, cumulative histogram-based merging, occlusion handling, and island detection are used to help group regions into meaningful objects. Compared to two reference methods, greater success and improved accuracy in segmenting video objects are first demonstrated by subjectively examining selected frames from a set of standard video sequences. Objective results are obtained through the use of a set of measures that aim at evaluating the accuracy of object boundaries and temporal stability through the use of color, motion, and histograms.

[1]  A. Murat Tekalp,et al.  Video object tracking with feedback of performance measures , 2003, IEEE Trans. Circuits Syst. Video Technol..

[2]  Michael G. Strintzis,et al.  Video object segmentation using Bayes-based temporal tracking and trajectory-based region merging , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Stella X. Yu.  Segmentation using multiscale cues , 2004, CVPR 2004.

[4]  Dong-Jo Park,et al.  Unsupervised video object segmentation and tracking based on new edge features , 2004, Pattern Recognit. Lett..

[5]  A. Murat Tekalp,et al.  Region-based affine motion segmentation using color information , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  A. Murat Tekalp,et al.  Performance measures for video object segmentation and tracking , 2003, IEEE Transactions on Image Processing.

[7]  Langis Gagnon,et al.  Video Object Segmentation Based on Object Enhancement and Region Merging , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[8]  Jitendra Malik,et al.  Textons, contours and regions: cue integration in image segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[10]  Theo Gevers,et al.  Robust segmentation and tracking of colored objects in video , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Alex Pentland,et al.  Cooperative Robust Estimation Using Layers of Support , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[13]  Çigdem Eroglu Erdem,et al.  Video object segmentation and tracking using region-based statistics , 2007, Signal Process. Image Commun..

[14]  Yi-Ping Hung,et al.  A Bayesian approach to video object segmentation via merging 3-D watershed volumes , 2005 .

[15]  Fernand Meyer,et al.  A novel approach to depth ordering in monocular image sequences , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  Qiong Wu,et al.  Robust Real-Time Bi-Layer Video Segmentation Using Infrared Video , 2008, 2008 Canadian Conference on Computer and Robot Vision.

[17]  Haifeng Xu,et al.  Automatic moving object extraction for content-based applications , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Shigang Wang,et al.  A stereo video segmentation algorithm combining disparity map and frame difference , 2008, 2008 3rd International Conference on Intelligent System and Knowledge Engineering.

[19]  Yu-Jin Zhang,et al.  Advances in image and video segmentation , 2006 .

[20]  Lance R. Williams,et al.  Local Parallel Computation of Stochastic Completion Fields , 1996, Neural Computation.

[21]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[22]  Ioannis Pitas,et al.  Temporal Video Segmentation by Graph Partitioning , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[23]  Nicole Vincent,et al.  A review of real-time segmentation of uncompressed video sequences for content-based search and retrieval , 2003, Real Time Imaging.

[24]  Demin Wang.  Unsupervised video segmentation based on watersheds and temporal tracking , 1998, IEEE Trans. Circuits Syst. Video Technol..

[25]  Domingo López-Rodríguez,et al.  A Dipolar Competitive Neural Network for Video Segmentation , 2008, IBERAMIA.

[26]  Mubarak Shah,et al.  Object based segmentation of video using color, motion and spatial information , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Tony Lindeberg,et al.  Scale-Space for Discrete Signals , 1990, IEEE Trans. Pattern Anal. Mach. Intell..