Video object segmentation using Bayes-based temporal tracking and trajectory-based region merging

A novel unsupervised video object segmentation algorithm is presented, aiming to segment a video sequence to objects: spatiotemporal regions representing a meaningful part of the sequence. The proposed algorithm consists of three stages: initial segmentation of the first frame using color, motion, and position information, based on a variant of the K-means-with-connectivity-constraint algorithm; a temporal tracking algorithm, using a Bayes classifier and rule-based processing to reassign changed pixels to existing regions and to efficiently handle the introduction of new regions; and a trajectory-based region merging procedure that employs the long-term trajectory of regions, rather than the motion at the frame level, so as to group them to objects with different motion. As shown by experimental evaluation, this scheme can efficiently segment video sequences with fast moving or newly appearing objects. A comparison with other methods shows segmentation results corresponding more accurately to the real objects appearing on the image sequence.

[1]  Kai-Kuang Ma,et al.  A new diamond search algorithm for fast block-matching motion estimation , 2000, IEEE Trans. Image Process..

[2]  Yutaka Yokoyama,et al.  Very low bit rate video coding using arbitrarily shaped region-based motion compensation , 1995, IEEE Trans. Circuits Syst. Video Technol..

[3]  B. S. Manjunath,et al.  Unsupervised Segmentation of Color-Texture Regions in Images and Video , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Georgios Tziritas,et al.  Moving object localisation using a multi-label fast marching algorithm , 2001, Signal Process. Image Commun..

[5]  King Ngi Ngan,et al.  Automatic segmentation of moving objects for video object plane generation , 1998, IEEE Trans. Circuits Syst. Video Technol..

[6]  Tianli Yu,et al.  Retrieval of video clips using global motion information , 2001 .

[7]  Michael G. Strintzis,et al.  Motion estimation based on spatiotemporal warping for very low bit-rate coding , 1997, IEEE Trans. Commun..

[8]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[9]  Levent Onural,et al.  Utilization of the recursive shortest spanning tree algorithm for video-object segmentation by 2-D affine motion modeling , 2000, IEEE Trans. Circuits Syst. Video Technol..

[10]  Aaron F. Bobick,et al.  Visual Tracking Using Closed-Worlds , 1995 .

[11]  Touradj Ebrahimi,et al.  Video segmentation based on multiple features for interactive multimedia applications , 1998, IEEE Trans. Circuits Syst. Video Technol..

[12]  Thomas Sikora,et al.  The MPEG-4 video standard verification model , 1997, IEEE Trans. Circuits Syst. Video Technol..

[13]  Levent Onural,et al.  Image sequence analysis for emerging interactive multimedia services-the European COST 211 framework , 1998, IEEE Trans. Circuits Syst. Video Technol..

[14]  Ferran Marqués,et al.  Region-based representations of image and video: segmentation tools for multimedia services , 1999, IEEE Trans. Circuits Syst. Video Technol..

[15]  Georgios Tziritas,et al.  Color and/or texture segmentation using deterministic relaxation and fast marching algorithms , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[16]  Michael G. Strintzis,et al.  Spatiotemporal segmentation and tracking of objects for visualization of videoconference image sequences , 2000, IEEE Trans. Circuits Syst. Video Technol..

[17]  Dimitrios Tzovaras,et al.  Three-dimensional camera motion estimation and foreground/background separation for stereoscopic image sequences , 1997 .

[18]  Seong-Dae Kim,et al.  Spatio-temporal video segmentation using a joint similarity measure , 1997, IEEE Trans. Circuits Syst. Video Technol..

[19]  Georgios Tziritas,et al.  Video Segmentation Using Fast Marching and Region Growing Algorithms , 2002, EURASIP J. Adv. Signal Process..

[20]  Steve Mann,et al.  Video orbits of the projective group a simple approach to featureless estimation of parameters , 1997, IEEE Trans. Image Process..

[21]  Mohammed Ghanbari,et al.  Key components for an advanced segmentation system , 2002, IEEE Trans. Multim..

[22]  Chein-Wei Jen,et al.  On the data reuse and memory bandwidth analysis for full-search block-matching VLSI architecture , 2002, IEEE Trans. Circuits Syst. Video Technol..

[23]  Fabrice Moscheni Spatio-temporal segmentation and object tracking: An applicaton to second generation video coding , 1997, Signal Process..

[24]  Michael G. Strintzis,et al.  Segmentation and Content-Based Watermarking for Color Image and Image Region Indexing and Retrieval , 2002, EURASIP J. Adv. Signal Process..

[25]  Leonardo Chiariglione MPEG and multimedia communications , 1997, IEEE Trans. Circuits Syst. Video Technol..

[26]  Demin Wang Unsupervised video segmentation based on watersheds and temporal tracking , 1998, IEEE Trans. Circuits Syst. Video Technol..

[27]  Walter Stechele,et al.  A video segmentation algorithm for hierarchical object representations and its implementation , 1999, IEEE Trans. Circuits Syst. Video Technol..

[28]  Chung-Lin Huang,et al.  A robust scene-change detection method for video segmentation , 2001, IEEE Trans. Circuits Syst. Video Technol..

[29]  Byung Cheol Song,et al.  A fast multi-resolution block matching algorithm and its LSI architecture for low bit-rate video coding , 2001, IEEE Trans. Circuits Syst. Video Technol..

[30]  Daniel Gatica-Perez,et al.  Semantic video object extraction using four-band watershed and partition lattice operators , 2001, IEEE Trans. Circuits Syst. Video Technol..

[31]  Michael G. Strintzis,et al.  3-D Camera Motion Estimation and Foreground / Background Separation for Stereoscopic Image Sequences , 1997 .

[32]  Yao Wang,et al.  Video Processing and Communications , 2001 .

[33]  Rangachar Kasturi,et al.  Machine vision , 1995 .

[34]  R. Koenen,et al.  MPEG-4 multimedia for our time , 1999 .

[35]  Ioannis Patras,et al.  Video Segmentation by MAP Labeling of Watershed Segments , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Michael G. Strintzis,et al.  Evaluation of multiresolution block matching techniques for motion and disparity estimation , 1994, Signal Process. Image Commun..

[37]  Fernando Pereira,et al.  MPEG-7: A standardised description of audiovisual content , 2000, Signal Process. Image Commun..

[38]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[39]  Michael G. Strintzis,et al.  Motion and disparity field estimation using rate-distortion optimization , 1998, IEEE Trans. Circuits Syst. Video Technol..

[40]  Fernando Pereira MPEG-4: Why, what, how and when? , 2000, Signal Process. Image Commun..

[41]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[42]  Shih-Fu Chang,et al.  Overview of the MPEG-7 standard , 2001, IEEE Trans. Circuits Syst. Video Technol..