A Time-Consistent Video Segmentation Algorithm designed for Real-Time Implementation

In this paper, we propose a time consistent video segmentation algorithm designed for real-time implementation. Our segmentation algorithm is based on a region merging process that combines both spatial and motion information. The spatial segmentation takes benefit of an adaptive decision rule and a specific order of merging. Our method has proven to be efficient for the segmentation of natural images (at or textured regions) with few parameters to be set. Temporal consistency of the segmentation is ensured by incorporating motion information through the use of an improved change detection mask. This mask is designed using both illumination differences between frames, and region segmentation of the previous frame. By considering both pixel and region levels, we obtain a particularly efficient algorithm at a low computational cost, allowing its implementation in real-time on the TriMedia processor for CIF image sequences.

[1]  Frank Nielsen,et al.  Statistical region merging , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Richard Nock,et al.  Sorted region merging to maximize test reliability , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[3]  Patrick Bouthemy,et al.  Computation and analysis of image motion: A synopsis of current problems and methods , 1996, International Journal of Computer Vision.

[4]  Philippe Salembier,et al.  Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval , 2000, IEEE Trans. Image Process..

[5]  Yll Haxhimusa,et al.  Evaluating Minimum Spanning Tree Based Segmentation Algorithms , 2005, CAIP.

[6]  B. S. Manjunath,et al.  Unsupervised Segmentation of Color-Texture Regions in Images and Video , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Jean-Marc Chassery,et al.  Robust fast extraction of video objects combining frame differences and adaptive reference image , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[8]  Arie Shoshani,et al.  Optimizing connected component labeling algorithms , 2005, SPIE Medical Imaging.

[9]  C. McDiarmid.  Concentration , 1862, The Dental register.

[10]  Christophe Fiorio,et al.  Two Linear Time Union-Find Strategies for Image Processing , 1996, Theor. Comput. Sci..

[11]  Lionel Moisan,et al.  Meaningful Alignments , 2000, International Journal of Computer Vision.

[12]  Murat Kunt,et al.  Spatiotemporal Segmentation Based on Region Merging , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Daniel DeMenthon,et al.  SPATIO-TEMPORAL SEGMENTATION OF VIDEO BY HIERARCHICAL MEAN SHIFT ANALYSIS , 2002 .

[14]  Ronen Basri,et al.  Fast multiscale image segmentation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Qi Tian,et al.  Mean shift based video segment representation and applications to replay detection , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  Ioannis Patras,et al.  Video Segmentation by MAP Labeling of Watershed Segments , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Giancarlo Iannizzotto,et al.  Fast and accurate edge-based segmentation with no contour smoothing in 2-D real images , 2000, IEEE Trans. Image Process..

[18]  Lior Wolf,et al.  Patch-Based Texture Edges and Segmentation , 2006, ECCV.

[19]  Stéphane Pateux.  Spatial segmentation of color images according to the MDL formalism , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[20]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[21]  M. Habib.  Probabilistic methods for algorithmic discrete mathematics , 1998 .

[22]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Demin Wang.  Unsupervised video segmentation based on watersheds and temporal tracking , 1998, IEEE Trans. Circuits Syst. Video Technol..

[24]  Gilles Bertrand,et al.  Quasi-Linear Algorithms for the Topological Watershed , 2005, Journal of Mathematical Imaging and Vision.

[25]  Gudrun Klinker,et al.  A physical approach to color image understanding , 1989, International Journal of Computer Vision.

[26]  Luc Brun,et al.  Hierarchical Watersheds Within the Combinatorial Pyramid Framework , 2005, DGCI.

[27]  Kai-Kuang Ma,et al.  Automatic video object segmentation via 3D structure tensor , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[28]  Luc Vincent,et al.  Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Jean-Michel Jolion,et al.  A test to control a region growing process within a hierarchical graph , 2003, Pattern Recognit..

[30]  Levent Onural,et al.  Image sequence analysis for emerging interactive multimedia services-the European COST 211 framework , 1998, IEEE Trans. Circuits Syst. Video Technol..