Unsupervised Segmentation of Color-Texture Regions in Images and Video

A method for unsupervised segmentation of color-texture regions in images and video is presented. This method, which we refer to as JSEG, consists of two independent steps: color quantization and spatial segmentation. In the first step, colors in the image are quantized to several representative classes that can be used to differentiate regions in the image. The image pixels are then replaced by their corresponding color class labels, thus forming a class-map of the image. The focus of this work is on spatial segmentation, where a criterion for "good" segmentation using the class-map is proposed. Applying the criterion to local windows in the class-map results in the "J-image," in which high and low values correspond to possible boundaries and interiors of color-texture regions. A region growing method is then used to segment the image based on the multiscale J-images. A similar approach is applied to video sequences. An additional region tracking scheme is embedded into the region growing process to achieve consistent segmentation and tracking results, even for scenes with nonrigid object motion. Experiments show the robustness of the JSEG algorithm on real images and video.

[1]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[2]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[3]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[4]  Glenn Healey,et al.  Markov Random Field Models for Unsupervised Segmentation of Textured Color Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Seong-Dae Kim,et al.  Spatio-temporal video segmentation using a joint similarity measure , 1997, IEEE Trans. Circuits Syst. Video Technol..

[7]  Dorin Comaniciu,et al.  Robust analysis of feature spaces: color image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  A. Murat Tekalp,et al.  Simultaneous motion estimation and segmentation , 1997, IEEE Trans. Image Process..

[9]  B. S. Manjunath,et al.  Edge flow: A framework of boundary detection and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Daphna Weinshall,et al.  Motion of disturbances: detection and tracking of multi-body non-rigid motion , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Abdelwaheb Marzouki,et al.  Estimation of generalized mixtures and its application in image segmentation , 1997, IEEE Trans. Image Process..

[12]  Christoph Stiller,et al.  Object-based estimation of dense motion fields , 1997, IEEE Trans. Image Process..

[13]  B. S. Manjunath,et al.  Content-based search of video using color, texture, and motion , 1997, Proceedings of International Conference on Image Processing.

[14]  Josef Kittler,et al.  Automatic watershed segmentation of randomly textured color images , 1997, IEEE Trans. Image Process..

[15]  King Ngi Ngan,et al.  Automatic segmentation of moving objects for video object plane generation , 1998, IEEE Trans. Circuits Syst. Video Technol..

[16]  Demin Wang.  Unsupervised video segmentation based on watersheds and temporal tracking , 1998, IEEE Trans. Circuits Syst. Video Technol..

[17]  Jitendra Malik,et al.  Motion segmentation and tracking using normalized cuts , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Paola Campadelli,et al.  Quantitative evaluation of color image segmentation results , 1998, Pattern Recognit. Lett..

[19]  Jia-Ping Wang,et al.  Stochastic Relaxation on Partitions With Connected Components and Its Application to Image Segmentation , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  B. S. Manjunath,et al.  NeTra-V: toward an object-based video representation , 1998, IEEE Trans. Circuits Syst. Video Technol..

[21]  Jun Zhang,et al.  Cluster validation for unsupervised stochastic model-based image segmentation , 1998, IEEE Trans. Image Process..

[22]  Murat Kunt,et al.  Spatiotemporal Segmentation Based on Region Merging , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Jitendra Malik,et al.  Color- and texture-based image segmentation using EM and its application to content-based image retrieval , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[24]  B. S. Manjunath,et al.  Peer group filtering and perceptual color image quantization , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[25]  Thomas de Quincey.  [C] , 2000, The Works of Thomas De Quincey, Vol. 1: Writings, 1799–1820.

[26]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..