Hierarchical segmentation of video sequences for content manipulation and adaptive coding

The problem of unsupervised hierarchical segmentation of video content is discussed. A segmentation approach combining local spatial homogeneity with motion homogeneity is proposed. Motion is the main characteristic which allows the decomposition of video sequences into a nested hierarchy of representation levels, each of them corresponding to a specified quality of motion compensation in terms of a proposed criterion. Two strategies of construction of such a hierarchy are studied. The first, a purely bottom-up approach, favours local motion homogeneity and constructs a hierarchical representation starting from the most detailed (gray-level based) segmentation then progressively merging the region primitives. The second, a cooperative strategy, uses a global motion analysis to guide the bottom-up approach. Results on test sequences MPEG ISO SC29/WG11 show interesting performances of such techniques for an automatic construction of hierarchical representations of time-varying images, allowing for flexible access and manipulation of video content.

[1]  Patrick Bouthemy,et al.  Multimodal motion estimation and segmentation using Markov random fields , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[2]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[4]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[5]  Dominique Barba,et al.  Spatio-temporal segmentation of image sequences for object-oriented low bit-rate image coding , 1996, Signal Process. Image Commun..

[6]  Montse Pardàs,et al.  Morphological operators for image and video compression , 1996, IEEE Trans. Image Process..

[7]  Henri Sanson.  Toward a robust parametric identification of motion on regions of arbitrary shape by nonlinear optimization , 1995, Proceedings., International Conference on Image Processing.

[8]  Jenny Benois-Pineau,et al.  Region-based coding scheme for the transmission of video sequences via channels of varying very low bit rate , 1997, Electronic Imaging.

[9]  Murat Kunt,et al.  Image sequence coding by split and merge , 1991, IEEE Trans. Commun..

[10]  Siu-Leong Iu,et al.  Robust estimation of motion vector fields with discontinuity and occlusion using local outliers rejection , 1993, Other Conferences.

[11]  Josef Bigün,et al.  Segmentation of moving objects by robust motion parameter estimation over multiple frames , 1994, ECCV.

[12]  Jörn Ostermann,et al.  Object-oriented analysis-synthesis coding of moving images , 1989, Signal Process. Image Commun..

[13]  Albert Oliveras,et al.  Motion analysis of image sequences using connected operators , 1997, Electronic Imaging.

[14]  Philippe Salembier,et al.  Morphological multiscale segmentation for image coding , 1994, Signal Process..

[15]  Philippe Salembier,et al.  Video sequence segmentation based on rate-distortion theory , 1996, Other Conferences.

[16]  Dominique Barba,et al.  Robust segmentation of moving image sequences , 1997, Proceedings of International Conference on Image Processing.

[17]  Jörn Ostermann,et al.  Object-based analysis-synthesis coding based on the source model of moving rigid 3D objects , 1994, Signal Process. Image Commun..

[18]  Jean-Marc Odobez,et al.  Detection of multiple moving objects using multiscale MRF with camera motion compensation , 1994, Proceedings of 1st International Conference on Image Processing.

[19]  Dominique Barba,et al.  Image segmentation by region-contour cooperation for image coding , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol. III. Conference C: Image, Speech and Signal Analysis,.

[20]  Elisa Sayrol.  Modeling the displaced frame difference as an alpha-stable distribution , 1997, Proceedings of International Conference on Image Processing.

[21]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[22]  Dominique Barba,et al.  Image segmentation by region-contour cooperation as a basis for efficient coding scheme , 1992, Other Conferences.

[23]  Emmanuel Reusens.  Joint optimization of representation model and frame segmentation for generic video compression , 1995, Signal Process..

[24]  Andrew Lippman,et al.  Spatio-temporal segmentation based on motion and static segmentation , 1995, Proceedings., International Conference on Image Processing.