Spatio-temporal segmentation of image sequences for object-oriented low bit-rate image coding

Abstract This paper describes a new method of segmentation of time-varying image sequences whose goal is object-oriented image coding. The segmentation represents a partition of each frame of the sequence into a set of regions which are homogeneous with regard to motion criterion. The region borders correspond to spatial contours of objects in the frame. Each spatio-temporal region is characterized by its temporal component, which is a model-dependent vector of motion parameters, and a structural component representing the polygonal approximation of the spatial contour of the region. The construction of spatio-temporal segmentation includes two phases: the initialization step and temporal tracking. The initialization step is based on the spatial segmentation of the first frame of the sequence. Then homogeneous spatial regions are merged through motion estimation in accordance with a motion-based criterion. The temporal tracking consists of the projection of the segmentation along the time axis, and its adjustment. Special attention is paid to the processing of occlusions. A predictive coding scheme is proposed which is based on the temporal coherence of the segmentation. This scheme is promising for a low bit-rate image compression. The results for teleconference and TV sequences show the high visual quality of reconstructed only by prediction images. Moreover, the bit-rates for motion coding are very low: from 0.002 to 0.007 bit/pixel for teleconference sequence and from 0.004 to 0.021 bit/pixel for complex TV sequence. A scheme for encoding of the structural information is proposed which requires 0.083 – 0.17 bit per pixel depending on the content of the sequence.

[1]  Dominique Barba,et al.  Image segmentation by region-contour cooperation as a basis for efficient coding scheme , 1992, Other Conferences.

[2]  Michael Hötter,et al.  Object-oriented analysis-synthesis coding based on moving two-dimensional objects , 1990, Signal Process. Image Commun..

[3]  Josef Kittler,et al.  A Gradient-Based Method for General Motion Estimation and Segmentation , 1993, J. Vis. Commun. Image Represent..

[4]  Siu-Leong Iu,et al.  Robust estimation of motion vector fields with discontinuity and occlusion using local outliers rejection , 1993, Other Conferences.

[5]  Murat Kunt,et al.  Image sequence coding by split and merge , 1991, IEEE Trans. Commun..

[6]  Patrick Bouthemy,et al.  Multimodal motion estimation and segmentation using Markov random fields , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[7]  Dominique Barba,et al.  Image segmentation by region-contour cooperation for image coding , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol. III. Conference C: Image, Speech and Signal Analysis,.

[8]  Kiyoharu Aizawa,et al.  Model-based analysis synthesis image coding (MBASIC) system for a person's face , 1989, Signal Process. Image Commun..

[9]  Henri Nicolas,et al.  Region-based motion estimation using deterministic relaxation schemes for image sequence coding , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[10]  Hiroshi Harashima,et al.  Three‐dimensional (3‐D) facial model‐based description and synthesis of facial expressions , 1991 .

[11]  Jörn Ostermann,et al.  Object-based analysis-synthesis coding based on the source model of moving rigid 3D objects , 1994, Signal Process. Image Commun..

[12]  Hans-Hellmut Nagel,et al.  Model-Based Object Tracking in Traffic Scenes , 1992, ECCV.

[13]  Dimitrios Tzovaras,et al.  Coding for monoscopic and stereoscopic 3D medical data visualization , 1994 .

[14]  Norbert Diehl,et al.  Object-oriented motion estimation and segmentation in image sequences , 1991, Signal Process. Image Commun..

[15]  Hiromi T. Tanaka,et al.  Time-varying homotopy and the animation of facial expressions for 3D virtual space teleconferencing , 1993, Other Conferences.

[16]  M. Kunt,et al.  Second-generation image-coding techniques , 1985, Proceedings of the IEEE.

[17]  Joseph Ronsin,et al.  Adaptive block truncation coding scheme using an edge following algorithm , 1982, ICASSP.