Improved video segmentation through robust statistics and MPEG-7 features

Video segmentation is an important task for a wide range of applications like content-based video coding or video retrieval. In this paper, a new spatio-temporal video segmentation framework is presented. It is based upon robust statistics, namely an M-estimator, and incorporates an MPEG-7 descriptor for consistent temporal labeling of identified textures. The algorithm is based on assumptions about the geometric modifications a given moving region undergoes with time as well as on its surface properties. Homogeneously moving segments are described using a parametric motion scheme. The latter is used to piecewise fit the optical flow field in order to extract rigid motion areas. Robust statistics are used to carefully constrain split, merge and contour refinement decisions. Experimental results show that regions detected by the proposed method are more reliable than the state-of-the-art. True region boundaries are moreover better detected.

[1]  Qian Huang,et al.  Quantitative methods of evaluating image segmentation , 1995, Proceedings., International Conference on Image Processing.

[2]  Michael Spann,et al.  A quad-tree approach to image segmentation which combines statistical and spatial information , 1985, Pattern Recognit..

[3]  Stuart C. Schwartz,et al.  A transform domain approach to real-time foreground segmentation in video sequences , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[4]  Jens-Rainer Ohm,et al.  Multimedia Communication Technology , 2004 .

[5]  Ferran Marqués,et al.  A motion-based binary partition tree approach to video object segmentation , 2005, IEEE International Conference on Image Processing 2005.

[6]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  B. S. Manjunath,et al.  Introduction to mpeg-7 , 2002 .

[9]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..