Entropy-based space-time segmentation in video sequences

This paper deals with motion-based segmentation of video sequences. Classical methods are based on the minimization of a penalty function of the prediction error (e.g. robust estimators). However, these estimators may not be efficient when the statistical distribution of the prediction error is not parametric. Instead, we propose to minimize the entropy of the prediction error while approximating its distribution by a non-parametric estimate. This approach only accounts for motion and cannot perform well in homogeneous regions. Therefore, a spatial term was added in the form of the entropy of the frame intensity. By combining motion and intensity (or color), it should be possible to segment objects which cannot be segmented correctly with one of these criterions alone. Minimization of the combined energy is achieved with an active contour algorithm. This method was applied to video segmentation and tracking on a synthetic, textured sequence and real world, standard sequences. Our algorithm show good results even when occlusions occur.

[1]  Ibrahim A. Ahmad,et al.  A nonparametric estimation of the entropy for absolutely continuous distributions (Corresp.) , 1976, IEEE Trans. Inf. Theory.

[2]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[3]  Josef Kittler,et al.  A Gradient-Based Method for General Motion Estimation and Segmentation , 1993, J. Vis. Commun. Image Represent..

[4]  Jean-Marc Odobez,et al.  Robust Multiresolution Estimation of Parametric Motion Models , 1995, J. Vis. Commun. Image Represent..

[5]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[6]  Michel Barlaud,et al.  Deterministic edge-preserving regularization in computed imaging , 1997, IEEE Trans. Image Process..

[7]  Rachid Deriche,et al.  Computing Optical Flow via Variational Techniques , 1999, SIAM J. Appl. Math..

[8]  Dorin Comaniciu,et al.  Mean shift analysis and applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Michel Barlaud,et al.  DREAM2S: Deformable Regions Driven by an Eulerian Accurate Minimization Method for Image and Video Segmentation , 2002, ECCV.

[10]  Daniel Cremers,et al.  Variational space-time motion segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Olivier D. Faugeras,et al.  Image Segmentation Using Active Contours: Calculus of Variations or Shape Gradients? , 2003, SIAM J. Appl. Math..

[12]  Michel Barlaud,et al.  DREAM2S: Deformable Regions Driven by an Eulerian Accurate Minimization Method for Image and Video Segmentation , 2002, International Journal of Computer Vision.

[13]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[14]  Joachim Weickert,et al.  Variational Optic Flow Computation with a Spatio-Temporal Smoothness Constraint , 2001, Journal of Mathematical Imaging and Vision.

[15]  John W. Fisher,et al.  Submitted to Ieee Transactions on Image Processing a Nonparametric Statistical Method for Image Segmentation Using Information Theory and Curve Evolution , 2022 .

[16]  Michel Barlaud,et al.  Segmentation of Vectorial Image Features Using Shape Gradients and Information Measures , 2006, Journal of Mathematical Imaging and Vision.