A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models

Describing a video sequence in terms of a small number of coherently moving segments is useful for tasks ranging from video compression to event perception. A promising approach is to view the motion segmentation problem in a mixture estimation framework. However, existing formulations generally use only the motion, data and thus fail to make use of static cues when segmenting the sequence. Furthermore, the number of models is either specified in advance or estimated outside the mixture model framework. In this work we address both of these issues. We show how to add spatial constraints to the mixture formulations and present a variant of the EM algorithm that males use of both the form and the motion constraints. Moreover this algorithm estimates the number of segments given knowledge about the level of model failure expected in the sequence. The algorithm's performance is illustrated on synthetic and real image sequences.

[1]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[2]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[3]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[4]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[5]  Federico Girosi,et al.  Parallel and Deterministic Algorithms from MRFs: Surface Reconstruction , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Edward H. Adelson,et al.  Probability distributions of optical flow , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Michal Irani,et al.  Image sequence enhancement using multiple motions analysis , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Yoshiaki Shirai,et al.  Segmentation and 2D motion estimation by region fragments , 1993, 1993 (4th) International Conference on Computer Vision.

[10]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[11]  Alan L. Yuille,et al.  Statistical Physics, Mixtures of Distributions, and the EM Algorithm , 1994, Neural Computation.

[12]  Joshua B. Tenenbaum,et al.  Factorial Learning by Clustering Features , 1994, NIPS.

[13]  Jun Zhang,et al.  Maximum-likelihood parameter estimation for unsupervised stochastic model-based image segmentation , 1994, IEEE Trans. Image Process..

[14]  Jean-Marc Odobez,et al.  Detection of multiple moving objects using multiscale MRF with camera motion compensation , 1994, Proceedings of 1st International Conference on Image Processing.

[15]  A. Jepson,et al.  Estimating multiple independent motions in segmented images using parametric models with local deformations , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[16]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.