Perceptually Organized Em: a Framework for Motion Segmentation That Combines Information about Form and Motion

Recent progress in motion analysis has been achieved with systems that estimate global pa-rameterized motion by integrating multiple constraints. The success of these approaches depends critically on the ability to segment constraints derived from diierent motions. Hence the problems of motion estimation and segmen-tation are tightly coupled. We believe it is impossible to solve these problems solely in the motion domain, and that mechanisms of spatial form analysis must be incorporated into the motion estimation procedure. We present a new framework which allows the incorporation of form information in a graceful manner. It combines concepts from perceptual organization with the powerful optimization technique of EM. We show that the algorithm is guaranteed to decrease a cost function at every iteration, and that in the absence of form information the cost function reduces to the one minimized by EM. We demonstrate that the approach can achieve good motion estimation and segmentation with challenging motion sequences. Recent progress in motion analysis has been achieved with systems that estimate global parameterized mo-These methods have advantages over local optic ow in that they overcome the local ill-posedness of the motion estimation problem by integrating multiple constraints. The sucess of these approaches , however, depends critically on the ability to segment constraints derived from diierent motions. Hence the problems of motion estimation and segmentation have become tightly coupled. The joint solution of these problems remains diicult, even for scenes that are very simple. Consider, for example, the scene shown in g 1(a) (see also Bergen et al., 1990]). Two bars of diierent grey shades are moving, one to the left and one to the right. We will consider how several kinds of motion analyses treat this input. First, the output of a standard least-squares optic ow routine is shown in g. 1(b), as an arrow plot; the x and y components of velocity are shown in g. 1(c) and (d) (velocities below some threshold conndence are set to zero, a b c d Figure 1: a A simple image sequence which causes problems for traditional motion estimation algorithms. b Least squares optical ow shown as an arrow plot c Least squares optical ow horizontal component. d Least squares optical ow vertical component. the algorithm is an implementation of Lucas and Kanade (1981) modiied according to Simoncelli et al., 1991]). Although this sequence is a synthetic one, it illustrates problems that occur frequently in analyzing real …

[1]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[4]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[5]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[7]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[9]  Lance R. Williams Perceptual organization of occluding contours , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[10]  Shmuel Peleg,et al.  Computing two motions from three frames , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[11]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[13]  Federico Girosi,et al.  Parallel and Deterministic Algorithms from MRFs: Surface Reconstruction , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Edward H. Adelson,et al.  Probability distributions of optical flow , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Michal Irani,et al.  Image sequence enhancement using multiple motions analysis , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Radford M. Neal A new view of the EM algorithm that justifies incremental and other variants , 1993 .

[17]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[19]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[20]  Michael J. Black,et al.  The outlier process: unifying line processes and robust statistics , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Alan L. Yuille,et al.  Statistical Physics, Mixtures of Distributions, and the EM Algorithm , 1994, Neural Computation.

[22]  Edward J. Delp,et al.  Parameter estimation and segmentation of noisy or textured images using the EM algorithm and MPM estimation , 1994, Proceedings of 1st International Conference on Image Processing.

[23]  Hans-Hellmut Nagel,et al.  Optical Flow Estimation: Advances and Comparisons , 1994, ECCV.

[24]  A. Jepson,et al.  Estimating multiple independent motions in segmented images using parametric models with local deformations , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[25]  P. Anandan,et al.  Accurate computation of optical flow by using layered motion representations , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[26]  L. Finkel,et al.  Intermediate-Level Visual Representations and the Construction of Surface Perception , 1995, Journal of Cognitive Neuroscience.