Recovery of Egomotion and Segmentation of Independent Object Motion Using the EM Algorithm

Motion in image sequences can result from the motion of the observer (egomotion) and from the presence of independently moving objects (IMOs) within the field of view of the observer. Any vision system intended for all observer capable of motion needs the ability to distinguish between these two possibilities in order to successfully perform navigation and collision avoidance tasks. One approach to motion segmentation is to perform a statistical clustering on a set of local constraints on 3-D motion in the image. This thesis proposes two new methods, based on the EM algorithm, to perform robust motion segmentation on image sequences that contain IMOs. The first method uses statistical clustering of linear and bilinear constraints (derived from computed optical flow using subspace methods) on 3-D translation and rotation. The problems of outlier detection and determining number of processes and their initial parameters for the EM algorithm are considered. Also, analysis of the effects of IMO boundaries on linear constraints, as well as a derivation for the removal of bias inherent in translation estimates from linear constraints, are presented. Effects of fixation on detection of IMOs are considered. A framework for hypothesizing about motions underlying a set of constraint clusters is detailed. There exist situations in which 3-D motion constraints are not sufficient to perform segmentation. The second method tracks depth-structure over time and evaluates rigidity allowing IMOs to be identified as outliers. Results obtained from four image sequences are presented. The first sequence is synthetic optic flow generated from a depth map and contains one IMO. The second sequence was captured from a robot moving in an industrial environment. The third sequence is similar to the second, except the flow has been generated using a regularly spaced grid which assumes no prior segmentation of the image. The fourth sequence illustrates a case in which the 3-D constraints are insufficient to perform the segmentation, and the estimation of depth structure is needed to solve the problem. Finally, directions for future research into this problem are presented.

[1]  R. Hetherington.  The Perception of the Visual World , 1952 .

[2]  J. Gibson,et al.  Motion parallax as a determinant of perceived depth. , 1959, Journal of experimental psychology.

[3]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[4]  W. Fulks.  Advanced Calculus: An Introduction to Analysis , 1969 .

[5]  Andrea J. van Doorn,et al.  Invariant Properties of the Motion Parallax Field due to the Movement of Rigid Bodies Relative to an Observer , 1975 .

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[8]  D. Whitteridge.  Movements of the eyes R. H. S. Carpenter, Pion Ltd, London (1977), 420 pp., $27.00 , 1979, Neuroscience.

[9]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[10]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[11]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  J H Rieger,et al.  Processing differential image motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[13]  S. McKee,et al.  Precise velocity discrimination despite random variations in temporal frequency and contrast , 1986, Vision Research.

[14]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[15]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[16]  J. W. Wolfe,et al.  Time-Optimal Control of Saccadic Eye Movements , 1987, IEEE Transactions on Biomedical Engineering.

[17]  C. Jennison,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[18]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[19]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[20]  David J. Heeger,et al.  Egomotion And The Stabilized World , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[21]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[22]  Narendra Ahuja,et al.  Motion and Structure From Two Perspective Views: Algorithms, Error Analysis, and Error Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  S. Maybank.  Properties of essential matrices , 1990, Int. J. Imaging Syst. Technol..

[24]  K. Hanna.  Direct multi-resolution estimation of ego-motion and structure from motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[25]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[26]  G D Paige,et al.  Eye movement responses to linear head motion in the squirrel monkey. II. Visual-vestibular interactions and kinematic considerations. , 1991, Journal of neurophysiology.

[27]  C. Tomasi,et al.  Factoring image sequences into shape and motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[28]  G. Paige,et al.  Eye movement responses to linear head motion in the squirrel monkey. I. Basic characteristics. , 1991, Journal of neurophysiology.

[29]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[30]  A. Jepson,et al.  A fast subspace algorithm for recovering rigid motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[31]  Michael J. Black.  Robust incremental optical flow , 1992 .

[32]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[33]  Allan D. Jepson,et al.  A lattice framework for integrating vision modules , 1992, IEEE Trans. Syst. Man Cybern..

[34]  David J. Fleet.  Measurement of image velocity , 1992 .

[35]  Ajit Singh,et al.  Incremental estimation of image flow using a Kalman filter , 1992, J. Vis. Commun. Image Represent..

[36]  D. Sinclair.  Motion segmentation and local structure , 1993, 1993 (4th) International Conference on Computer Vision.

[37]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 1993, 1993 (4th) International Conference on Computer Vision.

[38]  Kenichi Kanatani.  Renormalization for unbiased estimation , 1993, 1993 (4th) International Conference on Computer Vision.

[39]  Radford M. Neal.  A new view of the EM algorithm that justifies incremental and other variants , 1993 .

[40]  Philip H. S. Torr,et al.  Outlier detection and motion segmentation , 1993, Other Conferences.

[41]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[42]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[43]  David J. Fleet,et al.  Stability of Phase Information , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Alex Pentland,et al.  Recursive estimation of structure and motion using relative orientation constraints , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[45]  N. Lobo.  Computing egomotion, shape and detecting independent motion from image motion , 1993 .

[46]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Kenichi Kanatani,et al.  Geometric computation for machine vision , 1993 .

[48]  Shahriar Negahdaripour,et al.  A generalized brightness change model for computing optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[49]  Elizabeth R. Stuck,et al.  Detecting Moving Objects Using the Rigidity Constraint , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Roberto Cipolla,et al.  Robust Egomotion Estimation from Affine Motion Parallax , 1994, ECCV.

[51]  Philip H. S. Torr,et al.  Stochastic Motion Clustering , 1994, ECCV.

[52]  Michael R. M. Jenkin,et al.  Detecting Floor Anomalies , 1994, BMVC.

[53]  Allan D. Jepson,et al.  A new closed-form solution for absolute orientation , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Rama Chellappa,et al.  Time-to-X: analysis of motion through temporal parameters , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Allan D. Jepson,et al.  Linear subspace methods for recovering translational direction , 1994 .

[56]  A. Jepson,et al.  Estimating multiple independent motions in segmented images using parametric models with local deformations , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[57]  Thierry Viéville,et al.  Canonic Representations for the Geometries of Multiple Projective Views , 1994, ECCV.

[58]  Josef Bigün,et al.  Segmentation of moving objects by robust motion parameter estimation over multiple frames , 1994, ECCV.

[59]  Rachid Deriche,et al.  Robust Recovery of the Epipolar Geometry for an Uncalibrated Stereo Rig , 1994, ECCV.

[60]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[61]  Steve Rogers,et al.  Adaptive Filter Theory , 1996 .