A multi-body factorization method for motion analysis

The structure from motion problem has been extensively studied in the field of computer vision. Yet, the bulk of the existing work assumes that the scene contains only a single moving object. The more realistic case where an unknown number of objects move in the scene has received little attention, especially for its theoretical treatment. We present a new method for separating and recovering the motion and shape of multiple independently moving objects in a sequence of images. The method does not require prior knowledge of the number of objects, nor is dependent on any grouping of features into an object at the image level. For this purpose, we introduce a mathematical construct of object shapes, called the shape interaction matrix, which is invariant to both the object motions and the selection of coordinate systems. This invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects. Once the structure is computed, it allows for segmenting features into objects by the process of transforming it into a canonical form, as well as recovering the shape and motion of each object. >

[1]  E. Chaffee,et al.  Effects of Varying the Wave Length of the Stimulating Light Upon the Electrical Response of the Retina , 1924 .

[2]  Ralph Otto Schmidt,et al.  A signal subspace approach to multiple emitter location and spectral estimation , 1981 .

[3]  Andrew B. Watson,et al.  A look at motion in the frequency domain , 1983 .

[4]  S Ullman,et al.  Maximizing Rigidity: The Incremental Recovery of 3-D Structure from Rigid and Nonrigid Motion , 1984, Perception.

[5]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[6]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  J. Demmel.  The smallest perturbation of a submatrix which lowers the rank and constrained total least squares problems , 1987 .

[8]  Shmuel Peleg,et al.  Computing two motions from three frames , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[9]  Takeo Kanade,et al.  Shape and motion without depth , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[10]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[11]  T. Boult,et al.  Factorization-based segmentation of motions , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[12]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[13]  P. Burt,et al.  Mechanisms for isolating component patterns in the sequential analysis of multiple motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[14]  G. Stewart.  Determining rank in the presence of error , 1992 .

[15]  Nassir Navab,et al.  From Multiple Objects Motion Analysis To Behavior-Based Object Recognition , 1992, ECAI.

[16]  A. Rosenfeld,et al.  Perceptual motion transparency : the role of geometrical information , 1992 .

[17]  D. Sinclair.  Motion segmentation and local structure , 1993, 1993 (4th) International Conference on Computer Vision.

[18]  G. W. Stewart,et al.  On the Early History of the Singular Value Decomposition , 1993, SIAM Rev..

[19]  Reg G. Willson.  Modeling and calibration of automated zoom lenses , 1994, Other Conferences.

[20]  C. W. Gear,et al.  Feature grouping in moving objects , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.