Shape and motion from image streams: a factorization method.

Inferring scene geometry and camera motion from a stream of images is possible in principle, but it is an ill-conditioned problem when the objects are distant with respect to their size. We have developed a factorization method that can overcome this difficulty by recovering shape and motion without computing depth as an intermediate step. An image stream can be represented by the 2F x P measurement matrix of the image coordinates of P points tracked through F frames. Under orthographic projection this matrix is of rank 3. Using this observation, the factorization method uses the singular value decomposition technique to factor the measurement matrix into two matrices, which represent object shape and camera motion, respectively. The method can also handle and obtain a full solution from a partially filled-in measurement matrix, which occurs when features appear and disappear in the image sequence due to occlusions or tracking failures. The method gives accurate results and does not introduce smoothing in either shape or motion. We demonstrate this with a series of experiments on laboratory and outdoor image streams, with and without occlusions.

[1]  R. Dreisbach,et al.  STANFORD UNIVERSITY. , 1914, Science.

[2]  Michael A. Malcolm,et al.  Computer methods for mathematical computations , 1977 .

[3]  D Marr,et al.  Bandpass channels, zero-crossings, and early visual information processing. , 1979, Journal of the Optical Society of America.

[4]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[5]  Jake K. Aggarwal,et al.  Computer Tracking of Objects Moving in Space , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Hans P. Moravec.  Obstacle avoidance and navigation in the real world by a seeing robot rover , 1980 .

[7]  Thomas S. Huang,et al.  Estimating three-dimensional motion parameters of a rigid planar patch , 1981 .

[8]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[9]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[10]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[11]  Berthold K. P. Horn,et al.  Passive navigation , 1982, Computer Vision Graphics and Image Processing.

[12]  Thomas S. Huang,et al.  Estimating three-dimensional motion parameters of a rigid planar patch, II: Singular value decomposition , 1982 .

[13]  Azriel Rosenfeld,et al.  Gray-level corner detection , 1982, Pattern Recognit. Lett..

[14]  Hans-Hellmut Nagel,et al.  Volumetric model and 3D trajectory of a moving car derived from monocular TV frame sequences of a street scene , 1981, Comput. Graph. Image Process..

[15]  Ramesh C. Jain,et al.  Direct Computation of the Focus of Expansion , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Daryl T. Lawton,et al.  Processing translational motion sequences , 1983, Comput. Vis. Graph. Image Process..

[17]  Thomas S. Huang,et al.  Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surfaces , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Charles Elving Thorpe,et al.  Fido: vision and navigation for a robot rover , 1984 .

[19]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[20]  S Ullman,et al.  Maximizing Rigidity: The Incremental Recovery of 3-D Structure from Rigid and Nonrigid Motion , 1984, Perception.

[21]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Allen M. Waxman,et al.  Contour Evolution, Neighborhood Deformation, and Global Image Flow: Planar Surfaces in Motion , 1985 .

[23]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using orthonormal matrices , 1988 .

[24]  Josef Kittler,et al.  Proceedings of the 4th International Conference on Pattern Recognition , 1988 .

[25]  J. Aloimonos,et al.  Optimal motion estimation , 1989, [1989] Proceedings. Workshop on Visual Motion.

[26]  Joachim Heel,et al.  Dynamic Motion Vision , 1989, Other Conferences.

[27]  Narendra Ahuja,et al.  A direct data approximation based motion estimation algorithm , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[28]  Allan D. Jepson,et al.  Visual Perception of Three-Dimensional Motion , 1990, Neural Computation.

[29]  R. Chellappa,et al.  Recursive 3-D motion estimation from a monocular image sequence , 1990 .

[30]  Takeo Kanade,et al.  Shape and motion without depth , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[31]  Takeo Kanade,et al.  A locally adaptive window for signal matching , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[32]  T. Boult,et al.  Factorization-based segmentation of motions , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[33]  C. Tomasi.  Detection and Tracking of Point Features , 1991 .

[34]  James M. Rehg,et al.  Visual tracking with deformation models , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[35]  Narendra Ahuja,et al.  Motion and Structure Factorization and Segmentation of Long Multiple Motion Image Sequences , 1992, ECCV.