Rigid body segmentation and shape description from dense optical flow under weak perspective

We present an algorithm for identifying and tracking independently moving rigid objects from optical flow. The proposed method uses the fact that each distinct object has a unique epipolar constraint associated with its motion. This is in contrast to using local optical flow information for segmentation. Thus motion discontinuities based on self-occlusion are distinguished from those due to separate objects. The use of epipolar geometry allows for the determination of individual motion parameters for each object as well as the recovery of relative depth for each point on the object. The segmentation problem is formulated as a scene partitioning problem and a statistic-based algorithm which uses only nearest neighbor interactions and a finite number of iterations is developed. A Kalman filter based approach is used for tracking motion parameters with time. The algorithm assumes an affine camera where perspective effects are limited to changes in overall scale. No camera calibration parameters are required.<<ETX>>

[1]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[2]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[3]  P. Perona,et al.  Three dimensional transparent structure segmentation and multiple 3D motion estimation from monocular perspective image sequences , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[4]  Alessandro Verri,et al.  Identifying multiple motions from optical flow , 1992, ECCV.

[5]  Philip H. S. Torr,et al.  Stochastic Motion Clustering , 1994, ECCV.

[6]  David B. Cooper,et al.  Toward a Model-Based Bayesian Theory for Estimating and Recognizing Parameterized 3-D Objects Using Two or More Images Taken from Different Positions , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Claude L. Fennema,et al.  Scene Analysis Using Regions , 1970, Artif. Intell..

[8]  Brian G. Schunck,et al.  Image Flow Segmentation and Estimation by Constraint Line Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Michael J. Black,et al.  Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[11]  P. Belhumeur.  A Bayesian approach to the stereo correspondence problem , 1993 .

[12]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 1993, 1993 (4th) International Conference on Computer Vision.

[13]  Thomas S. Huang,et al.  Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surfaces , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[16]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[17]  Michael J. Black.  Combining Intensity and Motion for Incremental Segmentation and Tracking Over Long Image Sequences , 1992, ECCV.

[18]  P. Perona,et al.  Recursive Motion Estimation on the Essential Manifold , 1993 .

[19]  Federico Girosi,et al.  Parallel and Deterministic Algorithms from MRFs: Surface Reconstruction , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Hans-Hellmut Nagel,et al.  Motion Boundary Detection in Image Sequences by Local Stochastic Tests , 1994, ECCV.

[21]  Narendra Ahuja,et al.  Optimal motion and structure estimation , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Anselm Spoerri,et al.  The early detection of motion boundaries , 1990, ICCV 1987.

[23]  Thomas S. Huang,et al.  Motion and structure from orthographic projections , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[24]  Andrew Zisserman,et al.  Motion From Point Matches Using Affine Epipolar Geometry , 1994, ECCV.

[25]  S. Ullman.  The Interpretation of Visual Motion , 1979 .

[26]  David W. Murray,et al.  Detecting the image boundaries between optical flow fields from several moving planar facets , 1986, Pattern Recognit. Lett..

[27]  Valdis Berzins,et al.  Dynamic Occlusion Analysis in Optical Flow Fields , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[29]  Rachid Deriche,et al.  Robotique, Image Et Vision on Determining the Fundamental Matrix: Analysis of Different Methods and Experimental Results on Determining the Fundamental Matrix: Analysis of Diierent Methods and Experimental Results Programme 4: Robotique, Image Et Vision , 1993 .

[30]  Philip H. S. Torr,et al.  Outlier detection and motion segmentation , 1993, Other Conferences.

[31]  J. Weber.  Scene partitioning via statistic-based region growing , 1995, Electronic Imaging.

[32]  Michael J. Black,et al.  Constraints for the Early Detection of Discontinuity from Motion , 1990, AAAI.