Joint optical flow estimation, segmentation, and 3D interpretation with level sets

This paper describes a variational method with active curve evolution and level sets for the estimation, segmentation, and 3D interpretation of optical flow generated by independently moving rigid objects in space. Estimation, segmentation, and 3D interpretation are performed jointly. Segmentation is based on an estimate of optical flow consistent with a single rigid motion in each segmentation region. The method, which allows both viewing system and viewed objects to move, results in three steps iterated until convergence: (a) evolution of closed curves via level sets and, in each region of the segmentation, (b) linear least squares computation of the essential parameters of rigid motion, (c) estimation of optical flow consistent with a single rigid motion. The translational and rotational components of rigid motion and regularized relative depth are recovered analytically for each region of the segmentation from the estimated essential parameters and optical flow. Several examples with real image sequences are provided which verify the validity of the method.

[1]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[2]  W. James MacLean,et al.  Recovery of Egomotion and Segmentation of Independent Object Motion Using the EM Algorithm , 1994, BMVC.

[3]  Amar Mitiche,et al.  Concurrent 3-D motion segmentation and 3-D interpretation of temporal sequences of monocular images , 2006, IEEE Transactions on Image Processing.

[4]  Berthold K. P. Horn,et al.  Direct methods for recovering motion , 1988, International Journal of Computer Vision.

[5]  A. Mitiche.  Computational Analysis of Visual Motion , 1994, Advances in Computer Vision and Machine Intelligence.

[6]  Rachid Deriche,et al.  Computing Optical Flow via Variational Techniques , 1999, SIAM J. Appl. Math..

[7]  Carlos Vázquez,et al.  Approximation of images by basis functions for multiple region segmentation with level sets , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[8]  Allan D. Jepson,et al.  Subspace methods for recovering rigid motion I: Algorithm and implementation , 2004, International Journal of Computer Vision.

[9]  Amar Mitiche,et al.  Dense 3D interpretation of image sequences: a variational approach using anisotropic diffusion , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[10]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[11]  Steven A. Shafer,et al.  Dense Structure from a Dense Optical Flow Sequence , 1998, Comput. Vis. Image Underst..

[12]  Laveen N. Kanal,et al.  3-D Motion Estimation from Motion Field , 1995, Artif. Intell..

[13]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  M. Ali Taalebinezhaad.  Direct recovery of motion and shape in the general case by fixation , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[15]  Enrico De Micheli,et al.  Motion and structure from one dimensional optical flow , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  K. Nakayama,et al.  Optical Velocity Patterns, Velocity-Sensitive Neurons, and Space Perception: A Hypothesis , 1974, Perception.

[17]  Jitendra Malik,et al.  Rigid Body Segmentation and Shape Description from Dense Optical Flow Under Weak Perspective , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Sridhar Srinivasan,et al.  Extracting Structure from Optical Flow Using the Fast Error Search Technique , 2000, International Journal of Computer Vision.

[19]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[20]  Tony F. Chan,et al.  Variational PDE models in image processing , 2002 .

[21]  Carlos Vázquez,et al.  Image segmentation as regularized clustering: a fully global curve evolution method , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[22]  Gjlles Aubert,et al.  Mathematical problems in image processing , 2001 .

[23]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[24]  R. Deriche,et al.  A variational framework for active and adaptative segmentation of vector valued images , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[25]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[26]  Berthold K. P. Horn,et al.  Passive navigation , 1982, Computer Vision Graphics and Image Processing.

[27]  Yeung Sam Hung,et al.  A Kalman Filter Approach to Direct Depth Estimation Incorporating Surface Structure , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Jake K. Aggarwal,et al.  On the computation of motion from sequences of images-A review , 1988, Proc. IEEE.

[29]  Carlos Vázquez,et al.  Image partioning by level set multiregion competition , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[30]  Azriel Rosenfeld,et al.  A hierarchical approach for obtaining structure from two-frame optical flow , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[31]  Tony F. Chan,et al.  An Active Contour Model without Edges , 1999, Scale-Space.

[32]  Thomas S. Huang,et al.  Motion and structure from feature correspondences: a review , 1994, Proc. IEEE.

[33]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[34]  Yiannis Aloimonos,et al.  Structure from Motion: Beyond the Epipolar Constraint , 2000, International Journal of Computer Vision.

[35]  Narendra Ahuja,et al.  Segmentation and Factorization-Based Motion and Structure Estimation for Long Image Sequences , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Eric Dubois.  A projection method to generate anaglyph stereo images , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[37]  Larry S. Davis,et al.  What can projections of flow fields tell us about the visual motion , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[38]  Amar Mitiche,et al.  MDL estimation of a dense map of relative depth and 3D motion from a temporal sequence of images , 2003, Pattern Analysis & Applications.

[39]  Michael A. Malcolm,et al.  Computer methods for mathematical computations , 1977 .

[40]  Behzad Shahraray,et al.  Robust Depth Estimation From Optical Flow , 1988, [1988 Proceedings] Second International Conference on Computer Vision.