A Feature-Based Approach for Determining Dense Long Range Correspondences

Planar motion models can provide gross motion estimation and good segmentation for image pairs with large inter-frame disparity. However, as the disparity becomes larger, the resulting dense correspondences will become increasingly inaccurate for everything but purely planar objects. Flexible motion models, on the other hand, tend to overfit and thus make partitioning difficult. For this reason, to achieve dense optical flow for image sequences with large inter-frame disparity, we propose a two stage process in which a planar model is used to get an approximation for the segmentation and the gross motion, and then a spline is used to refine the fit. We present experimental results for dense optical flow estimation on image pairs with large inter-frame disparity that are beyond the scope of existing approaches.

[1]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[3]  Jean-Marc Odobez,et al.  Direct incremental model-based image motion segmentation for video analysis , 1998, Signal Process..

[4]  Andrew Zisserman,et al.  Motion Clustering using the Trilinear Constraint over Three Views , 1995 .

[5]  Mads Nielsen,et al.  Computer Vision — ECCV 2002 , 2002, Lecture Notes in Computer Science.

[6]  Richard Szeliski,et al.  Vision Algorithms: Theory and Practice , 2002, Lecture Notes in Computer Science.

[7]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  C. Tomasi,et al.  Factoring image sequences into shape and motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[9]  Serge J. Belongie,et al.  What went where , 2003, CVPR 2003.

[10]  David Sinclair.  Europe-China Workshop on Geometrical Modelling and Invariants for Computer Vision, 1995 , 1995 .

[11]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Michael J. Black,et al.  Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Richard Szeliski,et al.  Hierarchical spline-based image registration , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Anand Rangarajan,et al.  A new algorithm for non-rigid point matching , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  Allen R. Hanson,et al.  Trackability as a cue for potential obstacle identification and 3-D description , 1993, International Journal of Computer Vision.

[17]  Harry Shum,et al.  Motion estimation with quadtree splines , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[19]  Michal Irani,et al.  All About Direct Methods , 1999 .

[20]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Serge J. Belongie,et al.  What went where [motion segmentation] , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[23]  Stefano Soatto,et al.  DEFORMOTION: Deforming Motion, Shape Average and the Joint Registration and Segmentation of Images , 2002, ECCV.