A hierarchical approach for obtaining structure from two-frame optical flow

A hierarchical iterative algorithm is proposed for extracting structure from two-frame optical flow. The algorithm exploits two facts: one is that in many applications, such as face and gesture recognition, the depth variation of the visible surface of an object in a scene is small compared to the distance between the optical center and the object; the other is that the time aliasing problem is alleviated at the coarse level for any two-frame optical flow estimate so that the estimate tends to be more accurate. A hierarchical representation for the relationship between the optical flow, depth, and the motion parameters is derived, and the resulting non-linear system is iteratively solved through two linear subsystems. At the coarsest level, the surface of the object tends to be flat, so that the inverse depth tends to be a constant, which is used as the initial depth map. Inverse depth and motion parameters are estimated by the two linear subsystems at each level and the results are propagated to finer levels. Error analysis and experiments using both computer-rendered images and real images demonstrate the correctness and effectiveness of our algorithm.

[1]  R. Hetherington.  The Perception of the Visual World , 1952 .

[2]  J. Aloimonos,et al.  On the kinetic depth effect , 1989, Biological Cybernetics.

[3]  A. Mitiche.  Computational Analysis of Visual Motion , 1994, Advances in Computer Vision and Machine Intelligence.

[4]  S. Ullman.  The interpretation of structure from motion , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[5]  K. Nakayama,et al.  Optical Velocity Patterns, Velocity-Sensitive Neurons, and Space Perception: A Hypothesis , 1974, Perception.

[6]  Allen M. Waxman,et al.  Surface Structure and Three-Dimensional Motion from Image Flow Kinematics , 1985 .

[7]  Sridhar Srinivasan,et al.  Extracting Structure from Optical Flow Using the Fast Error Search Technique , 2000, International Journal of Computer Vision.

[8]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[9]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[10]  Jan J. Koenderink,et al.  Local structure of movement parallax of the plane , 1976 .

[11]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[13]  Takeo Kanade,et al.  A unified factorization algorithm for points, line segments and planes with uncertainty models , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  Allan D. Jepson,et al.  Linear subspace methods for recovering translational direction , 1994 .

[15]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[16]  Takeo Kanade,et al.  A sequential factorization method for recovering shape and motion from image streams , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Xinhua Zhuang,et al.  A simplified linear optic flow-motion algorithm , 1988, Comput. Vis. Graph. Image Process..

[18]  K. Hanna.  Direct multi-resolution estimation of ego-motion and structure from motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[19]  Laveen N. Kanal,et al.  3-D Motion Estimation from Motion Field , 1995, Artif. Intell..

[20]  Daryl T. Lawton,et al.  Processing translational motion sequences , 1983, Comput. Vis. Graph. Image Process..

[21]  John Oliensis,et al.  An Experimental Study of Projective Structure From Motion , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Berthold K. P. Horn,et al.  Direct methods for recovering motion , 1988, International Journal of Computer Vision.

[23]  K. Prazdny.  Determining The Instantaneous Direction Of Motion From Optical Flow Generated By A Curvilinearly Moving Observer , 1981, Other Conferences.

[24]  Thomas S. Huang,et al.  Motion and Structure from Image Sequences , 1992 .

[25]  John Oliensis,et al.  A Critique of Structure-from-Motion Algorithms , 2000, Comput. Vis. Image Underst..

[26]  Thomas S. Huang,et al.  Estimating three-dimensional motion parameters of a rigid planar patch , 1981 .

[27]  Berthold K. P. Horn,et al.  Passive navigation , 1982, Computer Vision Graphics and Image Processing.

[28]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Allen M. Waxman,et al.  Closed-form solutions to image flow equations for 3D structure and motion , 1988, International Journal of Computer Vision.

[30]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.