High Accuracy Optical Flow Estimation Based on a Theory for Warping

We study an energy functional for computing optical flow that combines three assumptions: a brightness constancy assumption, a gradient constancy assumption, and a discontinuity-preserving spatio-temporal smoothness constraint. In order to allow for large displacements, linearisations in the two data terms are strictly avoided. We present a consistent numerical scheme based on two nested fixed point iterations. By proving that this scheme implements a coarse-to-fine warping strategy, we give a theoretical foundation for warping which has been used on a mainly experimental basis so far. Our evaluation demonstrates that the novel method gives significantly smaller angular errors than previous techniques for optical flow estimation. We show that it is fairly insensitive to parameter variations, and we demonstrate its excellent robustness under noise.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[3]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  O. Faugeras,et al.  Computer Vision — ECCV 90 , 1990, Lecture Notes in Computer Science.

[5]  Hans-Hellmut Nagel,et al.  Extending the 'Oriented Smoothness Constraint' into the Temporal Domain and the Estimation of Derivatives of Optical Flow , 1990, ECCV.

[6]  Michael J. Black,et al.  Robust dynamic motion estimation over time , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[8]  Jan-Olof Eklundh,et al.  Computer Vision — ECCV '94 , 1994, Lecture Notes in Computer Science.

[9]  Massimo Tistarelli Multiple Constraints for Optical Flow , 1994, ECCV.

[10]  Christoph Schnörr Segmentation of visual motion by minimizing convex non-quadratic functionals , 1994, ICPR.

[11]  Rachid Deriche,et al.  Optical-Flow Estimation while Preserving Its Discontinuities: A Variational Approach , 1995, ACCV.

[12]  Han Wang,et al.  Recent Developments in Computer Vision , 1995, Lecture Notes in Computer Science.

[13]  Michael J. Black,et al.  Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[15]  Patrick Pérez,et al.  A multigrid approach for hierarchical motion estimation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  Julio Esclarín Monreal,et al.  A PDE model for computing the optimal flow , 1999 .

[17]  James M. Keller,et al.  Image Processing and Computer Vision , 1999 .

[18]  Gunnar Farnebäck Very High Accuracy Velocity Estimation using Orientation Tensors Parametric Motion and Simultaneous Segmentation of the Motion Field , 2001, ICCV.

[19]  G. Farneback Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Patrick Pérez,et al.  Hierarchical Estimation and Segmentation of Dense Motion Fields , 2002, International Journal of Computer Vision.

[21]  A. Verri,et al.  A computational approach to motion perception , 1988, Biological Cybernetics.

[22]  David Suter,et al.  Robust Optic Flow Computation , 1998, International Journal of Computer Vision.

[23]  Joachim Weickert,et al.  Reliable Estimation of Dense Optical Flow Fields with Large Displacements , 2000, International Journal of Computer Vision.

[24]  Joachim Weickert,et al.  Variational Optic Flow Computation with a Spatio-Temporal Smoothness Constraint , 2001, Journal of Mathematical Imaging and Vision.

[25]  Shang-Hong Lai,et al.  Reliable and Efficient Computation of Optical Flow , 1998, International Journal of Computer Vision.

[26]  Joachim Weickert,et al.  A Theoretical Framework for Convex Regularizers in PDE-Based Computation of Image Motion , 2001, International Journal of Computer Vision.

[27]  Joachim Weickert,et al.  Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods , 2004, International Journal of Computer Vision.

[28]  Laurent D. Cohen,et al.  Image Registration, Optical Flow and Local Rigidity , 2001, Journal of Mathematical Imaging and Vision.

[29]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[30]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[31]  J. Weickert,et al.  Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods , 2005 .

[32]  I. Cohen Nonlinear Variational Method for Optical Flow Computation , 2006 .