Reliable and Efficient Computation of Optical Flow

In this paper, we present two very efficient and accurate algorithms for computing optical flow. The first is a modified gradient-based regularization method, and the other is an SSD-based regularization method. For the gradient-based method, to amend the errors in the discrete image flow equation caused by numerical differentiation as well as temporal and spatial aliasing in the brightness function, we propose to selectively combine the image flow constraint and a contour-based flow constraint into the data constraint by using a reliability measure. Each data constraint is appropriately normalized to obtain an approximate minimum distance (of the data point to the linear flow equation) constraint instead of the conventional linear flow constraint. These modifications lead to robust and accurate optical flow estimation. We propose an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the resulting large and sparse linear system efficiently. Our SSD-based regularization method uses a normalized SSD measure (based on a similar reasoning as in the gradient-based scheme) as the data constraint in a regularization framework. The nonlinear conjugate gradient algorithm in conjunction with an incomplete Cholesky preconditioning is developed to solve the resulting nonlinear minimization problem. Experimental results on synthetic and real image sequences for these two algorithms are given to demonstrate their performance in comparison with competing methods reported in literature.

[1]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  Philip E. Gill,et al.  Practical optimization , 1981 .

[4]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[5]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[6]  Gene H. Golub,et al.  Matrix computations , 1983 .

[7]  Ellen C. Hildreth,et al.  Computations Underlying the Measurement of Visual Motion , 1984, Artif. Intell..

[8]  Allen M. Waxman,et al.  Contour Evolution, Neighborhood Deformation, and Global Image Flow: Planar Surfaces in Motion , 1985 .

[9]  Demetri Terzopoulos,et al.  Regularization of Inverse Visual Problems Involving Discontinuities , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  H. Elman.  A stability analysis of incomplete LU factorizations , 1986 .

[11]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Tomaso A. Poggio,et al.  Motion Field and Optical Flow: Qualitative Properties , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  J. H. Duncan,et al.  On the Detection of Motion and the Computation of Optical Flow , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[15]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 1993, 1993 (4th) International Conference on Computer Vision.

[16]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Shahriar Negahdaripour,et al.  A generalized brightness change model for computing optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[18]  Richard Szeliski,et al.  Hierarchical spline-based image registration , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Harry Shum,et al.  Motion estimation with quadtree splines , 1995, Proceedings of IEEE International Conference on Computer Vision.

[20]  Gary J. Balas,et al.  Optical flow: a curve evolution approach , 1995, Proceedings., International Conference on Image Processing.

[21]  Michael J. Black,et al.  Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Michael J. Black,et al.  Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[24]  Shang-Hong Lai,et al.  Physically Based Adaptive Preconditioning for Early Vision , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  S. Lai,et al.  Physically-based Adaptive Preconditioning for Early Vision 1 , 1997 .

[26]  A. Verri,et al.  A computational approach to motion perception , 1988, Biological Cybernetics.

[27]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[28]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[29]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 2005, International Journal of Computer Vision.