A Duality Based Approach for Realtime TV-L1 Optical Flow

Variational methods are among the most successful approaches to calculate the optical flow between two image frames. A particularly appealing formulation is based on total variation (TV) regularization and the robust L1 norm in the data fidelity term. This formulation can preserve discontinuities in the flow field and offers an increased robustness against illumination changes, occlusions and noise. In this work we present a novel approach to solve the TV-L1 formulation. Our method results in a very efficient numerical scheme, which is based on a dual formulation of the TV energy and employs an efficient point-wise thresholding step. Additionally, our approach can be accelerated by modern graphics processing units. We demonstrate the real-time performance (30 fps) of our approach for video inputs at a resolution of 320 × 240 pixels.

[1]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[2]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[4]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[5]  Ta Camus,et al.  Real-time quantized optical flow , 1995, Proceedings of Conference on Computer Architectures for Machine Perception.

[6]  Gene H. Golub,et al.  A Nonlinear primal dual method for TV-based image restoration , 1996 .

[7]  Joachim Weickert,et al.  A Scale-Space Approach to Nonlocal Optical Flow Calculations , 1999, Scale-Space.

[8]  Rachid Deriche,et al.  Computing Optical Flow via Variational Techniques , 1999, SIAM J. Appl. Math..

[9]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[10]  Otmar Scherzer,et al.  Inverse Problems, Image Analysis, and Medical Imaging , 2002 .

[11]  T. Brox,et al.  Diffusion and regularization of vector- and matrix-valued images , 2002 .

[12]  Jiří Matas,et al.  Computer Vision - ECCV 2004 , 2004, Lecture Notes in Computer Science.

[13]  Patrick Pérez,et al.  Hierarchical Estimation and Segmentation of Dense Motion Fields , 2002, International Journal of Computer Vision.

[14]  Robert Strzodka,et al.  Real-time motion estimation and visualization on graphics cards , 2004, IEEE Visualization 2004.

[15]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[16]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[17]  A. Chambolle.  Practical, Unified, Motion and Missing Data Treatment in Degraded Video , 2004, Journal of Mathematical Imaging and Vision.

[18]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[19]  Timo Kohlberger,et al.  Variational optical flow computation in real time , 2005, IEEE Transactions on Image Processing.

[20]  Thomas Brox,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Highly Accurate Optic Flow Computation with Theoretically Justified Warping Highly Accurate Optic Flow Computation with Theoretically Justified Warping , 2022 .

[21]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[22]  Timo Kohlberger,et al.  A Multigrid Platform for Real-Time Motion Computation with Discontinuity-Preserving Variational Methods , 2006, International Journal of Computer Vision.

[23]  Xavier Bresson,et al.  Fast Global Minimization of the Active Contour/Snake Model , 2007, Journal of Mathematical Imaging and Vision.