Regularization Strategies for Discontinuity-Preserving Optical Flow Methods

The aim of this paper is to study several strategies for the preservation of flow discontinuities in variational optical flow methods. We analyze the combination of robust functionals and diffusion tensors in the smoothness assumption. Our study includes the use of tensors based on decreasing functions, which has shown to provide good results. However, it presents several limitations and usually does not perform better than other basic approaches. It typically introduces instabilities in the computed motion fields in the form of independent blobs of vectors with large magnitude. We propose two alternatives to overcome these drawbacks: first, a simple approach that combines the decreasing function with a minimum isotropic smoothing, and second, a method that looks for the best parameter configuration that preserves the important motion contours and avoid instabilities. It relies on the input images and the regularization parameter. It is fully automatic, providing a near-optimal value for many sequences, as shown in the experiments. Both proposals allow to detect the contours of the motion field and produce more stable solutions for a large range of parameters. In the experimental results, we present a detailed study and comparison of the different strategies.

[1]  Horst Bischof,et al.  Motion estimation with non-local total variation regularization , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  J. Bigun,et al.  Optimal Orientation Detection of Linear Symmetry , 1987, ICCV 1987.

[3]  Stefano Soatto,et al.  Sparse Occlusion Detection with Optical Flow , 2012, International Journal of Computer Vision.

[4]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[5]  Agustín Salgado de la Nuez,et al.  Preserving accurate motion contours with reliable parameter selection , 2014, ICIP.

[6]  Patrick Pérez,et al.  Dense estimation and object-based segmentation of the optical flow with robust techniques , 1998, IEEE Trans. Image Process..

[7]  Michael J. Black,et al.  Learning Optical Flow , 2008, ECCV.

[8]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Julio Esclarín Monreal,et al.  A PDE model for computing the optimal flow , 1999 .

[10]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Daniel Cremers,et al.  Structure- and motion-adaptive regularization for high accuracy optic flow , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Gabriele Facciolo,et al.  TV-L 1 Optical Flow Estimation , 2013 .

[13]  Michael J. Black,et al.  A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them , 2013, International Journal of Computer Vision.

[14]  Michael J. Black,et al.  A Naturalistic Open Source Movie for Optical Flow Evaluation , 2012, ECCV.

[15]  Michael J. Black,et al.  Layered segmentation and optical flow estimation over time , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[17]  Horst Bischof,et al.  Joint motion estimation and segmentation of complex scenes with label costs and occlusion modeling , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Optic Flow in Harmony Optic Flow in Harmony Optic Flow in Harmony , 2022 .

[19]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Luc Van Gool,et al.  Determination of Optical Flow and its Discontinuities using Non-Linear Diffusion , 1994, ECCV.

[21]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[22]  Cordelia Schmid,et al.  EpicFlow: Edge-preserving interpolation of correspondences for optical flow , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[24]  Agustín Salgado de la Nuez,et al.  Efficient Mechanism for Discontinuity Preserving in Optical Flow Methods , 2014, ICISP.

[25]  Nelson Monzón,et al.  Optical Flow Estimation with Consistent Spatio-temporal Coherence Models , 2013, VISAPP 2013.

[26]  Thomas Pock,et al.  Non-local Total Generalized Variation for Optical Flow Estimation , 2014, ECCV.

[27]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[29]  Hans-Peter Seidel,et al.  Complementary Optic Flow , 2009, EMMCVPR.

[30]  Daniel Cremers,et al.  Anisotropic Huber-L1 Optical Flow , 2009, BMVC.

[31]  Joachim Weickert,et al.  A Review of Nonlinear Diffusion Filtering , 1997, Scale-Space.

[32]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[33]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Hui Cheng,et al.  Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection , 2006, ECCV.

[35]  Javier Sánchez Pérez,et al.  TV-L1 Optical Flow Estimation , 2013, Image Process. Line.

[36]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Joachim Weickert,et al.  Reliable Estimation of Dense Optical Flow Fields with Large Displacements , 2000, International Journal of Computer Vision.

[38]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[39]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[40]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[41]  I. Cohen Nonlinear Variational Method for Optical Flow Computation , 2006 .

[42]  Daniel Cremers,et al.  An Improved Algorithm for TV-L 1 Optical Flow , 2009, Statistical and Geometrical Approaches to Visual Motion Analysis.

[43]  H. Bischof,et al.  2 Anisotropic Huber-L 1 Optical Flow , 2009 .