Dense motion estimation using regularization constraints on local parametric models

This paper presents a method for dense optical flow estimation in which the motion field within patches that result from an initial intensity segmentation is parametrized with models of different order. We propose a novel formulation which introduces regularization constraints between the model parameters of neighboring patches. In this way, we provide the additional constraints for very small patches and for patches whose intensity variation cannot sufficiently constrain the estimation of their motion parameters. In order to preserve motion discontinuities, we use robust functions as a regularization mean. We adopt a three-frame approach and control the balance between the backward and forward constraints by a real-valued direction field on which regularization constraints are applied. An iterative deterministic relaxation method is employed in order to solve the corresponding optimization problem. Experimental results show that the proposed method deals successfully with motions large in magnitude, motion discontinuities, and produces accurate piecewise-smooth motion fields.

[1]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ioannis Patras,et al.  Video Segmentation by MAP Labeling of Watershed Segments , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[4]  C. Stiller,et al.  Estimating motion in image sequences , 1999, IEEE Signal Process. Mag..

[5]  Michael Spann,et al.  Robust Optical Flow Computation Based on Least-Median-of-Squares Regression , 2004, International Journal of Computer Vision.

[6]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[7]  Janusz Konrad,et al.  Motion estimation for region-based video coding , 1995, Proceedings., International Conference on Image Processing.

[8]  C. Stiller,et al.  Estimating Motion in Image Sequences A tutorial on modeling and computation of 2 D motion , 2022 .

[9]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[10]  Patrick Pérez,et al.  Hierarchical Estimation and Segmentation of Dense Motion Fields , 2002, International Journal of Computer Vision.

[11]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Joachim Weickert,et al.  Reliable Estimation of Dense Optical Flow Fields with Large Displacements , 2000, International Journal of Computer Vision.

[13]  Jean-Marc Odobez,et al.  Robust Multiresolution Estimation of Parametric Motion Models , 1995, J. Vis. Commun. Image Represent..

[14]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[16]  David Suter,et al.  Robust Optic Flow Computation , 1998, International Journal of Computer Vision.

[17]  Christoph Stiller,et al.  Object-based estimation of dense motion fields , 1997, IEEE Trans. Image Process..

[18]  Michael J. Black,et al.  Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Michael J. Black.  Recursive Non-Linear Estimation of Discontinuous Flow Fields , 1994, ECCV.

[21]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[23]  Patrick Bouthemy,et al.  A region-level motion-based graph representation and labeling for tracking a spatial image partition , 2000, Pattern Recognit..

[24]  Gérard G. Medioni,et al.  Accurate motion flow estimation with discontinuities , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[26]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[27]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[28]  Harry Shum,et al.  Motion estimation with quadtree splines , 1995, Proceedings of IEEE International Conference on Computer Vision.