The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields

Most approaches for estimating optical flow assume that, within a finite image region, only a single motion is present. Thissingle motion assumptionis violated in common situations involving transparency, depth discontinuities, independently moving objects, shadows, and specular reflections. To robustly estimate optical flow, the single motion assumption must be relaxed. This paper presents a framework based onrobust estimationthat addresses violations of the brightness constancy and spatial smoothness assumptions caused by multiple motions. We show how therobust estimation frameworkcan be applied to standard formulations of the optical flow problem thus reducing their sensitivity to violations of their underlying assumptions. The approach has been applied to three standard techniques for recovering optical flow: area-based regression, correlation, and regularization with motion discontinuities. This paper focuses on the recovery of multiple parametric motion models within a region, as well as the recovery of piecewise-smooth flow fields, and provides examples with natural and synthetic image sequences.

[1]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[2]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[3]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[4]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[5]  Gilad Adiv.  Recovering 2-D Motion Parameters in Scenes Containing Multiple Moving Objects. , 1983 .

[6]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Takeo Kanade,et al.  Adapting optical-flow to measure object motion in reflectance and x-ray image sequences (abstract only) , 1984, COMG.

[8]  Allen M. Waxman,et al.  Contour Evolution, Neighborhood Deformation, and Global Image Flow: Planar Surfaces in Motion , 1985 .

[9]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[10]  Valdis Berzins,et al.  Dynamic Occlusion Analysis in Optical Flow Fields , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[12]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Joseph K. Kearney,et al.  Optical Flow Estimation: An Error Analysis of Gradient-Based Methods with Local Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[15]  F. Glazer.  Hierarchical Motion Detection , 1987 .

[16]  Stuart Geman,et al.  Statistical methods for tomographic image reconstruction , 1987 .

[17]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[18]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[19]  Hans-Hellmut Nagel,et al.  On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results , 1987, Artif. Intell..

[20]  Tomaso Poggio,et al.  Probabilistic Solution of Ill-Posed Problems in Computational Vision , 1987 .

[21]  Bart W. Stuck,et al.  A Computer and Communication Network Performance Analysis Primer (Prentice Hall, Englewood Cliffs, NJ, 1985; revised, 1987) , 1987, Int. CMG Conference.

[22]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[23]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Eric Dubois,et al.  Multigrid Bayesian Estimation Of Image Motion Using Stochastic Relaxation , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[25]  D. Shulman,et al.  Regularization of discontinuous flow fields , 1989, [1989] Proceedings. Workshop on Visual Motion.

[26]  Brian G. Schunck,et al.  Image Flow Segmentation and Estimation by Constraint Line Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[28]  Michael J. Black,et al.  Constraints for the Early Detection of Discontinuity from Motion , 1990, AAAI.

[29]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Michael J. Black,et al.  A model for the detection of motion over time , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[31]  Joachim Heel,et al.  Temporally integrated surface reconstruction , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[32]  M. Shizawa,et al.  Principle of superposition: a common computational framework for analysis of multiple motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[33]  Federico Girosi,et al.  Parallel and Deterministic Algorithms from MRFs: Surface Reconstruction , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Michael J. Black,et al.  Robust dynamic motion estimation over time , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Michael J. Black.  Robust incremental optical flow , 1992 .

[36]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[37]  Shmuel Peleg,et al.  A Three-Frame Algorithm for Estimating Two-Component Image Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[39]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 1993, 1993 (4th) International Conference on Computer Vision.

[40]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[41]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Daniel J. Kersten,et al.  Multi-layer surface segmentation using energy minimization , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Shahriar Negahdaripour,et al.  A generalized brightness change model for computing optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[45]  Michael J. Black,et al.  The outlier process: unifying line processes and robust statistics , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.