Hierarchical Estimation and Segmentation of Dense Motion Fields

In this paper we present a comprehensive energy-based framework for the estimation and the segmentation of the apparent motion in image sequences. The robust cost functions and the associated hierarchical minimization techniques that we propose mix efficiently non-parametric (dense) representations, local interacting parametric representations, and global non-interacting parametric representations related to a partition into regions. Experimental comparisons, both on synthetic and real images, demonstrate the merit of the approach on different types of photometric and kinematic contents ranging from moving rigid objects to moving fluids.

[1]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[2]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[4]  B. G. Schunck The image flow constraint equation , 1986 .

[5]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[6]  Ronald A. Thisted,et al.  Elements of statistical computing , 1986 .

[7]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Wilfried Enkelmann,et al.  Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences , 1988, Comput. Vis. Graph. Image Process..

[9]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[10]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[11]  Olivier D. Faugeras,et al.  Robust and fast computation of unbiased intensity derivatives in images , 1992, ECCV.

[12]  Donald Geman,et al.  Constrained Restoration and the Recovery of Discontinuities , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Paolo Nesi,et al.  Variational approach to optical flow estimation managing discontinuities , 1993, Image Vis. Comput..

[14]  Richard Szeliski,et al.  Hierarchical spline-based image registration , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Michael J. Black Recursive Non-Linear Estimation of Discontinuous Flow Fields , 1994, ECCV.

[16]  Christoph Schnörr,et al.  A Variational Approach to the Design of Early Vision Algorithms , 1994, Theoretical Foundations of Computer Vision.

[17]  Rachid Deriche,et al.  Optical-Flow Estimation while Preserving Its Discontinuities: A Variational Approach , 1995, ACCV.

[18]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[19]  Xinhua Zhuang,et al.  Optic Flow Field Segmentation and Motion Estimation Using a Robust Genetic Partitioning Algorithm , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Michael J. Black,et al.  Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Michael J. Black,et al.  Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Richard Szeliski,et al.  Motion Estimation with Quadtree Splines , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[25]  A. Murat Tekalp,et al.  Simultaneous motion estimation and segmentation , 1997, IEEE Trans. Image Process..

[26]  Michel Barlaud,et al.  Deterministic edge-preserving regularization in computed imaging , 1997, IEEE Trans. Image Process..

[27]  Christoph Stiller,et al.  Object-based estimation of dense motion fields , 1997, IEEE Trans. Image Process..

[28]  Patrick Pérez,et al.  A multigrid approach for hierarchical motion estimation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  Patrick Pérez,et al.  Dense estimation and object-based segmentation of the optical flow with robust techniques , 1998, IEEE Trans. Image Process..

[30]  Yoram Bresler,et al.  Globally convergent edge-preserving regularized reconstruction: an application to limited-angle tomography , 1998, IEEE Trans. Image Process..

[31]  Anthony J. Yezzi,et al.  A statistical approach to snakes for bimodal and trimodal imagery , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[32]  Patrick Pérez,et al.  Fluid motion recovery by coupling dense and parametric vector fields , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[33]  Fabrice Heitz,et al.  3D deformable image matching using multiscale minimization of global energy functions , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[34]  Josiane Zerubia,et al.  Simultaneous image classification and restoration using a variational approach , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[35]  Lionel Oisel,et al.  Geometric Driven Optical Flow Estimation and Segmentation for 3D Reconstruction , 2000, ECCV.

[36]  P. Pérez,et al.  Dense Fluid Flow Estimation , 2000 .

[37]  Pierre Hellier,et al.  Hierarchical estimation of a dense deformation field for 3-D robust registration , 2001, IEEE Transactions on Medical Imaging.

[38]  Yvan G. Leclerc,et al.  Constructing simple stable descriptions for image partitioning , 1989, International Journal of Computer Vision.

[39]  Rachid Deriche,et al.  Image Sequence Analysis via Partial Differential Equations , 1999, Journal of Mathematical Imaging and Vision.

[40]  Isabelle Herlin,et al.  Non Uniform Multiresolution Method for Optical Flow and Phase Portrait Models: Environmental Applications , 2004, International Journal of Computer Vision.

[41]  David Suter,et al.  Robust Optic Flow Computation , 1998, International Journal of Computer Vision.

[42]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[43]  Patrick Bouthemy,et al.  Computation and analysis of image motion: A synopsis of current problems and methods , 1996, International Journal of Computer Vision.

[44]  Shang-Hong Lai,et al.  Reliable and Efficient Computation of Optical Flow , 1998, International Journal of Computer Vision.

[45]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[46]  Patrick Bouthemy,et al.  Motion segmentation and qualitative dynamic scene analysis from an image sequence , 1993, International Journal of Computer Vision.