Implicit Free-Form-Deformations for Multi-frame Segmentation and Tracking

In this paper, we propose a novel technique to address motion estimation and tracking. Such technique represents the motion field using a regular grid of thin-plate splines, and the moving objects using an implicit function on the image plane that is a cubic interpolation of a ”level set function” defined on this grid. Optical flow is determined through the deformation of the grid and consequently of the underlying image structures towards satisfying the constant brightness constraint. Tracking is performed in similar fashion through the consistent recovery in the temporal domain of the zero iso-surfaces of a level set that is the projection of the Free Form Deformation (FFD) implicit function according to the cubic spline formulation. Such an approach is a compromise between dense motion estimation and parametric motion models, introduces smoothness in an implicit fashion, is intrinsic, and can cope with important object deformations. Promising results demonstrate the potentials of our approach.

[1]  Rachid Deriche,et al.  Unifying boundary and region-based information for geodesic active tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[2]  Andrew Blake,et al.  Sparse Finite Elements for Geodesic Contours with Level-Sets , 2004, ECCV.

[3]  Daniel Cremers,et al.  A variational framework for image segmentation combining motion estimation and shape regularization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Rachid Deriche,et al.  Geodesic Active Contours and Level Sets for the Detection and Tracking of Moving Objects , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[6]  Michael J. Black,et al.  Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Thomas W. Sederberg,et al.  Free-form deformation of solid geometric models , 1986, SIGGRAPH.

[8]  Stanley Osher,et al.  Level Set Methods , 2003 .

[9]  Xin Li,et al.  Contour-based object tracking with occlusion handling in video acquired using mobile cameras , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  T. Chan,et al.  A Variational Level Set Approach to Multiphase Motion , 1996 .

[11]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[12]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[13]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[14]  J. Sethian,et al.  FRONTS PROPAGATING WITH CURVATURE DEPENDENT SPEED: ALGORITHMS BASED ON HAMILTON-JACOB1 FORMULATIONS , 2003 .

[15]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[16]  S. Osher,et al.  Geometric Level Set Methods in Imaging, Vision, and Graphics , 2011, Springer New York.

[17]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[18]  A. Yezzi,et al.  A variational framework for joint segmentation and registration , 2001, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA 2001).

[19]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[20]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[21]  Petros Faloutsos,et al.  Dynamic Free-Form Deformations for Animation Synthesis , 1997, IEEE Trans. Vis. Comput. Graph..

[22]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[23]  Jean-Marc Odobez,et al.  Robust Multiresolution Estimation of Parametric Motion Models , 1995, J. Vis. Commun. Image Represent..

[24]  Anthony J. Yezzi,et al.  Gradient flows and geometric active contour models , 1995, Proceedings of IEEE International Conference on Computer Vision.

[25]  Joachim Weickert,et al.  Variational Optic Flow Computation with a Spatio-Temporal Smoothness Constraint , 2001, Journal of Mathematical Imaging and Vision.

[26]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[27]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[28]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..