Motion segmentation with occlusions on the superpixel graph

We present a motion segmentation algorithm that partitions the image plane into disjoint regions based on their parametric motion. It relies on a finer partitioning of the image domain into regions of uniform photometric properties, with motion segments made of unions of such “superpixels.” We exploit recent advances in combinatorial graph optimization that yield computationally efficient estimates. The energy functional is built on a superpixel graph, and is iteratively minimized by computing a parametric motion model in closed-form, followed by a graph cut of the superpixel adjacency graph. It generalizes naturally to multi-label partitions that can handle multiple motions.

[1]  Rachid Deriche,et al.  Active unsupervised texture segmentation on a diffusion based feature space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[3]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Michael J. Black,et al.  Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[6]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[7]  Stefano Soatto,et al.  On the set of images modulo viewpoint and contrast changes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Wei Chen,et al.  Efficiently Solving the Piecewise Constant Mumford-Shah Model using Graph Cuts , 2005 .

[9]  Roberto Cipolla,et al.  Principled fusion of high-level model and low-level cues for motion segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Tony F. Chan,et al.  Aspects of Total Variation Regularized L[sup 1] Function Approximation , 2005, SIAM J. Appl. Math..

[11]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Christopher V. Alvino,et al.  Reformulating and Optimizing the Mumford-Shah Functional on a Graph - A Faster, Lower Energy Solution , 2008, ECCV.

[14]  P. Anandan,et al.  Accurate computation of optical flow by using layered motion representations , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[15]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Luc Van Gool,et al.  A Probabilistic Approach to Large Displacement Optical Flow and Occlusion Detection , 2004, ECCV Workshop SMVP.

[17]  Michael J. Black.  Explaining optical flow events with parameterized spatio-temporal models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[18]  Joachim Weickert,et al.  A Scale-Space Approach to Nonlocal Optical Flow Calculations , 1999, Scale-Space.

[19]  Max A. Viergever,et al.  Efficient and reliable schemes for nonlinear diffusion filtering , 1998, IEEE Trans. Image Process..

[20]  J. Sethian,et al.  FRONTS PROPAGATING WITH CURVATURE DEPENDENT SPEED: ALGORITHMS BASED ON HAMILTON-JACOB1 FORMULATIONS , 2003 .

[21]  Nebojsa Jojic,et al.  Consistent segmentation for optical flow estimation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[22]  Rachid Deriche,et al.  Geodesic Active Contours and Level Sets for the Detection and Tracking of Moving Objects , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Eric Dubois,et al.  Bayesian Estimation of Motion Vector Fields , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Adel Said Elmaghraby,et al.  Graph cut optimization for the Mumford-Shah model , 2007 .

[25]  Patrick Bouthemy,et al.  Motion segmentation and qualitative dynamic scene analysis from an image sequence , 1993, International Journal of Computer Vision.

[26]  Stefano Soatto,et al.  Quick Shift and Kernel Methods for Mode Seeking , 2008, ECCV.

[27]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Rachid Deriche,et al.  Optical-Flow Estimation while Preserving Its Discontinuities: A Variational Approach , 1995, ACCV.

[29]  Janusz Konrad,et al.  Occlusion-Aware Optical Flow Estimation , 2008, IEEE Transactions on Image Processing.

[30]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Michael J. Black.  Combining Intensity and Motion for Incremental Segmentation and Tracking Over Long Image Sequences , 1992, ECCV.

[32]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[33]  Christoph Schnörr,et al.  Computation of discontinuous optical flow by domain decomposition and shape optimization , 1992, International Journal of Computer Vision.

[34]  Hui Cheng,et al.  Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection , 2006, ECCV.