Robust Realtime Motion-Split-And-Merge for Motion Segmentation

In this paper, we analyze and modify the Motion-Split-and-Merge (MSAM) algorithm [3] for the motion segmentation of correspondences between two frames. Our goal is to make the algorithm suitable for practical use which means realtime processing speed at very low error rates. We compare our (robust realtime) RMSAM with J-Linkage [16] and Graph-Based Segmentation [5] and show that it is superior to both. Applying RMSAM in a multi-frame motion segmentation context to the Hopkins 155 benchmark, we show that compared to the original formulation, the error decreases from 2.05% to only 0.65% at a runtime reduced by 72%. The error is still higher than the best results reported so far, but RMSAM is dramatically faster and can handle outliers and missing data.

[1]  Bodo Rosenhahn,et al.  Multi-scale Clustering of Frame-to-Frame Correspondences for Motion Segmentation , 2012, ECCV.

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  Thomas Brox,et al.  Higher order motion models and spectral clustering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[5]  Charles V. Stewart,et al.  Bias in robust estimation caused by discontinuities and multiple structures , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[7]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[8]  Thomas Deselaers,et al.  ClassCut for Unsupervised Class Segmentation , 2010, ECCV.

[9]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Thomas Brox,et al.  Object segmentation in video: A hierarchical variational approach for turning point trajectories into dense regions , 2011, 2011 International Conference on Computer Vision.

[11]  René Vidal,et al.  Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[13]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[14]  David Suter,et al.  Two-View Multibody Structure-and-Motion with Outliers through Model Selection , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  P. Torr.  Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[16]  Patrick Pérez,et al.  Clustering Point Trajectories with Various Life-Spans , 2009, 2009 Conference for Visual Media Production.

[17]  Cordelia Schmid,et al.  Learning object class detectors from weakly annotated video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Vittorio Ferrari,et al.  Appearance Sharing for Collective Human Pose Estimation , 2012, ACCV.

[19]  Anil M. Cheriyadat,et al.  Non-negative matrix factorization of partial track data for motion segmentation , 2010, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Serge J. Belongie,et al.  A Feature-based Approach for Dense Segmentation and Estimation of Large Disparity Motion , 2006, International Journal of Computer Vision.

[21]  Luc Van Gool,et al.  Dynamic Objectness for Adaptive Tracking , 2012, ACCV.

[22]  Matthieu Guillaumin,et al.  Segmentation Propagation in ImageNet , 2012, ECCV.

[23]  Andrea Fusiello,et al.  Robust Multiple Structures Estimation with J-Linkage , 2008, ECCV.

[24]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.