Motion-based segmentation of objects using overlapping temporal windows

Motion segmentation refers to the problem of separating the objects in a video sequence according to their motion. It is a fundamental problem of computer vision, since various systems focusing on the analysis of dynamic scenes include motion segmentation algorithms. In this paper we present a novel approach, where a video shot is temporally divided in successive and overlapping windows and motion segmentation is performed on each window respectively. This attribute renders the algorithm suitable even for long video sequences. In the last stage of the algorithm the segmentation results for every window are aggregated into a final segmentation. The presented algorithm can handle effectively asynchronous trajectories on each window even when they have no temporal intersection. The evaluation of the proposed algorithm on the Berkeley motion segmentation benchmark demonstrates its scalability and accuracy compared to the state of the art.

[1]  M. Kendall.  Rank Correlation Methods , 1949 .

[2]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[3]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[4]  Elizabeth S. Spelke,et al.  Principles of Object Perception , 1990, Cogn. Sci..

[5]  Mark A. Georgeson,et al.  Visual perception : physiology, psychology, & ecology , 2003 .

[6]  Nikos Komodakis,et al.  Clustering via LP-based Stabilities , 2008, NIPS.

[7]  Thomas Brox,et al.  Higher order motion models and spectral clustering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Thomas Brox,et al.  Object segmentation in video: A hierarchical variational approach for turning point trajectories into dense regions , 2011, 2011 International Conference on Computer Vision.

[10]  Roberto Cipolla,et al.  Spatio-temporal clustering of probabilistic region trajectories , 2011, 2011 International Conference on Computer Vision.

[11]  Katerina Fragkiadaki,et al.  Detection free tracking: Exploiting motion and topology for segmenting and tracking under entanglement , 2011, CVPR 2011.

[12]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[13]  PAUL D. SAMPSON,et al.  Fitting conic sections to "very scattered" data: An iterative refinement of the bookstein algorithm , 1982, Comput. Graph. Image Process..

[14]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Tat-Jun Chin,et al.  The Ordered Residual Kernel for Robust Motion Subspace Clustering , 2009, NIPS.

[16]  Anastasios Delopoulos,et al.  Improved motion segmentation using Locally sampled Subspaces , 2012, 2012 19th IEEE International Conference on Image Processing.

[17]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[18]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Ben Carterette,et al.  On rank correlation and the distance between rankings , 2009, SIGIR.

[20]  René Vidal,et al.  Clustering disjoint subspaces via sparse representation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[22]  Kurt Keutzer,et al.  Long term video segmentation through pixel level spectral clustering on GPUs , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[23]  Sergei Vassilvitskii,et al.  Generalized distances between rankings , 2010, WWW '10.

[24]  Christoph Schnörr,et al.  Spectral clustering of linear subspaces for motion segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[26]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[27]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[28]  Katerina Fragkiadaki,et al.  Video segmentation by tracing discontinuities in a trajectory embedding , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  René Vidal,et al.  Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.