Motion Segmentation Based on Model Selection in Permutation Space for RGB Sensors

Motion segmentation is aimed at segmenting the feature point trajectories belonging to independently moving objects. Using the affine camera model, the motion segmentation problem can be viewed as a subspace clustering problem—clustering the data points drawn from a union of low-dimensional subspaces. In this paper, we propose a solution for motion segmentation that uses a multi-model fitting technique. We propose a data grouping method and a model selection strategy for obtaining more distinguishable data point permutation preferences, which significantly improves the clustering. We perform extensive testing on the Hopkins 155 dataset, and two real-world datasets. The experimental results illustrate that the proposed method can deal with incomplete trajectories and the perspective effect, comparing favorably with the current state of the art.

[1]  Nazrul Haque,et al.  Joint Semantic and Motion Segmentation for Dynamic Scenes using Deep Convolutional Networks , 2017, VISIGRAPP.

[2]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Christoph Schnörr,et al.  Spectral clustering of linear subspaces for motion segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[4]  Kenichi Kanatani,et al.  Evaluation and Selection of Models for Motion Segmentation , 2001, ECCV.

[5]  Tat-Jun Chin,et al.  Mode seeking over permutations for rapid geometric model fitting , 2013, Pattern Recognit..

[6]  Mengyin Fu,et al.  Semantic motion segmentation for urban dynamic scene understanding , 2016, CASE.

[7]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Zhuwen Li,et al.  Perspective Motion Segmentation via Collaborative Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Ying Wu,et al.  Multibody Grouping by Inference of Multiple Subspaces from High-Dimensional Data Using Oriented-Frames , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[11]  David Suter,et al.  Two-view multibody structure-and-motion with outliers through model selection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Andrea Fusiello,et al.  Multiple Models Fitting as a Set Coverage Problem , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Chu-Song Chen,et al.  Two-View Motion Segmentation with Model Selection and Outlier Removal by RANSAC-Enhanced Dirichlet Process Mixture Models , 2010, International Journal of Computer Vision.

[15]  Tat-Jun Chin,et al.  A simultaneous sample-and-filter strategy for robust multi-structure model fitting , 2013, Comput. Vis. Image Underst..

[16]  René Vidal,et al.  Segmenting Motions of Different Types by Unsupervised Manifold Clustering , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[18]  Wolfram Burgard,et al.  SMSnet: Semantic motion segmentation using deep convolutional neural networks , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Hongdong Li,et al.  Two-View Motion Segmentation from Linear Programming Relaxation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Andrea Fusiello,et al.  Robust Multiple Structures Estimation with J-Linkage , 2008, ECCV.

[21]  Andrea Fusiello,et al.  Robust Multiple Model Fitting with Preference Analysis and Low-rank Approximation , 2015, BMVC.

[22]  Tsung-Han Lin,et al.  Deep learning of spatio-temporal features with geometric-based moving point detection for motion segmentation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Jitendra Malik,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Segmentation of Moving Objects by Long Term Video Analysis , 2022 .

[24]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[25]  René Vidal,et al.  Motion Segmentation with Missing Data Using PowerFactorization and GPCA , 2004, CVPR.

[26]  Gilad Lerman,et al.  Median K-Flats for hybrid linear modeling with many outliers , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[27]  Kenichi Kanatani,et al.  Multi-Stage Unsupervised Learning for Multi-Body Motion Segmentation , 2004, IEICE Trans. Inf. Syst..

[28]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  Ehsan Elhamifar,et al.  Sparse subspace clustering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Wei Wu,et al.  Robust Trajectory Clustering for Motion Segmentation , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Kenichi Kanatani,et al.  Geometric Structure of Degeneracy for Multi-body Motion Segmentation , 2004, ECCV Workshop SMVP.

[32]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[33]  Tat-Jun Chin,et al.  Robust fitting of multiple structures: The statistical learning approach , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[34]  Ariel Shamir,et al.  Mode-detection via median-shift , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[36]  Tat-Jun Chin,et al.  Accelerated Hypothesis Generation for Multistructure Data via Preference Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Philip H. S. Torr,et al.  Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting , 2002, International Journal of Computer Vision.

[38]  Xun Xu,et al.  Motion Segmentation by Exploiting Complementary Geometric Models , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  P. Tseng Nearest q-Flat to m Points , 2000 .

[40]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  S. Shankar Sastry,et al.  Two-View Multibody Structure from Motion , 2005, International Journal of Computer Vision.

[42]  Lihi Zelnik-Manor,et al.  Degeneracies, dependencies and their implications in multi-body and multi-sequence factorizations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[43]  Yair Weiss,et al.  Incorporating Non-motion Cues into 3D Motion Segmentation , 2006, ECCV.

[44]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[45]  Tat-Jun Chin,et al.  The Random Cluster Model for robust geometric fitting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Bodo Rosenhahn,et al.  Multi-scale Clustering of Frame-to-Frame Correspondences for Motion Segmentation , 2012, ECCV.

[47]  Yair Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[48]  Andrea Fusiello,et al.  T-Linkage: A Continuous Relaxation of J-Linkage for Multi-model Fitting , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[50]  T. Boult,et al.  Factorization-based segmentation of motions , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[51]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[52]  Joaquim Salvi,et al.  Motion Segmentation: a Review , 2008, CCIA.

[53]  René Vidal,et al.  A new GPCA algorithm for clustering subspaces by fitting, differentiating and dividing polynomials , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[54]  Yan Yan,et al.  Motion Segmentation Via a Sparsity Constraint , 2017, IEEE Transactions on Intelligent Transportation Systems.

[55]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[56]  Andrea Fusiello,et al.  Multiple structure recovery with T-linkage , 2017, J. Vis. Commun. Image Represent..

[57]  Konrad Schindler,et al.  Perspective n-View Multibody Structure-and-Motion Through Model Selection , 2006, ECCV.

[58]  P. Torr Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[59]  René Vidal,et al.  Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Pei Chen,et al.  Optimization Algorithms on Subspaces: Revisiting Missing Data Problem in Low-Rank Matrix , 2008, International Journal of Computer Vision.

[61]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[62]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).