Motion Segmentation via Synchronization

In this paper we consider the problem of segmenting points in a collection of images that contain multiple moving objects. Our contribution is three-fold: (i) we propose a matrix representation of segmentation that permits to formulate the problem in terms of "synchronization" of binary matrices; (ii) we derive a spectral solution to solve such a problem, which is inspired by previous works on synchronization of rotations, homographies, rigid motions and permutations; (iii) we explain how our solution can be interpreted in terms of spectral clustering. The proposed approach is validated on both synthetic and real scenarios, in addition to the Hopkins benchmark.

[1]  A. Fusiello,et al.  SEAMLESS IMAGE MOSAICKING VIA SYNCHRONIZATION , 2018 .

[2]  René Vidal,et al.  Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Xiaowei Zhou,et al.  A Survey on Rotation Optimization in Structure from Motion , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Carl Olsson,et al.  Stable Structure from Motion for Unordered Image Collections , 2011, SCIA.

[5]  René Vidal,et al.  Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[6]  A. Singer.  Angular Synchronization by Eigenvectors and Semidefinite Programming. , 2009, Applied and computational harmonic analysis.

[7]  Xun Xu,et al.  Motion Segmentation by Exploiting Complementary Geometric Models , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[9]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[10]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yuri Boykov,et al.  Energy-Based Geometric Multi-model Fitting , 2012, International Journal of Computer Vision.

[12]  René Vidal,et al.  Three-View Multibody Structure from Motion , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Andrea Fusiello,et al.  Synchronization in the Symmetric Inverse Semigroup , 2017, ICIAP.

[14]  Hongdong Li,et al.  Shape Interaction Matrix Revisited and Robustified: Efficient Subspace Clustering with Corrupted and Incomplete Data , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Ian Davidson,et al.  On constrained spectral clustering and its applications , 2012, Data Mining and Knowledge Discovery.

[16]  Venu Madhav Govindu.  Lie-algebraic averaging for globally consistent motion estimation , 2004, CVPR 2004.

[17]  Andrea Fusiello,et al.  Robust Multiple Model Fitting with Preference Analysis and Low-rank Approximation , 2015, BMVC.

[18]  S. Shankar Sastry,et al.  Two-View Multibody Structure from Motion , 2005, International Journal of Computer Vision.

[19]  Sujay Sanghavi,et al.  Normalized Spectral Map Synchronization , 2016, NIPS.

[20]  Venu Madhav Govindu,et al.  Robust Relative Rotation Averaging , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Gui-Song Xia,et al.  Globally consistent correspondence of multiple feature sets using proximal Gauss-Seidel relaxation , 2016, Pattern Recognit..

[22]  B. S. Manjunath,et al.  The multiRANSAC algorithm and its application to detect planar homographies , 2005, IEEE International Conference on Image Processing 2005.

[23]  Andrea Fusiello,et al.  Practical and Efficient Multi-view Matching , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Yan Yan,et al.  Motion Segmentation Via a Sparsity Constraint , 2017, IEEE Transactions on Intelligent Transportation Systems.

[25]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Ronen Basri,et al.  A Survey on Structure from Motion , 2017, ArXiv.

[27]  P.R. Kumar,et al.  Distributed Clock Synchronization over Wireless Networks: Algorithms and Analysis , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[28]  Xiaowei Zhou,et al.  Fast Multi-image Matching via Density-Based Clustering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Hongdong Li,et al.  Rotation Averaging , 2013, International Journal of Computer Vision.

[31]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[32]  P. Torr.  Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[33]  Luc Van Gool,et al.  Multibody Structure-from-Motion in Practice , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[35]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[36]  Jana Kosecka,et al.  Nonparametric Estimation of Multiple Structures with Outliers , 2006, WDV.

[37]  Andrea Fusiello,et al.  Spectral Synchronization of Multiple Views in SE(3) , 2016, SIAM J. Imaging Sci..

[38]  Jiri Matas,et al.  Multi-Class Model Fitting by Energy Minimization and Mode-Seeking , 2017, ECCV.

[39]  Tat-Jun Chin,et al.  The Random Cluster Model for robust geometric fitting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Ira Kemelmacher-Shlizerman,et al.  Global Motion Estimation from Point Matches , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[41]  R. Reulke,et al.  Remote Sensing and Spatial Information Sciences , 2005 .

[42]  Yoel Shkolnisky,et al.  Three-Dimensional Structure Determination from Common Lines in Cryo-EM by Eigenvectors and Semidefinite Programming , 2011, SIAM J. Imaging Sci..

[43]  René Vidal,et al.  Projective Factorization of Multiple Rigid-Body Motions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Andrea Fusiello,et al.  T-Linkage: A Continuous Relaxation of J-Linkage for Multi-model Fitting , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Davide Scaramuzza,et al.  Monocular simultaneous multi-body motion segmentation and reconstruction from perspective views , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[47]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[48]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Deborah Estrin,et al.  Optimal and Global Time Synchronization in Sensornets , 2003 .

[50]  Haesun Park,et al.  SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering , 2014, Journal of Global Optimization.

[51]  Vikas Singh,et al.  Solving the multi-way matching problem by permutation synchronization , 2013, NIPS.

[52]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[53]  Nassir Navab,et al.  Closed-form solutions to multiple-view homography estimation , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[54]  Johan Thunberg,et al.  A solution for multi-alignment by transformation synchronisation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Mehryar Mohri,et al.  Learning Non-Linear Combinations of Kernels , 2009, NIPS.

[56]  Andrea Fusiello,et al.  Multiple Models Fitting as a Set Coverage Problem , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  John J. Leonard,et al.  SE-Sync: A certifiably correct algorithm for synchronization over the special Euclidean group , 2016, Int. J. Robotics Res..

[58]  Erkki Oja,et al.  A new curve detection method: Randomized Hough transform (RHT) , 1990, Pattern Recognit. Lett..

[59]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[60]  Johan Fredriksson,et al.  Simultaneous Multiple Rotation Averaging Using Lagrangian Duality , 2012, ACCV.

[61]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[62]  Zhuwen Li,et al.  Perspective Motion Segmentation via Collaborative Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[63]  É. Vincent,et al.  Detecting planar homographies in an image pair , 2001, ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat..

[64]  Alessio Del Bue,et al.  Joint estimation of segmentation and structure from motion , 2013, Comput. Vis. Image Underst..

[65]  Andrea Fusiello,et al.  Robust Multiple Structures Estimation with J-Linkage , 2008, ECCV.

[66]  U SaputraMuhamad Risqi,et al.  Visual SLAM and Structure from Motion in Dynamic Environments , 2018 .