Multi-Subspace Methods for Motion Segmentation from Affine, Perspective and Central Panoramic Cameras

Many robot navigation tasks require the computation of the motion of multiple objects moving in 3-D space from a collection of images taken by a moving robot. In this paper we present a unifying theoretical framework for both infinitesimal and discrete 3-D motion segmentation from optical flow or point correspondences in multiple affine, perspective or central panoramic views. We exploit the fact that for these motion and camera models, the image measurements associated with a single object live in a low dimensional subspace of a high dimensional space, hence motion segmentation is achieved by segmenting data living in multiple subspaces. We solve this problem in closed form using polynomial fitting and differentiation. Unlike previous work, our method does not restrict the motion of the objects to be full dimensional or fully independent. Instead, our approach deals gracefully with all the spectrum of possible motions: from low dimensional and partially dependent to full dimensional and fully independent. In addition, our method handles the case of missing data, meaning that point tracks do not have to be visible in all images. We test our algorithm on various real sequences with degenerate and nondegenerate motions, missing data, transparent motions, etc. Our algorithm achieves a misclassification error of less than 5% for sequences with up to 30% of missing data points.

[1]  Takeo Kanade,et al.  A multi-body factorization method for motion analysis , 1995, Proceedings of IEEE International Conference on Computer Vision.

[2]  Lihi Zelnik-Manor,et al.  Degeneracies, dependencies and their implications in multi-body and multi-sequence factorizations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[3]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  S. Shankar Sastry,et al.  Multibody motion estimation and segmentation from multiple central panoramic views , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[5]  S. Shankar Sastry,et al.  Infinitesimal motion estimation from multiple central panoramic views , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[6]  S. Shankar Sastry,et al.  Optimal segmentation of dynamic scenes from two perspective views , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[7]  Mei Han,et al.  Reconstruction of a Scene with Multiple Linearly Moving Objects , 2004, International Journal of Computer Vision.

[8]  Kenichi Kanatani,et al.  Multi-stage Optimization for Multi-body Motion Segmentation , 2003 .

[9]  Amnon Shashua,et al.  Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[10]  T. Boult,et al.  Factorization-based segmentation of motions , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[11]  Yi Ma,et al.  A new GPCA algorithm for clustering subspaces by fitting, differentiating and dividing polynomials , 2004, CVPR 2004.

[12]  René Vidal A Factorization Method for 3 D Multi-body Motion Estimation and Segmentation ∗ , 2002 .

[13]  Kenichi Kanatani,et al.  Motion segmentation by subspace separation and model selection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Lior Wolf,et al.  Affine 3-D reconstruction from two projective images of independently translating planes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Peter F. Sturm Structure and Motion for Dynamic Scenes - The Case of Points Moving in Planes , 2002, ECCV.

[17]  S. Shankar Sastry,et al.  Two-View Multibody Structure from Motion , 2005, International Journal of Computer Vision.

[18]  KanadeTakeo,et al.  Shape and motion from image streams under orthography , 1992 .

[19]  René Vidal,et al.  The multibody trifocal tensor: motion segmentation from 3 perspective views , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[20]  Ying Wu,et al.  Multibody grouping via orthogonal subspace decomposition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  René Vidal,et al.  A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation , 2004, ECCV.

[22]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[23]  Stefano Soatto,et al.  Structure from Motion Causally Integrated Over Time , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Michal Irani,et al.  Multi-frame optical flow estimation using subspace constraints , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  René Vidal,et al.  A new GPCA algorithm for clustering subspaces by fitting, differentiating and dividing polynomials , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..