Incorporating Non-motion Cues into 3D Motion Segmentation

We address the problem of segmenting an image sequence into rigidly moving 3D objects. An elegant solution to this problem is the multibody factorization approach in which the measurement matrix is factored into lower rank matrices. Despite progress in factorization algorithms, the performance is still far from satisfactory and in scenes with missing data and noise, most existing algorithms fail. In this paper we propose a method for incorporating 2D non-motion cues (such as spatial coherence) into multibody factorization. We formulate the problem in terms of constrained factor analysis and use the EM algorithm to find the segmentation. We show that adding these cues improves performance in real and synthetic sequences.

[1]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Y. Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, CVPR 2004.

[3]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  S. Shankar Sastry,et al.  Two-View Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix , 2002 .

[5]  Jitendra Malik,et al.  Motion segmentation and tracking using normalized cuts , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[6]  Yair Be'ery,et al.  Convergence analysis of turbo decoding of product codes , 2001, IEEE Trans. Inf. Theory.

[7]  Takeo Kanade,et al.  A multi-body factorization method for motion analysis , 1995, Proceedings of IEEE International Conference on Computer Vision.

[8]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Kenichi Kanatani,et al.  Evaluation and Selection of Models for Motion Segmentation , 2001, ECCV.

[10]  R. Zabih,et al.  Spatially coherent clustering using graph cuts , 2004, CVPR 2004.

[11]  P. Anandan,et al.  Factorization with Uncertainty , 2000, International Journal of Computer Vision.

[12]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[13]  R. Vidal,et al.  Motion segmentation with missing data using PowerFactorization and GPCA , 2004, CVPR 2004.

[14]  Yair Weiss,et al.  Factorization with Uncertainty and Missing Data: Exploiting Temporal Coherence , 2003, NIPS.

[15]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[16]  C. W. Gear,et al.  Multibody Grouping from Motion Images , 1998, International Journal of Computer Vision.

[17]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  T. Kanade,et al.  A multi-body factorization method for motion analysis , 1995, ICCV 1995.

[19]  W. James MacLean,et al.  Recovery of Egomotion and Segmentation of Independent Object Motion Using the EM Algorithm , 1994, BMVC.

[20]  Lihi Zelnik-Manor,et al.  Multi-body Factorization with Uncertainty: Revisiting Motion Consistency , 2005, International Journal of Computer Vision.

[21]  Pietro Perona,et al.  Scene segmentation from 3D motion , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[22]  M. Irani,et al.  Multi-body Segmentation : Revisiting Motion Consistency , 2002 .

[23]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[24]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .