An iterative multiresolution scheme for SFM with missing data: Single and multiple object scenes

Most of the techniques proposed for tackling the Structure from Motion problem (SFM) cannot deal with high percentages of missing data in the matrix of trajectories. Furthermore, an additional problem should be faced up when working with multiple object scenes: the rank of the matrix of trajectories should be estimated. This paper presents an iterative multiresolution scheme for SFM with missing data to be used in both the single and multiple object cases. The proposed scheme aims at recovering missing entries in the original input matrix. The objective is to improve the results by applying a factorization technique to the partially or totally filled in matrix instead of to the original input one. Experimental results obtained with synthetic and real data sequences, containing single and multiple objects, are presented to show the viability of the proposed approach.

[1]  Henrik Aanæs,et al.  Robust Factorization , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  K. Kanatani,et al.  Estimating the Number of Independent Motions for Multibody Motion Segmentation , 2002 .

[3]  Joan Serrat,et al.  Factorization with Missing and Noisy Data , 2006, International Conference on Computational Science.

[4]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[5]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[6]  Takeo Kanade,et al.  A factorization method for affine structure from line correspondences , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Bill Triggs,et al.  Linear projective reconstruction from matching tensors , 1997, Image Vis. Comput..

[8]  T. Boult,et al.  Factorization-based segmentation of motions , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[9]  Takayuki Okatani,et al.  On the Wiberg Algorithm for Matrix Factorization in the Presence of Missing Components , 2007, International Journal of Computer Vision.

[10]  Tomás Pajdla,et al.  3D reconstruction by fitting low-rank matrices with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  David W. Jacobs,et al.  Linear Fitting with Missing Data for Structure-from-Motion , 2001, Comput. Vis. Image Underst..

[12]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  T. Mexia,et al.  Author ' s personal copy , 2009 .

[14]  Andrew W. Fitzgibbon,et al.  Damped Newton algorithms for matrix factorization with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  R. Hartley,et al.  PowerFactorization : 3D reconstruction with missing or uncertain data , 2003 .

[16]  Joan Serrat,et al.  An Iterative Multiresolution Scheme for SFM with Missing Data , 2009, Journal of Mathematical Imaging and Vision.

[17]  Angel D. Sappa,et al.  Rank estimation in 3D multibody motion segmentation , 2008 .

[18]  Takeo Kanade,et al.  A unified factorization algorithm for points, line segments and planes with uncertainty models , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[19]  Lihi Zelnik-Manor,et al.  Degeneracies, dependencies and their implications in multi-body and multi-sequence factorizations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[20]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[21]  David Suter,et al.  Recovering the missing components in a large noisy low-rank matrix: application to SFM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[23]  Adrien Bartoli,et al.  Affine Approximation for Direct Batch Recovery of Euclidian Structure and Motion from Sparse Data , 2006, International Journal of Computer Vision.

[24]  Mei Han,et al.  Reconstruction of a Scene with Multiple Linearly Moving Objects , 2004, International Journal of Computer Vision.

[25]  Russ B. Altman,et al.  Missing value estimation methods for DNA microarrays , 2001, Bioinform..

[26]  Aleix M. Martínez,et al.  Perturbation Estimation of the Subspaces for Structure from Motion with Noisy and Missing Data , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[27]  Pedro M. Q. Aguiar,et al.  Estimation of Rank Deficient Matrices from Partial Observations: Two-Step Iterative Algorithms , 2003, EMMCVPR.

[28]  Takeo Kanade,et al.  A sequential factorization method for recovering shape and motion from image streams , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Shmuel Friedland,et al.  A simultaneous reconstruction of missing data in DNA microarrays , 2006 .

[30]  Joan Serrat,et al.  An Iterative Multiresolution Scheme for SFM , 2006, ICIAR.