Quasi-Perspective Factorization

Previous studies on structure and motion factorization are either based on simplified affine assumption or general perspective projection. The affine approximation is widely adopted due to its simplicity, whereas the extension to perspective model suffers from difficulties in projective depth recovery. To fill the gap between simplicity of affine and accuracy of perspective model, we propose a quasi-perspective factorization algorithm for structure and motion recovery of both rigid and nonrigid objects. Firstly, we establish a framework of rigid and nonrigid factorization under quasi-perspective assumption. Secondly, we propose an extended Cholesky decomposition to recover the rotation part of the Euclidean upgrading matrix. Finally, we prove that the last column of the upgrading matrix corresponds to a global scale and translation of the camera thus may be set freely. The proposed algorithm is validated and evaluated extensively on synthetic and real image sequences.

[1]  Mei Han,et al.  Creating 3D models with uncalibrated cameras , 2000, Proceedings Fifth IEEE Workshop on Applications of Computer Vision.

[2]  Long Quan,et al.  Self-calibration of an affine camera from multiple views , 1996, International Journal of Computer Vision.

[3]  Martial Hebert,et al.  Iterative projective reconstruction from multiple views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[6]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[7]  R. Hartley,et al.  PowerFactorization : 3D reconstruction with missing or uncertain data , 2003 .

[8]  Serge J. Belongie,et al.  Re-thinking non-rigid structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  René Vidal,et al.  Projective Factorization of Multiple Rigid-Body Motions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Jing Xiao,et al.  Uncalibrated perspective reconstruction of deformable structures , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Guanghui Wang,et al.  Perspective 3-D Euclidean Reconstruction With Varying Camera Parameters , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[13]  Jing Xiao,et al.  A Closed-Form Solution to Non-Rigid Shape and Motion Recovery , 2004, International Journal of Computer Vision.

[14]  Guanghui Wang,et al.  Quasi-perspective projection with applications to 3D factorization from uncalibrated image sequences , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Alessio Del Bue,et al.  Non-Rigid Metric Shape and Motion Recovery from Uncalibrated Images Using Priors , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Matthew Brand,et al.  A direct method for 3D factorization of nonrigid motion observed in 2D , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Aaron Hertzmann,et al.  Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[20]  Zhanyi Hu,et al.  Structure and motion of nonrigid object under perspective projection , 2007, Pattern Recognit. Lett..

[21]  Guanghui Wang,et al.  Rotation constrained power factorization for structure from motion of nonrigid objects , 2008, Pattern Recognit. Lett..

[22]  Marc Pollefeys,et al.  A Factorization-Based Approach to Articulated Motion Recovery , 2005, CVPR.

[23]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Richard I. Hartley,et al.  Iterative Extensions of the Sturm/Triggs Algorithm: Convergence and Nonconvergence , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Guanghui Wang,et al.  Quasi-perspective Projection Model: Theory and Application to Structure and Motion Factorization from Uncalibrated Image Sequences , 2010, International Journal of Computer Vision.

[26]  Andrew Zisserman,et al.  Robust Detection of Degenerate Configurations while Estimating the Fundamental Matrix , 1998, Comput. Vis. Image Underst..

[27]  René Vidal,et al.  Perspective Nonrigid Shape and Motion Recovery , 2008, ECCV.

[28]  Marc Pollefeys,et al.  A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Guanghui Wang,et al.  Stratification Approach for 3-D Euclidean Reconstruction of Nonrigid Objects From Uncalibrated Image Sequences , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).