Modelling Nonrigid Object from Video Sequence Based on Power Factorization

Recovering the 3D structure and motion of nonrigid object from a monocular image sequence is an important and difficult task in computer vision. Many previous methods on this problem utilize the extension technique of SVD factorization based on rank constraint to the tracking matrix. In this paper, we propose a constrained power factorization (CPF) algorithm that combines the orthonormal constraint and the replicated block structure of the motion matrix directly into the iterations. The proposed algorithm overcomes the limitations of previous SVD based methods. It is easy to implement and can even cope with the tracking matrix with missing data. Based on the solutions of the CPF, a novel sequential factorization technique is proposed to recover the shape and motion of new frames in realtime. Extensive experiments on synthetic data and real sequences validate the effectiveness of the algorithm and show noticeable improvements over the previous methods.

[1]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[3]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Jun Fujiki,et al.  Recursive factorization method for the paraperspective model based on the perspective projection , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[5]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.

[6]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[7]  Anders Heyden,et al.  An iterative factorization method for projective structure and motion from image sequences , 1999, Image Vis. Comput..

[8]  Alessio Del Bue,et al.  NON-RIGID 3D SHAPE RECOVERY USING STEREO FACTORIZATION , 2004 .

[9]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Takeo Kanade,et al.  A sequential factorization method for recovering shape and motion from image streams , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[12]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  Jing Xiao,et al.  A Closed-Form Solution to Non-rigid Shape and Motion Recovery , 2004, ECCV.

[14]  R. Vidal,et al.  Motion segmentation with missing data using PowerFactorization and GPCA , 2004, CVPR 2004.

[15]  Mei Han,et al.  Reconstruction of a Scene with Multiple Linearly Moving Objects , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  Andrew Blake,et al.  Separability of pose and expression in facial tracking and animation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Martial Hebert,et al.  Iterative projective reconstruction from multiple views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[20]  T. Kanade,et al.  Non-rigid shape and motion recovery: degenerate deformations , 2004, CVPR 2004.

[21]  Stéphane Christy,et al.  Euclidean Shape and Motion from Multiple Perspective Views by Affine Iterations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  R. Hartley,et al.  PowerFactorization : 3D reconstruction with missing or uncertain data , 2003 .