A factorization method in stereo motion for non-rigid objects

In this paper we propose a framework of factorization-based non-rigid shape modeling and tracking in stereo-motion. We construct a measurement matrix with the stereo-motion data captured from a stereo-rig. Organized in a particular way this matrix could be decomposed by singular value decomposition (SVD) into the 3D basis shapes, their configuration weights, rigid motion and camera geometry. Accordingly, the stereo correspondences can be inferred from motion correspondences only requiring that a minimum of 3K point stereo correspondences (where K is the dimension of shape basis space) are created in advance. Basically this framework still keeps the property of rank constraints, meanwhile it owns other advantages such as simpler correspondence and accurate reconstruction even with short image sequences. Results with real data are given to demonstrate its performance.

[1]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[2]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Michal Irani,et al.  Multi-frame optical flow estimation using subspace constraints , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Yu Huang,et al.  Facial tracking with head pose estimation in stereo vision , 2002, Proceedings. International Conference on Image Processing.

[5]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  P. Anandan,et al.  Factorization with Uncertainty , 2000, International Journal of Computer Vision.

[7]  F. Dornaika,et al.  Stereo correspondence from motion correspondence , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Alessio Del Bue,et al.  Non-Rigid Stereo Factorization , 2006, International Journal of Computer Vision.

[9]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[11]  Y. Aloimonos,et al.  Spatio-Temporal Stereo Using Multi-Resolution Subdivision Surfaces , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[12]  P.K. Ho,et al.  Stereo-motion that complements stereo and motion analyses , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Jing Xiao,et al.  A Closed-Form Solution to Non-Rigid Shape and Motion Recovery , 2004, International Journal of Computer Vision.

[14]  Kiriakos N. Kutulakos,et al.  Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion, Shape and Reflectance , 2002, International Journal of Computer Vision.

[15]  Kiriakos N. Kutulakos,et al.  Multi-view scene capture by surfel sampling: from video streams to non-rigid 3D motion, shape and reflectance , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Amnon Shashua,et al.  Direct estimation of motion and extended scene structure from a moving stereo rig , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[17]  Jing Xiao,et al.  A Closed-Form Solution to Non-rigid Shape and Motion Recovery , 2004, ECCV.