Recovery of Nonrigid Structures from 2D Observations

We present a new method for simultaneously determining three-dimensional (3D) motion and structure of a nonrigid object from its uncalibrated two-dimensional (2D) data with Gaussian or non-Gaussian distributions. A nonrigid motion can be treated as a combination of a rigid component and a nonrigid deformation. To reduce the high dimensionality of the deformable structure or shape, we estimate the probability distribution function (PDF) of the structure through random sampling, integrating an established probabilistic model. The fitting between the observations and the estimated 3D structure will be evaluated using the pooled variance estimator. The recovered structure is only available when the 2D feature points have been properly corresponded over two image frames. Applications of the proposed method to both synthetic and real image sequences are demonstrated with promising results.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[3]  Gang Xu,et al.  A General Expression of the Fundamental Matrix for Both Perspective and Affine Cameras , 1997, IJCAI.

[4]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[6]  Narendra Ahuja,et al.  Optimal motion and structure estimation , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[8]  Kiriakos N. Kutulakos,et al.  Multi-view 3D shape and motion recovery on the spatio-temporal curve manifold , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Alessio Del Bue,et al.  Non-Rigid Metric Shape and Motion Recovery from Uncalibrated Images Using Priors , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  Leonid Zhukov,et al.  Direct Pattern Tracking On Flexible Geometry , 2002, WSCG.

[11]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[12]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  E. Vonesh,et al.  Efficient inference for random-coefficient growth curve models with unbalanced data. , 1987, Biometrics.

[14]  Adrien Bartoli,et al.  A Batch Algorithm for Implicit Non-rigid Shape and Motion Recovery , 2006, WDV.

[15]  Edmond Boyer,et al.  Camera calibration and 3D reconstruction from single images using parallelepipeds , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[17]  Aaron Hertzmann,et al.  Learning Non-Rigid 3D Shape from 2D Motion , 2003, NIPS.

[18]  T. Kanade,et al.  Non-rigid shape and motion recovery: degenerate deformations , 2004, CVPR 2004.

[19]  Brian C. Lovell,et al.  A First Order Predicate Logic Formulation Of The 3d Reconstruction Problem And Its Solution Space , 2005, Int. J. Pattern Recognit. Artif. Intell..

[20]  Yoshinobu Sato,et al.  Acquiring 3D Models of Non-Rigid Moving Objects From Time and Viewpoint Varying Image Sequences: A Step Toward Left Ventricle Recovery , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Zhiwei Zhu,et al.  A Real-Time Human Stress Monitoring System Using Dynamic Bayesian Network , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[22]  Aaron Hertzmann,et al.  Automatic Non-rigid 3D Modeling from Video , 2004, ECCV.

[23]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).