Optimal Templates for Nonrigid Surface Reconstruction

This paper addresses the problem of reconstructing a deforming surface from point observations in a monocular video sequence. Recent state-of-the-art approaches divide the surface into smaller patches to simplify the problem. Among these, one very promising approach reconstructs the patches individually using a quadratic deformation model. In this paper, we demonstrate limitations that affect its applicability to real-world data and propose an approach that overcomes these problems. In particular, we show how to eliminate the need for manually picking a template that is used to model the deformations. We evaluate our algorithm on both synthetic and real-world data sets and show that it systematically reduces the reconstruction error by a factor of up to ten.

[1]  Adrien Bartoli,et al.  Monocular Template-based Reconstruction of Inextensible Surfaces , 2011, International Journal of Computer Vision.

[2]  Alessio Del Bue,et al.  Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences , 2010, ECCV.

[3]  David A. Forsyth,et al.  Capturing and animating occluded cloth , 2007, ACM Trans. Graph..

[4]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[5]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[6]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Pushmeet Kohli,et al.  Surface stereo with soft segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Nassir Navab,et al.  Monocular Template-Based Reconstruction of Smooth and Inextensible Surfaces , 2010, ACCV.

[9]  Aleix M. Martínez,et al.  Computing Smooth Time Trajectories for Camera and Deformable Shape in Structure from Motion with Occlusion , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Lourdes Agapito,et al.  Energy based multiple model fitting for non-rigid structure from motion , 2011, CVPR 2011.

[12]  René Vidal,et al.  Perspective Nonrigid Shape and Motion Recovery , 2008, ECCV.

[13]  Pascal Fua,et al.  Reconstructing sharply folding surfaces: A convex formulation , 2009, CVPR.

[14]  Aleix M. Martínez,et al.  Kernel non-rigid structure from motion , 2011, 2011 International Conference on Computer Vision.

[15]  Alessio Del Bue,et al.  Non-rigid Structure from Motion using Quadratic Deformation Models , 2009, BMVC.

[16]  Pascal Fua,et al.  Convex Optimization for Deformable Surface 3-D Tracking , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Pascal Fua,et al.  Reconstructing sharply folding surfaces: A convex formulation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Pascal Fua,et al.  Linear Local Models for Monocular Reconstruction of Deformable Surfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Raquel Urtasun,et al.  Implicitly Constrained Gaussian Process Regression for Monocular Non-Rigid Pose Estimation , 2010, NIPS.

[20]  Takeo Kanade,et al.  Nonrigid Structure from Motion in Trajectory Space , 2008, NIPS.

[21]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[22]  Lourdes Agapito,et al.  Factorization for non-rigid and articulated structure using metric projections , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Kiriakos N. Kutulakos,et al.  Non-rigid structure from locally-rigid motion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Raquel Urtasun,et al.  Physically-based motion models for 3D tracking: A convex formulation , 2011, 2011 International Conference on Computer Vision.

[25]  Markus H. Gross,et al.  Meshless deformations based on shape matching , 2005, ACM Trans. Graph..