Reference-free monocular 3D tacking of deformable surfaces

We address the problem of deformable surfaces tracking in a monocular video for the purpose of reconstructing the 3D shape of the intended surface. The proposed method is based on structure from motion. In this method, deformation of the surface is modeled as a Gaussian process, and hence the tracking problem is formulated using the predictive capability of stochastic processes. Based on our experimental results, the proposed method is even more accurate than template-based methods. Also, it is highly robust against the tracking initialization, which is a challenging problem in all tracking algorithms.

[1]  Alessio Del Bue,et al.  Non-rigid structure from motion using ranklet-based tracking and non-linear optimization , 2007, Image Vis. Comput..

[2]  Jing Xiao,et al.  Uncalibrated perspective reconstruction of deformable structures , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[3]  Henrik Aanæs,et al.  Estimation of Deformable Structure and Motion , 2002 .

[4]  Petros Faloutsos,et al.  Dynamic Free-Form Deformations for Animation Synthesis , 1997, IEEE Trans. Vis. Comput. Graph..

[5]  E. Malis,et al.  Deeper understanding of the homography decomposition for vision-based control , 2007 .

[6]  Pascal Fua,et al.  Deformable Surface 3D Reconstruction from Monocular Images , 2010, Synthesis Lectures on Computer Vision.

[7]  Serge J. Belongie,et al.  Re-thinking non-rigid structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Adrien Bartoli,et al.  Monocular Template-based Reconstruction of Inextensible Surfaces , 2011, International Journal of Computer Vision.

[9]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Adrien Bartoli,et al.  A Quasi-Minimal Model for Paper-Like Surfaces , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Juho Kannala,et al.  Uncalibrated non-rigid factorisation with automatic shape basis selection , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[12]  Aaron Hertzmann,et al.  Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Lieven Eeckhout,et al.  Deformable Surface 3D Reconstruction from Monocular Images , 2010 .

[14]  David J. Fleet,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Gaussian Process Dynamical Model , 2007 .

[15]  Pascal Fua,et al.  Local deformation models for monocular 3D shape recovery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Yaser Sheikh,et al.  In defense of orthonormality constraints for nonrigid structure from motion , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Pascal Fua,et al.  Reconstructing sharply folding surfaces: A convex formulation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jing Xiao,et al.  Non-rigid shape and motion recovery: degenerate deformations , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[19]  Jing Xiao,et al.  A Closed-Form Solution to Non-Rigid Shape and Motion Recovery , 2004, International Journal of Computer Vision.

[20]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[21]  Pascal Fua,et al.  Convex Optimization for Deformable Surface 3-D Tracking , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Vincent Lepetit,et al.  Deformable Surface Tracking Ambiguities , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Pascal Fua,et al.  Template-free monocular reconstruction of deformable surfaces , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[24]  René Vidal,et al.  Perspective Nonrigid Shape and Motion Recovery , 2008, ECCV.

[25]  Yuncai Liu,et al.  Monocular 3-D Tracking of Inextensible Deformable Surfaces Under $L_2$ -Norm , 2010, IEEE Transactions on Image Processing.