Resolving occlusion in multiframe reconstruction of deformable surfaces

Occlusion is troublesome for almost all computer vision algorithms. To a certain extent, the difficulty is alleviated when multiple frames are given. On the other hand, when we consider the recovery of shapes of moving deformable objects, observed using a monocular camera, the problem appears difficult again. In this paper, we show a method that outperforms previous approaches to reconstruction when feature data is unavailable, perhaps due to occlusion. Our key intuition is that portions of the surface that are visible in some frame can be reliably reconstructed in that frame; further, the reliable portions can be stitched together to find even missing portions, much the way a human eye would hallucinate. Our techniques are based on optimization in Riemannian shape spaces, and is demonstrated on isometric surfaces without involving any kind of machine learning methods.

[1]  Pascal Fua,et al.  Simultaneous point matching and 3D deformable surface reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  M. Salzmann,et al.  Learning and recovering 3D surface deformations , 2009 .

[3]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Lieven Eeckhout,et al.  Deformable Surface 3D Reconstruction from Monocular Images , 2010 .

[5]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[6]  Zenglin Xu,et al.  An Effective Approach to 3D Deformable Surface Tracking , 2008, ECCV.

[7]  Demetri Terzopoulos,et al.  Symmetry-seeking models and 3D object reconstruction , 1988, International Journal of Computer Vision.

[8]  Alessio Del Bue,et al.  Non-rigid Structure from Motion using Quadratic Deformation Models , 2009, BMVC.

[9]  M. Kilian,et al.  Geometric modeling in shape space , 2007, SIGGRAPH 2007.

[10]  T. Banchoff,et al.  Differential Geometry of Curves and Surfaces , 2010 .

[11]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Kiriakos N. Kutulakos,et al.  Non-rigid structure from locally-rigid motion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[14]  Alessio Del Bue,et al.  Non-rigid structure from motion using ranklet-based tracking and non-linear optimization , 2007, Image Vis. Comput..

[15]  Laurent D. Cohen,et al.  Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Jing Xiao,et al.  Uncalibrated perspective reconstruction of deformable structures , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[18]  Pascal Fua,et al.  Template-free monocular reconstruction of deformable surfaces , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  Dmitry B. Goldgof,et al.  Nonrigid motion analysis based on dynamic refinement of finite element models , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[20]  Aaron Hertzmann,et al.  Learning Non-Rigid 3D Shape from 2D Motion , 2003, NIPS.

[21]  Aaron Hertzmann,et al.  Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jessica K. Hodgins,et al.  Estimating cloth simulation parameters from video , 2003, SCA '03.

[23]  Pascal Fua,et al.  Linear Local Models for Monocular Reconstruction of Deformable Surfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Adrien Bartoli,et al.  Monocular Template-based Reconstruction of Inextensible Surfaces , 2011, International Journal of Computer Vision.

[25]  Sharat Chandran,et al.  Riemannian manifold optimisation for non-rigid structure from motion , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.