Capturing 3D stretchable surfaces from single images in closed form

We present a closed form solution to the problem of recovering the 3D shape of a nonrigid potentially stretchable surface from 3D-to-2D correspondences. In other words, we can reconstruct a surface from a single image without a priori knowledge of its deformations in that image. State of the art solutions to nonrigid 3D shape recovery rely on the fact that distances between neighboring surface points must be preserved and are therefore limited to inelastic surfaces. Here, we show that replacing the inextensibility constraints by shading ones removes this limitation while still allowing 3D reconstruction in closed-form. We demonstrate our method and compare it to an earlier one using both synthetic and real data.

[1]  René Vidal,et al.  Perspective Nonrigid Shape and Motion Recovery , 2008, ECCV.

[2]  Dmitry B. Goldgof,et al.  Nonrigid Motion Analysis Based on Dynamic Refinement of Finite Element Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Kiriakos N. Kutulakos,et al.  Semidefinite Programming Heuristics for Surface Reconstruction Ambiguities , 2008, ECCV.

[4]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[5]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[6]  Vincent Lepetit,et al.  Closed-Form Solution to Non-rigid 3D Surface Registration , 2008, ECCV.

[7]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Laurent D. Cohen,et al.  Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Jessica K. Hodgins,et al.  Estimating cloth simulation parameters from video , 2003, SCA '03.

[10]  David A. Forsyth,et al.  Combining Cues: Shape from Shading and Texture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Lei Zhang,et al.  Face synthesis and recognition from a single image under arbitrary unknown lighting using a spherical harmonic basis morphable model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Adrien Bartoli,et al.  Monocular Template-based Reconstruction of Inextensible Surfaces , 2008, BMVC.

[13]  Vincent Lepetit,et al.  Deformable Surface Tracking Ambiguities , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[15]  Katsushi Ikeuchi,et al.  Deformable surfaces: a free-form shape representation , 1991, Optics & Photonics.

[16]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[17]  Demetri Terzopoulos,et al.  A finite element model for 3D shape reconstruction and nonrigid motion tracking , 1993, 1993 (4th) International Conference on Computer Vision.

[18]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[19]  Andrew P. Witkin,et al.  Recovering Surface Shape and Orientation from Texture , 1981, Artif. Intell..

[20]  Dimitris N. Metaxas,et al.  Constrained deformable superquadrics and nonrigid motion tracking , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[22]  Aaron Hertzmann,et al.  Learning Non-Rigid 3D Shape from 2D Motion , 2003, NIPS.

[23]  Gang Hua,et al.  Face Re-Lighting from a Single Image under Harsh Lighting Conditions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.