Template-Based Monocular 3D Shape Recovery Using Laplacian Meshes

We show that by extending the Laplacian formalism, which was first introduced in the Graphics community to regularize 3D meshes, we can turn the monocular 3D shape reconstruction of a deformable surface given correspondences with a reference image into a much better-posed problem. This allows us to quickly and reliably eliminate outliers by simply solving a linear least squares problem. This yields an initial 3D shape estimate, which is not necessarily accurate, but whose 2D projections are. The initial shape is then refined by a constrained optimization problem to output the final surface reconstruction. Our approach allows us to reduce the dimensionality of the surface reconstruction problem without sacrificing accuracy, thus allowing for real-time implementations.

[1]  Adrien Bartoli,et al.  Using Isometry to Classify Correct/Incorrect 3D-2D Correspondences , 2014, ECCV.

[2]  Adrien Bartoli,et al.  On template-based reconstruction from a single view: Analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Alessio Del Bue,et al.  Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences , 2010, ECCV.

[4]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[5]  Nassir Navab,et al.  Monocular Template-Based Reconstruction of Smooth and Inextensible Surfaces , 2010, ACCV.

[6]  Vincent Lepetit,et al.  Closed-Form Solution to Non-rigid 3D Surface Registration , 2008, ECCV.

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  Pascal Fua,et al.  Convex Optimization for Deformable Surface 3-D Tracking , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Aaron Hertzmann,et al.  Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Daniel Pizarro-Perez,et al.  Feature-Based Deformable Surface Detection with Self-Occlusion Reasoning , 2011, International Journal of Computer Vision.

[11]  Vincent Lepetit,et al.  Capturing 3D stretchable surfaces from single images in closed form , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Kiriakos N. Kutulakos,et al.  Semidefinite Programming Heuristics for Surface Reconstruction Ambiguities , 2008, ECCV.

[13]  Pascal Fua,et al.  Accurate face models from uncalibrated and ill-lit video sequences , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[14]  Adrien Bartoli,et al.  Deformable 3D Reconstruction with an Object Database , 2012, BMVC.

[15]  Lourdes Agapito,et al.  Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Larry S. Davis,et al.  Structure of Applicable Surfaces from Single Views , 2004, ECCV.

[17]  Pascal Fua,et al.  Laplacian Meshes for Monocular 3D Shape Recovery , 2012, ECCV.

[18]  Lourdes Agapito,et al.  Energy based multiple model fitting for non-rigid structure from motion , 2011, CVPR 2011.

[19]  Michael R. Lyu,et al.  Progressive Finite Newton Approach To Real-time Nonrigid Surface Detection , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Pascal Fua,et al.  Linear Local Models for Monocular Reconstruction of Deformable Surfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Michael S. Brown,et al.  In Defence of RANSAC for Outlier Rejection in Deformable Registration , 2012, ECCV.

[22]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[23]  LiuYuncai,et al.  Monocular 3-D tracking of inextensible deformable surfaces under L2-norm , 2010 .

[24]  Markus Gross,et al.  Deformation Transfer for Detail-Preserving Surface Editing , 2006 .

[25]  Lieven Eeckhout,et al.  Deformable Surface 3D Reconstruction from Monocular Images , 2010 .

[26]  Olga Sorkine-Hornung,et al.  Bounded biharmonic weights for real-time deformation , 2011, Commun. ACM.

[27]  Adrien Bartoli,et al.  Monocular Template-Based 3D Reconstruction of Extensible Surfaces with Local Linear Elasticity , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Pascal Fua,et al.  Accurate face models from uncalibrated and ill-lit video sequences , 2004, CVPR 2004.

[29]  Pascal Fua,et al.  Using Dirichlet Free Form Deformation to Fit Deformable Models to Noisy 3-D Data , 2002, ECCV.

[30]  David S. Doermann,et al.  Flattening curved documents in images , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[33]  Vincent Lepetit,et al.  Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation , 2008, International Journal of Computer Vision.

[34]  Pascal Fua,et al.  Deformable Surface 3D Reconstruction from Monocular Images , 2010, Synthesis Lectures on Computer Vision.

[35]  Mathieu Perriollat,et al.  A computational model of bounded developable surfaces with application to image‐based three‐dimensional reconstruction , 2013, Comput. Animat. Virtual Worlds.

[36]  Vincent Lepetit,et al.  Fast Keypoint Recognition Using Random Ferns , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[38]  Alessio Del Bue,et al.  Multiview 3D warps , 2011, 2011 International Conference on Computer Vision.

[39]  Adrien Bartoli,et al.  Monocular Template-based Reconstruction of Inextensible Surfaces , 2011, International Journal of Computer Vision.

[40]  Pascal Fua,et al.  Least-Squares Minimization Under Constraints , 2010 .

[41]  Francesc Moreno-Noguer,et al.  Exploring Ambiguities for Monocular Non-rigid Shape Estimation , 2010, ECCV.

[42]  David A. Forsyth,et al.  Combining Cues: Shape from Shading and Texture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).