Stochastic Exploration of Ambiguities for Nonrigid Shape Recovery

Recovering the 3D shape of deformable surfaces from single images is known to be a highly ambiguous problem because many different shapes may have very similar projections. This is commonly addressed by restricting the set of possible shapes to linear combinations of deformation modes and by imposing additional geometric constraints. Unfortunately, because image measurements are noisy, such constraints do not always guarantee that the correct shape will be recovered. To overcome this limitation, we introduce a stochastic sampling approach to efficiently explore the set of solutions of an objective function based on point correspondences. This allows us to propose a small set of ambiguous candidate 3D shapes and then use additional image information to choose the best one. As a proof of concept, we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult nonlinear minimization problem. The advantages of our method are demonstrated on a variety of problems including both real and synthetic data.

[1]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[2]  Michael Isard,et al.  Active Contours , 2000, Springer London.

[3]  Stefano Soatto,et al.  Optimal Structure from Motion: Local Ambiguities and Global Estimates , 2004, International Journal of Computer Vision.

[4]  Gang Hua,et al.  Face Re-Lighting from a Single Image under Harsh Lighting Conditions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Richard Szeliski,et al.  Shape Ambiguities in Structure From Motion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Aaron Hertzmann,et al.  Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Jing Xiao,et al.  Uncalibrated perspective reconstruction of deformable structures , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[8]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[9]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[10]  Francesc Moreno-Noguer,et al.  Simultaneous pose, correspondence and non-rigid shape , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[12]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Zenglin Xu,et al.  An Effective Approach to 3D Deformable Surface Tracking , 2008, ECCV.

[14]  Vincent Lepetit,et al.  Closed-Form Solution to Non-rigid 3D Surface Registration , 2008, ECCV.

[15]  Greg Hamerly,et al.  Learning the k in k-means , 2003, NIPS.

[16]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[17]  Vincent Lepetit,et al.  Capturing 3D stretchable surfaces from single images in closed form , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Michael Isard,et al.  Partitioned Sampling, Articulated Objects, and Interface-Quality Hand Tracking , 2000, ECCV.

[19]  Katsushi Ikeuchi,et al.  Light source position and reflectance estimation from a single view without the distant illumination assumption , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Pascal Fua,et al.  Linear Local Models for Monocular Reconstruction of Deformable Surfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Michael J. Black,et al.  Shining a Light on Human Pose: On Shadows, Shading and the Estimation of Pose and Shape , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Dimitris N. Metaxas,et al.  Constrained deformable superquadrics and nonrigid motion tracking , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[24]  David J. Fleet,et al.  Model-based hand tracking with texture, shading and self-occlusions , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Laurent D. Cohen,et al.  Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  David S. Doermann,et al.  Flattening curved documents in images , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[28]  Alessio Del Bue,et al.  Optimal Metric Projections for Deformable and Articulated Structure-from-Motion , 2011, International Journal of Computer Vision.

[29]  Adrien Bartoli,et al.  Monocular Template-based Reconstruction of Inextensible Surfaces , 2011, International Journal of Computer Vision.

[30]  Kiriakos N. Kutulakos,et al.  Semidefinite Programming Heuristics for Surface Reconstruction Ambiguities , 2008, ECCV.

[31]  Francesc Moreno-Noguer,et al.  Exploring Ambiguities for Monocular Non-rigid Shape Estimation , 2010, ECCV.

[32]  Rama Chellappa,et al.  Structure from Motion Using Sequential Monte Carlo Methods , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[33]  David A. Forsyth,et al.  Combining Cues: Shape from Shading and Texture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[35]  Larry S. Davis,et al.  Structure of Applicable Surfaces from Single Views , 2004, ECCV.

[36]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[37]  M. Brand.  Morphable 3 D models from video , 2001 .

[38]  Dmitry B. Goldgof,et al.  Nonrigid motion analysis based on dynamic refinement of finite element models , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[39]  Antonio Criminisi,et al.  Accurate Visual Metrology from Single and Multiple Uncalibrated Images , 2001, Distinguished Dissertations.

[40]  Demetri Terzopoulos,et al.  A finite element model for 3D shape reconstruction and nonrigid motion tracking , 1993, 1993 (4th) International Conference on Computer Vision.