Robust single-view geometry and motion reconstruction

We present a framework and algorithms for robust geometry and motion reconstruction of complex deforming shapes. Our method makes use of a smooth template that provides a crude approximation of the scanned object and serves as a geometric and topological prior for reconstruction. Large-scale motion of the acquired object is recovered using a novel space-time adaptive, non-rigid registration method. Fine-scale details such as wrinkles and folds are synthesized with an efficient linear mesh deformation algorithm. Subsequent spatial and temporal filtering of detail coefficients allows transfer of persistent geometric detail to regions not observed by the scanner. We show how this two-scale process allows faithful recovery of small-scale shape and motion features leading to a high-quality reconstruction. We illustrate the robustness and generality of our algorithm on a variety of examples composed of different materials and exhibiting a large range of dynamic deformations.

[1]  Jörg Weule,et al.  Non-Linear Gaussian Filters Performing Edge Preserving Diffusion , 1995, DAGM-Symposium.

[2]  J A Sethian,et al.  Computing geodesic paths on manifolds. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Kari Pulli,et al.  Multiview registration for large data sets , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[4]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[5]  Steven M. Seitz,et al.  Image-based multiresolution shape recovery by surface deformation , 2000, IS&T/SPIE Electronic Imaging.

[6]  S. W. Roberts Control chart tests based on geometric moving averages , 2000 .

[7]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[8]  Marc Levoy,et al.  A hierarchical method for aligning warped meshes , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[9]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[10]  Szymon Rusinkiewicz,et al.  Non-rigid range-scan alignment using thin-plate splines , 2004 .

[11]  Sebastian Thrun,et al.  The Correlated Correspondence Algorithm for Unsupervised Registration of Nonrigid Surfaces , 2004, NIPS.

[12]  Li Zhang,et al.  Spacetime faces: high resolution capture for modeling and animation , 2004, SIGGRAPH 2004.

[13]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[14]  Ron Kimmel,et al.  Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Jessica K. Hodgins,et al.  Capturing and animating skin deformation in human motion , 2006, SIGGRAPH 2006.

[16]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Leonidas J. Guibas,et al.  Dynamic geometry registration , 2007, Symposium on Geometry Processing.

[18]  Szymon Rusinkiewicz,et al.  Global non-rigid alignment of 3-D scans , 2007, SIGGRAPH 2007.

[19]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Leonidas J. Guibas,et al.  Eurographics Symposium on Geometry Processing (2007) Reconstruction of Deforming Geometry from Time-varying Point Clouds , 2022 .

[21]  M. Pauly,et al.  Embedded deformation for shape manipulation , 2007, SIGGRAPH 2007.

[22]  M. Gross,et al.  Algebraic point set surfaces , 2007, SIGGRAPH 2007.

[23]  Olga Sorkine-Hornung,et al.  On Linear Variational Surface Deformation Methods , 2008, IEEE Transactions on Visualization and Computer Graphics.

[24]  Hans-Peter Seidel,et al.  Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Derek Bradley,et al.  Markerless garment capture , 2008, SIGGRAPH 2008.

[26]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, SIGGRAPH 2008.

[27]  Günther Greiner,et al.  Reconstructing Animated Meshes from Time‐Varying Point Clouds , 2008, Comput. Graph. Forum.

[28]  Matthias Zwicker,et al.  Automatic Registration for Articulated Shapes , 2008, Comput. Graph. Forum.

[29]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[30]  Dan A. Alcantara,et al.  Space-time surface reconstruction using incompressible flow , 2008, SIGGRAPH 2008.

[31]  Leonidas J. Guibas,et al.  Non-Rigid Registration Under Isometric Deformations , 2008 .

[32]  Jessica K. Hodgins,et al.  Data-driven modeling of skin and muscle deformation , 2008, SIGGRAPH 2008.

[33]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[34]  Hans-Peter Seidel,et al.  Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data , 2009, TOGS.

[35]  Matthias Zwicker,et al.  Range Scan Registration Using Reduced Deformable Models , 2009, Comput. Graph. Forum.