Geodesic Mapping for Dynamic Surface Alignment

This paper presents a novel approach that achieves dynamic surface alignment by geodesing mapping. The surfaces are 3D manifold meshes representing non-rigid objects in motion (e.g., humans) which can be obtained by multiview stereo reconstruction. The proposed framework consists of a geodesic mapping (i.e., geodesic diffeomorphism) between surfaces which carry a distance function (namely the global geodesic distance), and a geodesic-based coordinate system (namely the global geodesic coordinates) defined similarly to generalized barycentric coordinates. The coordinates are used to recursively choose correspondence points in non-ambiguous regions using a coarse-to-fine strategy to reliably locate all surface points and define a discrete mapping. Complete point-to-point surface alignment with smooth mapping is then derived by optimizing a piecewise objective function within a probabilistic framework. The proposed technique only relies on surface intrinsic geometrical properties, and does not require prior knowledge on surface appearance (e.g., color or texture), shape (e.g., topology) or parameterization (e.g., mesh connectivity or complexity). The method can be used for numerous applications, such as visual information (e.g., texture) transfer between surface models representing different objects, dense motion flow estimation of 3D dynamic surfaces, wide-timeframe matching, etc. Experiments show compelling results on challenging publicly available real-world datasets.

[1]  Slobodan Ilic,et al.  Probabilistic Deformable Surface Tracking from Multiple Videos , 2010, ECCV.

[2]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Valerio Pascucci,et al.  Time-varying reeb graphs for continuous space-time data , 2004, SCG '04.

[4]  Alexander M. Bronstein,et al.  Topology-Invariant Similarity of Nonrigid Shapes , 2009, International Journal of Computer Vision.

[5]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Bruno Raffin,et al.  A Distributed Approach for Real Time 3D Modeling , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[7]  Hans-Peter Seidel,et al.  Geodesics Guided Constrained Texture Deformation , 2006 .

[8]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[9]  Pieter Peers,et al.  Temporally coherent completion of dynamic shapes , 2012, TOGS.

[10]  Hujun Bao,et al.  3D Reconstruction of Dynamic Scenes with Multiple Handheld Cameras , 2012, ECCV.

[11]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, ACM Trans. Graph..

[12]  Hans-Peter Seidel,et al.  Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Lawrence H. Staib,et al.  Shape-based 3D surface correspondence using geodesics and local geometry , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Leonidas J. Guibas,et al.  Non-Rigid Registration Under Isometric Deformations , 2008 .

[15]  Takeo Kanade,et al.  Shape-From-Silhouette Across Time Part II: Applications to Human Modeling and Markerless Motion Tracking , 2005, International Journal of Computer Vision.

[16]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[17]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[18]  Takeo Kanade,et al.  A stereo machine for video-rate dense depth mapping and its new applications , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Anne Verroust-Blondet,et al.  Three-dimensional metamorphosis: a survey , 1998, The Visual Computer.

[20]  Thomas Ertl,et al.  Scale-Space Tracking of Critical Points in 3D Vector Fields , 2007, Topology-based Methods in Visualization.

[21]  Takashi Matsuyama,et al.  Topology Dictionary for 3D Video Understanding , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Alexander M. Bronstein,et al.  Calculus of Nonrigid Surfaces for Geometry and Texture Manipulation , 2007, IEEE Transactions on Visualization and Computer Graphics.

[23]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Slobodan Ilic,et al.  Iterative mesh deformation for dense surface tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[25]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[26]  Valerio Pascucci,et al.  Robust on-line computation of Reeb graphs: simplicity and speed , 2007, ACM Trans. Graph..

[27]  Jean Ponce,et al.  Dense 3D motion capture from synchronized video streams , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Jérémie Allard,et al.  Grimage: markerless 3D interactions , 2007, SIGGRAPH '07.

[29]  Christian Rössl,et al.  Dense correspondence finding for parametrization-free animation reconstruction from video , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, SIGGRAPH 2008.

[31]  Tony Tung,et al.  The Augmented Multiresolution Reeb Graph Approach for Content-based Retrieval of 3d Shapes , 2005, Int. J. Shape Model..

[32]  Takashi Matsuyama,et al.  3D Video and Its Applications , 2012, Springer London.

[33]  Xiaojun Wu,et al.  Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video , 2004, Comput. Vis. Image Underst..

[34]  Bernard D. Adelstein,et al.  Demand Characteristics in Assessing Motion Sickness in a Virtual Environment: Or Does Taking a Motion Sickness Questionnaire Make You Sick? , 2007 .

[35]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[36]  Eitan Grinspun,et al.  Discrete laplace operators: no free lunch , 2007, Symposium on Geometry Processing.

[37]  B. D. Adelstein,et al.  Calculus of Nonrigid Surfaces for Geometry and Texture Manipulation , 2007 .

[38]  Hlawka The calculus of variations in the large , 1939 .

[39]  Adrian Hilton,et al.  Correspondence labelling for wide-timeframe free-form surface matching , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[40]  Takashi Matsuyama,et al.  Dynamic surface matching by geodesic mapping for 3D animation transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[42]  Huamin Wang,et al.  Modeling deformable objects from a single depth camera , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Adrian Hilton,et al.  Shape Similarity for 3D Video Sequences of People , 2010, International Journal of Computer Vision.

[44]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, ACM Trans. Graph..

[46]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[47]  Leonidas J. Guibas,et al.  Gromov‐Hausdorff Stable Signatures for Shapes using Persistence , 2009, Comput. Graph. Forum.

[48]  Adrian Hilton,et al.  Spherical matching for temporal correspondence of non-rigid surfaces , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[49]  M. Morse The Calculus of Variations in the Large , 1934 .

[50]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Olga Sorkine-Hornung,et al.  Differential Representations for Mesh Processing , 2006, Comput. Graph. Forum.

[52]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[53]  H. Seidel,et al.  Isometric registration of ambiguous and partial data , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Radu Horaud,et al.  Temporal Surface Tracking Using Mesh Evolution , 2008, ECCV.

[55]  Leo Grady,et al.  FOCUSR: Feature Oriented Correspondence Using Spectral Regularization--A Method for Precise Surface Matching , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Tony Tung,et al.  Comparison of Skeleton and Non-Skeleton Shape Descriptors for 3D Video , 2010 .