Non‐Rigid Puzzles

Shape correspondence is a fundamental problem in computer graphics and vision, with applications in various problems including animation, texture mapping, robotic vision, medical imaging, archaeology and many more. In settings where the shapes are allowed to undergo non‐rigid deformations and only partial views are available, the problem becomes very challenging. To this end, we present a non‐rigid multi‐part shape matching algorithm. We assume to be given a reference shape and its multiple parts undergoing a non‐rigid deformation. Each of these query parts can be additionally contaminated by clutter, may overlap with other parts, and there might be missing parts or redundant ones. Our method simultaneously solves for the segmentation of the reference model, and for a dense correspondence to (subsets of) the parts. Experimental results on synthetic as well as real scans demonstrate the effectiveness of our method in dealing with this challenging matching scenario.

[1]  M. Bronstein,et al.  SHREC’16: Partial Matching of Deformable Shapes , 2016 .

[2]  Alexander M. Bronstein,et al.  Recent Trends, Applications, and Perspectives in 3D Shape Similarity Assessment , 2016, Comput. Graph. Forum.

[3]  Yücel Yemez,et al.  Partial 3‐D Correspondence from Shape Extremities , 2014, Comput. Graph. Forum.

[4]  Tobias Gurdan.  Sparse Modeling of Intrinsic Correspondences , 2014 .

[5]  Daniel Cohen-Or,et al.  4-points congruent sets for robust pairwise surface registration , 2008, ACM Trans. Graph..

[6]  Daniel Cremers,et al.  Robust Region Detection via Consensus Segmentation of Deformable Shapes , 2014, Comput. Graph. Forum.

[7]  Ron Kimmel,et al.  Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[9]  Alexander M. Bronstein,et al.  Not only size matters: Regularized partial matching of nonrigid shapes , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[10]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[11]  Ghassan Hamarneh,et al.  Bilateral Maps for Partial Matching , 2013, Comput. Graph. Forum.

[12]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[13]  Michael J. Black,et al.  FAUST: Dataset and Evaluation for 3D Mesh Registration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  J. Keyser,et al.  Multiple Shape Correspondence by Dynamic Programming , 2014 .

[15]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[17]  Alexander M. Bronstein,et al.  A game-theoretic approach to deformable shape matching , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Leonidas J. Guibas,et al.  Consistent Shape Maps via Semidefinite Programming , 2013, SGP '13.

[19]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[20]  Alexander M. Bronstein,et al.  Regularized Partial Matching of Rigid Shapes , 2008, ECCV.

[21]  Alfred M. Bruckstein,et al.  Partial Similarity of Objects, or How to Compare a Centaur to a Horse , 2009, International Journal of Computer Vision.

[22]  Maks Ovsjanikov,et al.  Functional maps , 2012, ACM Trans. Graph..

[23]  Xavier Bresson,et al.  Functional correspondence by matrix completion , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Daniel Cremers,et al.  Consistent Partial Matching of Shape Collections via Sparse Modeling , 2017, Comput. Graph. Forum.

[25]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[26]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[27]  Helmut Pottmann,et al.  Reassembling fractured objects by geometric matching , 2006, ACM Trans. Graph..

[28]  Leonidas J. Guibas,et al.  Functional map networks for analyzing and exploring large shape collections , 2014, ACM Trans. Graph..

[29]  Daniel Cremers,et al.  Large‐Scale Integer Linear Programming for Orientation Preserving 3D Shape Matching , 2011, Comput. Graph. Forum.

[30]  Yasuo Kuniyoshi,et al.  Elastic Net Constraints for Shape Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Daniel Cremers,et al.  Partial Matching of Deformable Shapes , 2016, 3DOR@Eurographics.

[32]  Qi-Xing Huang,et al.  Dense Human Body Correspondences Using Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Hans-Peter Seidel,et al.  A low-dimensional representation for robust partial isometric correspondences computation , 2013, Graph. Model..

[34]  Tony F. Chan,et al.  A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model , 2002, International Journal of Computer Vision.

[35]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[36]  Daniel Cremers,et al.  Partial Functional Correspondence , 2017 .

[37]  Alexander M. Bronstein,et al.  Coupled quasi‐harmonic bases , 2012, Comput. Graph. Forum.

[38]  H. Weyl.  Ueber die asymptotische Verteilung der Eigenwerte , 1911 .

[39]  Andrea Torsello,et al.  Multiview registration via graph diffusion of dual quaternions , 2011, CVPR 2011.

[40]  Alexander M. Bronstein,et al.  Putting the Pieces Together: Regularized Multi-part Shape Matching , 2012, ECCV Workshops.

[41]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[42]  Andrew W. Fitzgibbon,et al.  3D scanning deformable objects with a single RGBD sensor , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[45]  Andrea Torsello,et al.  Fast and accurate surface alignment through an isometry-enforcing game , 2015, Pattern Recognit..