3D Shape Registration

Registration is the problem of bringing together two or more 3D shapes, either of the same object or of two different but similar objects. This chapter first introduces the classical Iterative Closest Point (ICP) algorithm, which represents the gold standard registration method. Current limitations of ICP are addressed and the most popular variants are described to improve the basic implementation in several ways. Challenging registration scenarios are analyzed and a taxonomy of recent and promising alternative registration techniques is introduced. Three case studies are then described with an increasing level of problem difficulty. The first case study describes a simple but effective technique to detect outliers. The second case study uses the Levenberg-Marquardt optimization procedure to solve standard pairwise registration. The third case study focuses on the challenging problem of deformable object registration. Finally, open issues and directions for future work are discussed and conclusions are drawn.

[1]  Beno Benhabib,et al.  On-line Modeling for Real-Time, Model-Based, 3D Pose Tracking , 2007 .

[2]  Martial Hebert,et al.  Fully automatic registration of multiple 3D data sets , 2003, Image Vis. Comput..

[3]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[4]  Robert B. Fisher,et al.  Estimating 3-D rigid body transformations: a comparison of four major algorithms , 1997, Machine Vision and Applications.

[5]  Nick Pears,et al.  From 3D Point Clouds to Pose-Normalised Depth Maps , 2010, International Journal of Computer Vision.

[6]  Daniel Pizarro-Perez,et al.  Realtime Wide-Baseline Registration of the Uterus in Laparoscopic Videos Using Multiple Texture Maps , 2013, AE-CAI.

[7]  Remco C. Veltkamp,et al.  A survey of content based 3D shape retrieval methods , 2004, Proceedings Shape Modeling Applications, 2004..

[8]  Micha Sharir,et al.  On the performance of the ICP algorithm , 2008, Comput. Geom..

[9]  Robert Bergevin,et al.  A method for the registration of attributed range images , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[10]  Mohammed Bennamoun,et al.  Automatic Correspondence for 3d Modeling: an Extensive Review , 2005, Int. J. Shape Model..

[11]  Akihiro Sugimoto,et al.  Globally convergent range image registration by graph kernel algorithm , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[12]  Max A. Viergever,et al.  A survey of medical image registration , 1998, Medical Image Anal..

[13]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[14]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  James S. Duncan,et al.  A robust point-matching algorithm for autoradiograph alignment , 1997, Medical Image Anal..

[16]  Andrea Fusiello,et al.  Accurate and Automatic Alignment of Range Surfaces , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[17]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[18]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Federico Tombari,et al.  Performance Evaluation of 3D Keypoint Detectors , 2012, International Journal of Computer Vision.

[20]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[21]  David Fofi,et al.  A review of recent range image registration methods with accuracy evaluation , 2007, Image Vis. Comput..

[22]  Kari Pulli,et al.  Projective surface matching of colored 3D scans , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[23]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[24]  Leonidas J. Guibas,et al.  Dynamic geometry registration , 2007, Symposium on Geometry Processing.

[25]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[26]  Pascal Fua,et al.  Physically Valid Shape Parameterization for Monocular 3-D Deformable Surface Tracking , 2005, BMVC.

[27]  James S. Duncan,et al.  Rigid point feature registration using mutual information , 1999, Medical Image Anal..

[28]  Kim L. Boyer,et al.  Precision range image registration using a robust surface interpenetration measure and enhanced genetic algorithms , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Rodney A. Kennedy,et al.  A Survey of Medical Image Registration on Multicore and the GPU , 2010, IEEE Signal Processing Magazine.

[30]  Andrew W. Fitzgibbon,et al.  Single View Reconstruction of Curved Surfaces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[31]  Roberto Toldo,et al.  Global registration of multiple point clouds embedding the Generalized Procrustes Analysis into an ICP framework , 2010 .

[32]  Maks Ovsjanikov,et al.  Functional maps , 2012, ACM Trans. Graph..

[33]  Neill W Campbell,et al.  IEEE International Conference on Computer Vision and Pattern Recognition , 2008 .

[34]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[35]  Leonidas J. Guibas,et al.  Non-Rigid Registration Under Isometric Deformations , 2008 .

[36]  Vladimir G. Kim,et al.  Shape-based recognition of 3D point clouds in urban environments , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[37]  Christian Sandor,et al.  Improving Spatial Perception for Augmented Reality X-Ray Vision , 2009, 2009 IEEE Virtual Reality Conference.

[38]  Andrew W. Fitzgibbon.  Robust registration of 2D and 3D point sets , 2003, Image Vis. Comput..

[39]  Andrea Fusiello,et al.  Reconstruction of complex environments by robust pre-aligned ICP , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[40]  Bruno Lévy,et al.  Laplace-Beltrami Eigenfunctions Towards an Algorithm That "Understands" Geometry , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[41]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[42]  Takeo Kanade,et al.  Fast and accurate shape-based registration , 1996 .

[43]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[44]  Szymon Rusinkiewicz,et al.  Global non-rigid alignment of 3-D scans , 2007, SIGGRAPH 2007.

[45]  Sebastian Weik,et al.  Registration of 3-D partial surface models using luminance and depth information , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[46]  Sebastian Thrun,et al.  The Correlated Correspondence Algorithm for Unsupervised Registration of Nonrigid Surfaces , 2004, NIPS.

[47]  Babak Taati,et al.  Automatic Registration for Model Building using Variable Dimensional Local Shape Descriptors , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[48]  Mohammed Bennamoun,et al.  An Accurate and Robust Range Image Registration Algorithm for 3D Object Modeling , 2014, IEEE Transactions on Multimedia.

[49]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[50]  Sing Bing Kang,et al.  Registration and integration of textured 3-D data , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[51]  Sang Wook Lee,et al.  ICP Registration Using Invariant Features , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  N. Mitra,et al.  4-points congruent sets for robust pairwise surface registration , 2008, SIGGRAPH 2008.

[53]  Soon-Yong Park,et al.  An accurate and fast point-to-plane registration technique , 2003, Pattern Recognit. Lett..

[54]  Bernhard Schölkopf,et al.  Learning Dense 3D Correspondence , 2006, NIPS.

[55]  Andrea Fusiello,et al.  High resolution video mosaicing with global alignment , 2004, CVPR 2004.

[56]  Kostas Daniilidis,et al.  Fully Automatic Registration of 3D Point Clouds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[57]  Steven J. Gortler,et al.  Geometry images , 2002, SIGGRAPH.

[58]  Umberto Castellani,et al.  Sparse points matching by combining 3D mesh saliency with statistical descriptors , 2008, Comput. Graph. Forum.

[59]  Andrea Tagliasacchi,et al.  Sparse Iterative Closest Point , 2013, Comput. Graph. Forum.

[60]  Patrick J. Flynn,et al.  A Survey Of Free-Form Object Representation and Recognition Techniques , 2001, Comput. Vis. Image Underst..

[61]  Guido M. Cortelazzo,et al.  3D registration by textured spin-images , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[62]  Kiran Jude Fernandes,et al.  Reverse Engineering: An Industrial Perspective , 2007 .

[63]  Baba C. Vemuri,et al.  A robust algorithm for point set registration using mixture of Gaussians , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[64]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[65]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[66]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[67]  Björn Stenger,et al.  Demisting the Hough Transform for 3D Shape Recognition and Registration , 2014, International Journal of Computer Vision.

[68]  Leonidas J. Guibas,et al.  Robust global registration , 2005, SGP '05.

[69]  Yonghuai Liu,et al.  Automatic 3d free form shape matching using the graduated assignment algorithm , 2005, Pattern Recognit..

[70]  Shi-Min Hu,et al.  Geometry and Convergence Analysis of Algorithms for Registration of 3D Shapes , 2006, International Journal of Computer Vision.

[71]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[72]  Vladlen Koltun,et al.  Dense scene reconstruction with points of interest , 2013, ACM Trans. Graph..

[73]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[74]  Alan Watt,et al.  3D Computer Graphics , 1993 .

[75]  Benjamin B. Kimia,et al.  3D shape registration using regularized medial scaffolds , 2004 .

[76]  Luc Van Gool,et al.  Automatic Crude Patch Registration: Toward Automatic 3D Model Building , 2002, Comput. Vis. Image Underst..

[77]  Radu Horaud,et al.  Hand Motion from 3D Point Trajectories and a Smooth Surface Model , 2004, ECCV.

[78]  Remco C. Veltkamp,et al.  A Survey of 3D Face Recognition Methods , 2005, AVBPA.

[79]  Adrien Bartoli,et al.  Semantic Shape Context for the Registration of Multiple Partial 3D Views , 2009, BMVC.

[80]  Szymon Rusinkiewicz,et al.  Non-rigid range-scan alignment using thin-plate splines , 2004 .

[81]  Kari Pulli,et al.  Multiview registration for large data sets , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[82]  Matthias Zwicker,et al.  Automatic Registration for Articulated Shapes , 2008, Comput. Graph. Forum.

[83]  Harpreet S. Sawhney,et al.  Robust Video Mosaicing through Topology Inference and Local to Global Alignment , 1998, ECCV.

[84]  Andrew J. Stoddart,et al.  N-View Point Set Registration: A Comparison , 1999, BMVC.

[85]  Sergio Pissanetzky,et al.  Sparse Matrix Technology , 1984 .

[86]  Holly E. Rushmeier,et al.  The 3D Model Acquisition Pipeline , 2002, Comput. Graph. Forum.

[87]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[88]  Andrea Fusiello,et al.  Registration of Multiple Acoustic Range Views for Underwater Scene Reconstruction , 2002, Comput. Vis. Image Underst..

[89]  Martin D. Levine,et al.  Registering Multiview Range Data to Create 3D Computer Objects , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[91]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[92]  Joachim Hertzberg,et al.  Cached k-d tree search for ICP algorithms , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[93]  Ryutarou Ohbuchi,et al.  Merging virtual objects with the real world: seeing ultrasound imagery within the patient , 1992, SIGGRAPH.

[94]  Mohammed Bennamoun,et al.  Three-Dimensional Model-Based Object Recognition and Segmentation in Cluttered Scenes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[95]  Thomas A. Funkhouser,et al.  Shape-based retrieval and analysis of 3d models , 2005, CACM.

[96]  Umberto Castellani,et al.  Statistical 3D Shape Analysis by Local Generative Descriptors , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[97]  Anand Rangarajan,et al.  Groupwise point pattern registration using a novel CDF-based Jensen-Shannon Divergence , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[98]  J. M. M. Montiel,et al.  Visual SLAM for Handheld Monocular Endoscope , 2014, IEEE Transactions on Medical Imaging.

[99]  Nassir Navab,et al.  Model globally, match locally: Efficient and robust 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[100]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[101]  Ran Liu,et al.  Outlier Robust ICP for Minimizing Fractional RMSD , 2006, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[102]  Ko Nishino,et al.  Scale-hierarchical 3D object recognition in cluttered scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[103]  Matei T. Ciocarlie,et al.  Data-driven grasping with partial sensor data , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[104]  Martial Hebert,et al.  Directional Associative Markov Network for 3-D Point Cloud Classification , 2008 .

[105]  Kostas Daniilidis,et al.  Object Detection from Large-Scale 3D Datasets Using Bottom-Up and Top-Down Descriptors , 2008, ECCV.

[106]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[107]  Pavel Krsek,et al.  Differential Invariants as the Base of Triangulated Surface Registration , 2002, Comput. Vis. Image Underst..

[108]  U. Castellani,et al.  A complete system for on-line 3D modelling from acoustic images , 2005, Signal Process. Image Commun..

[109]  Takeo Kanade,et al.  A Correlation-Based Approach to Robust Point Set Registration , 2004, ECCV.

[110]  Anikó Ekárt,et al.  Pre-registration of arbitrarily oriented 3D surfaces using a genetic algorithm , 2006, Pattern Recognit. Lett..

[111]  Josef Kittler,et al.  Audio- and Video-Based Biometric Person Authentication, 5th International Conference, AVBPA 2005, Hilton Rye Town, NY, USA, July 20-22, 2005, Proceedings , 2005, AVBPA.

[112]  Hanspeter Pfister,et al.  Fast and automatic object pose estimation for range images on the GPU , 2009, Machine Vision and Applications.

[113]  Miguel Á. Carreira-Perpiñán,et al.  Non-rigid point set registration: Coherent Point Drift , 2006, NIPS.

[114]  S. Pissanetzky.  Sparse Matrix Algebra , 1984 .

[115]  Radim Sára,et al.  Finding the Largest Unambiguous Component of Stereo Matching , 2002, ECCV.

[116]  Radu Horaud,et al.  Rigid and Articulated Point Registration with Expectation Conditional Maximization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[117]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[118]  Umberto Castellani,et al.  Robust deformation capture from temporal range data for surface rendering , 2008 .

[119]  Bruno Baldaro.  Appunti delle lezioni , 2012 .

[120]  Matthias Zwicker,et al.  Range Scan Registration Using Reduced Deformable Models , 2009, Comput. Graph. Forum.

[121]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[122]  Andrea Torsello,et al.  A game-theoretic approach to fine surface registration without initial motion estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[123]  Ko Nishino,et al.  Scale-Dependent/Invariant Local 3D Shape Descriptors for Fully Automatic Registration of Multiple Sets of Range Images , 2008, ECCV.

[124]  Xavier Pennec,et al.  Multi-scale EM-ICP: A Fast and Robust Approach for Surface Registration , 2002, ECCV.

[125]  Alexander M. Bronstein,et al.  Three-Dimensional Face Recognition , 2005, International Journal of Computer Vision.

[126]  Szymon Rusinkiewicz,et al.  Non-rigid global alignment using thin-plate splines , 2005, SIGGRAPH '05.