Dependent landmark drift: robust point set registration with a Gaussian mixture model and a statistical shape model

The goal of point set registration is to find point-by-point correspondences between point sets, each of which characterizes the shape of an object. Because local preservation of object geometry is assumed, prevalent algorithms in the area can often elegantly solve the problems without using geometric information specific to the objects. This means that registration performance can be further improved by using prior knowledge of object geometry. In this paper, we propose a novel point set registration method using the Gaussian mixture model with prior shape information encoded as a statistical shape model. Our transformation model is defined as a combination of the similar transformation, motion coherence, and the statistical shape model. Therefore, the proposed method works effectively if the target point set includes outliers and missing regions, or if it is rotated. The computational cost can be reduced to linear, and therefore the method is scalable to large point sets. The effectiveness of the method will be verified through comparisons with existing algorithms using datasets concerning human body shapes, hands, and faces.

[1]  Alan L. Yuille,et al.  Non-Rigid Point Set Registration by Preserving Global and Local Structures , 2016, IEEE Transactions on Image Processing.

[2]  Timothy F. Cootes,et al.  Statistical models of appearance for computer vision , 1999 .

[3]  Michael J. Black,et al.  Learning a model of facial shape and expression from 4D scans , 2017, ACM Trans. Graph..

[4]  Michael J. Black,et al.  Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape , 2012, ECCV.

[5]  Didier Stricker,et al.  Extended coherent point drift algorithm with correspondence priors and optimal subsampling , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[6]  Pierre Alliez,et al.  State of the Art in Surface Reconstruction from Point Clouds , 2014, Eurographics.

[7]  Alla Sheffer,et al.  Template-based mesh completion , 2005, SGP '05.

[8]  Michael J. Black,et al.  Dyna: a model of dynamic human shape in motion , 2015, ACM Trans. Graph..

[9]  Hans-Peter Seidel,et al.  A Statistical Model of Human Pose and Body Shape , 2009, Comput. Graph. Forum.

[10]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[11]  Andrew W. Fitzgibbon,et al.  Robust Registration of 2D and 3D Point Sets , 2003, BMVC.

[12]  Vladlen Koltun,et al.  Fast Global Registration , 2016, ECCV.

[13]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[14]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Dimitrios Tzionas,et al.  Embodied hands , 2017, ACM Trans. Graph..

[16]  Pavel Krsek,et al.  Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm , 2005, Image Vis. Comput..

[17]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[18]  Larry S. Davis,et al.  Improved fast gauss transform and efficient kernel density estimation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[19]  Kostas Daniilidis,et al.  Fully Automatic Registration of 3D Point Clouds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Miguel Á. Carreira-Perpiñán,et al.  Non-rigid point set registration: Coherent Point Drift , 2006, NIPS.

[21]  Ming Xu,et al.  Towards Personalized Statistical Deformable Model and Hybrid Point Matching for Robust MR-TRUS Registration , 2016, IEEE Transactions on Medical Imaging.

[22]  Hans-Peter Seidel,et al.  MovieReshape: tracking and reshaping of humans in videos , 2010, ACM Trans. Graph..

[23]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[24]  Heinz Handels,et al.  Probabilistic Appearance Models for Segmentation and Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[25]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[26]  Nicholas Ayache,et al.  Generation of a statistical shape model with probabilistic point correspondences and the expectation maximization- iterative closest point algorithm , 2007, International Journal of Computer Assisted Radiology and Surgery.

[27]  Alan L. Yuille,et al.  A mathematical analysis of the motion coherence theory , 1989, International Journal of Computer Vision.

[28]  Radu Horaud,et al.  Articulated shape matching using Laplacian eigenfunctions and unsupervised point registration , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Patricio A. Vela,et al.  A Stochastic Approach to Diffeomorphic Point Set Registration with Landmark Constraints , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jessica K. Hodgins,et al.  Data-driven modeling of skin and muscle deformation , 2008, ACM Trans. Graph..

[31]  Helmut Pottmann,et al.  Registration of point cloud data from a geometric optimization perspective , 2004, SGP '04.

[32]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[33]  Michael J. Black,et al.  FAUST: Dataset and Evaluation for 3D Mesh Registration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Adrian Hilton,et al.  A FACS valid 3D dynamic action unit database with applications to 3D dynamic morphable facial modeling , 2011, 2011 International Conference on Computer Vision.

[35]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[36]  Guoliang Fan,et al.  Non-rigid articulated point set registration with Local Structure Preservation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[37]  Andriy Myronenko,et al.  On the closed-form solution of the rotation matrix arising in computer vision problems , 2009, ArXiv.

[38]  H. Chui,et al.  A feature registration framework using mixture models , 2000, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis. MMBIA-2000 (Cat. No.PR00737).

[39]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[40]  Alejandro F. Frangi,et al.  Robust group-wise rigid registration of point sets using t-mixture model , 2016, SPIE Medical Imaging.

[41]  Sim Heng Ong,et al.  A robust global and local mixture distance based non-rigid point set registration , 2015, Pattern Recognit..

[42]  Baba C. Vemuri,et al.  Robust Point Set Registration Using Gaussian Mixture Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Larry S. Davis,et al.  Automatic online tuning for fast Gaussian summation , 2008, NIPS.

[44]  V. Raykar,et al.  Fast Computation of Sums of Gaussians in High Dimensions , 2005 .

[45]  João Manuel R S Tavares,et al.  Medical image registration: a review , 2014, Computer methods in biomechanics and biomedical engineering.

[46]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[47]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[48]  Guoliang Fan,et al.  Non-rigid Articulated Point Set Registration for Human Pose Estimation , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[49]  Yaron Lipman,et al.  Point registration via efficient convex relaxation , 2016, ACM Trans. Graph..

[50]  Radu Horaud,et al.  Rigid and Articulated Point Registration with Expectation Conditional Maximization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[53]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[54]  Chia-Ling Tsai,et al.  The dual-bootstrap iterative closest point algorithm with application to retinal image registration , 2003, IEEE Transactions on Medical Imaging.

[55]  Mark Pauly,et al.  Dynamic 2D/3D registration for the Kinect , 2013, SIGGRAPH '13.

[56]  Takeo Kanade,et al.  A Correlation-Based Approach to Robust Point Set Registration , 2004, ECCV.

[57]  Andrea Tagliasacchi,et al.  Modern techniques and applications for real-time non-rigid registration , 2016, SIGGRAPH ASIA Courses.

[58]  Xavier Pennec,et al.  Multi-scale EM-ICP: A Fast and Robust Approach for Surface Registration , 2002, ECCV.

[59]  Johan Thunberg,et al.  Shape‐aware surface reconstruction from sparse 3D point‐clouds , 2016, Medical Image Anal..

[60]  Li Ma,et al.  Non-rigid point set registration via coherent spatial mapping , 2015, Signal Process..

[61]  Ruigang Yang,et al.  Semantic Parametric Reshaping of Human Body Models , 2014, 2014 2nd International Conference on 3D Vision.

[62]  Anand Rangarajan,et al.  The Softassign Procrustes Matching Algorithm , 1997, IPMI.

[63]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[64]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[65]  Purang Abolmaesumi,et al.  Group-Wise Registration of Point Sets for Statistical Shape Models , 2012, IEEE Transactions on Medical Imaging.

[66]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH '05.

[67]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[68]  Hans-Peter Meinzer,et al.  Statistical shape models for 3D medical image segmentation: A review , 2009, Medical Image Anal..

[69]  Lin Gao,et al.  Efficient and Flexible Deformation Representation for Data-Driven Surface Modeling , 2016, ACM Trans. Graph..