MeshMonk: Open-source large-scale intensive 3D phenotyping

Dense surface registration, commonly used in computer science, could aid the biological sciences in accurate and comprehensive quantification of biological phenotypes. However, few toolboxes exist that are openly available, non-expert friendly, and validated in a way relevant to biologists. Here, we report a customizable toolbox for reproducible high-throughput dense phenotyping of 3D images, specifically geared towards biological use. Given a target image, a template is first oriented, repositioned, and scaled to the target during a scaled rigid registration step, then transformed further to fit the specific shape of the target using a non-rigid transformation. As validation, we use n = 41 3D facial images to demonstrate that the MeshMonk registration is accurate, with 1.26 mm average error, across 19 landmarks, between placements from manual observers and using the MeshMonk toolbox. We also report no variation in landmark position or centroid size significantly attributable to landmarking method used. Though validated using 19 landmarks, the MeshMonk toolbox produces a dense mesh of vertices across the entire surface, thus facilitating more comprehensive investigations of 3D shape variation. This expansion opens up exciting avenues of study in assessing biological shapes to better understand their phenotypic variation, genetic and developmental underpinnings, and evolutionary history.

[1]  Tudor Groza,et al.  Phenotyping: Targeting genotype's rich cousin for diagnosis , 2014, Journal of paediatrics and child health.

[2]  B. Dorizzi,et al.  Precise Localization of Landmarks on 3D Faces using Gabor Wavelets , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[3]  A. Karmiloff-Smith,et al.  Discriminating power of localized three-dimensional facial morphology. , 2005, American journal of human genetics.

[4]  E. Hille,et al.  Analytic Function Theory. Volume II , 1973 .

[5]  F. Rohlf,et al.  A revolution morphometrics. , 1993, Trends in ecology & evolution.

[6]  Wiro Niessen,et al.  Ensemble landmarking of 3D facial surface scans , 2017, Scientific Reports.

[7]  R. Holbrooke,et al.  The Next Challenge , 1986 .

[8]  Kristina Aldridge,et al.  Precision and error of three‐dimensional phenotypic measures acquired from 3dMD photogrammetric images , 2005, American journal of medical genetics. Part A.

[9]  P. Claes,et al.  Estimating age and synthesising growth in children and adolescents using 3D facial prototypes. , 2018, Forensic science international.

[10]  Yan-Long Jia,et al.  Human rhinovirus internal ribosome entry site element enhances transgene expression in transfected CHO-S cells , 2018, Scientific Reports.

[11]  S Richmond,et al.  Reproducibility of facial soft tissue landmarks on 3D laser-scanned facial images. , 2009, Orthodontics & craniofacial research.

[12]  H. Cohen,et al.  Prevalence of monoclonal gammopathy of undetermined significance. , 2006, The New England journal of medicine.

[13]  Mads Nielsen,et al.  Non-rigid registration by geometry-constrained diffusion , 1999, Medical Image Anal..

[14]  L. Akarun,et al.  3D Facial Landmarking under Expression, Pose, and Occlusion Variations , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[15]  Anne E Carpenter,et al.  Visualization of image data from cells to organisms , 2010, Nature Methods.

[16]  M. Kendall.  Statistical Methods for Research Workers , 1937, Nature.

[17]  Ioannis A. Kakadiaris,et al.  3D Facial Landmark Detection & Face Registration A 3D Facial Landmark Model & 3D Local Shape Descriptors Approach , 2010 .

[18]  P. F. Vasconcelos,et al.  In situ immune response and mechanisms of cell damage in central nervous system of fatal cases microcephaly by Zika virus , 2018, Scientific Reports.

[19]  Douglas G. Altman,et al.  Measurement in Medicine: The Analysis of Method Comparison Studies , 1983 .

[20]  Mao Li,et al.  Rapid automated landmarking for morphometric analysis of three‐dimensional facial scans , 2017, Journal of anatomy.

[21]  Marcelo Romero,et al.  Automatic 3‐dimensional cephalometric landmarking based on active shape models in related projections , 2018, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[22]  Anil K. Jain,et al.  Detection of Anchor Points for 3D Face Veri.cation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[23]  Erik Otárola-Castillo,et al.  geomorph: an r package for the collection and analysis of geometric morphometric shape data , 2013 .

[24]  Stefanos Zafeiriou,et al.  A 3D Morphable Model Learnt from 10,000 Faces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  George Weiss,et al.  Analytic Function Theory, Volume 2 , 1962 .

[26]  Peter Hammond,et al.  Automated Registration of 3D Faces using Dense Surface Models , 2003, BMVC.

[27]  D. Adams,et al.  A method for analysis of phenotypic change for phenotypes described by high-dimensional data , 2014, Heredity.

[28]  Thomas A. Funkhouser,et al.  Algorithms to automatically quantify the geometric similarity of anatomical surfaces , 2011, Proceedings of the National Academy of Sciences.

[29]  Maurício Pamplona Segundo,et al.  Automatic Face Segmentation and Facial Landmark Detection in Range Images , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Timothy F. Cootes,et al.  A minimum description length approach to statistical shape modeling , 2002, IEEE Transactions on Medical Imaging.

[31]  S. Omholt,et al.  Phenomics: the next challenge , 2010, Nature Reviews Genetics.

[32]  Andrea Cavallaro,et al.  3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model , 2009, IEEE Transactions on Multimedia.

[33]  Paul Suetens,et al.  Development and comparison of non-rigid surface registration algorithms and extensions , 2014 .

[34]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  A. Ponniah,et al.  Large Scale 3D Morphable Models , 2017, International Journal of Computer Vision.

[36]  Mary L Marazita,et al.  Digital Three-Dimensional Photogrammetry: Evaluation of Anthropometric Precision and Accuracy Using a Genex 3D Camera System , 2004, The Cleft palate-craniofacial journal : official publication of the American Cleft Palate-Craniofacial Association.

[37]  P. Claes,et al.  Augmentation of linear facial anthropometrics through modern morphometrics: a facial convexity example. , 2011, Australian dental journal.

[38]  Syed Zulqarnain Gilani,et al.  Dense 3D Face Correspondence , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  F. Rohlf,et al.  Extensions of the Procrustes Method for the Optimal Superimposition of Landmarks , 1990 .

[40]  Anil K. Jain,et al.  Automatic feature extraction for multiview 3D face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[41]  G. Verbeke,et al.  Sexual dimorphism in multiple aspects of 3D facial symmetry and asymmetry defined by spatially dense geometric morphometrics , 2012, Journal of anatomy.

[42]  Stefan Zachow,et al.  Fully Automated and Highly Accurate Dense Correspondence for Facial Surfaces , 2016, ECCV Workshops.

[43]  Stefanos Zafeiriou,et al.  Large Scale 3D Morphable Models , 2017, International Journal of Computer Vision.

[44]  Carrie L Heike,et al.  3D digital stereophotogrammetry: a practical guide to facial image acquisition , 2010, Head & face medicine.

[45]  Stefan Milz,et al.  Editorial – Journal of Anatomy January 2017 , 2017, Journal of anatomy.

[46]  Jens Fagertun,et al.  3D facial landmarks: Inter-operator variability of manual annotation , 2014, BMC Medical Imaging.

[47]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Gary F. Rogers,et al.  Validity and Reliability of Craniofacial Anthropometric Measurement of 3D Digital Photogrammetric Images , 2008, The Cleft palate-craniofacial journal : official publication of the American Cleft Palate-Craniofacial Association.

[49]  W. Hoeffding,et al.  Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling. , 1962 .

[50]  Marta Mirazón Lahr,et al.  The problem of assessing landmark error in geometric morphometrics: theory, methods, and modifications. , 2007, American journal of physical anthropology.

[51]  Peter Claes,et al.  Toward DNA-based facial composites: preliminary results and validation. , 2014, Forensic science international. Genetics.

[52]  David Dunaway,et al.  An Automatic 3D Facial Landmarking Algorithm Using 2D Gabor Wavelets , 2016, IEEE Transactions on Image Processing.

[53]  Tsz-Ho Yu,et al.  A Novel Genetic Algorithm for 3D Facial Landmark Localization , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[54]  Manish Agarwal,et al.  Computerized Medical Imaging and Graphics Automated Identification of Anatomical Landmarks on 3d Bone Models Reconstructed from Ct Scan Images , 2022 .

[55]  Kun Tang,et al.  Automatic landmark annotation and dense correspondence registration for 3D human facial images , 2012, BMC Bioinformatics.

[56]  Rajiv Balachandran,et al.  A knowledge-based algorithm for automatic detection of cephalometric landmarks on CBCT images , 2015, International Journal of Computer Assisted Radiology and Surgery.

[57]  F. Rohlf,et al.  Geometric morphometrics: Ten years of progress following the ‘revolution’ , 2004 .

[58]  Liming Chen,et al.  A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[59]  Marcel J. T. Reinders,et al.  EXTRACTION OF SEMANTIC 3D MODELS OF HUMAN FACES FROM STEREOSCOPIC IMAGE SEQUENCES , 2007 .

[60]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[61]  Shu Liang,et al.  Improved detection of landmarks on 3D human face data , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[62]  P. Claes,et al.  Improved facial outcome assessment using a 3D anthropometric mask. , 2012, International journal of oral and maxillofacial surgery.

[63]  Makiko Kouchi,et al.  Three-dimensional analyses of aging-induced alterations in facial shape: a longitudinal study of 171 Japanese males , 2015, International Journal of Legal Medicine.