WISH: efficient 3D biological shape classification through Willmore flow and Spherical Harmonics decomposition

Shape analysis of cell nuclei, enabled by the recent advances in nano-scale digital imaging and reconstruction methods, is emerging as a very important tool to understand low-level biological processes. Current analysis techniques, however, are performed on 2D slices or assume very simple 3D shape approximations , limiting their discrimination capabilities. In this work, we introduce a compact rotation-invariant frequency-based representation of genus-0 3D shapes represented by manifold triangle meshes, that we apply to cell nuclei envelopes reconstructed from electron micrographs. The representation is robustly obtained through Spherical Harmonics coefficients over a spherical parameterization of the input mesh obtained through Willmore flow. Our results show how our method significantly improves the state-of-the-art in the classification of nuclear envelopes of rodent brain samples. Moreover, while our method is motivated by the analysis of specific biological shapes, the framework is of general use for the compact frequency encoding of any genus-0 surface.

[1]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[2]  Giuseppe Patanè,et al.  STAR ‐ Laplacian Spectral Kernels and Distances for Geometry Processing and Shape Analysis , 2016, Comput. Graph. Forum.

[3]  Peter Schröder,et al.  Discrete Willmore flow , 2005, SIGGRAPH Courses.

[4]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Robin Green,et al.  Spherical Harmonic Lighting: The Gritty Details , 2003 .

[6]  Marcin Novotni,et al.  3D zernike descriptors for content based shape retrieval , 2003, SM '03.

[7]  Alla Sheffer,et al.  Fundamentals of spherical parameterization for 3D meshes , 2003, ACM Trans. Graph..

[8]  Mohammad H. Mahoor,et al.  A Robust Method for Rotation Estimation Using Spherical Harmonics Representation , 2013, IEEE Transactions on Image Processing.

[9]  Luc Van Gool,et al.  Hough Transform and 3D SURF for Robust Three Dimensional Classification , 2010, ECCV.

[10]  Yalin Wang,et al.  Conformal mapping of nuclei in 3D tomographic cell images to assess shape heterogeneity , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[11]  Mark Fricker,et al.  Interphase Nuclei of Many Mammalian Cell Types Contain Deep, Dynamic, Tubular Membrane-bound Invaginations of the Nuclear Envelope , 1997, The Journal of cell biology.

[12]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Gernot Riegler,et al.  OctNet: Learning Deep 3D Representations at High Resolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Henrik Wann Jensen,et al.  Importance Sampling Spherical Harmonics , 2009, Comput. Graph. Forum.

[15]  Gabriel Wittum,et al.  Filtering, reconstruction, and measurement of the geometry of nuclei from hippocampal neurons based on confocal microscopy data. , 2008, Journal of biomedical optics.

[16]  Lin Yang,et al.  Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review , 2016, IEEE Reviews in Biomedical Engineering.

[17]  Hugues Hoppe,et al.  Spherical parametrization and remeshing , 2003, ACM Trans. Graph..

[18]  K. Rohr,et al.  Shape normalization of 3D cell nuclei using elastic spherical mapping , 2008, Journal of microscopy.

[19]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[20]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  A. Ben Hamza,et al.  Deep learning with geodesic moments for 3D shape classification , 2018, Pattern Recognit. Lett..

[22]  Yalin Wang,et al.  Shape Analysis with Hyperbolic Wasserstein Distance , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[24]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[25]  Heikki Lehväslaiho,et al.  Three‐dimensional immersive virtual reality for studying cellular compartments in 3D models from EM preparations of neural tissues , 2015, The Journal of comparative neurology.

[26]  Hongdong Li,et al.  Conformal spherical representation of 3D genus-zero meshes , 2007, Pattern Recognit..

[27]  Bruno Lévy,et al.  Mesh parameterization: theory and practice , 2007, SIGGRAPH Courses.

[28]  Polina Golland,et al.  BrainPrint: A discriminative characterization of brain morphology , 2015, NeuroImage.

[29]  Tatsuya Harada,et al.  Neural 3D Mesh Renderer , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Leonidas J. Guibas,et al.  GRASS: Generative Recursive Autoencoders for Shape Structures , 2017, ACM Trans. Graph..

[31]  M. McPeek,et al.  Modeling Three-Dimensional Morphological Structures Using Spherical Harmonics , 2009, Evolution; international journal of organic evolution.

[32]  Bruno Lévy,et al.  Spectral Geometry Processing with Manifold Harmonics , 2008, Comput. Graph. Forum.

[33]  José M. Bioucas-Dias,et al.  Improving point cloud to surface reconstruction with generalized Tikhonov regularization , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[34]  Yang Liu,et al.  O-CNN , 2017, ACM Trans. Graph..

[35]  Markus Hadwiger,et al.  A Process for Digitizing and Simulating Biologically Realistic Oligocellular Networks Demonstrated for the Neuro-Glio-Vascular Ensemble , 2018, Front. Neurosci..

[36]  Nick Barnes,et al.  3-D Shape Matching and Non-Rigid Correspondence for Hippocampi Based on Markov Random Fields , 2018, IEEE Transactions on Image Processing.

[37]  Ioannis Pratikakis,et al.  Efficient 3D shape matching and retrieval using a concrete radialized spherical projection representation , 2007, Pattern Recognit..

[38]  Markus Hadwiger,et al.  Shape analysis of 3D nanoscale reconstructions of brain cell nuclear envelopes by implicit and explicit parametric representations , 2019, Comput. Graph. X.

[39]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[40]  Steve Marschner,et al.  Efficient multiple scattering in hair using spherical harmonics , 2008, ACM Trans. Graph..

[41]  Wei Zeng,et al.  Spherical Parameterization Balancing Angle and Area Distortions , 2017, IEEE Transactions on Visualization and Computer Graphics.

[42]  Gabriel Wittum,et al.  Synaptic Activity Induces Dramatic Changes in the Geometry of the Cell Nucleus: Interplay between Nuclear Structure, Histone H3 Phosphorylation, and Nuclear Calcium Signaling , 2009, The Journal of Neuroscience.

[43]  J. L. Nazareth,et al.  Cholesky-based Methods for Sparse Least Squares : The Benefits of Regularization ∗ , 1996 .

[44]  P. Magistretti,et al.  3D cellular reconstruction of cortical glia and parenchymal morphometric analysis from Serial Block-Face Electron Microscopy of juvenile rat , 2019, Progress in Neurobiology.

[45]  Jean-Philippe Thiran,et al.  Sampling theorems and compressive sensing on the sphere , 2011, Optical Engineering + Applications.

[46]  Keenan Crane,et al.  Robust fairing via conformal curvature flow , 2013, ACM Trans. Graph..

[47]  Markus Hadwiger,et al.  Hyperquadrics for Shape Analysis of 3D Nanoscale Reconstructions of Brain Cell Nuclear Envelopes , 2018, STAG.

[48]  Gerhard Dziuk,et al.  Computational parametric Willmore flow , 2008, Numerische Mathematik.

[49]  Bernard Ng,et al.  Shape Analysis for Brain Structures , 2014 .

[50]  Raphaëlle Chaine,et al.  Toward an efficient triangle-based spherical harmonics representation of 3D objects , 2008, Comput. Aided Geom. Des..

[51]  Guido Gerig,et al.  Parametrization of Closed Surfaces for 3-D Shape Description , 1995, Comput. Vis. Image Underst..

[52]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[53]  Douglas W. Jones,et al.  Shape analysis of brain ventricles using SPHARM , 2001, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA 2001).

[54]  Moo K. Chung,et al.  Large-Scale Modeling of Parametric Surfaces Using Spherical Harmonics , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[55]  Jean-Christophe Olivo-Marin,et al.  Characterization of cell shape and deformation in 3D using Spherical Harmonics , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[56]  Robert F Murphy,et al.  Deformation‐based nuclear morphometry: Capturing nuclear shape variation in HeLa cells , 2008, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[57]  Rolf Backofen,et al.  Jupyter and Galaxy: Easing entry barriers into complex data analyses for biomedical researchers , 2017, PLoS Comput. Biol..

[58]  Ivo D Dinov,et al.  3D Shape Modeling for Cell Nuclear Morphological Analysis and Classification , 2018, Scientific Reports.

[59]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[60]  Martin Rumpf,et al.  Two Step Time Discretization of Willmore Flow , 2009, IMA Conference on the Mathematics of Surfaces.

[61]  Haibin Ling,et al.  Shape Classification Using the Inner-Distance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[64]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[65]  Karthik Ramani,et al.  Deep Learning 3D Shape Surfaces Using Geometry Images , 2016, ECCV.

[66]  S. Marschner,et al.  Efficient multiple scattering in hair using spherical harmonics , 2008, SIGGRAPH 2008.

[67]  Kai Hormann,et al.  Surface Parameterization: a Tutorial and Survey , 2005, Advances in Multiresolution for Geometric Modelling.

[68]  Keenan Crane,et al.  Spin transformations of discrete surfaces , 2011, ACM Trans. Graph..

[69]  Ulrich Pinkall,et al.  Computing Discrete Minimal Surfaces and Their Conjugates , 1993, Exp. Math..

[70]  Jorge Jara,et al.  3D morpho-topological analysis of asymmetric neuronal morphogenesis in developing zebrafish , 2006, CompIMAGE.

[71]  Karthik Ramani,et al.  SurfNet: Generating 3D Shape Surfaces Using Deep Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[72]  Leonidas J. Guibas,et al.  SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).