InShaDe: Invariant Shape Descriptors for Visual Analysis of Histology 2D Cellular and Nuclear Shapes

We present a shape processing framework for visual exploration of cellular nuclear envelopes extracted from histology images. The framework is based on a novel shape descriptor of closed contours relying on a geodesically uniform resampling of discrete curves to allow for discrete differential-geometry-based computation of unsigned curvature at vertices and edges. Our descriptor is, by design, invariant under translation, rotation and parameterization. Moreover, it additionally offers the option for uniform-scale-invariance. The optional scale-invariance is achieved by scaling features to z-scores, while invariance under parameterization shifts is achieved by using elliptic Fourier analysis (EFA) on the resulting curvature vectors. These invariant shape descriptors provide an embedding into a fixed-dimensional feature space that can be utilized for various applications: (i) as input features for deep and shallow learning techniques; (ii) as input for dimension reduction schemes for providing a visual reference for clustering collection of shapes. The capabilities of the proposed framework are demonstrated in the context of visual analysis and unsupervised classification of histology images. CCS Concepts • Applied computing → Imaging; • Computing methodologies → Shape representations; Cluster analysis;

[1]  H. Irshad,et al.  Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review—Current Status and Future Potential , 2014, IEEE Reviews in Biomedical Engineering.

[2]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Keenan Crane,et al.  A Glimpse into Discrete Diff erential Geometry , 2017 .

[4]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[5]  Fillia Makedon,et al.  Spherical mapping for processing of 3D closed surfaces , 2006, Image Vis. Comput..

[6]  Kang-Hyun Jo,et al.  A survey of 2D shape representation: Methods, evaluations, and future research directions , 2018, Neurocomputing.

[7]  Guojun Lu,et al.  A comparative study of curvature scale space and Fourier descriptors for shape-based image retrieval , 2003, J. Vis. Commun. Image Represent..

[8]  Leonidas J. Guibas,et al.  SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[10]  Yang Liu,et al.  O-CNN , 2017, ACM Trans. Graph..

[11]  Markus Hadwiger,et al.  Interactive Volumetric Visual Analysis of Glycogen‐derived Energy Absorption in Nanometric Brain Structures , 2019, Comput. Graph. Forum.

[12]  J. Angel Arul Jothi,et al.  A survey on automated cancer diagnosis from histopathology images , 2017, Artificial Intelligence Review.

[13]  Saeid Belkasim,et al.  Invariant curvature-based Fourier shape descriptors , 2012, J. Vis. Commun. Image Represent..

[14]  Dong Xu,et al.  Geometric moment invariants , 2008, Pattern Recognit..

[15]  Masayuki Nakajima,et al.  Spherical Wavelet Descriptors for Content-based 3D Model Retrieval , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[16]  Dinesh Manocha,et al.  Collision queries using oriented bounding boxes , 2000 .

[17]  Sim Heng Ong,et al.  Image Analysis by Tchebichef Moments , 2001, IEEE Trans. Image Process..

[18]  Marco Agus,et al.  Forget About Electron Micrographs: A Novel Guide for Using for Quantitative Analysis of Dense Reconstructions , 2020 .

[19]  S. Abbas,et al.  Efficient Shape Classification using Zernike Moments and Geometrical Features on MPEG-7 Dataset , 2019 .

[20]  Raveendran Paramesran,et al.  An efficient method for the computation of Legendre moments , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Farzin Mokhtarian,et al.  Silhouette-Based Isolated Object Recognition through Curvature Scale Space , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Karthik Ramani,et al.  Deep Learning 3D Shape Surfaces Using Geometry Images , 2016, ECCV.

[24]  Thomas J. Fuchs,et al.  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images , 2019, Nature Medicine.

[25]  Minh-Triet Tran,et al.  Enhanced Rotation-Equivariant U-Net for Nuclear Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Sergio Escalera,et al.  Blurred Shape Model for binary and grey-level symbol recognition , 2009, Pattern Recognit. Lett..

[27]  Alexander Bobenko Geometry II Discrete Differential Geometry , 2010 .

[28]  Nasir M. Rajpoot,et al.  PanNuke: An Open Pan-Cancer Histology Dataset for Nuclei Instance Segmentation and Classification , 2019, ECDP.

[29]  Nasir Rajpoot,et al.  PanNuke Dataset Extension, Insights and Baselines , 2020, ArXiv.

[30]  B. H. Shekar,et al.  Shape Representation and Classification through Pattern Spectrum and Local Binary Pattern -- A Decision Level Fusion Approach , 2014, 2014 Fifth International Conference on Signal and Image Processing.

[31]  Ricardo J. G. B. Campello,et al.  Density-Based Clustering Based on Hierarchical Density Estimates , 2013, PAKDD.

[32]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction , 2018, ArXiv.

[33]  Pascal Fua,et al.  The effects of aging on neuropil structure in mouse somatosensory cortex—A 3D electron microscopy analysis of layer 1 , 2018, PloS one.

[34]  Kpalma Kidiyo,et al.  A Survey of Shape Feature Extraction Techniques , 2008 .

[35]  Corrado Calì,et al.  WISH: efficient 3D biological shape classification through Willmore flow and Spherical Harmonics decomposition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[36]  Lin Yang,et al.  Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review , 2016, IEEE Reviews in Biomedical Engineering.

[37]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[38]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[39]  G. Diaz,et al.  Elliptic fourier analysis of cell and nuclear shapes. , 1989, Computers and biomedical research, an international journal.

[40]  Markus Hadwiger,et al.  GLAM: Glycogen-derived Lactate Absorption Map for visual analysis of dense and sparse surface reconstructions of rodent brain structures on desktop systems and virtual environments , 2018, Comput. Graph..

[41]  Ellery Wulczyn,et al.  Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer , 2018, npj Digital Medicine.

[42]  Markus Hadwiger,et al.  Shape analysis of 3D nanoscale reconstructions of brain cell nuclear envelopes by implicit and explicit parametric representations , 2019, Comput. Graph. X.

[43]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[44]  Thierry Blu,et al.  Linear interpolation revitalized , 2004, IEEE Transactions on Image Processing.

[45]  Saeid Belkasim,et al.  A novel curvature-based shape Fourier Descriptor , 2008, 2008 15th IEEE International Conference on Image Processing.

[46]  Robert F Murphy,et al.  Deformation‐based nuclear morphometry: Capturing nuclear shape variation in HeLa cells , 2008, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[47]  Tamara Munzner,et al.  A Taxonomy of Visual Cluster Separation Factors , 2012, Comput. Graph. Forum.

[48]  Hao Chen,et al.  A Multi-Organ Nucleus Segmentation Challenge , 2020, IEEE Transactions on Medical Imaging.

[49]  Nima Tajbakhsh,et al.  Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation , 2019, Medical Image Anal..

[50]  Charles R. Giardina,et al.  Elliptic Fourier features of a closed contour , 1982, Comput. Graph. Image Process..

[51]  Max Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[52]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..