Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations

We consider the task of representation learning for unsupervised segmentation of 3D voxel-grid biomedical images. We show that models that capture implicit hierarchical relationships between subvolumes are better suited for this task. To that end, we consider encoder-decoder architectures with a hyperbolic latent space, to explicitly capture hierarchical relationships present in subvolumes of the data. We propose utilizing a 3D hyperbolic variational autoencoder with a novel gyroplane convolutional layer to map from the embedding space back to 3D images. To capture these relationships, we introduce an essential self-supervised loss -- in addition to the standard VAE loss -- which infers approximate hierarchies and encourages implicitly related subvolumes to be mapped closer in the embedding space. We present experiments on both synthetic data and biomedical data to validate our hypothesis.

[1]  Board , 2023, Médecine des Maladies Métaboliques.

[2]  M. Mahdi Roozbahani,et al.  ScaleNet: An Unsupervised Representation Learning Method for Limited Information , 2023, GCPR.

[3]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[4]  Dacheng Tao,et al.  Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task , 2019, BrainLes@MICCAI.

[5]  Lior Wolf,et al.  Unsupervised Microvascular Image Segmentation Using an Active Contours Mimicking Neural Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Jakub Nalepa,et al.  Unsupervised Segmentation of Hyperspectral Images Using 3-D Convolutional Autoencoders , 2019, IEEE Geoscience and Remote Sensing Letters.

[7]  Wei Xiong,et al.  VesselNet: A deep convolutional neural network with multi pathways for robust hepatic vessel segmentation , 2019, Comput. Medical Imaging Graph..

[8]  Paul J. Kennedy,et al.  Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges , 2019, Journal of Digital Imaging.

[9]  Marc Niethammer,et al.  DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation , 2019, MICCAI.

[10]  Adrian V. Dalca,et al.  Data Augmentation Using Learned Transformations for One-Shot Medical Image Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Shoichiro Yamaguchi,et al.  A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning , 2019, ICML.

[12]  Charline Le Lan,et al.  Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders , 2019, NeurIPS.

[13]  Ivan Ovinnikov,et al.  Poincar\'e Wasserstein Autoencoder , 2019, 1901.01427.

[14]  Tuan Leng Tay,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[15]  Heinz Handels,et al.  Unsupervised pathology detection in medical images using conditional variational autoencoders , 2018, International Journal of Computer Assisted Radiology and Surgery.

[16]  Lin Yang,et al.  A New Ensemble Learning Framework for 3D Biomedical Image Segmentation , 2018, AAAI.

[17]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[18]  João F. Henriques,et al.  Invariant Information Clustering for Unsupervised Image Classification and Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[20]  Mert R. Sabuncu,et al.  Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Thomas Hofmann,et al.  Hyperbolic Neural Networks , 2018, NeurIPS.

[22]  Nassir Navab,et al.  Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images , 2018, BrainLes@MICCAI.

[23]  Holger Roth,et al.  Unsupervised segmentation of 3D medical images based on clustering and deep representation learning , 2018, Medical Imaging.

[24]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[25]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[26]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[27]  Trevor Darrell,et al.  Learning Features by Watching Objects Move , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Lin Yang,et al.  Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation , 2016, NIPS.

[29]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[30]  Dhruv Batra,et al.  Joint Unsupervised Learning of Deep Representations and Image Clusters , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Paolo Favaro,et al.  Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles , 2016, ECCV.

[32]  Seyed-Ahmad Ahmadi,et al.  Hough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound , 2016, Comput. Vis. Image Underst..

[33]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[34]  Alexei A. Efros,et al.  Unsupervised Visual Representation Learning by Context Prediction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Rik Sarkar,et al.  Low Distortion Delaunay Embedding of Trees in Hyperbolic Plane , 2011, GD.

[37]  Abraham Albert Ungar,et al.  A Gyrovector Space Approach to Hyperbolic Geometry , 2009, A Gyrovector Space Approach to Hyperbolic Geometry.

[38]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[39]  A. Ungar.  Hyperbolic trigonometry and its application in the Poincaré ball model of hyperbolic geometry , 2001 .