Hyperbolic Deep Learning in Computer Vision: A Survey

Deep representation learning is a ubiquitous part of modern computer vision. While Euclidean space has been the de facto standard manifold for learning visual representations, hyperbolic space has recently gained rapid traction for learning in computer vision. Specifically, hyperbolic learning has shown a strong potential to embed hierarchical structures, learn from limited samples, quantify uncertainty, add robustness, limit error severity, and more. In this paper, we provide a categorization and in-depth overview of current literature on hyperbolic learning for computer vision. We research both supervised and unsupervised literature and identify three main research themes in each direction. We outline how hyperbolic learning is performed in all themes and discuss the main research problems that benefit from current advances in hyperbolic learning for computer vision. Moreover, we provide a high-level intuition behind hyperbolic geometry and outline open research questions to further advance research in this direction.

[1]  Justin Johnson,et al.  Hyperbolic Image-Text Representations , 2023, ArXiv.

[2]  Tejas Anvekar,et al.  GPr-Net: Geometric Prototypical Network for Point Cloud Few-Shot Learning , 2023, 2304.06007.

[3]  Yunde Jia,et al.  Exploring Data Geometry for Continual Learning , 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Wee Peng Tay,et al.  HypLiLoc: Towards Effective LiDAR Pose Regression with Hyperbolic Fusion , 2023, ArXiv.

[5]  Cewu Lu,et al.  From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding , 2023, 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Niels Landwehr,et al.  Hyperbolic Geometry in Computer Vision: A Novel Framework for Convolutional Neural Networks , 2023, ArXiv.

[7]  P. Mettes,et al.  Poincar\'e ResNet , 2023, 2303.14027.

[8]  Fabio Galasso,et al.  HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representations , 2023, ICLR.

[9]  Kazunori D. Yamada,et al.  Hyperbolic Contrastive Learning , 2023, ArXiv.

[10]  Yu-Chien Kong,et al.  Ancestor Search: Generalized Open Set Recognition via Hyperbolic Side Information Learning , 2023, 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[11]  Teng Long,et al.  Hierarchical Explanations for Video Action Recognition , 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Cheng Deng,et al.  Adaptive Hierarchical Similarity Metric Learning With Noisy Labels , 2021, IEEE Transactions on Image Processing.

[13]  Albert K Lee,et al.  Hippocampal spatial representations exhibit a hyperbolic geometry that expands with experience , 2022, Nature Neuroscience.

[14]  Suha Kwak,et al.  HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization , 2022, ArXiv.

[15]  Shlok Kumar Mishra,et al.  Hyperbolic Contrastive Learning for Visual Representations beyond Objects , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Yi Zhang,et al.  The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation , 2022, ArXiv.

[17]  Fan Yang,et al.  Hyperbolic Cosine Transformer for LiDAR 3D Object Detection , 2022, ArXiv.

[18]  Jimson Mathew,et al.  Deep Semantic Hashing with Structure-Semantic Disagreement Correction via Hyperbolic Metric Learning , 2022, 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP).

[19]  E. Magli,et al.  Rethinking the compositionality of point clouds through regularization in the hyperbolic space , 2022, NeurIPS.

[20]  Toan N. Nguyen,et al.  Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddings , 2022, MICCAI.

[21]  M. Harandi,et al.  Curved Geometric Networks for Visual Anomaly Recognition , 2022, IEEE transactions on neural networks and learning systems.

[22]  Yawen Cui,et al.  Rethinking Few-Shot Class-Incremental Learning with Open-Set Hypothesis in Hyperbolic Geometry , 2022, ArXiv.

[23]  Hao Jiang,et al.  Hyperbolic Knowledge Transfer with Class Hierarchy for Few-Shot Learning , 2022, IJCAI.

[24]  M. Harandi,et al.  Adaptive Poincaré Point to Set Distance for Few-Shot Classification , 2022, AAAI.

[25]  Elise van der Pol,et al.  Maximum Class Separation as Inductive Bias in One Matrix , 2022, NeurIPS.

[26]  Nurendra Choudhary,et al.  Towards Scalable Hyperbolic Neural Networks using Taylor Series Approximations , 2022, ArXiv.

[27]  Juyong Lee,et al.  A Rotated Hyperbolic Wrapped Normal Distribution for Hierarchical Representation Learning , 2022, NeurIPS.

[28]  Biljana Mileva-Boshkoska,et al.  Face recognition with a hyperbolic metric classification model , 2022, 2022 45th Jubilee International Convention on Information, Communication and Electronic Technology (MIPRO).

[29]  Zenglin Xu,et al.  Contrastive Multi-view Hyperbolic Hierarchical Clustering , 2022, IJCAI.

[30]  Irwin King,et al.  HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization , 2022, WWW.

[31]  N. Sebe,et al.  Hyperbolic Vision Transformers: Combining Improvements in Metric Learning , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  J. Röning,et al.  Hyperbolic Uncertainty Aware Semantic Segmentation , 2022, ArXiv.

[33]  F. Lécué,et al.  FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition , 2022, AAAI.

[34]  Mina Ghadimi Atigh,et al.  Hyperbolic Image Segmentation , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  J. Han,et al.  Meta hyperbolic networks for zero-shot learning , 2022, Neurocomputing.

[36]  Dongmian Zou,et al.  Autoencoding Hyperbolic Representation for Adversarial Generation , 2022, 2201.12825.

[37]  Stella X. Yu,et al.  Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Guoying Zhao,et al.  Hyperbolic Deep Neural Networks: A Survey , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Yanfang Ye,et al.  Hyperbolic Graph Attention Network , 2019, IEEE Transactions on Big Data.

[40]  Yunde Jia,et al.  Hyperbolic Feature Augmentation via Distribution Estimation and Infinite Sampling on Manifolds , 2022, NeurIPS.

[41]  Abhinav Valada,et al.  On Hyperbolic Embeddings in Object Detection , 2022, GCPR.

[42]  Carlo Vercellis,et al.  Multimodal sentiment and emotion recognition in hyperbolic space , 2021, Expert Syst. Appl..

[43]  Heng Huang,et al.  Learning Better Visual Data Similarities via New Grouplet Non-Euclidean Embedding , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[44]  Mehrtash Harandi,et al.  Kernel Methods in Hyperbolic Spaces , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[45]  M. Harandi,et al.  Curvature Generation in Curved Spaces for Few-Shot Learning , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  Michael S. Bernstein,et al.  On the Opportunities and Risks of Foundation Models , 2021, ArXiv.

[47]  Shu Wu,et al.  Fully Hyperbolic Graph Convolution Network for Recommendation , 2021, CIKM.

[48]  Pascal Mettes,et al.  Hyperbolic Busemann Learning with Ideal Prototypes , 2021, NeurIPS.

[49]  Heng Huang,et al.  Unsupervised Hyperbolic Metric Learning , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Jiuyang Tang,et al.  Multi-modal Entity Alignment in Hyperbolic Space , 2021, Neurocomputing.

[51]  Julien Mairal,et al.  Emerging Properties in Self-Supervised Vision Transformers , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[52]  Tingjun Hou,et al.  Hyperbolic relational graph convolution networks plus: a simple but highly efficient QSAR-modeling method , 2021, Briefings Bioinform..

[53]  Chuan Shi,et al.  Lorentzian Graph Convolutional Networks , 2021, WWW.

[54]  Yunde Jia,et al.  A Hyperbolic-to-Hyperbolic Graph Convolutional Network , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Mehmet Giray Ogut,et al.  Supplementary Material for Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision , 2021 .

[56]  Ilya Sutskever,et al.  Learning Transferable Visual Models From Natural Language Supervision , 2021, ICML.

[57]  Carl Vondrick,et al.  Learning the Predictability of the Future , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Joy Hsu,et al.  Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations , 2020, NeurIPS.

[59]  Yi Jiang,et al.  Sparse R-CNN: End-to-End Object Detection with Learnable Proposals , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[61]  Daniel Cohen-Or,et al.  Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Werner Creixell,et al.  HGAN: Hyperbolic Generative Adversarial Network , 2021, IEEE Access.

[63]  Qun Liu,et al.  HyperText: Endowing FastText with Hyperbolic Geometry , 2020, FINDINGS.

[64]  Albert Gu,et al.  From Trees to Continuous Embeddings and Back: Hyperbolic Hierarchical Clustering , 2020, NeurIPS.

[65]  Shiliang Pu,et al.  Learning Open Set Network with Discriminative Reciprocal Points , 2020, ECCV.

[66]  Alexander Tuzhilin,et al.  Performance of Hyperbolic Geometry Models on Top-N Recommendation Tasks , 2020, RecSys.

[67]  Yu-Gang Jiang,et al.  Hyperbolic Visual Embedding Learning for Zero-Shot Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[68]  Shyam Visweswaran,et al.  Semi-Supervised Hierarchical Drug Embedding in Hyperbolic Space , 2020, J. Chem. Inf. Model..

[69]  Heng Tao Shen,et al.  Searching for Actions on the Hyperbole , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Ce Liu,et al.  Supervised Contrastive Learning , 2020, NeurIPS.

[71]  Dario Pavllo,et al.  Hierarchical Image Classification using Entailment Cone Embeddings , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[72]  Renjie Liao,et al.  Latent Variable Modelling with Hyperbolic Normalizing Flows , 2020, ICML.

[73]  Pascal Chossat,et al.  The hyperbolic model for edge and texture detection in the primary visual cortex , 2020, The Journal of Mathematical Neuroscience.

[74]  Tero Karras,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[75]  Ross B. Girshick,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[76]  Octavian-Eugen Ganea,et al.  Constant Curvature Graph Convolutional Networks , 2019, ICML.

[77]  Dingkang Wang,et al.  An Improved Cost Function for Hierarchical Cluster Trees , 2018, J. Comput. Geom..

[78]  Ling Shao,et al.  Learning Attentive and Hierarchical Representations for 3D Shape Recognition , 2020, ECCV.

[79]  Timothy M. Hospedales,et al.  Multi-relational Poincaré Graph Embeddings , 2019, NeurIPS.

[80]  Douwe Kiela,et al.  Hyperbolic Graph Neural Networks , 2019, NeurIPS.

[81]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[82]  Andrew McCallum,et al.  Gradient-based Hierarchical Clustering using Continuous Representations of Trees in Hyperbolic Space , 2019, KDD.

[83]  David Lopez-Paz,et al.  Poincaré maps for analyzing complex hierarchies in single-cell data , 2019, Nature Communications.

[84]  Ross B. Girshick,et al.  LVIS: A Dataset for Large Vocabulary Instance Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[85]  Renjie Liao,et al.  Lorentzian Distance Learning for Hyperbolic Representations , 2019, ICML.

[86]  Shoichiro Yamaguchi,et al.  A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning , 2019, ICML.

[87]  Charline Le Lan,et al.  Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders , 2019, NeurIPS.

[88]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[89]  Bonnie Berger,et al.  Large-Margin Classification in Hyperbolic Space , 2018, AISTATS.

[90]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[91]  Matthias Leimeister,et al.  Skip-gram word embeddings in hyperbolic space , 2018, ArXiv.

[92]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[93]  Thomas Hofmann,et al.  Hyperbolic Neural Networks , 2018, NeurIPS.

[94]  Christopher De Sa,et al.  Representation Tradeoffs for Hyperbolic Embeddings , 2018, ICML.

[95]  Thomas Hofmann,et al.  Hyperbolic Entailment Cones for Learning Hierarchical Embeddings , 2018, ICML.

[96]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[97]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[98]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[99]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[100]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[101]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[102]  Yannis Avrithis,et al.  Efficient Diffusion on Region Manifolds: Recovering Small Objects with Compact CNN Representations , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[103]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[104]  Samy Bengio,et al.  Density estimation using Real NVP , 2016, ICLR.

[105]  T. Takata,et al.  Mathematical Proceedings of the Cambridge Philosophical Society , 2017 .

[106]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[107]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[108]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[109]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[110]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[111]  Abraham Albert Ungar,et al.  A Gyrovector Space Approach to Hyperbolic Geometry , 2009, A Gyrovector Space Approach to Hyperbolic Geometry.

[112]  Martha Palmer,et al.  Verbnet: a broad-coverage, comprehensive verb lexicon , 2005 .

[113]  William L. Jorgensen,et al.  Journal of Chemical Information and Modeling , 2005, J. Chem. Inf. Model..

[114]  A. Ungar Beyond the Einstein Addition Law and its Gyroscopic Thomas Precession: The Theory of Gyrogroups and Gyrovector Spaces , 2001 .

[115]  Carole D. Hafner,et al.  The State of the Art in Ontology Design: A Survey and Comparative Review , 1997, AI Mag..

[116]  W. Floyd,et al.  HYPERBOLIC GEOMETRY , 1996 .