GeomCA: Geometric Evaluation of Data Representations

Evaluating the quality of learned representations without relying on a downstream task remains one of the challenges in representation learning. In this work, we present Geometric Component Analysis (GeomCA) algorithm that evaluates representation spaces based on their geometric and topological properties. GeomCA can be applied to representations of any dimension, independently of the model that generated them. We demonstrate its applicability by analyzing representations obtained from a variety of scenarios, such as contrastive learning models, generative models and supervised learning models.

[1]  Valentin Khrulkov,et al.  Geometry Score: A Method For Comparing Generative Adversarial Networks , 2018, ICML.

[2]  Lars Kai Hansen,et al.  Latent Space Oddity: on the Curvature of Deep Generative Models , 2017, ICLR.

[3]  Afra Zomorodian,et al.  Computing Persistent Homology , 2004, SCG '04.

[4]  Andriy Mnih,et al.  Disentangling by Factorising , 2018, ICML.

[5]  Pieter Abbeel,et al.  CURL: Contrastive Unsupervised Representations for Reinforcement Learning , 2020, ICML.

[6]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[7]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[8]  Andrew Gordon Wilson,et al.  Why Normalizing Flows Fail to Detect Out-of-Distribution Data , 2020, NeurIPS.

[9]  Danica Kragic,et al.  Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[11]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[12]  D. Pfau,et al.  Disentangling by Subspace Diffusion , 2020, NeurIPS.

[13]  Karsten Borgwardt,et al.  Topological Autoencoders , 2019, ICML.

[14]  Danica Kragic,et al.  Data-efficient visuomotor policy training using reinforcement learning and generative models , 2020, ArXiv.

[15]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Bernhard Schölkopf,et al.  Geometrically Enriched Latent Spaces , 2020, AISTATS.

[17]  Bernhard Schölkopf,et al.  Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations , 2018, ICML.

[18]  Simon Schönenberger.  Witness Autoencoder: Shaping the Latent Space with Witness Complexes , 2020 .

[19]  Weihong Deng,et al.  Very deep convolutional neural network based image classification using small training sample size , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[20]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[21]  Huaping Liu,et al.  Unsupervised Representation Learning by InvariancePropagation , 2020, NeurIPS.

[22]  Aric Hagberg,et al.  Exploring Network Structure, Dynamics, and Function using NetworkX , 2008 .

[23]  Alan F. Smeaton,et al.  Contrastive Representation Learning: A Framework and Review , 2020, IEEE Access.

[24]  M. Lippi,et al.  State Representations in Robotics: Identifying Relevant Factors of Variation using Weak Supervision , 2020 .

[25]  Luca Oneto,et al.  Exploiting MMD and Sinkhorn Divergences for Fair and Transferable Representation Learning , 2020, NeurIPS.

[26]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[27]  Marc G. Bellemare,et al.  Representations for Stable Off-Policy Reinforcement Learning , 2020, ICML.

[28]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Jaakko Lehtinen,et al.  Improved Precision and Recall Metric for Assessing Generative Models , 2019, NeurIPS.