A Topology Layer for Machine Learning

Topology applied to real world data using persistent homology has started to find applications within machine learning, including deep learning. We present a differentiable topology layer that computes persistent homology based on level set filtrations and distance-bases filtrations. We present three novel applications: the topological layer can (i) serve as a regularizer directly on data or the weights of machine learning models, (ii) construct a loss on the output of a deep generative network to incorporate topological priors, and (iii) perform topological adversarial attacks on deep networks trained with persistence features. The code is publicly available and we hope its availability will facilitate the use of persistent homology in deep learning and other gradient based applications.

[1]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[2]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[3]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[4]  Leonidas J. Guibas,et al.  Persistence barcodes for shapes , 2004, SGP '04.

[5]  Afra Zomorodian,et al.  Computing Persistent Homology , 2005, Discret. Comput. Geom..

[6]  H. Edelsbrunner.  Alpha Shapes — a Survey , 2009 .

[7]  Herbert Edelsbrunner,et al.  Computational Topology - an Introduction , 2009 .

[8]  Tamal K. Dey,et al.  Persistent Heat Signature for Pose‐oblivious Matching of Incomplete Models , 2010, Comput. Graph. Forum.

[9]  Leonidas J. Guibas,et al.  Persistence-based segmentation of deformable shapes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[10]  Dmitriy Morozov,et al.  Dualities in persistent (co)homology , 2011, ArXiv.

[11]  Primoz Skraba,et al.  Zigzag persistent homology in matrix multiplication time , 2011, SoCG '11.

[12]  Robert Tibshirani,et al.  Nearly-Isotonic Regression , 2011, Technometrics.

[13]  Aaron B. Adcock,et al.  The Ring of Algebraic Functions on Persistence Bar Codes , 2013, 1304.0530.

[14]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[15]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[16]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[17]  Yi-Hsuan Yang,et al.  Applying Topological Persistence in Convolutional Neural Network for Music Audio Signals , 2016, ArXiv.

[18]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[19]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[20]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[21]  Andreas Uhl,et al.  Deep Learning with Topological Signatures , 2017, NIPS.

[22]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[23]  Mason A. Porter,et al.  A roadmap for the computation of persistent homology , 2015, EPJ Data Science.

[24]  Paul Schrater,et al.  Adversary Detection in Neural Networks via Persistent Homology , 2017, ArXiv.

[25]  Chi Seng Pun,et al.  Persistent-Homology-Based Machine Learning and Its Applications -- A Survey , 2018, 1811.00252.

[26]  Ruslan Salakhutdinov,et al.  On Characterizing the Capacity of Neural Networks using Algebraic Topology , 2018, ArXiv.

[27]  Gunnar E. Carlsson,et al.  Topological Approaches to Deep Learning , 2018, Topological Data Analysis.

[28]  Debdeep Mukhopadhyay,et al.  Adversarial Attacks and Defences: A Survey , 2018, ArXiv.

[29]  Maks Ovsjanikov,et al.  Topological Function Optimization for Continuous Shape Matching , 2018, Comput. Graph. Forum.

[30]  Leonidas J. Guibas,et al.  Learning Representations and Generative Models for 3D Point Clouds , 2017, ICML.

[31]  Chao Chen,et al.  A Topological Regularizer for Classifiers via Persistent Homology , 2019, AISTATS.

[32]  Karsten M. Borgwardt,et al.  Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology , 2018, ICLR.

[33]  Ilkay Öksüz,et al.  Explicit topological priors for deep-learning based image segmentation using persistent homology , 2019, IPMI.

[34]  Rickard Brüel Gabrielsson,et al.  Exposition and Interpretation of the Topology of Neural Networks , 2018, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA).

[35]  Alan Hylton,et al.  Characterizing the Shape of Activation Space in Deep Neural Networks , 2019, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA).

[36]  Paul Schrater,et al.  Adversarial Examples Target Topological Holes in Deep Networks , 2019, ArXiv.

[37]  Chul Moon,et al.  Persistent Homology Machine Learning for Fingerprint Classification , 2017, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA).

[38]  Gard Spreemann,et al.  Topology of Learning in Artificial Neural Networks , 2019, ArXiv.

[39]  Primoz Skraba,et al.  Randomly Weighted d-Complexes: Minimal Spanning Acycles and Persistence Diagrams , 2017, Electron. J. Comb..

[40]  Rickard Brüel Gabrielsson,et al.  Topology‐Aware Surface Reconstruction for Point Clouds , 2018, Comput. Graph. Forum.