Gamma-convergence of a nonlocal perimeter arising in adversarial machine learning

In this paper we prove Gamma-convergence of a nonlocal perimeter of Minkowski type to a local anisotropic perimeter. The nonlocal model describes the regularizing effect of adversarial training in binary classifications. The energy essentially depends on the interaction between two distributions modelling likelihoods for the associated classes. We overcome typical strict regularity assumptions for the distributions by only assuming that they have bounded $BV$ densities. In the natural topology coming from compactness, we prove Gamma-convergence to a weighted perimeter with weight determined by an anisotropic function of the two densities. Despite being local, this sharp interface limit reflects classification stability with respect to adversarial perturbations. We further apply our results to deduce Gamma-convergence of the associated total variations, to study the asymptotics of adversarial training, and to prove Gamma-convergence of graph discretizations for the nonlocal perimeter.

[1]  Jonathan Niles-Weed,et al.  The Consistency of Adversarial Training for Binary Classification , 2022, ArXiv.

[2]  M. Jacobs,et al.  The Multimarginal Optimal Transport Formulation of Adversarial Multiclass Classification , 2022, J. Mach. Learn. Res..

[3]  JEFF CALDER,et al.  Improved spectral convergence rates for graph Laplacians on ε-graphs and k-NN graphs , 2022, Applied and Computational Harmonic Analysis.

[4]  Muni Sreenivas Pydi.  The Many Faces of Adversarial Risk: An Expanded Study , 2022, IEEE Transactions on Information Theory.

[5]  Martin Burger,et al.  Variational Regularization in Inverse Problems and Machine Learning , 2021, ArXiv.

[6]  Mehryar Mohri,et al.  On the Existence of the Adversarial Bayes Classifier (Extended Version) , 2021, NeurIPS.

[7]  Leon Bungert,et al.  The Geometry of Adversarial Training in Binary Classification , 2021, ArXiv.

[8]  Camilo A. Garcia Trillos,et al.  On the regularized risk of distributionally robust learning over deep neural networks , 2021, ArXiv.

[9]  Mehryar Mohri,et al.  Calibration and Consistency of Adversarial Surrogate Losses , 2021, NeurIPS.

[10]  B. Wen,et al.  Recent Advances in Adversarial Training for Adversarial Robustness , 2021, IJCAI.

[11]  Leon Bungert,et al.  Continuum Limit of Lipschitz Learning on Graphs , 2020, Foundations of Computational Mathematics.

[12]  Ryan W. Murray,et al.  Adversarial Classification: Necessary conditions and geometric flows , 2020, J. Mach. Learn. Res..

[13]  Jeff Calder,et al.  Rates of convergence for Laplacian semi-supervised learning with low labeling rates , 2020, Research in the Mathematical Sciences.

[14]  Ryan W. Murray,et al.  From Graph Cuts to Isoperimetric Inequalities: Convergence Rates of Cheeger Cuts on Data Clouds , 2020, Archive for Rational Mechanics and Analysis.

[15]  Muni Sreenivas Pydi,et al.  Adversarial Risk via Optimal Transport and Optimal Couplings , 2019, IEEE Transactions on Information Theory.

[16]  Daniel Cullina,et al.  Lower Bounds on Adversarial Robustness from Optimal Transport , 2019, NeurIPS.

[17]  Adam M. Oberman,et al.  Scaleable input gradient regularization for adversarial robustness , 2019, Machine Learning with Applications.

[18]  Martin Burger,et al.  Modern regularization methods for inverse problems , 2018, Acta Numerica.

[19]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[20]  Enrico Valdinoci,et al.  Minimizers for nonlocal perimeters of Minkowski type , 2017, Calculus of Variations and Partial Differential Equations.

[21]  M. Novaga,et al.  Isoperimetric problems for a nonlocal perimeter of Minkowski type , 2017, 1709.05284.

[22]  Irene Fonseca,et al.  The Weighted Ambrosio-Tortorelli Approximation Scheme , 2016, SIAM J. Math. Anal..

[23]  D. Slepčev,et al.  On the Rate of Convergence of Empirical Measures in ∞-transportation Distance , 2015, Canadian Journal of Mathematics.

[24]  Xavier Bresson,et al.  Consistency of Cheeger and Ratio Graph Cuts , 2014, J. Mach. Learn. Res..

[25]  Antonin Chambolle,et al.  Nonlocal Curvature Flows , 2014, Archive for Rational Mechanics and Analysis.

[26]  Nicolás García Trillos,et al.  Continuum Limit of Total Variation on Point Clouds , 2014, Archive for Rational Mechanics and Analysis.

[27]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[28]  A. Chambolle,et al.  A remark on the anisotropic outer Minkowski content , 2012, 1203.5190.

[29]  Antonin Chambolle,et al.  A Nonlocal Mean Curvature Flow and Its Semi-implicit Time-Discrete Approximation , 2012, SIAM J. Math. Anal..

[30]  Daniel Spector.  SIMPLE PROOFS OF SOME RESULTS OF RESHETNYAK , 2011 .

[31]  Marcello Ponsiglione,et al.  A Variational Model for Infinite Perimeter Segmentations Based on Lipschitz Level Set Functions: Denoising while Keeping Finely Oscillatory Boundaries , 2010, Multiscale Model. Simul..

[32]  A. Chambolle,et al.  Continuous limits of discrete perimeters , 2009, ESAIM: Mathematical Modelling and Numerical Analysis.

[33]  I. Fonseca,et al.  Modern Methods in the Calculus of Variations: L^p Spaces , 2007 .

[34]  G. Friesecke,et al.  A theorem on geometric rigidity and the derivation of nonlinear plate theory from three‐dimensional elasticity , 2002 .

[35]  Andrea Braides.  Gamma-Convergence for Beginners , 2002 .

[36]  Andrea Braides.  Approximation of Free-Discontinuity Problems , 1998 .

[37]  L. Modica.  The gradient theory of phase transitions and the minimal interface criterion , 1987 .

[38]  T. Schuster,et al.  A note on Γ-convergence of Tikhonov functionals for nonlinear inverse problems , 2022, arXiv.org.

[39]  N. Fusco,et al.  On the approximation of SBV functions , 2017 .

[40]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[41]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[42]  G. D. Maso,et al.  An Introduction to-convergence , 1993 .

[43]  L. Evans.  Measure theory and fine properties of functions , 1992 .

[44]  Robert L. Foote,et al.  Regularity of the distance function , 1984 .