The Multimarginal Optimal Transport Formulation of Adversarial Multiclass Classification

We study a family of adversarial multiclass classification problems and provide equivalent reformulations in terms of: 1) a family of generalized barycenter problems introduced in the paper and 2) a family of multimarginal optimal transport problems where the number of marginals is equal to the number of classes in the original classification problem. These new theoretical results reveal a rich geometric structure of adversarial learning problems in multiclass classification and extend recent results restricted to the binary classification setting. A direct computational implication of our results is that by solving either the barycenter problem and its dual, or the MOT problem and its dual, we can recover the optimal robust classification rule and the optimal adversarial strategy for the original adversarial problem. Examples with synthetic and real data illustrate our results.

[1]  Natalie Frank.  Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification , 2022, ArXiv.

[2]  Jonathan Niles-Weed,et al.  The Consistency of Adversarial Training for Binary Classification , 2022, ArXiv.

[3]  G. Carlier.  On the Linear Convergence of the Multimarginal Sinkhorn Algorithm , 2022, SIAM J. Optim..

[4]  Muni Sreenivas Pydi.  The Many Faces of Adversarial Risk: An Expanded Study , 2022, IEEE Transactions on Information Theory.

[5]  Mehryar Mohri,et al.  On the Existence of the Adversarial Bayes Classifier (Extended Version) , 2021, NeurIPS.

[6]  G. Steidl,et al.  Unbalanced Multi-marginal Optimal Transport , 2021, Journal of Mathematical Imaging and Vision.

[7]  Yann Chevaleyre,et al.  Mixed Nash Equilibria in the Adversarial Examples Game , 2021, ICML.

[8]  Enric Boix-Adsera,et al.  Wasserstein barycenters are NP-hard to compute , 2021, SIAM J. Math. Data Sci..

[9]  Ryan W. Murray,et al.  Adversarial Classification: Necessary conditions and geometric flows , 2020, J. Mach. Learn. Res..

[10]  William L. Hamilton,et al.  Adversarial Example Games , 2020, NeurIPS.

[11]  Enric Boix-Adsera,et al.  Wasserstein barycenters can be computed in polynomial time in fixed dimension , 2020, J. Mach. Learn. Res..

[12]  Pavel Dvurechensky,et al.  Multimarginal Optimal Transport by Accelerated Alternating Minimization , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[13]  Muni Sreenivas Pydi,et al.  Adversarial Risk via Optimal Transport and Optimal Couplings , 2019, IEEE Transactions on Information Theory.

[14]  Simone Di Marino,et al.  An Optimal Transport Approach for the Schrödinger Bridge Problem and Convergence of Sinkhorn Algorithm , 2019, Journal of Scientific Computing.

[15]  Mingkui Tan,et al.  Multi-marginal Wasserstein GAN , 2019, NeurIPS.

[16]  Michael I. Jordan,et al.  On the Complexity of Approximating Multimarginal Optimal Transport , 2019, J. Mach. Learn. Res..

[17]  Daniel Cullina,et al.  Lower Bounds on Adversarial Robustness from Optimal Transport , 2019, NeurIPS.

[18]  Julie Delon,et al.  A Wasserstein-Type Distance in the Space of Gaussian Mixture Models , 2019, SIAM J. Imaging Sci..

[19]  Preetum Nakkiran,et al.  Adversarial Robustness May Be at Odds With Simplicity , 2019, ArXiv.

[20]  P. Chiappori,et al.  Multi‐to One‐Dimensional Optimal Transport , 2017 .

[21]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Jean-David Benamou,et al.  Generalized incompressible flows, multi-marginal transport and Sinkhorn algorithm , 2017, Numerische Mathematik.

[23]  J. Blanchet,et al.  Robust Wasserstein profile inference and applications to machine learning , 2016, J. Appl. Probab..

[24]  Karthyek R. A. Murthy,et al.  Quantifying Distributional Model Risk Via Optimal Transport , 2016, Math. Oper. Res..

[25]  David B. Dunson,et al.  Scalable Bayes via Barycenter in Wasserstein Space , 2015, J. Mach. Learn. Res..

[26]  Simone Di Marino,et al.  Multimarginal Optimal Transport Maps for One–dimensional Repulsive Costs , 2015, Canadian Journal of Mathematics.

[27]  Gabriel Peyré,et al.  Iterative Bregman Projections for Regularized Transportation Problems , 2014, SIAM J. Sci. Comput..

[28]  Adam M. Oberman,et al.  NUMERICAL METHODS FOR MATCHING FOR TEAMS AND WASSERSTEIN BARYCENTERS , 2014, 1411.3602.

[29]  Brendan Pass.  Multi-marginal optimal transport: theory and applications , 2014, 1406.0026.

[30]  Brendan Pass,et al.  THE MULTI-MARGINAL OPTIMAL PARTIAL TRANSPORT PROBLEM , 2014, Forum of Mathematics, Sigma.

[31]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[32]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[33]  Brendan Pass,et al.  Multi-marginal optimal transport on Riemannian manifolds , 2013, 1303.6251.

[34]  Lin Lin,et al.  Kantorovich dual solution for strictly correlated electrons in atoms and molecules , 2012, 1210.7117.

[35]  G. Buttazzo,et al.  Optimal-transport formulation of electronic density-functional theory , 2012, 1205.4514.

[36]  Codina Cotar,et al.  Density Functional Theory and Optimal Transportation with Coulomb Cost , 2011, 1104.0603.

[37]  Guillaume Carlier,et al.  Barycenters in the Wasserstein Space , 2011, SIAM J. Math. Anal..

[38]  G. Carlier,et al.  Matching for teams , 2010 .

[39]  P. Chiappori,et al.  Hedonic price equilibria, stable matching, and optimal transport: equivalence, topology, and uniqueness , 2007 .

[40]  P. Gori-Giorgi,et al.  Strictly correlated electrons in density-functional theory: A general formulation with applications to spherical densities , 2007, cond-mat/0701025.

[41]  I. Ekeland.  An optimal matching problem , 2003, math/0308206.

[42]  C. Villani.  Topics in Optimal Transportation , 2003 .

[43]  Yongxin Chen,et al.  Multimarginal Optimal Transport with a Tree-Structured Cost and the Schrödinger Bridge Problem , 2021, SIAM Journal of Control and Optimization.

[44]  W. Gangbo,et al.  Optimal maps for the multidimensional Monge-Kantorovich problem , 1998 .