Adversarial Computation of Optimal Transport Maps

Computing optimal transport maps between high-dimensional and continuous distributions is a challenging problem in optimal transport (OT). Generative adversarial networks (GANs) are powerful generative models which have been successfully applied to learn maps across high-dimensional domains. However, little is known about the nature of the map learned with a GAN objective. To address this problem, we propose a generative adversarial model in which the discriminator's objective is the $2$-Wasserstein metric. We show that during training, our generator follows the $W_2$-geodesic between the initial and the target distributions. As a consequence, it reproduces an optimal map at the end of training. We validate our approach empirically in both low-dimensional and high-dimensional continuous settings, and show that it outperforms prior methods on image data.

[1]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[2]  Sebastian Nowozin,et al.  Which Training Methods for GANs do actually Converge? , 2018, ICML.

[3]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[4]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Gabriel Peyré,et al.  Fast Optimal Transport Averaging of Neuroimaging Data , 2015, IPMI.

[6]  Philip Bachman,et al.  Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data , 2018, ICML.

[7]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[8]  Shoichiro Yamaguchi,et al.  Distributional Concavity Regularization for GANs , 2018, ICLR.

[9]  Kelvin Shuangjian Zhang,et al.  Implicit Manifold Learning on Generative Adversarial Networks , 2017, 1710.11260.

[10]  Roberto Cominetti,et al.  Asymptotic analysis of the exponential penalty trajectory in linear programming , 1994, Math. Program..

[11]  David Lopez-Paz,et al.  Geometrical Insights for Implicit Generative Modeling , 2017, Braverman Readings in Machine Learning.

[12]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[13]  Vivien Seguy,et al.  Smooth and Sparse Optimal Transport , 2017, AISTATS.

[14]  Gabriel Peyré,et al.  Learning Generative Models with Sinkhorn Divergences , 2017, AISTATS.

[15]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[16]  Yong Yu,et al.  Guiding the One-to-one Mapping in CycleGAN via Optimal Transport , 2018, AAAI.

[17]  Han Zhang,et al.  Improving GANs Using Optimal Transport , 2018, ICLR.

[18]  L. Ambrosio,et al.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures , 2005 .

[19]  Jason D. Lee,et al.  On the Convergence and Robustness of Training GANs with Regularized Optimal Transport , 2018, NeurIPS.

[20]  Jason D. Lee,et al.  Solving Approximate Wasserstein GANs to Stationarity , 2018, ArXiv.

[21]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[22]  L. Ambrosio,et al.  A User’s Guide to Optimal Transport , 2013 .

[23]  Shing-Tung Yau,et al.  A Geometric View of Optimal Transportation and Generative Model , 2017, Comput. Aided Geom. Des..

[24]  C. Villani.  Optimal Transport: Old and New , 2008 .

[25]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[26]  Gabriel Peyré,et al.  Stochastic Optimization for Large-scale Optimal Transport , 2016, NIPS.

[27]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[28]  Tong Zhang,et al.  Composite Functional Gradient Learning of Generative Adversarial Models , 2018, ICML.

[29]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Jacob Abernethy,et al.  On Convergence and Stability of GANs , 2018 .

[31]  F. Santambrogio.  {Euclidean, metric, and Wasserstein} gradient flows: an overview , 2016, 1609.03890.

[32]  Hongyuan Zha,et al.  A Fast Proximal Point Method for Wasserstein Distance , 2018, ArXiv.

[33]  J. Zico Kolter,et al.  Gradient descent GAN optimization is locally stable , 2017, NIPS.

[34]  Denis Lukovnikov,et al.  On the regularization of Wasserstein GANs , 2017, ICLR.

[35]  Ashish Khetan,et al.  PacGAN: The Power of Two Samples in Generative Adversarial Networks , 2017, IEEE Journal on Selected Areas in Information Theory.

[36]  Y. Brenier.  Polar Factorization and Monotone Rearrangement of Vector-Valued Functions , 1991 .

[37]  Hongyuan Zha,et al.  A Fast Proximal Point Method for Computing Exact Wasserstein Distance , 2018, UAI.

[38]  Nicolas Courty,et al.  Large Scale Optimal Transport and Mapping Estimation , 2017, ICLR.