Learning Cost Functions for Optimal Transport

Learning the cost function for optimal transport from observed transport plan or its samples has been cast as a bi-level optimization problem. In this paper, we derive an unconstrained convex optimization formulation for the problem which can be further augmented by any customizable regularization. This novel framework avoids repeatedly solving a forward optimal transport problem in each iteration which has been a thorny computational bottleneck for the bi-level optimization approach. To validate the effectiveness of this framework, we develop two numerical algorithms, one is a fast matrix scaling method based on the Sinkhorn-Knopp algorithm for the discrete case, and the other is a supervised learning algorithm that realizes the cost function as a deep neural network in the continuous case. Numerical results demonstrate promising efficiency and accuracy advantages of the proposed algorithms over existing state of the art methods.

[1]  Tam Le,et al.  Unsupervised Riemannian Metric Learning for Histograms Using Aitchison Transformations , 2015, ICML.

[2]  Gabriel Peyré,et al.  Sample Complexity of Sinkhorn Divergences , 2018, AISTATS.

[3]  Nicolas Papadakis,et al.  Parameter Estimation in Finite Mixture Models by Regularized Optimal Transport: A Unified Framework for Hard and Soft Clustering , 2017, ArXiv.

[4]  Matt J. Kusner,et al.  Supervised Word Mover's Distance , 2016, NIPS.

[5]  Daniel Kuhn,et al.  Regularization via Mass Transportation , 2017, J. Mach. Learn. Res..

[6]  N. Papadakis Optimal Transport for Image Processing , 2015 .

[7]  A. Dupuy,et al.  Personality Traits and the Marriage Market , 2014, Journal of Political Economy.

[8]  David Avis,et al.  Ground metric learning , 2011, J. Mach. Learn. Res..

[9]  Viet Anh Nguyen,et al.  Wasserstein Distributionally Robust Kalman Filtering , 2018, NeurIPS.

[10]  Shun-ichi Amari,et al.  Information geometry connecting Wasserstein distance and Kullback–Leibler divergence via the entropy-relaxed transportation problem , 2017, Information Geometry.

[11]  Han Sun,et al.  Learning With Batch-Wise Optimal Transport Loss for 3D Shape Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Marc Sebban,et al.  A Survey on Metric Learning for Feature Vectors and Structured Data , 2013, ArXiv.

[13]  Nicolas Courty,et al.  Large Scale Optimal Transport and Mapping Estimation , 2017, ICLR.

[14]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[15]  Yuan Jiang,et al.  Complex Object Classification: A Multi-Modal Multi-Instance Multi-Label Deep Network with Optimal Transport , 2018, KDD.

[16]  Gualdani,et al.  Optimal transportation and applications , 2010 .

[17]  Leonidas J. Guibas,et al.  Supervised Earth Mover's Distance Learning and Its Computer Vision Applications , 2012, ECCV.

[18]  James Zou,et al.  Learning transport cost from subset correspondence , 2020, ICLR.

[19]  Ruilin Li,et al.  Learning to Match via Inverse Optimal Transport , 2018, J. Mach. Learn. Res..

[20]  Umberto Castellani,et al.  Supervised Learning of Diffusion Distance to Improve Histogram Matching , 2015, SIMBAD.

[21]  Gabriel Peyré,et al.  Fast Dictionary Learning with a Smoothed Wasserstein Loss , 2016, AISTATS.

[22]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Bernhard Schmitzer,et al.  Stabilized Sparse Scaling Algorithms for Entropy Regularized Transport Problems , 2016, SIAM J. Sci. Comput..

[24]  A. Galichon,et al.  Cupid’s Invisible Hand: Social Surplus and Identification in Matching Models , 2015, 2106.02371.

[25]  Marie-Therese Wolfram,et al.  Inverse Optimal Transport , 2019, SIAM J. Appl. Math..

[26]  Tara Abrishami,et al.  Geometry of Graph Partitions via Optimal Transport , 2019, SIAM J. Sci. Comput..

[27]  Nicolas Papadakis,et al.  Regularized Optimal Transport and the Rot Mover's Distance , 2016, J. Mach. Learn. Res..

[28]  C. Villani Optimal Transport: Old and New , 2008 .

[29]  Zhi-Hua Zhou,et al.  Label Distribution Learning by Optimal Transport , 2018, AAAI.