Stochastic Incremental Algorithms for Optimal Transport with SON Regularizer

We introduce a new regularizer for optimal transport (OT) which is tailored to better preserve the class structure. We give the first theoretical guarantees for an OT scheme that respects class structure. We give an accelerated proximal--projection scheme for this formulation with the proximal operator in closed form to give a highly scalable algorithm for computing optimal transport plans. We give a novel argument for the uniqueness of the optimum even in the absence of strong convexity. Our experiments show that the new regularizer preserves class structure better and is more robust compared to previous regularizers.

[1]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[3]  L. Kantorovitch,et al.  On the Translocation of Masses , 1958 .

[4]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[5]  Dit-Yan Yeung,et al.  Robust path-based spectral clustering , 2008, Pattern Recognit..

[6]  Ion Necoara,et al.  Nonasymptotic convergence of stochastic proximal point methods for constrained convex optimization , 2017, J. Mach. Learn. Res..

[7]  Justin Solomon,et al.  Optimal Transport on Discrete Domains , 2018, Proceedings of Symposia in Applied Mathematics.

[8]  Gabriel Peyré,et al.  Stochastic Optimization for Large-scale Optimal Transport , 2016, NIPS.

[9]  Gustavo K. Rohde,et al.  Optimal Mass Transport: Signal processing and machine-learning applications , 2017, IEEE Signal Processing Magazine.

[10]  Mark W. Schmidt,et al.  Minimizing finite sums with the stochastic average gradient , 2013, Mathematical Programming.

[11]  Aaron Defazio,et al.  A Simple Practical Accelerated Method for Finite Sums , 2016, NIPS.

[12]  L. Kantorovich.  On the Translocation of Masses , 2006 .

[13]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[14]  Filippo Santambrogio,et al.  Optimal Transport for Applied Mathematicians , 2015 .

[15]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[16]  Nicolas Courty,et al.  Mapping Estimation for Discrete Optimal Transport , 2016, NIPS.

[17]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..