Revisiting Fixed Support Wasserstein Barycenter: Computational Hardness and Efficient Algorithms

We study the fixed-support Wasserstein barycenter problem (FS-WBP), which consists in computing the Wasserstein barycenter of m discrete probability measures supported on a finite metric space of size n. We show first that the constraint matrix arising from the linear programming (LP) representation of the FS-WBP is totally unimodular when m ≥ 3 and n = 2, but not totally unimodular when m ≥ 3 and n ≥ 3. This result answers an open problem, since it shows that the FS-WBP is not a minimum-cost flow problem and therefore cannot be solved efficiently using linear programming. Building on this negative result, we propose and analyze a simple and efficient variant of the iterative Bregman projection (IBP) algorithm, currently the most widely adopted algorithm to solve the FS-WBP. The algorithm is an accelerated IBP algorithm which achieves the complexity bound of Õ(mn7/3/ε). This bound is better than that obtained for the standard IBP algorithm—Õ(mn2/ε2)—in terms of ε, and that of accelerated primal-dual gradient algorithm—Õ(mn5/2/ε)—in terms of n. Empirical studies on simulated datasets demonstrate that the acceleration promised by the theory is real in practice.

[1]  Kevin Tian,et al.  A Direct Õ(1/ε) Iteration Parallel Algorithm for Optimal Transport , 2019, ArXiv.

[2]  Gabriel Peyré,et al.  Iterative Bregman Projections for Regularized Transportation Problems , 2014, SIAM J. Sci. Comput..

[3]  Sayan Mukherjee,et al.  Probabilistic Fréchet Means and Statistics on Vineyards , 2013, ArXiv.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Peter Richtárik,et al.  Accelerated, Parallel, and Proximal Coordinate Descent , 2013, SIAM J. Optim..

[6]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[7]  Gabriel Peyré,et al.  Stochastic Optimization for Large-scale Optimal Transport , 2016, NIPS.

[8]  Fred B. Schneider,et al.  A Theory of Graphs , 1993 .

[9]  Michael I. Jordan,et al.  On the Complexity of Approximating Multimarginal Optimal Transport , 2019, J. Mach. Learn. Res..

[10]  Lin Xiao,et al.  On the complexity analysis of randomized block-coordinate descent methods , 2013, Mathematical Programming.

[11]  Julien Rabin,et al.  Wasserstein Barycenter and Its Application to Texture Mixing , 2011, SSVM.

[12]  James Zijun Wang,et al.  Fast Discrete Distribution Clustering Using Wasserstein Barycenter With Sparse Support , 2015, IEEE Transactions on Signal Processing.

[13]  Guillaume Carlier,et al.  Barycenters in the Wasserstein Space , 2011, SIAM J. Math. Anal..

[14]  Vahab S. Mirrokni,et al.  Accelerating Greedy Coordinate Descent Methods , 2018, ICML.

[15]  Lei Yang,et al.  A Fast Globally Linearly Convergent Algorithm for the Computation of Wasserstein Barycenters , 2018, J. Mach. Learn. Res..

[16]  Darina Dvinskikh,et al.  On the Complexity of Approximating Wasserstein Barycenters , 2019, ICML.

[17]  G. Dantzig,et al.  Integral Extreme Points , 1968 .

[18]  Lin Xiao,et al.  An Accelerated Randomized Proximal Coordinate Gradient Method and its Application to Regularized Empirical Risk Minimization , 2015, SIAM J. Optim..

[19]  Angelia Nedic,et al.  Distributed Computation of Wasserstein Barycenters Over Networks , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[20]  Alexander Gasnikov,et al.  Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm , 2018, ICML.

[21]  Michael I. Jordan,et al.  On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms , 2019, ICML.

[22]  Makoto Yamada,et al.  On Scalable Variant of Wasserstein Barycenter , 2019, ArXiv.

[23]  Julien Rabin,et al.  Sliced and Radon Wasserstein Barycenters of Measures , 2014, Journal of Mathematical Imaging and Vision.

[24]  Yinyu Ye,et al.  Interior-Point Methods Strike Back: Solving the Wasserstein Barycenter Problem , 2019, NeurIPS.

[25]  Ted K. Ralphs,et al.  Integer and Combinatorial Optimization , 2013 .

[26]  L. Rüschendorf,et al.  On the Computation of Wasserstein Barycenters , 2019, J. Multivar. Anal..

[27]  A. Bonato,et al.  Graphs and Hypergraphs , 2022 .

[28]  Jason Altschuler,et al.  Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration , 2017, NIPS.

[29]  Michael I. Jordan,et al.  On the Acceleration of the Sinkhorn and Greenkhorn Algorithms for Optimal Transport , 2019, ArXiv.

[30]  Adam M. Oberman,et al.  NUMERICAL METHODS FOR MATCHING FOR TEAMS AND WASSERSTEIN BARYCENTERS , 2014, 1411.3602.

[31]  Yin Tat Lee,et al.  Path Finding Methods for Linear Programming: Solving Linear Programs in Õ(vrank) Iterations and Faster Algorithms for Maximum Flow , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[32]  Aaron Sidford,et al.  Towards Optimal Running Times for Optimal Transport , 2018, ArXiv.

[33]  W. Gangbo,et al.  Optimal maps for the multidimensional Monge-Kantorovich problem , 1998 .

[34]  Alain Trouvé,et al.  Local Geometry of Deformable Templates , 2005, SIAM J. Math. Anal..

[35]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[36]  Péter Kovács,et al.  Minimum-cost flow algorithms: an experimental evaluation , 2015, Optim. Methods Softw..

[37]  Darina Dvinskikh,et al.  Decentralize and Randomize: Faster Algorithm for Wasserstein Barycenters , 2018, NeurIPS.

[38]  F. G. Commoner,et al.  A sufficient condition for a matrix to be totally unimodular , 1973, Networks.

[39]  Nathaniel Lahn,et al.  A Graph Theoretic Additive Approximation of Optimal Transport , 2019, NeurIPS.

[40]  Alan J. Hoffman,et al.  Integral Boundary Points of Convex Polyhedra , 2010, 50 Years of Integer Programming.

[41]  Gabriel Peyré,et al.  Wasserstein barycentric coordinates , 2016, ACM Trans. Graph..

[42]  Daniel A. Spielman,et al.  Faster approximate lossy generalized flow via interior point algorithms , 2008, STOC.

[43]  Steffen Borgwardt,et al.  On the computational complexity of finding a sparse Wasserstein barycenter , 2019, Journal of Combinatorial Optimization.

[44]  Kent Quanrud,et al.  Approximating optimal transport with linear programs , 2018, SOSA.

[45]  Dinh Q. Phung,et al.  Multilevel Clustering via Wasserstein Means , 2017, ICML.

[46]  P. Camion.  Characterization of totally unimodular matrices , 1965 .

[47]  Steffen Borgwardt,et al.  Improved Linear Programs for Discrete Barycenters , 2018, INFORMS Journal on Optimization.

[48]  C. Villani.  Topics in Optimal Transportation , 2003 .

[49]  P. Chiappori,et al.  Hedonic price equilibria, stable matching, and optimal transport: equivalence, topology, and uniqueness , 2007 .

[50]  Justin Solomon,et al.  Stochastic Wasserstein Barycenters , 2018, ICML.

[51]  Codina Cotar,et al.  Density Functional Theory and Optimal Transportation with Coulomb Cost , 2011, 1104.0603.

[52]  David B. Dunson,et al.  Scalable Bayes via Barycenter in Wasserstein Space , 2015, J. Mach. Learn. Res..

[53]  G. Carlier,et al.  Matching for teams , 2010 .

[54]  Steffen Borgwardt,et al.  Discrete Wasserstein barycenters: optimal transport for discrete data , 2015, Mathematical Methods of Operations Research.

[55]  Justin Solomon,et al.  Parallel Streaming Wasserstein Barycenters , 2017, NIPS.

[56]  Gabriel Peyré,et al.  Semi-dual Regularized Optimal Transport , 2018, SIAM Rev..

[57]  A. Tamir,et al.  On totally unimodular matrices , 1976, Networks.

[58]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[59]  Gabriel Peyré,et al.  A Smoothed Dual Approach for Variational Wasserstein Problems , 2015, SIAM J. Imaging Sci..