Fast Unbalanced Optimal Transport on Tree

This study examines the time complexities of the unbalanced optimal transport problems from an algorithmic perspective for the first time. We reveal which problems in unbalanced optimal transport can/cannot be solved efficiently. Specifically, we prove that the Kantrovich Rubinstein distance and optimal partial transport in Euclidean metric cannot be computed in strongly subquadratic time under the strong exponential time hypothesis. Then, we propose an algorithm that solves a more general unbalanced optimal transport problem exactly in quasi-linear time on a tree metric. The proposed algorithm processes a tree with one million nodes in less than one second. Our analysis forms a foundation for the theoretical study of unbalanced optimal transport algorithms and opens the door to the applications of unbalanced optimal transport to million-scale datasets.

[1]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[2]  Liam Roditty,et al.  Fast approximation algorithms for the diameter and radius of sparse graphs , 2013, STOC '13.

[3]  James B. Orlin.  A Faster Strongly Polynomial Minimum Cost Flow Algorithm , 1993, Oper. Res..

[4]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[5]  Moses Charikar,et al.  Similarity estimation techniques from rounding algorithms , 2002, STOC '02.

[6]  S. Kondratyev,et al.  A new optimal transport distance on the space of finite Radon measures , 2015, Advances in Differential Equations.

[7]  Ryan Williams,et al.  An Equivalence Class for Orthogonal Vectors , 2018, SODA.

[8]  G. Peyré,et al.  Unbalanced optimal transport: Dynamic and Kantorovich formulations , 2015, Journal of Functional Analysis.

[9]  Ryan Williams,et al.  A new algorithm for optimal 2-constraint satisfaction and its implications , 2005, Theor. Comput. Sci..

[10]  Carola-Bibiane Schönlieb,et al.  Imaging with Kantorovich-Rubinstein Discrepancy , 2014, SIAM J. Imaging Sci..

[11]  Péter Kovács,et al.  LEMON - an Open Source C++ Graph Template Library , 2011, WGT@ETAPS.

[12]  Karl Bringmann,et al.  More consequences of falsifying SETH and the orthogonal vectors conjecture , 2018, STOC.

[13]  Kenji Fukumizu,et al.  Tree-Sliced Variants of Wasserstein Distances , 2019, NeurIPS.

[14]  V. V. Williams.  ON SOME FINE-GRAINED QUESTIONS IN ALGORITHMS AND COMPLEXITY , 2019, Proceedings of the International Congress of Mathematicians (ICM 2018).

[15]  Robert E. Tarjan,et al.  Better Approximation Algorithms for the Graph Diameter , 2014, SODA.

[16]  Piotr Indyk,et al.  Edit Distance Cannot Be Computed in Strongly Subquadratic Time (unless SETH is false) , 2014, STOC.

[17]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[18]  Lili Yu,et al.  Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport , 2020, ACL.

[19]  J. Benamou.  NUMERICAL RESOLUTION OF AN \UNBALANCED" MASS TRANSPORT PROBLEM , 2003 .

[20]  Steve Oudot,et al.  Large Scale computation of Means and Clusters for Persistence Diagrams using Optimal Transport , 2018, NeurIPS.

[21]  Piotr Indyk,et al.  Scalable Nearest Neighbor Search for Optimal Transport , 2019, ICML.

[22]  R. McCann,et al.  Free boundaries in optimal transport and Monge-Ampere obstacle problems , 2010 .

[23]  Yang Zou,et al.  Sliced Wasserstein Kernels for Probability Distributions , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Alessandro Rudi,et al.  Massively scalable Sinkhorn distances via the Nyström method , 2018, NeurIPS.

[25]  Nicola Gigli,et al.  A new transportation distance between non-negative measures, with applications to gradients flows with Dirichlet boundary conditions , 2010 .

[26]  Alexander Mielke,et al.  Optimal Transport in Competition with Reaction: The Hellinger-Kantorovich Distance and Geodesic Curves , 2015, SIAM J. Math. Anal..

[27]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[28]  A. Figalli.  The Optimal Partial Transport Problem , 2010 .

[29]  Steve Oudot,et al.  Sliced Wasserstein Kernel for Persistence Diagrams , 2017, ICML.

[30]  K. Guittet.  Extended Kantorovich norms : a tool for optimization , 2001 .

[31]  Yann Brenier,et al.  A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem , 2000, Numerische Mathematik.

[32]  B. Piccoli,et al.  On Properties of the Generalized Wasserstein Distance , 2013, Archive for Rational Mechanics and Analysis.

[33]  Paul Tseng,et al.  On Computing the Nested Sums and Infimal Convolutions of Convex Piecewise-Linear Functions , 1996, J. Algorithms.

[34]  Gabriel Peyré,et al.  Stochastic Optimization for Large-scale Optimal Transport , 2016, NIPS.

[35]  Hisashi Kashima,et al.  Fast and Robust Comparison of Probability Measures in Heterogeneous Spaces , 2020, ArXiv.

[36]  Gabriel Peyré,et al.  Convolutional wasserstein distances , 2015, ACM Trans. Graph..

[37]  David Coeurjolly,et al.  SPOT , 2019, ACM Trans. Graph..

[38]  Matt J. Kusner,et al.  From Word Embeddings To Document Distances , 2015, ICML.

[39]  François-Xavier Vialard,et al.  Scaling algorithms for unbalanced optimal transport problems , 2017, Math. Comput..

[40]  S. Evans,et al.  The phylogenetic Kantorovich–Rubinstein metric for environmental sequence samples , 2010, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[41]  Gabriel Peyré,et al.  Fast Dictionary Learning with a Smoothed Wasserstein Loss , 2016, AISTATS.

[42]  L. Kantorovich.  On the Translocation of Masses , 2006 .

[43]  Russell Impagliazzo,et al.  On the Complexity of k-SAT , 2001, J. Comput. Syst. Sci..

[44]  Ryan Williams,et al.  Probabilistic Polynomials and Hamming Nearest Neighbors , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[45]  Mathieu Desbrun,et al.  Blue noise through optimal transport , 2012, ACM Trans. Graph..

[46]  Han Zhang,et al.  Improving GANs Using Optimal Transport , 2018, ICLR.

[47]  François-Xavier Vialard,et al.  An Interpolating Distance Between Optimal Transport and Fisher–Rao Metrics , 2010, Foundations of Computational Mathematics.

[48]  Piotr Indyk,et al.  On the Fine-Grained Complexity of Empirical Risk Minimization: Kernel Methods and Neural Networks , 2017, NIPS.

[49]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Julien Tierny,et al.  Progressive Wasserstein Barycenters of Persistence Diagrams , 2019, IEEE Transactions on Visualization and Computer Graphics.

[51]  Tomasz Malisiewicz,et al.  SuperPoint: Self-Supervised Interest Point Detection and Description , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[52]  Tomasz Malisiewicz,et al.  SuperGlue: Learning Feature Matching With Graph Neural Networks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Xavier Bresson,et al.  Local Histogram Based Segmentation Using the Wasserstein Distance , 2009, International Journal of Computer Vision.

[54]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[55]  Michael Werman,et al.  A Linear Time Histogram Metric for Improved SIFT Matching , 2008, ECCV.

[56]  B. Piccoli,et al.  Generalized Wasserstein Distance and its Application to Transport Equations with Source , 2012, 1206.3219.

[57]  P. Rigollet,et al.  Optimal-Transport Analysis of Single-Cell Gene Expression Identifies Developmental Trajectories in Reprogramming , 2019, Cell.

[58]  Julien Rabin,et al.  Wasserstein Barycenter and Its Application to Texture Mixing , 2011, SSVM.

[59]  Caroline Uhler,et al.  Scalable Unbalanced Optimal Transport using Generative Adversarial Networks , 2018, ICLR.

[60]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[61]  R. Knight,et al.  UniFrac: a New Phylogenetic Method for Comparing Microbial Communities , 2005, Applied and Environmental Microbiology.