Multiscale Strategies for Computing Optimal Transport

This paper presents a multiscale approach to efficiently compute approximate optimal transport plans between point sets. It is particularly well-suited for point sets that are in high-dimensions, but are close to being intrinsically low-dimensional. The approach is based on an adaptive multiscale decomposition of the point sets. The multiscale decomposition yields a sequence of optimal transport problems, that are solved in a top-to-bottom fashion from the coarsest to the finest scale. We provide numerical evidence that this multiscale approach scales approximately linearly, in time and memory, in the number of nodes, instead of quadratically or worse for a direct solution. Empirically, the multiscale approach results in less than one percent relative error in the objective function. Furthermore, the multiscale plans constructed are of interest by themselves as they may be used to introduce novel features and notions of distances between point sets. An analysis of sets of brain MRI based on optimal transport distances illustrates the effectiveness of the proposed method on a real world data set. The application demonstrates that multiscale optimal transport distances have the potential to improve on state-of-the-art metrics currently used in computational anatomy.

[1]  George B. Dantzig,et al.  Decomposition Principle for Linear Programs , 1960 .

[2]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Adam M. Oberman,et al.  Numerical solution of the Optimal Transportation problem using the Monge-Ampère equation , 2012, J. Comput. Phys..

[4]  Jacques Desrosiers,et al.  Accelerating Strategies in Column Generation Methods for Vehicle Routing and Crew Scheduling Problems , 2002 .

[5]  L. Younes,et al.  On the metrics and euler-lagrange equations of computational anatomy. , 2002, Annual review of biomedical engineering.

[6]  Gabriel Peyré,et al.  Optimal Transport with Proximal Splitting , 2013, SIAM J. Imaging Sci..

[7]  L. Kantorovich.  On the Translocation of Masses , 2006 .

[8]  Eiji Oki,et al.  GLPK (GNU Linear Programming Kit) , 2012 .

[9]  Leonidas J. Guibas,et al.  Functional map networks for analyzing and exploring large shape collections , 2014, ACM Trans. Graph..

[10]  M. Klein.  A Primal Method for Minimal Cost Flows with Applications to the Assignment and Transportation Problems , 1966 .

[11]  T. Glimm,et al.  Iterative Scheme for Solving Optimal Transportation Problems Arising in Reflector Design , 2011, 1110.3061.

[12]  Alexandr Andoni,et al.  Earth mover distance over high-dimensional spaces , 2008, SODA '08.

[13]  Leonidas J. Guibas,et al.  Consistent Shape Maps via Semidefinite Programming , 2013, SGP '13.

[14]  H. Kuhn.  The Hungarian method for the assignment problem , 1955 .

[15]  D. R. Fulkerson,et al.  Solving a Transportation Problem , 1956 .

[16]  L. Rosasco,et al.  Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature , 2017 .

[17]  Chris L. Jackins,et al.  Oct-trees and their use in representing three-dimensional objects , 1980 .

[18]  M. Cullen.  A Mathematical Theory of Large-scale Atmosphere/ocean Flow , 2006 .

[19]  David K. Smith.  Network Flows: Theory, Algorithms, and Applications , 1994 .

[20]  Christoph Schnörr,et al.  A Hierarchical Approach to Optimal Transport , 2013, SSVM.

[21]  Arthur Cayley,et al.  The Collected Mathematical Papers: On Monge's “Mémoire sur la théorie des déblais et des remblais” , 2009 .

[22]  Mauro Maggioni,et al.  Approximation of Points on Low-Dimensional Manifolds Via Random Linear Projections , 2012, ArXiv.

[24]  Mauro Maggioni,et al.  Adaptive Geometric Multiscale Approximations for Intrinsically Low-dimensional Data , 2016, Journal of machine learning research.

[25]  James B. Orlin,et al.  A polynomial time primal network simplex algorithm for minimum cost flows , 1996, SODA '96.

[26]  Andrew V. Goldberg,et al.  Solving minimum-cost flow problems by successive approximation , 1987, STOC.

[27]  Franz Aurenhammer,et al.  Minkowski-Type Theorems and Least-Squares Clustering , 1998, Algorithmica.

[28]  Quentin Mérigot,et al.  A Multiscale Approach to Optimal Transport , 2011, Comput. Graph. Forum.

[29]  Y. Brenier.  Polar Factorization and Monotone Rearrangement of Vector-Valued Functions , 1991 .

[30]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[31]  Ross T. Whitaker,et al.  Manifold modeling for brain population analysis , 2010, Medical Image Anal..

[32]  Robert E. Tarjan,et al.  Dynamic trees as search trees via euler tours, applied to the network simplex algorithm , 1997, Math. Program..

[33]  David W. Jacobs,et al.  Approximate earth mover’s distance in linear time , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[35]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transportation , 2013, NIPS 2013.

[36]  Robert Krauthgamer,et al.  Navigating nets: simple algorithms for proximity search , 2004, SODA '04.

[37]  Lei Zhu,et al.  Optimal Mass Transport for Registration and Warping , 2004, International Journal of Computer Vision.

[38]  M. Beckmann.  A Continuous Model of Transportation , 1952 .

[39]  Jon Louis Bentley,et al.  Quad trees a data structure for retrieval on composite keys , 1974, Acta Informatica.

[40]  William H. Cunningham,et al.  A network simplex method , 1976, Math. Program..

[41]  John Langford,et al.  Cover trees for nearest neighbor , 2006, ICML.

[42]  Angelo Iollo,et al.  A lagrangian scheme for the solution of the optimal mass transfer problem , 2011, J. Comput. Phys..

[43]  J. Desrosiers,et al.  A Primer in Column Generation , 2005 .

[44]  Paul Tseng,et al.  Relaxation Methods for Minimum Cost Ordinary and Generalized Network Flow Problems , 1988, Oper. Res..

[45]  Patrice Assouad.  Plongements lipschitziens dans ${\mathbb {R}}^n$ , 1983 .

[46]  Guangliang Chen,et al.  A fast multiscale framework for data in high-dimensions: Measure estimation, anomaly detection, and compressive measurements , 2012, 2012 Visual Communications and Image Processing.

[47]  M. Maggioni,et al.  Multiscale Geometric Methods for Data Sets II: Geometric Multi-Resolution Analysis , 2011, 1105.4924.

[48]  David Avis,et al.  Ground metric learning , 2011, J. Mach. Learn. Res..

[49]  Yann Brenier,et al.  A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem , 2000, Numerische Mathematik.

[50]  Leonidas J. Guibas,et al.  One Point Isometric Matching with the Heat Kernel , 2010, Comput. Graph. Forum.

[51]  Guillaume Carlier,et al.  Optimal Transportation with Traffic Congestion and Wardrop Equilibria , 2006, SIAM J. Control. Optim..

[52]  Steven Haker,et al.  Minimizing Flows for the Monge-Kantorovich Problem , 2003, SIAM J. Math. Anal..

[53]  Bernhard Schmitzer.  A sparse algorithm for dense optimal transport , 2015, SSVM.

[54]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[55]  Adam M. Oberman,et al.  An efficient linear programming method for Optimal Transportation , 2015, 1509.03668.