Scaling Algorithms for Unbalanced Transport Problems

This article introduces a new class of fast algorithms to approx-imate variational problems involving unbalanced optimal transport. While classical optimal transport considers only normalized probability distributions, it is important for many applications to be able to compute some sort of re-laxed transportation between arbitrary positive measures. A generic class of such “unbalanced” optimal transport problems has been recently proposed by several authors. In this paper, we show how to extend the, now classical, entropic regularization scheme to these unbalanced problems. This gives rise to fast, highly parallelizable algorithms that operate by performing only diagonal scaling (i.e. pointwise multiplications) of the transportation couplings. They are generalizations of the celebrated Sinkhorn algorithm. We show how these methods can be used to solve unbalanced transport, unbalanced gradient flows, and to compute unbalanced barycenters. We showcase applications to 2-D shape modification, color transfer, and growth models.

[1]  W. Deming,et al.  On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known , 1940 .

[2]  A. C. Thompson.  ON CERTAIN CONTRACTION MAPPINGS IN A PARTIALLY ORDERED VECTOR SPACE , 1963 .

[3]  Richard Sinkhorn.  A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[4]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[5]  R. Rockafellar,et al.  Duality and stability in extremum problems involving convex functions. , 1967 .

[6]  R. Rockafellar.  Integrals which are convex functionals. II , 1968 .

[7]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[8]  D. Bertsekas,et al.  The auction algorithm for the transportation problem , 1989 .

[9]  J. Lorenz,et al.  On the scaling of multidimensional matrices , 1989 .

[10]  Y. Brenier.  Polar Factorization and Monotone Rearrangement of Vector-Valued Functions , 1991 .

[11]  J. Borwein,et al.  Duality relationships for entropy-like minimization problems , 1991 .

[12]  L. Hanin.  Kantorovich-Rubinstein norm and its application in the theory of Lipschitz spaces , 1992 .

[13]  Jonathan Eckstein,et al.  Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming , 1993, Math. Oper. Res..

[14]  Alan L. Yuille,et al.  The invisible hand algorithm: Solving the assignment problem with statistical physics , 1994, Neural Networks.

[15]  D. Kinderlehrer,et al.  THE VARIATIONAL FORMULATION OF THE FOKKER-PLANCK EQUATION , 1996 .

[16]  Yair Censor,et al.  The Dykstra algorithm with Bregman projec-tions , 1998 .

[17]  Franz Aurenhammer,et al.  Minkowski-Type Theorems and Least-Squares Clustering , 1998, Algorithmica.

[18]  Mauro Dell'Amico,et al.  Assignment Problems , 1998, IFIP Congress: Fundamentals - Foundations of Computer Science.

[19]  Yann Brenier,et al.  A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem , 2000, Numerische Mathematik.

[20]  Heinz H. Bauschke,et al.  Dykstras algorithm with bregman projections: A convergence proof , 2000 .

[21]  F. Otto.  THE GEOMETRY OF DISSIPATIVE EVOLUTION EQUATIONS: THE POROUS MEDIUM EQUATION , 2001 .

[22]  C. Villani.  Topics in Optimal Transportation , 2003 .

[23]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[24]  L. Ambrosio,et al.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures , 2005 .

[25]  Julie Delon,et al.  Movie and video scale-time equalization application to flicker reduction , 2006, IEEE Transactions on Image Processing.

[26]  François Pitié,et al.  Automated colour grading using colour distribution transfer , 2007, Comput. Vis. Image Underst..

[27]  C. Villani.  Optimal Transport: Old and New , 2008 .

[28]  A. Galichon,et al.  Matching with Trade-Offs: Revealed Preferences Over Competing Characteristics , 2009, 2102.12811.

[29]  F. Santambrogio,et al.  A MACROSCOPIC CROWD MOTION MODEL OF GRADIENT FLOW TYPE , 2010, 1002.0686.

[30]  Christian L'eonard,et al.  O C ] 1 1 N ov 2 01 0 FROM THE SCHRÖDINGER PROBLEM TO THE MONGE-KANTOROVICH , 2010 .

[31]  R. McCann,et al.  Free boundaries in optimal transport and Monge-Ampere obstacle problems , 2010 .

[32]  A. Figalli.  The Optimal Partial Transport Problem , 2010 .

[33]  M. Burger,et al.  A mixed finite element method for nonlinear diffusion equations , 2010 .

[34]  S. Gaubert,et al.  A parallel preprocessing for the optimal assignment problem based on diagonal scaling , 2011 .

[35]  Guillaume Carlier,et al.  Barycenters in the Wasserstein Space , 2011, SIAM J. Math. Anal..

[36]  Quentin Mérigot,et al.  A Multiscale Approach to Optimal Transport , 2011, Comput. Graph. Forum.

[37]  Bertrand Maury,et al.  Handling congestion in crowd motion modeling , 2011, Networks Heterog. Media.

[38]  M. V. D. Panne,et al.  Displacement Interpolation Using Lagrangian Mass Transport , 2011 .

[39]  Carola-Bibiane Schönlieb,et al.  Regularized Regression and Density Estimation based on Optimal Transport , 2012 .

[40]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[41]  Christian L'eonard.  A survey of the Schr\"odinger problem and some of its connections with optimal transport , 2013, 1308.0215.

[42]  J. Carrillo,et al.  A Finite-Volume Method for Nonlinear Nonlocal Equations with a Gradient Flow Structure , 2014, 1402.4252.

[43]  Leonidas J. Guibas,et al.  Wasserstein Propagation for Semi-Supervised Learning , 2014, ICML.

[44]  B. Piccoli,et al.  Generalized Wasserstein Distance and its Application to Transport Equations with Source , 2012, 1206.3219.

[45]  B. Perthame,et al.  The Hele–Shaw Asymptotics for Mechanical Models of Tumor Growth , 2013, Archive for Rational Mechanics and Analysis.

[46]  Gabriel Peyré,et al.  Optimal Transport with Proximal Splitting , 2013, SIAM J. Imaging Sci..

[47]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[48]  B. Lévy.  A Numerical Algorithm for L2 Semi-Discrete Optimal Transport in 3D , 2014, 1409.1279.

[49]  Arindam Banerjee,et al.  Bregman Alternating Direction Method of Multipliers , 2013, NIPS.

[50]  Jonathan Zinsl,et al.  Transport distances and geodesic convexity for systems of degenerate diffusion equations , 2014, 1409.6520.

[51]  Q. Nguyen.  Forward-Backward Splitting with Bregman Distances , 2015, 1505.05198.

[52]  G. Peyré,et al.  Unbalanced Optimal Transport: Geometry and Kantorovich Formulation , 2015 .

[53]  Julien Rabin,et al.  Convex Color Image Segmentation with Optimal Transport Distances , 2015, SSVM.

[54]  M. Rumpf,et al.  A generalized model for optimal transport of images including dissipation and density modulation , 2015, 1504.01988.

[55]  Gabriel Peyré,et al.  Entropic Approximation of Wasserstein Gradient Flows , 2015, SIAM J. Imaging Sci..

[56]  F. Santambrogio.  Optimal Transport for Applied Mathematicians: Calculus of Variations, PDEs, and Modeling , 2015 .

[57]  M. Laborde.  On some non linear evolution systems which are perturbations of Wasserstein gradient flows , 2015, 1506.00126.

[58]  M. Agueh,et al.  Optimal transport for particle image velocimetry , 2015 .

[59]  Giuseppe Savaré,et al.  Optimal Entropy-Transport problems and a new Hellinger–Kantorovich distance between positive measures , 2015, 1508.07941.

[60]  Filippo Santambrogio,et al.  Optimal Transport for Applied Mathematicians , 2015 .

[61]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[62]  Gabriel Peyré,et al.  Iterative Bregman Projections for Regularized Transportation Problems , 2014, SIAM J. Sci. Comput..

[63]  Amir Beck,et al.  On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes , 2015, SIAM J. Optim..

[64]  Alexander Mielke,et al.  Optimal Transport in Competition with Reaction: The Hellinger-Kantorovich Distance and Geodesic Curves , 2015, SIAM J. Math. Anal..

[65]  Antonin Chambolle,et al.  On the ergodic convergence rates of a first-order primal–dual algorithm , 2016, Math. Program..

[66]  Quentin Mérigot,et al.  Discretization of functionals involving the Monge–Ampère operator , 2014, Numerische Mathematik.

[67]  Jean-David Benamou,et al.  An augmented Lagrangian approach to Wasserstein gradient flows and applications , 2016 .

[68]  Bernhard Schmitzer,et al.  A Sparse Multiscale Algorithm for Dense Optimal Transport , 2015, Journal of Mathematical Imaging and Vision.

[69]  Gabriel Peyré,et al.  Convergence of Entropic Schemes for Optimal Transport and Gradient Flows , 2015, SIAM J. Math. Anal..

[70]  François-Xavier Vialard,et al.  An Interpolating Distance Between Optimal Transport and Fisher–Rao Metrics , 2010, Foundations of Computational Mathematics.