A note on overrelaxation in the Sinkhorn algorithm

We derive an a priori parameter range for overrelaxation of the Sinkhorn algorithm, which guarantees global convergence and a strictly faster asymptotic local convergence. Guided by the spectral analysis of the linearized problem we pursue a zero cost procedure to choose a near optimal relaxation parameter.

[1]  Roger D. Nussbaum,et al.  An elementary proof of the Birkhoff-Hopf theorem , 1995, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  Nicolas Papadakis,et al.  Overrelaxed Sinkhorn-Knopp Algorithm for Regularized Optimal Transport , 2017, Algorithms.

[3]  Richard Sinkhorn,et al.  Concerning nonnegative matrices and doubly stochastic matrices , 1967 .

[4]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[5]  D. Lorenz,et al.  A Sinkhorn-Newton method for entropic optimal transport , 2017, 1710.06635.

[6]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[7]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[8]  J. Lorenz,et al.  On the scaling of multidimensional matrices , 1989 .

[9]  Richard Sinkhorn.  Diagonal equivalence to matrices with prescribed row and column sums. II , 1967 .

[10]  Daniel Ruiz,et al.  A Fast Algorithm for Matrix Balancing , 2013, Web Information Retrieval and Linear Algebra Algorithms.

[11]  J. Solomon,et al.  Quantum entropic regularization of matrix-valued optimal transport , 2017, European Journal of Applied Mathematics.

[12]  Codina Cotar,et al.  Density Functional Theory and Optimal Transportation with Coulomb Cost , 2011, 1104.0603.

[13]  T. A. Porsching,et al.  Aspects of Nonlinear Block Successive Overrelaxation , 1975 .

[14]  Philip A. Knight,et al.  The Sinkhorn-Knopp Algorithm: Convergence and Applications , 2008, SIAM J. Matrix Anal. Appl..

[15]  V. Pawlowsky-Glahn,et al.  Modeling and Analysis of Compositional Data , 2015 .

[16]  W. Hackbusch.  Iterative Solution of Large Sparse Systems of Equations , 1993 .

[17]  C. Barceló-Vidal,et al.  The mathematics of compositional analysis , 2016 .

[18]  Nicolas Courty,et al.  POT: Python Optimal Transport , 2021, J. Mach. Learn. Res..

[19]  Richard Sinkhorn.  Diagonal equivalence to matrices with prescribed row and column sums. II , 1974 .