Learning Probability Measures with respect to Optimal Transport Metrics

We study the problem of estimating, in the sense of optimal transport metrics, a measure which is assumed supported on a manifold embedded in a Hilbert space. By establishing a precise connection between optimal transport metrics, optimal quantization, and learning theory, we derive new probabilistic bounds for the performance of a classic algorithm in unsupervised learning (k-means), when used to produce a probability measure derived from the data. In the course of the analysis, we arrive at new lower bounds, as well as probabilistic upper bounds on the convergence rate of the empirical law of large numbers, which, unlike existing bounds, are applicable to a wide class of measures.

[1]  J. Steele.  Probability theory and combinatorial optimization , 1987 .

[2]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[3]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[4]  P. Gruber,et al.  Optimum Quantization and Its Applications , 2004 .

[5]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[6]  Massimiliano Pontil,et al.  $K$ -Dimensional Coding Schemes in Hilbert Spaces , 2010, IEEE Transactions on Information Theory.

[7]  C. Villani,et al.  Quantitative Concentration Inequalities for Empirical Measures on Non-compact Spaces , 2005, math/0503123.

[8]  David Pollard,et al.  Quantization and the method of k -means , 1982, IEEE Trans. Inf. Theory.

[9]  Bruno Pelletier.  Kernel density estimation on Riemannian manifolds , 2005 .

[10]  C. Villani,et al.  Weighted Csiszár-Kullback-Pinsker inequalities and applications to transportation inequalities , 2005 .

[11]  J. Yukich,et al.  Asymptotics for transportation cost in high dimensions , 1995 .

[12]  C. Bordenave,et al.  Combinatorial Optimization Over Two Random Point Sets , 2011, 1103.2734.

[13]  J. Alonso,et al.  Convex and Discrete Geometry , 2009 .

[14]  C. Villani.  Optimal Transport: Old and New , 2008 .

[15]  Y. Ollivier.  Ricci curvature of Markov chains on metric spaces , 2007, math/0701886.

[16]  Daniela Rodriguez,et al.  Kernel Density Estimation on Riemannian Manifolds: Asymptotic Results , 2009, Journal of Mathematical Imaging and Vision.

[17]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[18]  Emmanuel Boissard.  Simple Bounds for the Convergence of Empirical and Occupation Measures in 1-Wasserstein Distance , 2011, 1103.3188.

[19]  Frédéric Chazal,et al.  Deconvolution for the Wasserstein Metric and Geometric Inference , 2011, GSI.

[20]  Amiel Feinstein,et al.  Information and information stability of random variables and processes , 1964 .

[21]  S. Graf,et al.  Foundations of Quantization for Probability Distributions , 2000 .

[22]  Kenneth L. Clarkson,et al.  Building triangulations using ε-nets , 2006, STOC '06.

[23]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[24]  M. Talagrand.  Transportation cost for Gaussian and other product measures , 1996 .

[25]  Alexander G. Gray,et al.  Submanifold density estimation , 2009, NIPS.

[26]  Joseph Horowitz,et al.  Mean rates of convergence of empirical measures in the Wasserstein metric , 1994 .

[27]  János Komlós,et al.  On optimal matchings , 1984, Comb..

[28]  Xavier Pennec,et al.  Intrinsic Statistics on Riemannian Manifolds: Basic Tools for Geometric Measurements , 2006, Journal of Mathematical Imaging and Vision.

[29]  Gordon Blower,et al.  The Gaussian Isoperimetric Inequality and Transportation , 2003 .

[30]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[31]  S. Bobkov,et al.  Exponential Integrability and Transportation Cost Related to Logarithmic Sobolev Inequalities , 1999 .

[32]  Jim Freeman.  Probability Metrics and the Stability of Stochastic Models , 1991 .

[33]  Luc Devroye,et al.  Combinatorial methods in density estimation , 2001, Springer series in statistics.

[34]  Harald Luschgy,et al.  DISTORTION MISMATCH IN THE QUANTIZATION OF PROBABILITY MEASURES , 2006, math/0602381.

[35]  Pascal Vincent,et al.  Manifold Parzen Windows , 2002, NIPS.