Inference for empirical Wasserstein distances on finite spaces

The Wasserstein distance is an attractive tool for data analysis but statistical inference is hindered by the lack of distributional limits. To overcome this obstacle, for probability measures supported on finitely many points, we derive the asymptotic distribution of empirical Wasserstein distances as the optimal value of a linear programme with random objective function. This facilitates statistical inference (e.g. confidence intervals for sample‐based Wasserstein distances) in large generality. Our proof is based on directional Hadamard differentiability. Failure of the classical bootstrap and alternatives are discussed. The utility of the distributional results is illustrated on two data sets.

[1]  R. Dobrushin Prescribing a System of Random Variables by Conditional Distributions , 1970 .

[2]  C. Mallows A Note on Asymptotic Joint Normality , 1972 .

[3]  D. Freedman,et al.  Some Asymptotic Theory for the Bootstrap , 1981 .

[4]  R. Rockafellar Directional differentiability of the optimal value function in a nonlinear programming problem , 1984 .

[5]  János Komlós,et al.  On optimal matchings , 1984, Comb..

[6]  S. Rachev The Monge–Kantorovich Mass Transference Problem and Its Stochastic Applications , 1985 .

[7]  R. Gill Non- and semi-parametric maximum likelihood estimators and the Von Mises method , 1986 .

[8]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[9]  N. Bingham EMPIRICAL PROCESSES WITH APPLICATIONS TO STATISTICS (Wiley Series in Probability and Mathematical Statistics) , 1987 .

[10]  J. Wellner,et al.  Empirical Processes with Applications to Statistics , 2009 .

[11]  Robert M. Gray,et al.  Probability, Random Processes, And Ergodic Properties , 1987 .

[12]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[13]  D. Donoho,et al.  Pathologies of some Minimum Distance Estimators , 1988 .

[14]  M. Gelbrich On a Formula for the L2 Wasserstein Metric between Measures on Euclidean and Hilbert Spaces , 1990 .

[15]  Alexander Shapiro,et al.  Asymptotic analysis of stochastic programs , 1991, Ann. Oper. Res..

[16]  A. Shapiro Perturbation analysis of optimization problems in banach spaces , 1992 .

[17]  Michel Talagrand,et al.  Matching Random Samples in Many Dimensions , 1992 .

[18]  Lutz Diimbgen On nondifferentiable functions and the bootstrap , 1993 .

[19]  N. H. Anderson,et al.  Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates , 1994 .

[20]  Joseph Horowitz,et al.  Mean rates of convergence of empirical measures in the Wasserstein metric , 1994 .

[21]  M. Talagrand THE TRANSPORTATION COST FROM THE UNIFORM MEASURE TO THE EMPIRICAL MEASURE IN DIMENSION > 3 , 1994 .

[22]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[23]  D. Kinderlehrer,et al.  THE VARIATIONAL FORMULATION OF THE FOKKER-PLANCK EQUATION , 1996 .

[24]  J. Cima,et al.  On weak* convergence in ¹ , 1996 .

[25]  C. Czado,et al.  Nonparametric validation of similar distributions and assessment of goodness of fit , 1998 .

[26]  John M. Wilson,et al.  Advances in Sensitivity Analysis and Parametric Programming , 1998, J. Oper. Res. Soc..

[27]  S. Rachev,et al.  Mass transportation problems , 1998 .

[28]  J. A. Cuesta-Albertos,et al.  Tests of goodness of fit based on the $L_2$-Wasserstein distance , 1999 .

[29]  Dario Maio,et al.  Synthetic fingerprint-image generation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[30]  W. Gangbo,et al.  Shape recognition via Wasserstein distance , 2000 .

[31]  F. Otto THE GEOMETRY OF DISSIPATIVE EVOLUTION EQUATIONS: THE POROUS MEDIUM EQUATION , 2001 .

[32]  Anil K. Jain,et al.  FVC2002: Second Fingerprint Verification Competition , 2002, Object recognition supported by user interaction for service robots.

[33]  Halil Mete Soner,et al.  Mathematical Aspects of Evolving Interfaces , 2003 .

[34]  L. Ambrosio Lecture Notes on Optimal Transport Problems , 2003 .

[35]  C. Villani Topics in Optimal Transportation , 2003 .

[36]  Anil K. Jain,et al.  Handbook of Fingerprint Recognition , 2005, Springer Professional Computing.

[37]  O. Mangasarian On Concepts of Directional Differentiability , 2004 .

[38]  R. Samworth,et al.  Central limit theorem and convergence to stable laws in Mallows distance , 2004, math/0406218.

[39]  Larry Wasserman,et al.  All of Statistics , 2004 .

[40]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[41]  R. Samworth,et al.  Convergence of the empirical process in Mallows distance, with an application to bootstrap performance , 2004, math/0406603.

[42]  E. Giné,et al.  Asymptotics for L2 functionals of the empirical quantile process, with applications to tests of fit based on weighted Wasserstein distances , 2005 .

[43]  P. Rosenbaum An exact distribution‐free test comparing two multivariate distributions based on adjacency , 2005 .

[44]  A. Munk,et al.  On Hadamard differentiability in k -sample semiparametric models: with applications to the assessment of structural relationships , 2005 .

[45]  R. Samworth,et al.  The empirical process in Mallows distance, with application to goodness-of-fit tests , 2005, math/0504424.

[46]  Werner Römisch,et al.  Delta Method, Infinite Dimensional , 2006 .

[47]  C. Matr,et al.  Tests of Goodness of Fit Based on the L 2 -wasserstein Distance , 2007 .

[48]  C. Czado,et al.  A nonparametric test for similarity of marginals—With applications to the assessment of population bioequivalence , 2007 .

[49]  Anil K. Jain Technology: Biometric recognition , 2007, Nature.

[50]  R. Knight,et al.  The Human Microbiome Project , 2007, Nature.

[51]  Xavier Bresson,et al.  Local Histogram Based Segmentation Using the Wasserstein Distance , 2009, International Journal of Computer Vision.

[52]  Martin Hartmann,et al.  Introducing mothur: Open-Source, Platform-Independent, Community-Supported Software for Describing and Comparing Microbial Communities , 2009, Applied and Environmental Microbiology.

[53]  R. Knight,et al.  Bacterial Community Variation in Human Body Habitats Across Space and Time , 2009, Science.

[54]  R. Knight,et al.  Microbial community resemblance methods differ in their ability to detect biologically relevant patterns , 2010, Nature Methods.

[55]  Abhishek Halder,et al.  Model validation: A probabilistic formulation , 2011, IEEE Conference on Decision and Control and European Control Conference.

[56]  Guillaume Carlier,et al.  Barycenters in the Wasserstein Space , 2011, SIAM J. Math. Anal..

[57]  L. Duembgen,et al.  APPROXIMATION BY LOG-CONCAVE DISTRIBUTIONS, WITH APPLICATIONS TO REGRESSION , 2010, 1002.3448.

[58]  Thibaut Le Gouic,et al.  Distribution's template estimate with Wasserstein metrics , 2011, 1111.5927.

[59]  Thibaut Le Gouic,et al.  On the mean speed of convergence of empirical and occupation measures in Wasserstein distance , 2011, 1105.5263.

[60]  Patrick D. Schloss,et al.  Reducing the Effects of PCR Amplification and Sequencing Artifacts on 16S rRNA-Based Studies , 2011, PloS one.

[61]  Pascal Bianchi,et al.  Classification of Periodic Activities Using the Wasserstein Distance , 2012, IEEE Transactions on Biomedical Engineering.

[62]  C. Dorea,et al.  Conditions for equivalence between Mallows distance and convergence to stable laws , 2012 .

[63]  Paul-Marie Samson,et al.  Displacement convexity of entropy and related inequalities on graphs , 2012, Probability Theory and Related Fields.

[64]  J. Maas,et al.  Ricci Curvature of Finite Markov Chains via Convexity of the Entropy , 2011, 1111.2687.

[65]  S. Evans,et al.  The phylogenetic Kantorovich–Rubinstein metric for environmental sequence samples , 2010, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[66]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[67]  Ambuj K. Singh,et al.  Quantifying spatial relationships from whole retinal images , 2013, Bioinform..

[68]  A. Guillin,et al.  On the rate of convergence in Wasserstein distance of the empirical measure , 2013, 1312.2128.

[69]  J'er'emie Bigot,et al.  Geodesic PCA in the Wasserstein space , 2013, 1307.7721.

[70]  Carsten Gottschlich,et al.  The Shortlist Method for Fast Computation of the Earth Mover's Distance and Finding Optimal Solutions to Transportation Problems , 2014, PloS one.

[71]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[72]  Carsten Gottschlich,et al.  Separating the real from the synthetic: minutiae histograms as fingerprints of fingerprints , 2013, IET Biom..

[73]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[74]  Julien Rabin,et al.  Sliced and Radon Wasserstein Barycenters of Measures , 2014, Journal of Mathematical Imaging and Vision.

[75]  Jean-Michel Loubes,et al.  A parametric registration model for warped distributions with Wasserstein's distance , 2015, J. Multivar. Anal..

[76]  E. Barrio,et al.  A statistical analysis of a deformation model with Wasserstein barycenters : estimation procedure and goodness of fit test , 2015, 1508.06465.

[77]  An Algorithmic Approach to Compute Principal Geodesics in the Wasserstein Space , 2015 .

[78]  Benoît R. Kloeckner A geometric study of Wasserstein spaces: ultrametrics , 2013, 1304.5219.

[79]  Marco Cuturi,et al.  Principal Geodesic Analysis for Probability Measures under the Optimal Transport Metric , 2015, NIPS.

[80]  A. Kasue,et al.  EXPANSION CONSTANTS AND HYPERBOLIC EMBEDDINGS OF FINITE GRAPHS , 2015 .

[81]  D. Mason A Weighted Approximation Approach to the Study of the Empirical Wasserstein Distance , 2016 .

[82]  Axel Munk,et al.  Limit laws of the empirical Wasserstein distance: Gaussian distributions , 2015, J. Multivar. Anal..

[83]  Phillipp Kaestner,et al.  Linear And Nonlinear Programming , 2016 .

[84]  G. Walther,et al.  Earth Mover’s Distance (EMD): A True Metric for Comparing Biomarker Expression Levels in Cell Populations , 2016, PloS one.

[85]  Benjamin Pfaff,et al.  Perturbation Analysis Of Optimization Problems , 2016 .

[86]  Jérémie Bigot,et al.  Geodesic PCA in the Wasserstein space by Convex PCA , 2017 .

[87]  David B. Dunson,et al.  Scalable Bayes via Barycenter in Wasserstein Space , 2015, J. Mach. Learn. Res..

[88]  Andrés Santos,et al.  Inference on Directionally Differentiable Functions , 2014, The Review of Economic Studies.