Wasserstein distance error bounds for the multivariate normal approximation of the maximum likelihood estimator

We obtain explicit Wasserstein distance error bounds between the distribution of the multi-parameter MLE and the multivariate normal distribution. Our general bounds are given for possibly high-dimensional, independent and identically distributed random vectors. Our general bounds are of the optimal $\mathcal{O}(n^{-1/2})$ order. We apply our general bounds to derive Wasserstein distance error bounds for the multivariate normal approximation of the MLE in several settings; these being single-parameter exponential families, the normal distribution under canonical parametrisation, and the multivariate normal distribution under non-canonical parametrisation.

[1]  G. Peccati,et al.  The law of iterated logarithm for subordinated Gaussian sequences: uniform Wasserstein bounds , 2014, 1403.7003.

[2]  L_1 bounds for asymptotic normality of m-dependent sums using Stein's technique , 1974 .

[3]  I. Pinelis.  Optimal-order uniform and nonuniform bounds on the rate of convergence to normality for maximum likelihood estimators , 2017 .

[4]  Multivariate normal approximation of the maximum likelihood estimator via the delta method , 2016, 1609.03970.

[5]  Feng Qi (祁锋),et al.  TWO NEW PROOFS OF THE COMPLETE MONOTONICITY OF A FUNCTION INVOLVING THE PSI FUNCTION , 2009, 0902.2520.

[6]  Ronald F. Boisvert,et al.  NIST Handbook of Mathematical Functions , 2010 .

[7]  N. L. Johnson,et al.  Linear Statistical Inference and Its Applications , 1966 .

[8]  Andreas Anastasiou.  Assessing the multivariate normal approximation of the maximum likelihood estimator from high-dimensional, heterogeneous data , 2015, 1510.03679.

[9]  Michael I. Jordan.  Graphical Models , 2003 .

[10]  Andreas Anastasiou.  Bounds for the normal approximation of the maximum likelihood estimator , 2014, 1411.2391.

[11]  Yuta Koike.  High-Dimensional Central Limit Theorems for Homogeneous Sums , 2019, Journal of Theoretical Probability.

[12]  G. Styan,et al.  On the Existence and Uniqueness of the Maximum Likelihood Estimate of a Vector-Valued Parameter in Fixed-Size Samples , 1981 .

[13]  M. Kendall,et al.  The Advanced Theory of Statistics: Volume 1, Distribution Theory , 1978 .

[14]  Necdet Batır.  Inequalities for the inverses of the polygamma functions , 2017, 1705.06547.

[15]  Bounds for the asymptotic normality of the maximum likelihood estimator using the Delta method , 2015, 1508.04948.

[16]  Thomas A. Courtade,et al.  Existence of Stein Kernels under a Spectral Gap, and Discrepancy Bound , 2017, Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.

[17]  Robert E. Gaunt,et al.  Chi-square approximation by Stein's method with application to Pearson's statistic , 2015, 1507.01707.

[18]  Qi-Man Shao,et al.  Normal approximation for nonlinear statistics using a concentration inequality approach , 2007, 0708.4272.

[19]  Q. Shao,et al.  Multivariate approximations in Wasserstein distance by Stein’s method and Bismut’s formula , 2018, Probability Theory and Related Fields.

[20]  Yoav Zemel,et al.  Statistical Aspects of Wasserstein Distances , 2018, Annual Review of Statistics and Its Application.

[21]  A. Xia,et al.  On Stein’s Factors for Poisson Approximation in Wasserstein Distance with Nonlinear Transportation Costs , 2006, Journal of Theoretical Probability.

[22]  Thomas Bonis.  Stein’s method for normal approximation in Wasserstein distances with application to the multivariate central limit theorem , 2019, Probability Theory and Related Fields.

[23]  M. Reiß,et al.  Wasserstein and total variation distance between marginals of L\'evy processes , 2017, 1710.02715.

[24]  A. C. Davison,et al.  Statistical models: Name Index , 2003 .

[25]  Martin Raivc.  A multivariate central limit theorem for Lipschitz and smooth test functions , 2018, 1812.08268.

[26]  John Pike,et al.  Stein's Method and the Laplace Distribution , 2012, 1210.5775.

[27]  Nathan Ross.  Fundamentals of Stein's method , 2011, 1109.1880.

[28]  Yvik Swan,et al.  Regularity of solutions of the Stein equation and rates in the multivariate central limit theorem , 2018 .

[29]  C. Stein.  A bound for the error in the normal approximation to the distribution of a sum of dependent random variables , 1972 .

[30]  Andrzej Rucinski,et al.  A central limit theorem for decomposable random variables with applications to random graphs , 1989, J. Comb. Theory B.

[31]  C. Villani.  Optimal Transport: Old and New , 2008 .

[32]  P. Billingsley,et al.  Statistical Methods in Markov Chains , 1961 .

[33]  B. Hoadley.  Asymptotic Properties of Maximum Likelihood Estimators for the Independent Not Identically Distributed Case , 1971 .

[34]  Rory A. Fisher,et al.  Theory of Statistical Estimation , 1925, Mathematical Proceedings of the Cambridge Philosophical Society.

[35]  R. Fisher.  The Advanced Theory of Statistics , 1943, Nature.

[36]  G. Peccati,et al.  Multivariate Normal Approximation on the Wiener Space: New Bounds in the Convex Distance , 2020, Journal of Theoretical Probability.

[37]  Gesine Reinert,et al.  Couplings for normal approximations with Stein's method , 1997, Microsurveys in Discrete Probability.

[38]  I. Pinelis,et al.  Optimal-order bounds on the rate of convergence to normality in the multivariate delta method , 2009, 0906.0177.

[39]  M. Fathi.  Stein kernels and moment maps , 2018, The Annals of Probability.

[40]  H. Massam,et al.  Bayesian precision and covariance matrix estimation for graphical Gaussian models with edge and vertex symmetries , 2018 .

[41]  N. Yoshida.  Polynomial type large deviation inequalities and quasi-likelihood analysis for stochastic differential equations , 2011 .

[42]  E. Rio.  Upper bounds for minimal distances in the central limit theorem , 2009 .

[43]  Martin Raič,et al.  Normal Approximation by Stein ’ s Method , 2003 .