On Choosing and Bounding Probability

Summary When studying convergence of measures, an important issue is the choice of probability metric. We provide a summary and some new results concerning bounds among some important probability metrics/distances that are used by statisticians and probabilists. Knowledge of other metrics can provide a means of deriving bounds for another one in an applied problem. Considering other metrics can also provide alternate insights. We also give examples that show that rates of convergence can strongly depend on the metric chosen. Careful consideration is necessary when choosing a metric.

[1]  P. Diaconis,et al.  LOGARITHMIC SOBOLEV INEQUALITIES FOR FINITE MARKOV CHAINS , 1996 .

[2]  A. Szulga.  On Minimal Metrics in the Space of Random Variables , 1983 .

[3]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[4]  J. Hartigan.  The maximum likelihood prior , 1998 .

[5]  F. Su.  Discrepancy Convergence for the Drunkard's Walk on the Sphere , 2001, math/0102205.

[6]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[7]  Yu. V. Prokhorov.  Convergence of Random Processes and Limit Theorems in Probability Theory , 1956 .

[8]  B. Lindsay.  Efficiency versus robustness : the case for minimum Hellinger distance and related methods , 1994 .

[9]  Walter R. Gilks,et al.  Introduction to general state-space Markov chain theory , 1995 .

[10]  T. N. Sriram.  Asymptotics in Statistics–Some Basic Concepts , 2002 .

[11]  R. Dudley.  Distances of Probability Measures and Random Variables , 1968 .

[12]  Alison L. Gibbs,et al.  Convergence of Markov chain Monte Carlo algorithms with applications to image restoration , 2000 .

[13]  R. Reiss.  Approximate Distributions of Order Statistics , 1989 .

[14]  F. Su.  Convergence of random walks on the circle generated by an irrational rotation , 1998 .

[15]  Ludger Rüschendorf,et al.  Distributions with fixed marginals and related topics , 1999 .

[16]  S. Orey.  Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities , 1971 .

[17]  Jim Freeman.  Probability Metrics and the Stability of Stochastic Models , 1991 .

[18]  V. Strassen.  The Existence of Probability Measures with Given Marginals , 1965 .

[19]  I. Vajda,et al.  Convex Statistical Distances , 2018, Statistical Inference for Engineers and Data Scientists.

[20]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[21]  P. Diaconis,et al.  Updating Subjective Probability , 1982 .

[22]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[23]  L. Cam,et al.  Théorie asymptotique de la décision statistique , 1969 .

[24]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[25]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[26]  S. Kullback,et al.  A lower bound for discrimination information in terms of variation (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[27]  S. Kakutani.  On Equivalence of Infinite Product Measures , 1948 .

[28]  V. V. Petrov.  Limit Theorems of Probability Theory: Sequences of Independent Random Variables , 1995 .

[29]  J. Linnik.  An Information-Theoretic Proof of the Central Limit Theorem with Lindeberg Conditions , 1959 .

[30]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[31]  P. Diaconis.  Group representations in probability and statistics , 1988 .

[32]  A. Shiryaev,et al.  Limit Theorems for Stochastic Processes , 1987 .

[33]  P. Diaconis,et al.  Strong uniform times and finite random walks , 1987 .

[34]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[35]  T. Lindvall.  Lectures on the Coupling Method , 1992 .

[36]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[37]  V. M. Zolotarev,et al.  Addendum: Probability Metrics , 1984 .