The consistency of posterior distributions in nonparametric problems

We give conditions that guarantee that the posterior probability of every Hellinger neighborhood of the true distribution tends to 1 almost surely. The conditions are (1) a requirement that the prior not put high mass near distributions with very rough densities and (2) a requirement that the prior put positive mass in Kullback-Leibler neighborhoods of the true distribution. The results are based on the idea of approximating the set of distributions with a finite-dimensional set of distributions with sufficiently small Hellinger bracketing metric entropy. We apply the results to some examples.

[1]  J. Fabius.  Asymptotic behavior of bayes' estimates , 1963 .

[2]  D. Freedman.  On the Asymptotic Behavior of Bayes' Estimates in the Discrete Case , 1963 .

[3]  C. Kraft.  A class of distribution function processes which have derivatives , 1964, Journal of Applied Probability.

[4]  Tom Leonard.  Density Estimation, Stochastic Processes and Prior Information , 1978 .

[5]  S. Geman,et al.  Nonparametric Maximum Likelihood Estimation by the Method of Sieves , 1982 .

[6]  K. Alexander,et al.  Probability Inequalities for Empirical Processes and a Law of the Iterated Logarithm , 1984 .

[7]  D. Freedman,et al.  On the consistency of Bayes estimates , 1986 .

[8]  P. Lenk.  The Logistic Normal Distribution for Bayesian, Nonparametric, Predictive Densities , 1988 .

[9]  Grace L. Yang,et al.  On Bayes Procedures , 1990 .

[10]  Peter J. Lenk,et al.  Towards a practicable Bayesian nonparametric density estimator , 1991 .

[11]  W. Sudderth,et al.  Polya Trees and Random Distributions , 1992 .

[12]  S. Geer.  Hellinger-Consistency of Certain Nonparametric Maximum Likelihood Estimators , 1993 .

[13]  M. Lavine.  More Aspects of Polya Tree Distributions for Statistical Modelling , 1992 .

[14]  M. Schervish.  Theory of Statistics , 1995 .

[15]  W. Wong,et al.  Probability inequalities for likelihood ratios and convergence rates of sieve MLEs , 1995 .

[16]  D. Freedman,et al.  Consistency of Bayes Estimates for Nonparametric Regression: A Review , 1997 .

[17]  Tj Sweeting,et al.  Invited discussion of A. R. Barron: Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems , 1998 .

[18]  D. Freedman,et al.  Consistency of Bayes estimates for nonparametric regression: normal theory , 1998 .

[19]  J. Ghosh,et al.  Consistent semiparametric Bayesian inference about a location parameter , 1999 .

[20]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions , 2000 .

[21]  L. Wasserman,et al.  Rates of convergence of posterior distributions , 2001 .