Convergence rates of posterior distributions for non-i.i.d. observations

We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion. We then specialize our results to independent, nonidentically distributed observations, Markov processes, stationary Gaussian time series and the white noise model. We apply our general results to several examples of infinite-dimensional statistical models including nonparametric regression with normal errors, binary regression, Poisson regression, an interval censoring model, Whittle estimation of the spectral density of a time series and a nonlinear autoregressive model. © Institute of Mathematical Statistics, 2007.

[1]  Le Cam,et al.  On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates , 1953 .

[2]  J. Doob.  Stochastic processes , 1953 .

[3]  P. Whittle.  Curve and Periodogram Smoothing , 1957 .

[4]  I. Ibragimov,et al.  Some Limit Theorems for Stationary Processes , 1962 .

[5]  Michel Loève,et al.  Probability Theory I , 1977 .

[6]  L. Schwartz.  On Bayes procedures , 1965 .

[7]  Richard A. Johnson.  Asymptotic Expansions Associated with Posterior Distributions , 1970 .

[8]  L. Lecam.  Convergence of Estimates Under Dimensionality Restrictions , 1973 .

[9]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[10]  H. Teicher,et al.  Probability theory: Independence, interchangeability, martingales , 1978 .

[11]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[12]  Lucien Birgé,et al.  Robust Testing for Independent Non Identically Distributed Variables and Markov Chains , 1983 .

[13]  Lucien Birgé.  Approximation dans les espaces métriques et théorie de l'estimation , 1983 .

[14]  Albert Y. Lo,et al.  Weak convergence for Dirichlet processes , 1983 .

[15]  Hani Doss.  Bayesian Nonparametric Estimation of the Median; Part II: Asymptotic Properties of the Estimates , 1985 .

[16]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[17]  R. Dahlhaus.  Empirical spectral processes and their applications to time series analysis , 1988 .

[18]  C. J. Stone,et al.  Large-Sample Inference for Log-Spline Models , 1990 .

[19]  D. Pollard.  Empirical Processes: Theory and Applications , 1990 .

[20]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[21]  Richard A. Davis,et al.  Time Series: Theory and Methods (2nd ed.). , 1992 .

[22]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[23]  Emmanuel Rio,et al.  Covariance inequalities for strongly mixing processes , 1993 .

[24]  D. Cox.  An Analysis of Bayesian Inference for Nonparametric Regression , 1993 .

[25]  C. J. Stone,et al.  The Use of Polynomial Splines and Their Tensor Products in Multivariate Function Estimation , 1994 .

[26]  K. Do,et al.  Efficient and Adaptive Estimation for Semiparametric Models. , 1994 .

[27]  M. Schervish.  Theory of Statistics , 1995 .

[28]  W. Wong,et al.  Probability inequalities for likelihood ratios and convergence rates of sieve MLEs , 1995 .

[29]  J. Ghosh,et al.  On convergence of posterior distributions , 1995 .

[30]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[31]  Piet Groeneboom,et al.  Lectures on inverse problems , 1996 .

[32]  Larry Wasserman,et al.  Asymptotic Properties of Nonparametric Bayesian Procedures , 1998 .

[33]  D. Freedman.  On the Bernstein-von Mises Theorem with Infinite Dimensional Parameters , 1999 .

[34]  L. Wasserman,et al.  The consistency of posterior distributions in nonparametric problems , 1999 .

[35]  J. Ghosh,et al.  Consistent semiparametric Bayesian inference about a location parameter , 1999 .

[36]  J. Ghosh,et al.  POSTERIOR CONSISTENCY OF DIRICHLET MIXTURES IN DENSITY ESTIMATION , 1999 .

[37]  Sonia Petrone.  Bayesian density estimation using bernstein polynomials , 1999 .

[38]  Linda H. Zhao.  Bayesian aspects of some nonparametric problems , 2000 .

[39]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions , 2000 .

[40]  L. Wasserman,et al.  Rates of convergence of posterior distributions , 2001 .

[41]  S. Ghosal.  Convergence rates for density estimation with Bernstein polynomials , 2001 .

[42]  A. V. D. Vaart,et al.  Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities , 2001 .

[43]  Xiaotong Shen.  Asymptotic Normality of Semiparametric and Nonparametric Posterior Distributions , 2002 .

[44]  S. Ghosal,et al.  Bayesian Estimation of the Spectral Density of a Time Series , 2004 .

[45]  S. Ghosal,et al.  Contiguity of the Whittle measure for a Gaussian time series , 2004 .

[46]  Van Der Vaart,et al.  Convergence rates of posterior distributions for Brownian semimartingale models , 2006 .

[47]  A. V. D. Vaart,et al.  Misspecification in infinite-dimensional Bayesian statistics , 2006, math/0607023.

[48]  L. Birge,et al.  Model selection via testing: an alternative to (penalized) maximum likelihood estimators , 2006 .

[49]  A. V. D. Vaart,et al.  Posterior convergence rates of Dirichlet mixtures at smooth densities , 2007, 0708.1885.