Bernstein von Mises Theorems for Gaussian Regression with increasing number of regressors

This paper brings a contribution to the Bayesian theory of nonparametric and semiparametric estimation. We are interested in the asymptotic normality of the posterior distribution in Gaussian linear regression models when the number of regressors increases with the sample size. Two kinds of Bernstein-von Mises Theorems are obtained in this framework: nonparametric theorems for the parameter itself, and semiparametric theorems for functionals of the parameter. We apply them to the Gaussian sequence model and to the regression of functions in Sobolev and $C^{\alpha}$ classes, in which we get the minimax convergence rates. Adaptivity is reached for the Bayesian estimators of functionals in our applications.

[1]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[2]  Andrew R. Barron,et al.  Information-theoretic asymptotics of Bayes methods , 1990, IEEE Trans. Inf. Theory.

[3]  A. V. D. Vaart.  Asymptotic Statistics: Delta Method , 1998 .

[4]  D. Freedman.  On the Bernstein-von Mises Theorem with Infinite Dimensional Parameters , 1999 .

[5]  Subhashis Ghosal,et al.  Asymptotic normality of posterior distributions in high-dimensional linear models , 1999 .

[6]  S. Ghosal.  Asymptotic Normality of Posterior Distributions for Exponential Families when the Number of Parameters Tends to Infinity , 2000 .

[7]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions , 2000 .

[8]  L. Wasserman,et al.  Rates of convergence of posterior distributions , 2001 .

[9]  Xiaotong Shen.  Asymptotic Normality of Semiparametric and Nonparametric Posterior Distributions , 2002 .

[10]  A. Tsybakov,et al.  Introduction à l'estimation non-paramétrique , 2003 .

[11]  Yongdai Kim,et al.  A Bernstein–von Mises theorem in the nonparametric right-censoring model , 2004, math/0410083.

[12]  Yongdai Kim.  The Bernstein–von Mises theorem for the proportional hazard model , 2006, math/0611230.

[13]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions for non-i.i.d. observations , 2007, 0708.0491.

[14]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[15]  S. Boucheron,et al.  A Bernstein-Von Mises Theorem for discrete probability distributions , 2008, 0807.2096.

[16]  J. Rousseau,et al.  BERNSTEIN-VON MISES THEOREM FOR LINEAR FUNCTIONALS OF THE DENSITY , 2009, 0908.4167.

[17]  B. Clarke,et al.  Reference priors for exponential families with increasing dimension , 2010 .

[18]  Dominique Bontemps.  Universal Coding on Infinite Alphabets: Exponentially Decreasing Envelopes , 2011, IEEE Transactions on Information Theory.

[19]  P. Bickel,et al.  The semiparametric Bernstein-von Mises theorem , 2010, 1007.0179.