Bayesian aspects of some nonparametric problems

We study the Bayesian approach to nonparametric function estimation problems such as nonparametric regression and signal estimation. We consider the asymptotic properties of Bayes procedures for conjugate (=Gaussian) priors. We show that so long as the prior puts nonzero measure on the very large parameter set of interest then the Bayes estimators are not satisfactory. More specifically, we show that these estimators do not achieve the correct minimax rate over norm bounded sets in the parameter space. Thus all Bayes estimators for proper Gaussian priors have zero asymptotic efficiency in this minimax sense. We then present a class of priors whose Bayes procedures attain the optimal minimax rate of convergence. These priors may be viewed as compound, or hierarchical, mixtures of suitable Gaussian distributions.

[1]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[2]  H. Kuo.  Gaussian Measures in Banach Spaces , 1975 .

[3]  G. Wahba.  Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression , 1978 .

[4]  P. Diaconis,et al.  Conjugate Priors for Exponential Families , 1979 .

[5]  All Admissible Linear Estimators of the Mean of a Gaussian Distribution on a Hilbert Space , 1984 .

[6]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[7]  D. Freedman,et al.  On the consistency of Bayes estimates , 1986 .

[8]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[9]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[10]  G. Wahba.  Spline models for observational data , 1990 .

[11]  D. Donoho,et al.  Minimax Risk Over Hyperrectangles, and Implications , 1990 .

[12]  D. Donoho,et al.  Geometrizing Rates of Convergence, III , 1991 .

[13]  D. Donoho,et al.  Renormalization Exponents and Optimal Pointwise Rates of Convergence , 1992 .

[14]  A NOTE ON SMOOTHING SPLINES AS BAYESIAN ESTIMATES , 1993 .

[15]  D. Cox.  An Analysis of Bayesian Inference for Nonparametric Regression , 1993 .

[16]  A. Linde,et al.  Splines from a Bayesian point of view , 1995 .

[17]  L. Brown,et al.  Asymptotic equivalence of nonparametric regression and white noise , 1996 .

[18]  M. Nussbaum.  Asymptotic Equivalence of Density Estimation and Gaussian White Noise , 1996 .

[19]  I. Johnstone,et al.  Minimax estimation via wavelet shrinkage , 1998 .

[20]  M. Nussbaum,et al.  Constructive asymptotic equivalence of density estimation and Gaussian white noise , 1998 .

[21]  D. Freedman.  On the Bernstein-von Mises Theorem with Infinite Dimensional Parameters , 1999 .

[22]  L. Wasserman,et al.  Rates of convergence of posterior distributions , 2001 .

[23]  M. Nussbaum,et al.  Asymptotic equivalence for nonparametric regression , 2002 .