An Analysis of Bayesian Inference for Nonparametric Regression

Abstract : The observation model Y sub i = Beta(i/n) + epsilon sub i, 1 or = n, is considered, where the epsilon's are i.i.d. mean zero and variance sigma-sq and beta is an unknown smooth function. A Gaussian prior distribution is specified by assuming beta is the solution of a high order stochastic differential equation. The estimation error delta = beta - beta-average is analyzed, where beta-average is the posterior expectation of beta. Asymptotic posterior and sampling distributional approximations are given for (abs. val del)square when (abs. val)square is one of a family of norms natural to the problem. It is shown that the frequentist coverage probability of a variety of (1 - alpha) posterior probability regions tends to be larger than 1 - alpha, but will be infinitely often less than any epsilon 0 as n approaches infinity with prior probability 1. A related continuous time signal estimation problem is also studied. Keywords: Bayesian inference; Nonparametric regression; Confidence regions; Signal extraction: Smoothing splices.