Adaptive Bayesian density estimation using Pitman-Yor or normalized inverse-Gaussian process kernel mixtures

We consider Bayesian nonparametric density estimation using a Pitman-Yor or a normalized inverse-Gaussian process kernel mixture as the prior distribution for a density. The procedure is studied from a frequentist perspective. Using the stick-breaking representation of the Pitman-Yor process or the expression of the finite-dimensional distributions for the normalized-inverse Gaussian process, we prove that, when the data are replicates from an infinitely smooth density, the posterior distribution concentrates on any shrinking $L^p$-norm ball, $1\leq p\leq\infty$, around the sampling density at a \emph{nearly parametric} rate, up to a logarithmic factor. The resulting hierarchical Bayesian procedure, with a fixed prior, is thus shown to be adaptive to the infinite degree of smoothness of the sampling density.

[1]  E. C. Titchmarsh Introduction to the Theory of Fourier Integrals , 1938 .

[2]  A. Offord Introduction to the Theory of Fourier Integrals , 1938, Nature.

[3]  M. R. Leadbetter,et al.  On the Estimation of the Probability Density, I , 1963 .

[4]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[5]  A. G. Greenhill,et al.  Handbook of Mathematical Functions with Formulas, Graphs, , 1971 .

[6]  T. Kawata Fourier analysis in probability theory , 1972 .

[7]  K. B. Davis Mean Integrated Square Error Properties of Density Estimates , 1977 .

[8]  I. Ibragimov,et al.  Estimation of distribution density , 1983 .

[9]  T. Ferguson BAYESIAN DENSITY ESTIMATION BY MIXTURES OF NORMAL DISTRIBUTIONS , 1983 .

[10]  I. Ibragimov,et al.  On density estimation in the view of Kolmogorov's ideas in approximation theory , 1990 .

[11]  L. Devroye A Note on the Usefulness of Superkernels in Density Estimation , 1992 .

[12]  W. Wong,et al.  Probability inequalities for likelihood ratios and convergence rates of sieve MLEs , 1995 .

[13]  B. Levit,et al.  Asymptotically efficient estimation of analytic functions in Gaussian noise , 1996 .

[14]  J. Pitman,et al.  The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator , 1997 .

[15]  A. Tsybakov,et al.  Exact asymptotic minimax constants for the estimation of analytical functions in Lp , 1998 .

[16]  L. Wasserman,et al.  The consistency of posterior distributions in nonparametric problems , 1999 .

[17]  J. Ghosh,et al.  POSTERIOR CONSISTENCY OF DIRICHLET MIXTURES IN DENSITY ESTIMATION , 1999 .

[18]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions , 2000 .

[19]  H. Ishwaran,et al.  Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models , 2000 .

[20]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[21]  S. Ghosal Convergence rates for density estimation with Bernstein polynomials , 2001 .

[22]  A. V. D. Vaart,et al.  Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities , 2001 .

[23]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[24]  Ramsés H. Mena,et al.  Hierarchical Mixture Modeling With Normalized Inverse-Gaussian Priors , 2005 .

[25]  K. Athreya,et al.  Measure Theory and Probability Theory , 2006 .

[26]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions for non-i.i.d. observations , 2007, 0708.0491.

[27]  A. V. D. Vaart,et al.  Posterior convergence rates of Dirichlet mixtures at smooth densities , 2007, 0708.1885.

[28]  Cristina Butucea,et al.  Sharp Optimality in Density Deconvolution with Dominating Bias. II , 2008 .

[29]  Van Der Vaart,et al.  Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth , 2009, 0908.3556.

[30]  J. H. Zanten,et al.  Adaptive nonparametric Bayesian inference using location-scale mixture priors , 2010, 1211.2121.

[31]  A. V. D. Vaart,et al.  Adaptive Bayesian density estimation with location-scale mixtures , 2010 .

[32]  X. Nguyen Convergence of latent mixing measures in nonparametric and mixture models , 2011 .

[33]  M. Cathy,et al.  Adaptive density estimation for clustering with Gaussian mixtures , 2011, 1103.4253.

[34]  M. Cathy,et al.  Adaptive density estimation using finite Gaussian mixtures , 2011 .

[35]  Richard Nickl,et al.  Rates of contraction for posterior distributions in Lr-metrics, 1 ≤ r ≤ ∞ , 2011, 1203.2043.

[36]  J. Norris Appendix: probability and measure , 1997 .

[37]  C. Maugis-Rabusseau,et al.  Adaptive density estimation for clustering with gaussian mixtures , 2013 .

[38]  S. Ghosal,et al.  Adaptive Bayesian multivariate density estimation with Dirichlet mixtures , 2011, 1109.6406.

[39]  Andriy Norets,et al.  POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES , 2011, Econometric Theory.