The Gaussian Process Density Sampler

We present the Gaussian Process Density Sampler (GPDS), an exchangeable generative model for use in nonparametric Bayesian density estimation. Samples drawn from the GPDS are consistent with exact, independent samples from a fixed density function that is a transformation of a function drawn from a Gaussian process prior. Our formulation allows us to infer an unknown density from data using Markov chain Monte Carlo, which gives samples from the posterior distribution over density functions and from the predictive distribution on data space. We can also infer the hyperparameters of the Gaussian process. We compare this density modeling technique to several existing techniques on a toy problem and a skull-reconstruction task.

[1]  Radford M. Neal,et al.  Suppressing Random Walks in Markov Chain Monte Carlo Using Ordered Overrelaxation , 1995, Learning in Graphical Models.

[2]  S. Chib,et al.  Marginal Likelihood From the Metropolis–Hastings Output , 2001 .

[3]  P. Fearnhead,et al.  Exact and computationally efficient likelihood‐based estimation for discretely observed diffusion processes (with discussion) , 2006 .

[4]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[5]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[6]  J. Ghosh,et al.  Posterior consistency of logistic Gaussian process priors in density estimation , 2007 .

[7]  Zoubin Ghahramani,et al.  MCMC for Doubly-intractable Distributions , 2006, UAI.

[8]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[9]  J. Møller,et al.  An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants , 2006 .

[10]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[11]  Iain Murray.  Advances in Markov chain Monte Carlo methods , 2007 .

[12]  Benedikt Hallgrímsson,et al.  THE RELATIONSHIP BETWEEN FLUCTUATING ASYMMETRY AND ENVIRONMENTAL VARIANCE IN RHESUS MACAQUE SKULLS , 2005, Evolution; international journal of organic evolution.

[13]  Peter J. Lenk,et al.  Towards a practicable Bayesian nonparametric density estimator , 1991 .

[14]  J. Propp,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996 .

[15]  LawrenceNeil.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005 .

[16]  Thorburn Daniel,et al.  A Bayesian approach to density estimation , 1986 .

[17]  D. Mackay,et al.  Bayesian neural networks and density networks , 1995 .

[18]  Thomas E. Wehrly,et al.  An Invariant Approach to Statistical Analysis of Shapes , 2004, Technometrics.

[19]  Tom Leonard.  Density Estimation, Stochastic Processes and Prior Information , 1978 .