Nonparametric empirical Bayes for the Dirichlet process mixture model

The Dirichlet process prior allows flexible nonparametric mixture modeling. The number of mixture components is not specified in advance and can grow as new data arrive. However, analyses based on the Dirichlet process prior are sensitive to the choice of the parameters, including an infinite-dimensional distributional parameter G0. Most previous applications have either fixed G0 as a member of a parametric family or treated G0 in a Bayesian fashion, using parametric prior specifications. In contrast, we have developed an adaptive nonparametric method for constructing smooth estimates of G0. We combine this method with a technique for estimating α, the other Dirichlet process parameter, that is inspired by an existing characterization of its maximum-likelihood estimator. Together, these estimation procedures yield a flexible empirical Bayes treatment of Dirichlet process mixtures. Such a treatment is useful in situations where smooth point estimates of G0 are of intrinsic interest, or where the structure of G0 cannot be conveniently modeled with the usual parametric prior families. Analysis of simulated and real-world datasets illustrates the robustness of this approach.

[1]  H. Robbins.  An Empirical Bayes Approach to Statistics , 1956 .

[2]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[3]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[4]  D. Berry,et al.  Empirical Bayes Estimation of a Binomial Parameter Via Mixtures of Dirichlet Processes , 1979 .

[5]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[6]  M. Postman,et al.  Probes of large-scale structure in the Corona Borealis region. , 1986 .

[7]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[8]  A. Izenman,et al.  Philatelic Mixtures and Multimodal Densities , 1988 .

[9]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[10]  K. Roeder.  Density estimation with confidence sets exemplified by superclusters and voids in the galaxies , 1990 .

[11]  Dennis V. Lindley,et al.  Empirical Bayes Methods , 1974 .

[12]  M. C. Jones,et al.  A reliable data-based bandwidth selection method for kernel density estimation , 1991 .

[13]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[14]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[15]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[16]  Jun S. Liu.  Nonparametric hierarchical Bayes via sequential imputations , 1996 .

[17]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[18]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[19]  G. McLachlan,et al.  Modelling the distribution of stamp paper thickness via finite normal mixtures: The 1872 Hidalgo stamp issue of Mexico revisited , 1997 .

[20]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[21]  G. Tomlinson.  Analysis of densities , 1998 .

[22]  Carl E. Rasmussen,et al.  The Infinite Gaussian Mixture Model , 1999, NIPS.

[23]  Radford M. Neal.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[24]  M. Stephens.  Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[25]  Carl E. Rasmussen,et al.  Infinite Mixtures of Gaussian Process Experts , 2001, NIPS.

[26]  Alan E. Gelfand,et al.  A Computational Approach for Full Nonparametric Bayesian Inference Under Dirichlet Process Mixture Models , 2002 .

[27]  D. D. Castro.  Semiparametric regression for count data , 2002 .

[28]  Lancelot F. James,et al.  Approximate Dirichlet Process Computing in Finite Normal Mixtures , 2002 .

[29]  J. Ibrahim,et al.  A Bayesian semiparametric joint hierarchical model for longitudinal and survival data. , 2003, Biometrics.

[30]  S. MacEachern,et al.  An ANOVA Model for Dependent Random Measures , 2004 .

[31]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[32]  Volker Tresp,et al.  A nonparametric hierarchical bayesian framework for information filtering , 2004, SIGIR '04.

[33]  Acknowledgments , 2006, Molecular and Cellular Endocrinology.

[34]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[35]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .