An Autoregressive Approach to Nonparametric Hierarchical Dependent Modeling

We propose a conditional autoregression framework for a collection of random probability measures. Under this framework, we devise a conditional autoregressive Dirichlet process (DP) that we call one-parameter dependent DP (ωDDP). The appealing properties of this specification are that it has two equivalent representations and its inference can be implemented in a conditional Polya urn scheme. Moreover, these two representations bear a resemblance to the Polya urn scheme and the stick-breaking representation in the conventional DP. We apply this ωDDP to Bayesian multivariate-response regression problems. An efficient Markov chain Monte Carlo algorithm is developed for Bayesian computation and prediction.

[1]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[2]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[3]  W. Eric L. Grimson,et al.  Construction of Dependent Dirichlet Processes based on Poisson Processes , 2010, NIPS.

[4]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[5]  S. MacEachern,et al.  An ANOVA Model for Dependent Random Measures , 2004 .

[6]  B. Skagerberg,et al.  Multivariate data analysis applied to low-density polyethylene reactors , 1992 .

[7]  S. MacEachern,et al.  A semiparametric Bayesian model for randomised block designs , 1996 .

[8]  J. Friedman,et al.  Predicting Multivariate Responses in Multiple Linear Regression , 1997 .

[9]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[10]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[11]  Warren B. Powell,et al.  Dirichlet Process Mixtures of Generalized Linear Models , 2009, J. Mach. Learn. Res..

[12]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[13]  C. Braak.  Discussion to 'Predicting multivariate responses in multiple linear regression' by L. Breiman & J.H. Friedman , 1997 .

[14]  N. Pillai,et al.  Bayesian density regression , 2007 .

[15]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[16]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[17]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[18]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[19]  Yee Whye Teh,et al.  Semiparametric latent factor models , 2005, AISTATS.

[20]  Steven N. MacEachern,et al.  Computational Methods for Mixture of Dirichlet Process Models , 1998 .

[21]  T. Fearn,et al.  Bayesian Wavelet Regression on Curves With Application to a Spectroscopic Calibration Problem , 2001 .

[22]  S. MacEachern,et al.  Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing , 2005 .

[23]  Zhihua Zhang,et al.  Matrix-Variate Dirichlet Process Mixture Models , 2010, AISTATS.

[24]  A. Gelfand,et al.  Hybrid Dirichlet mixture models for functional data , 2009 .

[25]  S. MacEachern.  Decision Theoretic Aspects of Dependent Nonparametric Processes , 2000 .

[26]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[27]  Babak Shahbaba,et al.  Nonlinear Models Using Dirichlet Process Mixtures , 2007, J. Mach. Learn. Res..