Matrix-Variate Dirichlet Process Mixture Models

We are concerned with a multivariate response regression problem where the interest is in considering correlations both across response variates and across response samples. In this paper we develop a new Bayesian nonparametric model for such a setting based on Dirichlet process priors. Building on an additive kernel model, we allow each sample to have its own regression matrix. Although this overcomplete representation could in principle sufier from severe overfltting problems, we are able to provide efiective control over the model via a matrix-variate Dirichlet process prior on the regression matrices. Our model is able to share statistical strength among regression matrices due to the clustering property of the Dirichlet process. We make use of a Markov chain Monte Carlo algorithm for inference and prediction. Compared with other Bayesian kernel models, our model has advantages in both computational and statistical e‐ciency.

[1]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[2]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[3]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[4]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[5]  B. Skagerberg,et al.  Multivariate data analysis applied to low-density polyethylene reactors , 1992 .

[6]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[7]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[8]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[9]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[10]  S. MacEachern,et al.  A semiparametric Bayesian model for randomised block designs , 1996 .

[11]  J. Friedman,et al.  Predicting Multivariate Responses in Multiple Linear Regression , 1997 .

[12]  C. Braak.  Discussion to 'Predicting multivariate responses in multiple linear regression' by L. Breiman & J.H. Friedman , 1997 .

[13]  Steven N. MacEachern,et al.  Computational Methods for Mixture of Dirichlet Process Models , 1998 .

[14]  Joseph G. Ibrahim,et al.  Semiparametric Bayesian Methods for Random Effects Models , 1998 .

[15]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[16]  A. Rukhin.  Matrix Variate Distributions , 1999, The Multivariate Normal Distribution.

[17]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[18]  S. MacEachern.  Decision Theoretic Aspects of Dependent Nonparametric Processes , 2000 .

[19]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[20]  S. MacEachern,et al.  An ANOVA Model for Dependent Random Measures , 2004 .

[21]  Marcus R. Frean,et al.  Dependent Gaussian Processes , 2004, NIPS.

[22]  S. MacEachern,et al.  Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing , 2005 .

[23]  Yee Whye Teh,et al.  Semiparametric latent factor models , 2005, AISTATS.

[24]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[25]  Zhihua Zhang,et al.  Bayesian Multicategory Support Vector Machines , 2006, UAI.

[26]  N. Pillai,et al.  Bayesian density regression , 2007 .

[27]  Lawrence Carin,et al.  Multi-Task Learning for Classification with Dirichlet Process Priors , 2007, J. Mach. Learn. Res..

[28]  L. Carin,et al.  The Matrix Stick-Breaking Process , 2008 .

[29]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[30]  A. P. Dawid,et al.  Regression and Classification Using Gaussian Process Priors , 2009 .

[31]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.