Generalized smooth finite mixtures

We propose a general class of models and a unified Bayesian inference methodology for flexibly estimating the density of a response variable conditional on a possibly high-dimensional set of covariates. Our model is a finite mixture of component models with covariate-dependent mixing weights. The component densities can belong to any parametric family, with each model parameter being a deterministic function of covariates through a link function. Our MCMC methodology allows for Bayesian variable selection among the covariates in the mixture components and in the mixing weights. The model’s parameterization and variable selection prior are chosen to prevent overfitting. We use simulated and real data sets to illustrate the methodology.

[1]  David J. Nott,et al.  Adaptive sampling for Bayesian variable selection , 2005 .

[2]  P. Dellaportas,et al.  Bayesian variable and link determination for generalised linear models , 2003 .

[3]  D. Dunson,et al.  Nonparametric Bayes Conditional Distribution Modeling With Variable Selection , 2009, Journal of the American Statistical Association.

[4]  R. Kohn,et al.  Regression Density Estimation Using Smooth Adaptive Gaussian Mixtures , 2007 .

[5]  P. Consul,et al.  A Generalization of the Poisson Distribution , 1973 .

[6]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[7]  Gordon K. Smyth,et al.  Generalized linear models with varying dispersion , 1989 .

[8]  Achim Wambach,et al.  Incentive effects in the demand for health care: a bivariate panel count data estimation , 2003 .

[9]  José M. R. Murteira,et al.  Alternative Estimating and Testing Empirical Strategies for Fractional Regression Models , 2011 .

[10]  Robert Kohn,et al.  Nonparametric regression using linear combinations of basis functions , 2001, Stat. Comput..

[11]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[12]  Sally Wood,et al.  Bayesian mixture of splines for spatially adaptive nonparametric regression , 2002 .

[13]  J. Geweke,et al.  Smoothly mixing regressions , 2007 .

[14]  R. Kass.  Bayes Factors in Practice , 1993 .

[15]  Andriy Norets,et al.  POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES , 2011, Econometric Theory.

[16]  J. Hilbe.  Negative Binomial Regression: Preface , 2007 .

[17]  R. Casarin,et al.  Bayesian Model Selection for Beta Autoregressive Processes , 2010, 1008.0121.

[18]  Felix Famoye,et al.  Zero-Inflated Generalized Poisson Regression Model with an Application to Domestic Violence Data , 2021, Journal of Data Science.

[19]  Paolo Giordani,et al.  Taking the Twists into Account: Predicting Firm Bankruptcy Risk with Splines of Financial Ratios , 2011, Journal of Financial and Quantitative Analysis.

[20]  Bruce D. McCullough,et al.  Regression analysis of proportions in finance with self selection , 2008 .

[21]  B. McCullough,et al.  Author ' s personal copy Regression analysis of proportions in fi nance with self selection , 2008 .

[22]  Dani Gamerman,et al.  Sampling from the posterior distribution in generalized linear mixed models , 1997, Stat. Comput..

[23]  M. Tanner,et al.  Hierarchical mixtures-of-experts for exponential family regression models: approximation and maximum , 1999 .

[24]  M. Stephens.  Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[25]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[26]  Babak Shahbaba,et al.  Nonlinear Models Using Dirichlet Process Mixtures , 2007, J. Mach. Learn. Res..

[27]  P. Müller,et al.  Bayesian curve fitting using multivariate normal mixtures , 1996 .

[28]  Andriy Norets,et al.  Approximation of conditional densities by smooth mixtures of regressions , 2010, 1010.0581.

[29]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[30]  Sveriges Riksbank.  Flexible modeling of conditional distributions using smooth mixtures of asymmetric student T densities , 2009 .

[31]  R. Rajan,et al.  What Do We Know About Capital Structure? Some Evidence from International Data , 1994 .

[32]  P. Green,et al.  Modelling Heterogeneity With and Without the Dirichlet Process , 2001 .

[33]  David J Nott,et al.  Sampling Schemes for Bayesian Variable Selection in Generalized Linear Models , 2004 .

[34]  Fengchun Peng,et al.  Bayesian Inference in Mixtures-of-Experts and Hierarchical Mixtures-of-Experts Models With an Applic , 1996 .

[35]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[36]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[37]  John Geweke,et al.  Federal Reserve Bank of Minneapolis Research Department Staff Report 249 Using Simulation Methods for Bayesian Econometric Models: Inference, Development, and Communication , 2022 .

[38]  W. DeSarbo,et al.  A maximum likelihood methodology for clusterwise linear regression , 1988 .

[39]  Christopher Holmes,et al.  Bayesian Methods for Nonlinear Classification and Regressing , 2002 .

[40]  C. Czado,et al.  Zero-inflated generalized Poisson models with regression effects on the mean, dispersion and zero-inflation level applied to patent outsourcing rates , 2007 .

[41]  Robert Kohn,et al.  Locally Adaptive Nonparametric Binary Regression , 2007, 0709.3545.

[42]  N. Pillai,et al.  Bayesian density regression , 2007 .