Bayesian inference for finite mixtures of generalized linear models with random effects

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.

[1]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[2]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[3]  R. Luce,et al.  Individual Choice Behavior: A Theoretical Analysis. , 1960 .

[4]  J. B. Ramsey,et al.  Estimating Mixtures of Normal Distributions and Switching Regressions , 1978 .

[5]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[8]  P. Damlen,et al.  Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables , 1999 .

[9]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[10]  Greg M. Allenby,et al.  On the Heterogeneity of Demand , 1998 .

[11]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[12]  Geoffrey J. McLachlan,et al.  FITTING FINITE MIXTURE MODELS IN A REGRESSION CONTEXT , 1992 .

[13]  Michel Wedel,et al.  A Latent Class Poisson Regression Model for Heterogeneous Count Data , 1993 .

[14]  T. Lwin,et al.  Probits of mixtures. , 1989, Biometrics.

[15]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[16]  David David.  Maximum likelihood estimates of the parameters of a mixture of two regression lines , 1974 .

[17]  R. Quandt.  A New Approach to Estimating Switching Regressions , 1972 .

[18]  Gary J. Russell,et al.  A Probabilistic Choice Model for Market Segmentation and Elasticity Structure , 1989 .

[19]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[20]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[21]  W. DeSarbo,et al.  An Empirical Pooling Approach for Estimating Marketing Mix Elasticities with PIMS Data , 1993 .

[22]  Adrian F. M. Smith,et al.  Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus , 1993 .

[23]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[24]  A. Raftery,et al.  Estimating Bayes Factors via Posterior Simulation with the Laplace—Metropolis Estimator , 1997 .

[25]  A. Cohen,et al.  Finite Mixture Distributions , 1982 .

[26]  S. R. Searle.  Linear Models , 1971 .

[27]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[28]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[29]  Wagner A. Kamakura,et al.  Estimating flexible distributions of ideal-points with external analysis of preferences , 1991 .

[30]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[31]  P. Lenk,et al.  Hierarchical Bayes Conjoint Analysis: Recovery of Partworth Heterogeneity from Reduced Experimental Designs , 1996 .

[32]  Nicholas G. Polson,et al.  On the Geometric Convergence of the Gibbs Sampler , 1994 .

[33]  Andrew L. Rukhin,et al.  Tools for statistical inference , 1991 .

[34]  Scott L. Zeger,et al.  Generalized linear models with random e ects: a Gibbs sampling approach , 1991 .

[35]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[36]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[37]  H. Jeffreys,et al.  Theory of probability , 1896 .

[38]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[39]  Wayne S. DeSarbo,et al.  A latent class probit model for analyzing pick any/N data , 1991 .

[40]  Michel Wedel,et al.  Latent class metric conjoint analysis , 1992 .

[41]  S. Newcomb.  A Generalized Theory of the Combination of Observations so as to Obtain the Best Result , 1886 .

[42]  Wayne S. DeSarbo,et al.  A Latent Class Binomial Logit Methodology for the Analysis of Paired Comparison Choice Data: An Application Reinvestigating the Determinants of Perceived Risk , 1993 .

[43]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[44]  W. DeSarbo,et al.  A maximum likelihood methodology for clusterwise linear regression , 1988 .

[45]  M. Puterman,et al.  Mixed Poisson regression models with covariate dependent rates. , 1996, Biometrics.

[46]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[47]  L. Wasserman,et al.  Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio , 1995 .

[48]  D. McFadden.  Conditional logit analysis of qualitative choice behavior , 1972 .

[49]  W. DeSarbo,et al.  A mixture likelihood approach for generalized linear models , 1995 .

[50]  L. A. Goodman.  Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[51]  Martin L. Puterman,et al.  Mixed logistic regression models , 1998 .

[52]  Martin L. Puterman,et al.  Analysis of Patent Data—A Mixed-Poisson-Regression-Model Approach , 1998 .

[53]  D. Hosmer.  Estimating Mixtures of Normal Distributions and Switching Regressions: Comment , 1978 .

[54]  Alison L. Gibbs,et al.  Convergence of Markov chain Monte Carlo algorithms with applications to image restoration , 2000 .