A mixture likelihood approach for generalized linear models

A mixture model approach is developed that simultaneously estimates the posterior membership probabilities of observations to a number of unobservable groups or latent classes, and the parameters of a generalized linear model which relates the observations, distributed according to some member of the exponential family, to a set of specified covariates within each Class. We demonstrate how this approach handles many of the existing latent class regression procedures as special cases, as well as a host of other parametric specifications in the exponential family heretofore not mentioned in the latent class literature. As such we generalize the McCullagh and Nelder approach to a latent class framework. The parameters are estimated using maximum likelihood, and an EM algorithm for estimation is provided. A Monte Carlo study of the performance of the algorithm for several distributions is provided, and the model is illustrated in two empirical applications.

[1]  S. Newcomb.  A Generalized Theory of the Combination of Observations so as to Obtain the Best Result , 1886 .

[2]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[3]  C. V. L. Charlier,et al.  On the dissection of frequency functions , 1923 .

[4]  A. Gardner.  Methods of Statistics , 1941 .

[5]  J. P. Harding,et al.  The Use of Probability Paper for the Graphical Analysis of Polymodal Frequency Distributions , 1949, Journal of the Marine Biological Association of the United Kingdom.

[6]  R. Cassie,et al.  Some uses of probability paper in the analysis of size frequency distributions , 1954 .

[7]  R. McHugh.  Efficient estimation and local identification in latent class analysis , 1956 .

[8]  Richard B. McHugh.  Note on “efficient estimation and local identification in latent class analysis” , 1958 .

[9]  H. Teicher.  On the Mixture of Distributions , 1960 .

[10]  H. Teicher.  Identifiability of Mixtures , 1961 .

[11]  A. C. Nielsen.  The Impact of Retail Coupons , 1965 .

[12]  V. Hasselblad.  Estimation of parameters for a mixture of normal distributions , 1966 .

[13]  E. A. Thomas,et al.  Mathematical models for the clustered firing of single cortical neurones. , 1966, The British journal of mathematical and statistical psychology.

[14]  C G Bhattacharya,et al.  A simple method of resolution of a distribution into gaussian components. , 1967, Biometrics.

[15]  V. Hasselblad.  Finite mixtures of distributions from the exponential family , 1969 .

[16]  A. Hope.  A Simplified Monte Carlo Significance Test Procedure , 1968 .

[17]  Paul F. Lazarsfeld,et al.  Latent Structure Analysis. , 1969 .

[18]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[19]  N. E. Day.  Estimating the components of a mixture of normal distributions , 1969 .

[20]  J. Wolfe.  PATTERN CLUSTERING BY MULTIVARIATE MIXTURE ANALYSIS. , 1970, Multivariate behavioral research.

[21]  Vithala R. Rao,et al.  Conjoint Measurement- for Quantifying Judgmental Data , 1971 .

[22]  A. Dempster.  An overview of multivariate data analysis , 1971 .

[23]  S. R. Searle.  Linear Models , 1971 .

[24]  R. Quandt.  A New Approach to Estimating Switching Regressions , 1972 .

[25]  C. Robertson,et al.  A comparison of some methods for estimating mixed normal distributions , 1972 .

[26]  Robert H. Berk,et al.  Consistency and Asymptotic Normality of MLE's for Exponential Models , 1972 .

[27]  L. A. Goodman.  Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[28]  H. Akaike.  A new look at the statistical model identification , 1974 .

[29]  Draper Daniels.  The Second Meaning of the Word “Creative” Should Be First in the Hearts of Advertising People , 1974 .

[30]  David David.  Maximum likelihood estimates of the parameters of a mixture of two regression lines , 1974 .

[31]  C. F. Banfield,et al.  Algorithm AS 113: A Transfer for Non-Hierarchical Classification , 1977 .

[32]  R. Davies.  Hypothesis testing when a nuisance parameter is present only under the alternative , 1977 .

[33]  Shelby J. Haberman,et al.  Maximum Likelihood Estimates in Exponential Response Models , 1977 .

[34]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[35]  R. Davies.  Hypothesis testing when a nuisance parameter is present only under the alternative , 1977 .

[36]  E. M. Babb,et al.  Consumer Response to Promotional Deals , 1978 .

[37]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[38]  Ronald W. Ward,et al.  A Pooled Cross-Section Time Series Model of Coupon Promotions , 1978 .

[39]  H. Walker,et al.  An iterative procedure for obtaining maximum-likelihood estimates of the parameters for a mixture of normal distributions , 1978 .

[40]  Kendall E. Atkinson.  An introduction to numerical analysis , 1978 .

[41]  Alice M. Tybout,et al.  Impact of Deals and Deal Retraction on Brand Switching , 1978 .

[42]  Robert C. Blattberg,et al.  Identifying the Deal Prone Segment , 1978 .

[43]  E. M. Babb,et al.  Consumer Response to Promotional Deals , 1978 .

[44]  J. B. Ramsey,et al.  Estimating Mixtures of Normal Distributions and Switching Regressions , 1978 .

[45]  Edward B. Fowlkes,et al.  Some Methods for Studying the Mixture of Two Normal (Lognormal) Distributions , 1979 .

[46]  William O. Bearden,et al.  Correlates of Consumer Susceptibility to Coupons in New Grocery Product Introductions , 1980 .

[47]  R. Oliver.  A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions , 1980 .

[48]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo Method , 1981 .

[49]  Michael J. Symons,et al.  Clustering criteria and multivariate normal mixtures , 1981 .

[50]  Murray Aitkin,et al.  Statistical Modelling of Data on Teaching Styles , 1981 .

[51]  B. Everitt,et al.  Finite Mixture Distributions , 1981 .

[52]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[53]  G. J. McLachlan,et al.  9 The classification and mixture maximum likelihood approaches to cluster analysis , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[54]  J. Hammersley.  SIMULATION AND THE MONTE CARLO METHOD , 1982 .

[55]  R. A. Boyles.  On the Convergence of the EM Algorithm , 1983 .

[56]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[57]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[58]  Brian Everitt,et al.  Maximum Likelihood Estimation of the Parameters in a Mixture of Two Univariate Normal Distributions; a Comparison of Different Algorithms , 1984 .

[59]  Lewis G. Pringle.  A Comment , 1984 .

[60]  Terence A. Shimp,et al.  The Theory of Reasoned Action Applied to Coupon Usage , 1984 .

[61]  H. Bozdogan,et al.  Multi-sample cluster analysis using Akaike's Information Criterion , 1984 .

[62]  C. Narasimhan.  A Price Discrimination Theory of Coupons , 1984 .

[63]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[64]  P. Sen,et al.  On the asymptotic performance of the log likelihood ratio statistic for the mixture model and related results , 1984 .

[65]  P. Green.  Iteratively reweighted least squares for maximum likelihood estimation , 1984 .

[66]  B. Jørgensen.  The Delta Algorithm and GLIM , 1984 .

[67]  Geoffrey J. McLachlan,et al.  The mixture method of clustering applied to three-way data , 1985 .

[68]  D. Rubin,et al.  Estimation and Hypothesis Testing in Finite Mixture Models , 1985 .

[69]  Wayne S. DeSarbo,et al.  A Probabilistic Multidimensional Scaling Vector Model , 1986 .

[70]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[71]  G. McLachlan.  On Bootstrapping the Likelihood Ratio Test Statistic for the Number of Components in a Normal Mixture , 1987 .

[72]  Robert W. Shoemaker,et al.  The Coupon-Prone Consumer: Some Findings Based on Purchase Behavior across Product Classes , 1987 .

[73]  S. Sclove.  Application of model-selection criteria to some problems in multivariate analysis , 1987 .

[74]  Dick R. Wittink,et al.  SUPPORTING A HIGHER SHELF PRICE THROUGH COUPON DISTRIBUTIONS , 1987 .

[75]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[76]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[77]  W. DeSarbo,et al.  A maximum likelihood methodology for clusterwise linear regression , 1988 .

[78]  W. DeSarbo,et al.  Response Determinants in Satisfaction Judgments , 1988 .

[79]  N. Sedransk,et al.  Mixtures of Distributions: A Topological Approach , 1988 .

[80]  Robert W. Shoemaker,et al.  Analyzing Incremental Sales from a Direct Mail Coupon Promotion , 1989 .

[81]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[82]  T. Lwin,et al.  Probits of mixtures. , 1989, Biometrics.

[83]  I. Meilijson.  A fast improvement to the EM algorithm on its own terms , 1989 .

[84]  Wayne S. DeSarbo,et al.  A simulated annealing methodology for clusterwise linear regression , 1989 .

[85]  Gary J. Russell,et al.  A Probabilistic Choice Model for Market Segmentation and Elasticity Structure , 1989 .

[86]  Scot Burton,et al.  Distinguishing Coupon Proneness from Value Consciousness: An Acquisition-Transaction Utility Theory Perspective , 1990 .

[87]  D. Titterington.  Some recent research in the analysis of mixture distributions , 1990 .

[88]  Wagner A. Kamakura,et al.  Estimating flexible distributions of ideal-points with external analysis of preferences , 1991 .

[89]  Wayne S. DeSarbo,et al.  A latent class probit model for analyzing pick any/N data , 1991 .

[90]  Geoffrey J. McLachlan,et al.  Improving the convergence rate of the em algorithm for a mixture model fitted to grouped truncated data , 1992 .

[91]  Michel Wedel,et al.  Latent class metric conjoint analysis , 1992 .

[92]  Geoffrey J. McLachlan,et al.  FITTING FINITE MIXTURE MODELS IN A REGRESSION CONTEXT , 1992 .

[93]  Michel Wedel,et al.  A Latent Class Poisson Regression Model for Heterogeneous Count Data , 1993 .

[94]  Peter G. M. van der Heijden,et al.  The EM algorithm for latent class analysis with equality constraints , 1992 .

[95]  W. DeSarbo,et al.  An Empirical Pooling Approach for Estimating Marketing Mix Elasticities with PIMS Data , 1993 .

[96]  Wayne S. DeSarbo,et al.  A Latent Class Binomial Logit Methodology for the Analysis of Paired Comparison Choice Data: An Application Reinvestigating the Determinants of Perceived Risk , 1993 .

[97]  W. DeSarbo,et al.  A Review of Recent Developments in Latent Class Regression Models , 1994 .

[98]  Rolf Langeheine,et al.  Latent Trait and Latent Class Models , 2013 .