Composite mixture of log-linear models for categorical data

Multivariate categorical data are routinely collected in many application areas. As the number of cells in the table grows exponentially with the number of variables, many or even most cells will contain zero observations. This severe sparsity motivates appropriate statistical methodologies that effectively reduce the number of free parameters, with penalized log-linear models and latent structure analysis being popular options. This article proposes a fundamentally new class of methods, which we refer to as Mixture of Log Linear models (mills). Combining latent class analysis and log-linear models, mills defines a novel Bayesian methodology to model complex multivariate categorical with flexibility and interpretability. Mills is shown to have key advantages over alternative methods for contingency tables in simulations and an application investigating the relation among suicide attempts and empathy.

[1]  B. Efron Bayes and likelihood calculations from confidence intervals , 1993 .

[2]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[3]  D. Dunson,et al.  Simplex Factor Models for Multivariate Unordered Categorical Data , 2012, Journal of the American Statistical Association.

[4]  Bruce G. Lindsay,et al.  ISSUES AND STRATEGIES IN THE SELECTION OF COMPOSITE LIKELIHOODS , 2011 .

[5]  Stephen E. Fienberg,et al.  Three centuries of categorical data analysis: Log-linear models and maximum likelihood estimation , 2007 .

[6]  D. Lawley,et al.  XXIII.—On Problems connected with Item Selection and Test Construction , 1943, Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Sciences.

[7]  A. Rinaldo,et al.  The Log-Linear Group Lasso Estimator and Its Asymptotic Properties , 2007, 0709.3526.

[8]  José Manoel Bertolote,et al.  Definitions of Suicidal Behaviour. , 2004 .

[9]  Adrian E. Raftery,et al.  Accounting for Model Uncertainty in Survival Analysis Improves Predictive Performance , 1995 .

[10]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[11]  Alan Agresti,et al.  Categorical Data Analysis , 2003 .

[12]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[13]  Trevor Hastie,et al.  Statistical Learning with Sparsity: The Lasso and Generalizations , 2015 .

[14]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[15]  Alfred O. Hero,et al.  Distributed Learning of Gaussian Graphical Models via Marginal Likelihoods , 2013, AISTATS.

[16]  Christian P. Robert,et al.  Handbook of Mixture Analysis , 2018 .

[17]  Michael I. Jordan Graphical Models , 2003 .

[18]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[19]  David R. Williams,et al.  Cross-national prevalence and risk factors for suicidal ideation, plans and attempts , 2008, British Journal of Psychiatry.

[20]  D. Cox,et al.  A note on pseudolikelihood constructed from marginal densities , 2004 .

[21]  E. Erosheva Comparing Latent Structures of the Grade of Membership, Rasch, and Latent Class Models , 2005 .

[22]  K. Mengersen,et al.  Asymptotic behaviour of the posterior distribution in overfitted mixture models , 2011 .

[23]  Helene Massam,et al.  Bayes factors and the geometry of discrete hierarchical loglinear models , 2011, 1103.5381.

[24]  Jonathan Lachal,et al.  Empathy in adolescent suicidal behaviors: Perspectives from the adolescents, their parents and their healthcare professionals , 2016 .

[25]  Jeffrey W. Miller Asymptotic Normality, Concentration, and Coverage of Generalized Posteriors , 2019, J. Mach. Learn. Res..

[26]  D. Dunson,et al.  Bayesian cumulative shrinkage for infinite factorizations. , 2019, Biometrika.

[27]  R. Nathan Spreng,et al.  Altered empathic responding in major depressive disorder: Relation to symptom severity, illness burden, and psychosocial outcome , 2011, Psychiatry Research.

[28]  Martin J Wainwright,et al.  Graphical models, exponential families, and variational inference. [electronic resource] / Wainwright, Martin J , 2008 .

[29]  Tom A. B. Snijders,et al.  Markov Chain Monte Carlo Estimation of Exponential Random Graph Models , 2002, J. Soc. Struct..

[30]  K. Mardia,et al.  Maximum likelihood estimation using composite likelihoods for closed exponential families , 2009 .

[31]  H. Massam,et al.  The mode oriented stochastic search (MOSS) algorithm for log-linear models with conjugate priors , 2010 .

[32]  A. Raftery A model for high-order Markov chains , 1985 .

[33]  M. Amir,et al.  Posttraumatic symptoms and suicide risk , 2004 .

[34]  Jorge Moll,et al.  Empathy and symptoms dimensions of patients with obsessive-compulsive disorder. , 2009, Journal of psychiatric research.

[35]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[36]  N. Lazar Bayesian empirical likelihood , 2003 .

[37]  James E. Johndrow,et al.  Optimal Gaussian approximations to the posterior for log-linear models with Diaconis-Ylvisaker priors , 2015, 1511.00764.

[38]  Laura Ventura,et al.  Robust likelihood functions in Bayesian inference , 2008 .

[39]  James G. Scott,et al.  Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables , 2012, 1205.0310.

[40]  Wicher P. Bergsma,et al.  Marginal models for categorical data , 2002 .

[41]  Davide Ferrari,et al.  Fast construction of efficient composite likelihood equations , 2017, 1709.03234.

[42]  A. Davison,et al.  Bayesian Inference from Composite Likelihoods, with an Application to Spatial Extremes , 2009, 0911.5357.

[43]  Lise Laporte,et al.  Alexithymia, empathy, and psychological symptoms in a family context. , 2002, Comprehensive psychiatry.

[44]  Edoardo M. Airoldi,et al.  Handbook of Mixed Membership Models and Their Applications , 2014 .

[45]  H. Massam,et al.  A conjugate prior for discrete hierarchical log-linear models , 2006, 0711.1609.

[46]  David Dunson,et al.  Bayesian Factorizations of Big Sparse Tensors , 2013, Journal of the American Statistical Association.

[47]  D. Dunson,et al.  Nonparametric Bayes Modeling of Multivariate Categorical Data , 2009, Journal of the American Statistical Association.

[48]  Hélène Massam,et al.  Local conditional and marginal approach to parameter estimation in discrete graphical models , 2018, J. Multivar. Anal..

[49]  Daniele Durante,et al.  Bayesian inference on group differences in multivariate categorical data , 2016, Comput. Stat. Data Anal..

[50]  Mark H. Davis,et al.  A Multidimensional Approach to Individual Differences in Empathy , 1980 .

[51]  N. Reid,et al.  AN OVERVIEW OF COMPOSITE LIKELIHOOD METHODS , 2011 .

[52]  Luigi Pace,et al.  Efficient composite likelihood for a scalar parameter of interest , 2019, Stat.

[53]  K. Hawton,et al.  Suicide, and Other Causes of Death, Following Attempted Suicide , 1988, British Journal of Psychiatry.

[54]  Davide Ferrari,et al.  Parsimonious and Efficient Likelihood Composition by Gibbs Sampling , 2015, 1502.04800.

[55]  Laura Ventura,et al.  Bayesian composite marginal likelihoods , 2011 .

[56]  L. Covi,et al.  SCL-90: an outpatient psychiatric rating scale--preliminary report. , 1973, Psychopharmacology bulletin.

[57]  M. aan het Rot,et al.  Empathy in adults with clinical or subclinical depressive symptoms. , 2013, Journal of affective disorders.

[58]  C. Wikle,et al.  Bayesian Hierarchical Models With Conjugate Full-Conditional Distributions for Dependent Data From the Natural Exponential Family , 2017, 1701.07506.

[59]  Rahul Mukerjee,et al.  Probability matching property of adjusted likelihoods , 2006 .

[60]  Ó. Gonçalves,et al.  Effects of Empathy and Conflict Resolution Strategies on Psychophysiological Arousal and Satisfaction in Romantic Relationships , 2013, Applied Psychophysiology and Biofeedback.

[61]  Nancy Reid,et al.  Combining likelihood and significance functions , 2019, Statistica Sinica.

[62]  David B Dunson,et al.  TENSOR DECOMPOSITIONS AND SPARSE LOG-LINEAR MODELS. , 2014, Annals of statistics.

[63]  A. Roverato,et al.  Log-mean linear models for binary data , 2011, 1109.6239.