Bayesian model-based clustering for longitudinal ordinal data

Traditional cluster analysis methods used in ordinal data, for instance k-means and hierarchical clustering, are mostly heuristic and lack statistical inference tools to compare among competing models. To address this we propose a latent transitional model, a finite mixture model that includes both observed and latent covariates and apply it for the first time to the case of longitudinal ordinal data. This model-based clustering model is an extension of the proportional odds model and includes a first-order transitional term, occasion effects and interactions which provide flexible ways to capture different time patterns by cluster as well as time-heterogeneous transitions. We estimate model parameters within a Bayesian setting using a Markov chain Monte Carlo scheme and block-wise Metropolis–Hastings sampling. We illustrate the model using 2001–2011 self-reported health status (SRHS) from the Household, Income and Labour Dynamics in Australia survey. SRHS is recorded as an ordinal variable with five levels: poor, fair, good, very good and excellent. Using the Widely Applicable Information Criterion for model comparison, we find evidence for six latent groups. Transitions in the original data and the estimated groups are visualized using heatmaps.

[1]  Rebecca A Betensky,et al.  A penalized latent class model for ordinal data. , 2007, Biostatistics.

[2]  Fernando A. Quintana,et al.  Bayesian Nonparametric Data Analysis , 2015 .

[3]  Julien Jacques,et al.  Model-based clustering of multivariate ordinal data relying on a stochastic binary search algorithm , 2016, Stat. Comput..

[4]  Gérard Govaert,et al.  Block clustering with Bernoulli mixture models: Comparison of different approaches , 2008, Comput. Stat. Data Anal..

[5]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[6]  A. Kottas,et al.  Bayesian Nonparametric Modeling for Multivariate Ordinal Regression , 2014, 1408.1027.

[7]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[8]  S. Frühwirth-Schnatter,et al.  Labor Market Entry and Earnings Dynamics: Bayesian Inference Using Mixtures-of-Experts Markov Chain Clustering , 2012 .

[9]  S. Pledger.  Unified Maximum Likelihood Estimates for Closed Capture–Recapture Models Using Mixtures , 2000, Biometrics.

[10]  S S Stevens,et al.  On the Theory of Scales of Measurement. , 1946, Science.

[11]  T. McKinley,et al.  Bayesian Model Choice in Cumulative Link Ordinal Regression Models , 2015, 1503.07642.

[12]  J. P. McKeone,et al.  Investigation of the widely applicable Bayesian information criterion , 2015, Stat. Comput..

[13]  Jean-Michel Marin,et al.  Bayesian Modelling and Inference on Mixtures of Distributions , 2005 .

[14]  Francis K. C. Hui,et al.  Variational Approximations for Generalized Linear Latent Variable Models , 2017 .

[15]  Sumio Watanabe,et al.  A widely applicable Bayesian information criterion , 2012, J. Mach. Learn. Res..

[16]  S. Frühwirth-Schnatter,et al.  Model-based clustering of categorical time series , 2010 .

[17]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[18]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[19]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[20]  Halina Frydman.  Estimation in the Mixture of Markov Chains Moving With Different Speeds , 2003 .

[21]  M. Plummer,et al.  A Bayesian information criterion for singular models , 2013, 1309.0911.

[22]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[23]  P. McCullagh.  Regression Models for Ordinal Data , 1980 .

[24]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[25]  A. Agresti,et al.  The analysis of ordered categorical data: An overview and a survey of recent developments , 2005 .

[26]  Yu Hayakawa,et al.  Capture–Recapture Estimation Using Finite Mixtures of Arbitrary Dimension , 2010, Biometrics.

[27]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[28]  Aki Vehtari,et al.  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC , 2015, Statistics and Computing.

[29]  P. McNicholas.  Mixture Model-Based Classification , 2016 .

[30]  D. Balding,et al.  Approximate Bayesian computation in population genetics. , 2002, Genetics.

[31]  D. Fernández,et al.  Model selection for mixture‐based clustering for ordinal data , 2016 .

[32]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[33]  Mohamed Nadif,et al.  Co-clustering for Binary and Categorical Data with Maximum Modularity , 2011, 2011 IEEE 11th International Conference on Data Mining.

[34]  D. Fernández,et al.  Mixture-based clustering for the ordered stereotype model , 2016, Comput. Stat. Data Anal..

[35]  B. Manly.  Multivariate Statistical Methods : A Primer , 1986 .

[36]  A. Agresti.  Analysis of Ordinal Categorical Data: Agresti/Analysis , 2010 .

[37]  Sumio Watanabe,et al.  Algebraic Geometry and Statistical Learning Theory: Contents , 2009 .

[38]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[39]  Gérard Govaert,et al.  Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  P. Diggle,et al.  Analysis of Longitudinal Data. , 1997 .

[41]  C. Robert,et al.  Deviance information criteria for missing data models , 2006 .

[42]  Christian P. Robert,et al.  Monte Carlo Statistical Methods (Springer Texts in Statistics) , 2005 .

[43]  Ritabrata Dutta,et al.  Likelihood-free inference via classification , 2014, Stat. Comput..

[44]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[45]  Paul S Albert,et al.  A mixture of transition models for heterogeneous longitudinal ordinal data: with applications to longitudinal bacterial vaginosis data , 2014, Statistics in medicine.

[46]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[47]  N. G. Best,et al.  The deviance information criterion: 12 years on , 2014 .

[48]  Benjamin Kedem,et al.  Regression Models for Time Series Analysis: Kedem/Time Series Analysis , 2005 .

[49]  Brian Everitt,et al.  Cluster analysis , 1974 .

[50]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[51]  Leland Wilkinson,et al.  The History of the Cluster Heat Map , 2009 .

[52]  M. Stephens.  Dealing with label switching in mixture models , 2000 .

[53]  Torsten Hothorn,et al.  A unified framework of constrained regression , 2014, Stat. Comput..

[54]  Volodymyr Melnykov,et al.  Finite mixture models and model-based clustering , 2010 .

[55]  S. Chib,et al.  Bayesian residual analysis for binary response regression models , 1995 .

[56]  Ivy Liu,et al.  Biclustering Models for Two-Mode Ordinal Data , 2016, Psychometrika.

[57]  Richard Arnold,et al.  Multivariate methods using mixtures: Correspondence analysis, scaling and pattern-detection , 2014, Comput. Stat. Data Anal..

[58]  Aki Vehtari,et al.  Understanding predictive information criteria for Bayesian models , 2013, Statistics and Computing.

[59]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .