Unsupervised learning of mixture regression models for longitudinal data

This paper is concerned with learning of mixture regression models for individuals that are measured repeatedly. The adjective "unsupervised" implies that the number of mixing components is unknown and has to be determined, ideally by data driven tools. For this purpose, a novel penalized method is proposed to simultaneously select the number of mixing components and to estimate the mixing proportions and unknown parameters in the models. The proposed method is capable of handling both continuous and discrete responses by only requiring the first two moment conditions of the model distribution. It is shown to be consistent in both selecting the number of components and estimating the mixing proportions and unknown regression parameters. Further, a modified EM algorithm is developed to seamlessly integrate model selection and estimation. Simulation studies are conducted to evaluate the finite sample performance of the proposed procedure. And it is further illustrated via an analysis of a primary biliary cirrhosis data set.

[1]  Arnovst Kom'arek,et al.  Clustering for multivariate continuous and discrete longitudinal data , 2013, 1304.4448.

[2]  Emmanuel Lesaffre,et al.  Generalized linear mixed model with a penalized Gaussian mixture as a random effects distribution , 2008, Comput. Stat. Data Anal..

[3]  G. Celeux,et al.  Mixture of linear mixed models for clustering gene expression profiles from repeated microarray experiments , 2005 .

[4]  E. Gassiat,et al.  Testing in locally conic models, and application to mixture models , 1997 .

[5]  Weixin Yao,et al.  Label switching and its solutions for frequentist mixture models , 2015 .

[6]  G. Casella,et al.  Clustering using objective functions and stochastic search , 2008 .

[7]  Lan Wang,et al.  GEE analysis of clustered binary data with diverging number of covariates , 2011, 1103.1795.

[8]  Peirong Xu,et al.  Estimation for a marginal generalized single-index longitudinal model , 2012, J. Multivar. Anal..

[9]  Gerhard Tutz,et al.  Clustering in linear‐mixed models with a group fused lasso penalty , 2014, Biometrical journal. Biometrische Zeitschrift.

[10]  E. Gassiat,et al.  Testing the order of a model using locally conic parametrization : population mixtures and stationary ARMA processes , 1999 .

[11]  A. Maruotti.  Mixed Hidden Markov Models for Longitudinal Data: An Overview , 2011 .

[12]  Christophe Genolini,et al.  KmL: k-means for longitudinal data , 2010, Comput. Stat..

[13]  Jiahua Chen,et al.  Order Selection in Finite Mixture Models With a Nonsmooth Penalty , 2008 .

[14]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[15]  Peirong Xu,et al.  Efficient estimation for marginal generalized partially linear single-index models with longitudinal data , 2016 .

[16]  Runze Li,et al.  Tuning parameter selectors for the smoothly clipped absolute deviation method. , 2007, Biometrika.

[17]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[18]  P. Grambsch,et al.  Prognosis in primary biliary cirrhosis: Model for decision making , 1989, Hepatology.

[19]  K. Lindor,et al.  Primary biliary cirrhosis , 2003, The Lancet.

[20]  P. Grambsch,et al.  Primary biliary cirrhosis: Prediction of short‐term survival based on repeated patient visits , 1994, Hepatology.

[21]  M. Kaplan,et al.  Primary biliary cirrhosis , 1998, Hepatology.

[22]  A. Pickles,et al.  Latent mixture models for multivariate and longitudinal outcomes , 2010, Statistical methods in medical research.

[23]  J. Hoofnagle,et al.  Randomized trial of chlorambucil for primary biliary cirrhosis. , 1986, Gastroenterology.

[24]  J. Wooldridge,et al.  Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances , 1992 .

[25]  C. Hennig.  Breakdown points for maximum likelihood estimators of location–scale mixtures , 2004, math/0410073.

[26]  G. Verbeke,et al.  A Linear Mixed-Effects Model with Heterogeneity in the Random-Effects Population , 1996 .

[27]  Gerhard Tutz,et al.  Clustering in linear mixed models with approximate Dirichlet process mixtures using EM algorithm , 2013 .

[28]  T. Ferguson.  A Course in Large Sample Theory , 1996 .

[29]  S. Zeger,et al.  Longitudinal data analysis using generalized linear models , 1986 .

[30]  Yanan Fan,et al.  Relabelling algorithms for mixture models with applications for large data sets , 2016 .

[31]  Tao Huang,et al.  Model Selection for Gaussian Mixture Models , 2013, 1301.3558.

[32]  Annie Qu,et al.  Efficient classification for longitudinal data , 2014, Comput. Stat. Data Anal..

[33]  J. A. Roth,et al.  A patient with primary biliary cirrhosis and multiple sclerosis. , 1992, The American journal of medicine.

[34]  A. Raftery,et al.  Detecting features in spatial point processes with clutter via model-based clustering , 1998 .

[35]  Fernando A. Quintana,et al.  Model-based clustering for longitudinal data , 2008, Comput. Stat. Data Anal..

[36]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[37]  Jianhua Z. Huang,et al.  Efficient estimation in marginal partially linear models for longitudinal/clustered data using splines , 2007 .

[38]  B. Leroux.  Consistent estimation of a mixing distribution , 1992 .

[39]  R. W. Wedderburn.  Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method , 1974 .

[40]  E. Erosheva,et al.  Breaking Bad: Two Decades of Life-Course Data Analysis in Criminology, Developmental Psychology, and Beyond , 2014 .

[41]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .