Clustering bivariate mixed-type data via the cluster-weighted model

The cluster-weighted model (CWM) is a mixture model with random covariates that allows for flexible clustering/classification and distribution estimation of a random vector composed of a response variable and a set of covariates. Within this class of models, the generalized linear exponential CWM is here introduced especially for modeling bivariate data of mixed-type. Its natural counterpart in the family of latent class models is also defined. Maximum likelihood parameter estimates are derived using the expectation-maximization algorithm and some computational issues are detailed. Through Monte Carlo experiments, the classification performance of the proposed model is compared with other mixture-based approaches, consistency of the estimators of the regression coefficients is evaluated, and several likelihood-based information criteria are compared for selecting the number of mixture components. An application to real data is also finally considered.

[1]  Chun Yu,et al.  Robust mixture regression using the t-distribution , 2014, Comput. Stat. Data Anal..

[2]  Peter Schlattmann,et al.  Medical Applications of Finite Mixture Models , 2009 .

[3]  Margarida G. M. S. Cardoso,et al.  Retail Clients Latent Segments , 2005, EPIA.

[4]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[5]  H. Joe.  Asymptotic efficiency of the two-stage estimation method for copula-based models , 2005 .

[6]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[7]  Dimitris Karlis,et al.  Choosing Initial Values for the EM Algorithm for Finite Mixtures , 2003, Comput. Stat. Data Anal..

[8]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[9]  W. DeSarbo,et al.  A mixture likelihood approach for generalized linear models , 1995 .

[10]  Antonio Punzo,et al.  Finite mixtures of unimodal beta and gamma densities and the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k$$\end{d , 2012, Computational Statistics.

[11]  Gérard Govaert,et al.  Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  B. Lindsay,et al.  The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family , 1994 .

[13]  F. Leisch.  FlexMix: A general framework for finite mixture models and latent class regression in R , 2004 .

[14]  A. Punzo.  Flexible mixture modelling with the polynomial Gaussian cluster-weighted model , 2012, 1207.0939.

[15]  Ryan P. Browne,et al.  Model-based clustering, classification, and discriminant analysis of data with mixed type , 2012 .

[16]  M. Stephens.  Dealing with label switching in mixture models , 2000 .

[17]  Salvatore Ingrassia,et al.  Parsimonious Generalized Linear Gaussian Cluster-Weighted Models , 2015 .

[18]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[19]  C. Robert,et al.  Computational and Inferential Difficulties with Mixture Posterior Distributions , 2000 .

[20]  Grant Henning,et al.  Meanings and implications of the principle of local independence , 1989 .

[21]  F. Leisch,et al.  FlexMix Version 2: Finite Mixtures with Concomitant Variables and Varying and Constant Parameters , 2008 .

[22]  Jaime R. S. Fonseca,et al.  On the Performance of Information Criteria in Latent Segment Models , 2010 .

[23]  M. Sklar.  Fonctions de repartition a n dimensions et leurs marges , 1959 .

[24]  Athanasios Tsanas,et al.  Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools , 2012 .

[25]  Christian Hennig,et al.  Identifiablity of Models for Clusterwise Linear Regression , 2000, J. Classif..

[26]  M. Wedel,et al.  Market Segmentation: Conceptual and Methodological Foundations , 1997 .

[27]  R. Nelsen.  An Introduction to Copulas , 1998 .

[28]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[29]  Giorgio Vittadini,et al.  Local Statistical Modeling via a Cluster-Weighted Approach with Elliptical Distributions , 2012, J. Classif..

[30]  Lynette A. Hunt,et al.  Fitting a Mixture Model to Three-Mode Three-Way Data with Categorical and Continuous Variables , 1999 .

[31]  Christophe Biernacki,et al.  Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models , 2003, Comput. Stat. Data Anal..

[32]  Weixin Yao,et al.  Model based labeling for mixture models , 2012, Stat. Comput..

[33]  R. Nelsen.  An Introduction to Copulas (Springer Series in Statistics) , 2006 .

[34]  Paul D. McNicholas,et al.  Robust Clustering in Regression Analysis via the Contaminated Gaussian Cluster-Weighted Model , 2014, J. Classif..

[35]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[36]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[37]  Dimitris Karlis,et al.  A finite mixture of bivariate Poisson regression models with an application to insurance ratemaking , 2012, Comput. Stat. Data Anal..

[38]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[39]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[40]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[41]  Stanley L. Sclove,et al.  Theory and methodology of time series analysis , 1994 .

[42]  Lynette A. Hunt,et al.  Clustering mixed data , 2011, WIREs Data Mining Knowl. Discov..

[43]  Robert H. Shumway,et al.  The model selection criterion AICu , 1997 .

[44]  N. Balakrishnan,et al.  Continuous Bivariate Distributions , 2009 .

[45]  Salvatore Ingrassia,et al.  Model-based clustering via linear cluster-weighted models , 2012, Comput. Stat. Data Anal..

[46]  C. Hennig,et al.  How to find an appropriate clustering for mixed‐type variables with application to socio‐economic stratification , 2013 .

[47]  Paul D. McNicholas,et al.  Clustering and classification via cluster-weighted factor analyzers , 2012, Advances in Data Analysis and Classification.

[48]  Giorgio Vittadini,et al.  The Generalized Linear Mixed Cluster-Weighted Model , 2015, Journal of Classification.

[49]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[50]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[51]  Neil Gershenfeld,et al.  Nonlinear Inference and Cluster‐Weighted Modeling , 1997 .

[52]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[53]  Paul D. McNicholas,et al.  Cluster-weighted $$t$$t-factor analyzers for robust model-based clustering and dimension reduction , 2015, Stat. Methods Appl..

[54]  A. Raftery,et al.  Model-based Gaussian and non-Gaussian clustering , 1993 .

[55]  J. Vermunt,et al.  Latent class cluster analysis , 2002 .

[56]  C. Genest,et al.  A Primer on Copulas for Count Data , 2007, ASTIN Bulletin.