The infinite Student's t-factor mixture analyzer for robust clustering and classification

Recently, the Student's t-factor mixture analyzer (tFMA) has been proposed. Compared with the mixture of Student's t-factor analyzers (MtFA), the tFMA has better performance when processing high-dimensional data. Moreover, the factors estimated by the tFMA can be visualized in a low-dimensional latent space, which is not shared by the MtFA. However, as the tFMA belongs to finite mixtures and the related parameter estimation method is based on the maximum likelihood criterion, it could not automatically determine the appropriate model complexity according to the observed data, leading to overfitting. In this paper, we propose an infinite Student's t-factor mixture analyzer (itFMA) to handle this issue. The itFMA is based on the nonparametric Bayesian statistics which assumes infinite number of mixing components in advance, and automatically determines the proper number of components after observing the high-dimensional data. Moreover, we derive an efficient variational inference algorithm for the itFMA. The proposed itFMA and the related variational inference algorithm are used to cluster and classify high-dimensional data. Experimental results of some applications show that the itFMA has good generalization capacity, offering a more robust and powerful performance than other competing approaches.

[1]  Volkan Cevher,et al.  Learning Low-Dimensional Signal Models , 2011, IEEE Signal Processing Magazine.

[2]  Geoffrey J. McLachlan,et al.  Robust mixture modelling using the t distribution , 2000, Stat. Comput..

[3]  B. Lindsay.  Mixture models : theory, geometry, and applications , 1995 .

[4]  Chunguang Li,et al.  The Student's $t$-Hidden Markov Model With Truncated Stick-Breaking Priors , 2011, IEEE Signal Processing Letters.

[5]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[6]  B. Lindsay,et al.  The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family , 1994 .

[7]  Sotirios Chatzis,et al.  Signal Modeling and Classification Using a Robust Latent Space Model Based on $t$ Distributions , 2008, IEEE Transactions on Signal Processing.

[8]  David J. Bartholomew,et al.  Latent Variable Models and Factor Analysis: A Unified Approach , 2011 .

[9]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[10]  Pierrick Bruneau,et al.  Parsimonious reduction of Gaussian mixture models with a variational-Bayes approach , 2010, Pattern Recognit..

[11]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[12]  Chunguang Li,et al.  The infinite Student's t-mixture for robust modeling , 2012, Signal Process..

[13]  Cinzia Viroli,et al.  Dimensionally Reduced Model-Based Clustering Through Mixtures of Factor Mixture Analyzers , 2010, J. Classif..

[14]  Nizar Bouguila,et al.  A Dirichlet Process Mixture of Generalized Dirichlet Distributions for Proportional Data Modeling , 2010, IEEE Transactions on Neural Networks.

[15]  P. McNicholas,et al.  Mixtures of modified t-factor analyzers for model-based clustering, classification, and discriminant , 2011 .

[16]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[17]  Dit-Yan Yeung,et al.  Time series clustering with ARMA mixtures , 2004, Pattern Recognit..

[18]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[19]  Paul D. McNicholas,et al.  Parsimonious Gaussian mixture models , 2008, Stat. Comput..

[20]  Geoffrey J. McLachlan,et al.  Mixtures of common t-factor analyzers for clustering high-dimensional microarray data , 2011, Bioinform..

[21]  Paul D. McNicholas,et al.  Model-based clustering of microarray expression data via latent Gaussian mixture models , 2010, Bioinform..

[22]  M. Opper,et al.  Advanced mean field methods: theory and practice , 2001 .

[23]  Sumio Watanabe,et al.  Singularities in mixture models and upper bounds of stochastic complexity , 2003, Neural Networks.

[24]  Alfred Binet.  Spearman The proof and measurement of association between two things; General intelligence objectively determined and measured , 1904 .

[25]  Geoffrey J. McLachlan,et al.  A mixture model-based approach to the clustering of microarray expression data , 2002, Bioinform..

[26]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[27]  Geoffrey J. McLachlan,et al.  Mixtures of Factor Analyzers with Common Factor Loadings: Applications to the Clustering and Visualization of High-Dimensional Data , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  A. Montanari,et al.  Heteroscedastic factor mixture analysis , 2010 .

[29]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[30]  L. Hubert,et al.  Comparing partitions , 1985 .

[31]  Geoffrey J. McLachlan,et al.  Robust Cluster Analysis via Mixtures of Multivariate t-Distributions , 1998, SSPR/SPR.

[32]  C. Spearman.  The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[33]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[34]  Paul D. McNicholas,et al.  Serial and parallel implementations of model-based clustering via parsimonious Gaussian mixture models , 2010, Comput. Stat. Data Anal..

[35]  Geoffrey J. McLachlan,et al.  Extension of the mixture of factor analyzers model to incorporate the multivariate t-distribution , 2007, Comput. Stat. Data Anal..

[36]  Angela Montanari,et al.  Maximum likelihood estimation of mixtures of factor analyzers , 2011, Comput. Stat. Data Anal..

[37]  Samuel J. Gershman,et al.  A Tutorial on Bayesian Nonparametric Models , 2011, 1106.2697.

[38]  Volodymyr Melnykov,et al.  Finite mixture models and model-based clustering , 2010 .

[39]  P. McNicholas,et al.  Extending mixtures of multivariate t-factor analyzers , 2011, Stat. Comput..

[40]  J. Downing,et al.  Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. , 2002, Cancer cell.