Evaluate the number of clusters in finite mixture models with the penalized histogram difference criterion

Abstract Aimed at the determination of the number of mixtures for finite mixture models (FMMs), in this work, a new method called the penalized histogram difference criterion (PHDC) is proposed and evaluated with other criteria such as Akaike information criterion (AIC), the minimum message length (MML), the information complexity (ICOMP) and the evidence of data criterion (EDC). The new method, which calculates the penalized histogram difference between the data generated from estimated FMMs and those for modeling purpose, turns out to be better than others for data with complicate mixtures patterns. It is demonstrated in this work that the PHDC can determine the optimal number of clusters of the FMM. Furthermore, the estimated FMMs asymptotically approximate the true model. The utility of the new method is demonstrated through synthetic data sets analysis and the batch-wise comparison of citric acid fermentation processes.

[1]  Otto Opitz,et al.  Information and Classification , 1993 .

[2]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[3]  Si-Zhao Joe Qin,et al.  Monitoring Non-normal Data with Principal Component Analysis and Adaptive Density Estimation , 2007, 2007 46th IEEE Conference on Decision and Control.

[4]  E. Martin,et al.  Pre-analysis of Multi-batch Bioprocesses Data with Finite Mixture Models in the Reduced Feature Subspace , 2010 .

[5]  Gene H. Golub,et al.  Matrix computations , 1983 .

[6]  William D. Penny,et al.  Bayesian Approaches to Gaussian Mixture Modeling , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[8]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Jin Hyun Park,et al.  Process monitoring using a Gaussian mixture model via principal component analysis and discriminant analysis , 2004, Comput. Chem. Eng..

[10]  Mark J. van der Laan,et al.  Fitting of mixtures with unspecified number of components using cross validation distance estimate , 2003, Comput. Stat. Data Anal..

[11]  S. Qin,et al.  Multimode process monitoring with Bayesian inference‐based finite Gaussian mixture models , 2008 .

[12]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[13]  Adelino R. Ferreira da Silva,et al.  A Dirichlet process mixture model for brain MRI tissue classification , 2007, Medical Image Anal..

[14]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[15]  N. J. A. Sloane,et al.  Sphere Packings, Lattices and Groups , 1987, Grundlehren der mathematischen Wissenschaften.

[16]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[18]  Elaine Martin,et al.  Batch process modelling with mixtures of Gaussian processes , 2008, Neural Computing and Applications.

[19]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[20]  John D. Kalbfleisch,et al.  Penalized minimum‐distance estimates in finite mixture models , 1996 .

[21]  H. Bozdogan.  Choosing the Number of Component Clusters in the Mixture-Model Using a New Informational Complexity Criterion of the Inverse-Fisher Information Matrix , 1993 .

[22]  H. Akaike.  A new look at the statistical model identification , 1974 .

[23]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .

[24]  D. Harville.  Matrix Algebra From a Statistician's Perspective , 1998 .

[25]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[26]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[27]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[28]  H. Bozdogan,et al.  Akaike's Information Criterion and Recent Developments in Information Complexity. , 2000, Journal of mathematical psychology.