A method for training finite mixture models under a fuzzy clustering principle

In this paper, we establish a novel regard towards fuzzy clustering, showing it provides a sound framework for fitting finite mixture models. We propose a novel fuzzy clustering-type methodology for finite mixture model fitting, effected by utilizing a regularized form of the fuzzy c-means (FCM) algorithm, and introducing a proper dissimilarity functional for the algorithm with respect to the probabilistic properties of the model being treated. We apply the proposed methodology in a number of popular finite mixture models, and the corresponding expressions of the fuzzy model fitting algorithm are derived. We examine the efficacy of our novel approach in both clustering and classification applications of benchmark data sets, and we demonstrate the advantages of the proposed approach over maximum-likelihood.

[1]  R. Hathaway Another interpretation of the EM algorithm for mixture distributions , 1986 .

[2]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[3]  Sadaaki Miyamoto,et al.  Fuzzy Clustering by Quadratic Regularization , 1998 .

[4]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[5]  Enrique H. Ruspini,et al.  A New Approach to Clustering , 1969, Inf. Control..

[6]  James M. Keller,et al.  Fuzzy Models and Algorithms for Pattern Recognition and Image Processing , 1999 .

[7]  Geoffrey J. McLachlan,et al.  Extension of the mixture of factor analyzers model to incorporate the multivariate t-distribution , 2007, Comput. Stat. Data Anal..

[8]  Hidetomo Ichihashi,et al.  Gaussian Mixture PDF Approximation and Fuzzy c-Means Clustering with Entropy Regularization , 2000 .

[9]  Hidetomo Ichihashi,et al.  Regularized linear fuzzy clustering and probabilistic PCA mixture models , 2005, IEEE Transactions on Fuzzy Systems.

[10]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[11]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[12]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[13]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Naonori Ueda,et al.  Deterministic annealing EM algorithm , 1998, Neural Networks.

[15]  Ai Hui Tan,et al.  From a Gaussian mixture model to additive fuzzy systems , 2005, IEEE Transactions on Fuzzy Systems.

[16]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[17]  D. Rubin,et al.  ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME , 1999 .