Adaptive parameter estimation of GMM and its application in clustering

Abstract Parameter estimation of Gaussian mixture model (GMM) has begun to gain attention in the field of science and engineering, and it has gradually become one of the most popular research topics. In particular, the rapid development of theoretical progress on globally optimal convergence promotes the widespread application of Gaussian mixtures in data clustering. So this paper introduces a novel parameter estimation algorithm called TDAVBEM, which combines the Tsallis entropy and a deterministic annealing (DA) algorithm on the basis of the variational bayesian expected maximum (VBEM) to simultaneously implement the parameter estimation and select the optimal components of GMM. We experimentally certified the effectiveness and robustness of our proposed algorithm passes through comparing it with several parameter evaluation methods and its application in data clustering.

[1]  Fredrik Lindsten,et al.  Interacting Particle Markov Chain Monte Carlo , 2016, ICML.

[2]  Veronika Rockova,et al.  EMVS: The EM Approach to Bayesian Variable Selection , 2014 .

[3]  Peng Chen,et al.  Uncertainty propagation using infinite mixture of Gaussian processes and variational Bayesian inference , 2015, J. Comput. Phys..

[4]  Xiangtao Li,et al.  Evolutionary Multiobjective Clustering and Its Applications to Patient Stratification , 2019, IEEE Transactions on Cybernetics.

[5]  Naonori Ueda,et al.  Deterministic annealing EM algorithm , 1998, Neural Networks.

[6]  Débora C. Muchaluat-Saade,et al.  Thermal Signal Analysis for Breast Cancer Risk Verification , 2015, MedInfo.

[7]  Sanghamitra Bandyopadhyay,et al.  A New Symmetry Based Genetic Clustering Technique for Automatic Evolution of Clusters , 2006 .

[8]  Daniel Gildea,et al.  Convergence of the EM Algorithm for Gaussian Mixtures with Unbalanced Mixing Coefficients , 2012, ICML.

[9]  Florence Forbes,et al.  Location and scale mixtures of Gaussians with flexible tail behaviour: Properties, inference and application to multivariate clustering , 2015, Comput. Stat. Data Anal..

[10]  Yuan Yan Tang,et al.  Maximum correntropy criterion for convex anc semi-nonnegative matrix factorization , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[11]  Satyam Maheshwari,et al.  Phrase based Clustering Scheme of Suffix Tree Document Clustering Model , 2013 .

[12]  Yuhang Liu,et al.  Frame-Based Variational Bayesian Learning for Independent or Dependent Source Separation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Abbas Babajani-Feremi,et al.  Development of a variational Bayesian expectation maximization (VBEM) method for model inversion of multi-area E/MEG model , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[14]  Pierre Alquier,et al.  1-Bit matrix completion: PAC-Bayesian analysis of a variational approximation , 2016, Machine Learning.

[15]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Lin Li,et al.  Simulated Annealing Algorithm Coupled With a Deterministic Method for Parameter Extraction of Energetic Hysteresis Model , 2018, IEEE Transactions on Magnetics.

[17]  Olatz Arbelaitz,et al.  An extensive comparative study of cluster validity indices , 2013, Pattern Recognit..

[18]  Biao Huang,et al.  Variational Bayesian Approach for Causality and Contemporaneous Correlation Features Inference in Industrial Process Data , 2019, IEEE Transactions on Cybernetics.

[19]  Liang Chen,et al.  Silhouette Index Supervised Affinity Propagation Clustering , 2013 .

[20]  John B. Thomas,et al.  Maximum likelihood estimation of mixture constants and locally optimal detection of contaminants , 1989 .

[21]  Koji Tsumura,et al.  Deterministic quantum annealing expectation-maximization algorithm , 2017, Journal of Statistical Mechanics: Theory and Experiment.

[22]  Makoto Yasuda,et al.  Deterministic Annealing Approach to Fuzzy C-Means Clustering Based on Entropy Maximization , 2011, Adv. Fuzzy Syst..

[23]  Donald C. Wunsch,et al.  A Comparison Study of Validity Indices on Swarm-Intelligence-Based Clustering , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Lennart Svensson,et al.  Variational Bayesian Expectation Maximization for Radar Map Estimation , 2016, IEEE Transactions on Signal Processing.

[25]  Farhan Abrol,et al.  Deterministic Annealing for Stochastic Variational Inference , 2014, ArXiv.

[26]  Seungjin Choi,et al.  Probabilistic Models for Common Spatial Patterns: Parameter-Expanded EM and Variational Bayes , 2012, AAAI.

[27]  Juan Manuel Sáez,et al.  EBEM: An Entropy-based EM Algorithm for Gaussian Mixture Models , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[28]  Virginia L. Ballarin,et al.  Analysis of genetic association in Listeria and Diabetes using Hierarchical Clustering and Silhouette Index , 2016 .

[29]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[30]  Kaare Brandt Petersen,et al.  On the Slow Convergence of EM and VBEM in Low-Noise Linear Models , 2005, Neural Computation.

[31]  Pierrick Bruneau,et al.  Parsimonious variational-Bayes mixture aggregation with a Poisson prior , 2009, 2009 17th European Signal Processing Conference.

[32]  Ethem Alpaydin,et al.  Soft vector quantization and the EM algorithm , 1998, Neural Networks.

[33]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[34]  Djamel Bouchaffra,et al.  Genetic-based EM algorithm for learning Gaussian mixture models , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  N. Nasios,et al.  Variational learning for Gaussian mixture models , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Arnaud Doucet,et al.  Expectation-maximization algorithms for inference in Dirichlet processes mixture , 2011, Pattern Analysis and Applications.

[37]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[38]  Miin-Shen Yang,et al.  A robust EM clustering algorithm for Gaussian mixture models , 2012, Pattern Recognit..

[39]  Fabio Valente,et al.  Variational Bayesian GMM for speech recognition , 2003, INTERSPEECH.

[40]  Peng Yu,et al.  A data mining paradigm for identifying key factors in biological processes using gene expression data , 2018, Scientific Reports.

[41]  Ali Ghaffari,et al.  A routing protocol for vehicular ad hoc networks using simulated annealing algorithm and neural networks , 2018, The Journal of Supercomputing.

[42]  Maria Teresinha Arns Steiner,et al.  Clustering of solar energy facilities using a hybrid fuzzy c-means algorithm initialized by metaheuristics , 2018, Journal of Cleaner Production.

[43]  Nikos A. Vlassis,et al.  A Greedy EM Algorithm for Gaussian Mixture Learning , 2002, Neural Processing Letters.

[44]  Maryam Fatemi,et al.  Variational Bayesian EM for SLAM , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).