Applying fuzzy EM algorithm with a fast convergence to GMMs

Inspired from the mechanism of Fuzzy C-means (FCMs) which introduces a degree of fuzziness on the dissimilarity function based on distances, a fuzzy Expectation Maximization (EM) algorithm for Gaussian Mixture Models (GMMs) is proposed in this paper. In the fuzzy EM algorithm, the dissimilarity function is defined as the multiplicative inverse of probability density function. Different from FCMs, the defined dissimilarity function is based on the exponential function of the distance. The fuzzy EM algorithm is compared with normal EM algorithm in terms of fitting degree and convergence speed. The experimental results in modeling random data and various characters demonstrate the ability of the proposed algorithm in reducing the computational cost of GMMs.

[1]  Jeff A. Bilmes,et al.  A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[2]  M. Montagnuolo,et al.  Automatic Genre Classification of TV Programmes Using Gaussian Mixture Models and Neural Networks , 2007 .

[3]  Weixin Xie,et al.  Weighting exponent m in fuzzy C-means (FCM) clustering algorithm , 2001, International Symposium on Multispectral Image Processing and Pattern Recognition.

[4]  Aladdin M. Ariyaeeinia,et al.  Efficient Speaker Change Detection Using Adapted Gaussian Mixture Models , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  Hiok Chai Quek,et al.  Driving Profile Modeling and Recognition Based on Soft Computing Approach , 2009, IEEE Transactions on Neural Networks.

[6]  Peter Vojtáš,et al.  Integrating user and group preferences for top-k search from distributed web resources , 2007 .

[7]  Constantine Kotropoulos,et al.  Gaussian Mixture Modeling by Exploiting the Mahalanobis Distance , 2008, IEEE Transactions on Signal Processing.

[8]  James C. Bezdek,et al.  On cluster validity for the fuzzy c-means model , 1995, IEEE Trans. Fuzzy Syst..

[9]  Martin Vondra,et al.  Recognition of Emotions in German Speech Using Gaussian Mixture Models , 2008, COST 2102 School.

[10]  Sadaaki Miyamoto,et al.  Fuzzy c-means as a regularization and maximum entropy approach , 1997 .

[11]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[12]  Hidetomo Ichihashi,et al.  Fuzzy c-means clustering with regularization by K-L information , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[13]  J. B. Jordan,et al.  On the optimal choice of parameters in a fuzzy c-means algorithm , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[14]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[15]  Katsuhiro HONDA,et al.  Fuzzy c-Means Clustering with Regularization by KL Information , 2001 .

[16]  Jian Yu,et al.  Analysis of the weighting exponent in the FCM , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[17]  Ronald R. Yager,et al.  Participatory Learning With Granular Observations , 2009, IEEE Transactions on Fuzzy Systems.

[18]  Yannis Stylianou,et al.  Wrapped Gaussian Mixture Models for Modeling and High-Rate Quantization of Phase Data of Speech , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Honghai Liu,et al.  Fuzzy Qualitative Robot Kinematics , 2008, IEEE Transactions on Fuzzy Systems.

[20]  R. Hathaway.  Another interpretation of the EM algorithm for mixture distributions , 1986 .

[21]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Shuji Tsukiyama,et al.  A New Statistical Timing Analysis Using Gaussian Mixture Models for Delay and Slew Propagated Together , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[23]  Thomas Eriksson,et al.  Packet Video Error Concealment With Gaussian Mixture Models , 2008, IEEE Transactions on Image Processing.

[24]  Honghai Liu,et al.  Advances in View-Invariant Human Motion Analysis: A Review , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[25]  Xiangyang Zhu,et al.  Dynamic Grasp Recognition Using Time Clustering, Gaussian Mixture Models and Hidden Markov Models , 2008, ICIRA.

[26]  Joon-Hyuk Chang,et al.  Analysis and Improvement of Speech/Music Classification for 3GPP2 SMV Based on GMM , 2008, IEEE Signal Processing Letters.

[27]  Honghai Liu,et al.  Fuzzy Qualitative Human Motion Analysis , 2009, IEEE Transactions on Fuzzy Systems.

[28]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[29]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[30]  Dat Tran,et al.  Fuzzy Gaussian mixture models for speaker recognition , 1998, ICSLP.

[31]  Honghai Liu,et al.  A Fuzzy Qualitative Framework for Connecting Robot Qualitative and Quantitative Representations , 2008, IEEE Transactions on Fuzzy Systems.

[32]  Xiaoyan Wang,et al.  Adaptive Gaussian Mixture Models Based Facial Actions Tracking , 2008, 2008 International Conference on Computer Science and Software Engineering.

[33]  Herbert Gish,et al.  Discriminatively Trained GMMs for Language Classification Using Boosting Methods , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[34]  Mattias Nilsson,et al.  On entropy-constrained vector quantization using gaussian mixture models , 2008, IEEE Transactions on Communications.

[35]  Joseph Tabrikian,et al.  MIMO-AR System Identification and Blind Source Separation for GMM-Distributed Sources , 2009, IEEE Transactions on Signal Processing.