Fuzzy c-means clustering using Jeffreys-divergence based similarity measure

Abstract In clustering, similarity measure has been one of the major factors for discovering the natural grouping of a given dataset by identifying hidden patterns. To determine a suitable similarity measure is an open problem in clustering analysis for several years. The purpose of this study is to make known a divergence based similarity measure. The notion of the proposed similarity measure is derived from Jeffrey-divergence. Various features of the proposed similarity measure are explained. Afterwards we develop fuzzy c-means (FCM) by making use of the proposed similarity measure, which guarantees to converge to local minima. The various characteristics of the modified FCM algorithm are also addressed. Some well known real-world and synthetic datasets are considered for the experiments. In addition to that two remote sensing image datasets are also adopted in this work to illustrate the effectiveness of the proposed FCM over some existing methods. All the obtained results demonstrate that FCM with divergence based proposed similarity measure outperforms three latest FCM algorithms.

[1]  Ondrej Krejcar,et al.  Fuzzy K-Means Using Non-Linear S-Distance , 2019, IEEE Access.

[2]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[3]  Shinto Eguchi,et al.  Spontaneous Clustering via Minimum Gamma-Divergence , 2014, Neural Computation.

[4]  Ujjwal Maulik,et al.  Performance Evaluation of Some Clustering Algorithms and Validity Indices , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jianbin Qiu,et al.  State Estimation in Nonlinear System Using Sequential Evolutionary Filter , 2016, IEEE Transactions on Industrial Electronics.

[6]  Jianbin Qiu,et al.  Fault Detection for Nonlinear Process With Deterministic Disturbances: A Just-In-Time Learning Based Data Driven Method , 2017, IEEE Transactions on Cybernetics.

[7]  Mita Nasipuri,et al.  Multi-scale RoIs selection for classifying multi-spectral images , 2019, Multidimens. Syst. Signal Process..

[8]  W. Peizhuang.  Pattern Recognition with Fuzzy Objective Function Algorithms (James C. Bezdek) , 1983 .

[9]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[10]  Eric Grivel,et al.  Jeffrey's divergence between moving-average models that are real or complex, noise-free or disturbed by additive white noises , 2017, Signal Process..

[11]  Swagatam Das,et al.  Geometric divergence based fuzzy clustering with strong resilience to noise features , 2016, Pattern Recognit. Lett..

[12]  Frank Nielsen,et al.  Total Jensen divergences: Definition, properties and clustering , 2013, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Dongdai Lin,et al.  Robust Face Clustering Via Tensor Decomposition , 2015, IEEE Transactions on Cybernetics.

[14]  James Bailey,et al.  Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance , 2010, J. Mach. Learn. Res..

[15]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[16]  Ayan Seal,et al.  Modeling uncertain data using Monte Carlo integration method for clustering , 2019, Expert Syst. Appl..

[17]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[18]  Shen Yin,et al.  Performance Monitoring for Vehicle Suspension System via Fuzzy Positivistic C-Means Clustering Based on Accelerometer Measurements , 2015, IEEE/ASME Transactions on Mechatronics.

[19]  Guo Li,et al.  A Big Data Clustering Algorithm for Mitigating the Risk of Customer Churn , 2016, IEEE Transactions on Industrial Informatics.

[20]  J. Jäkel,et al.  A New Convergence Proof of Fuzzy c-Means , 2005, IEEE Transactions on Fuzzy Systems.

[21]  Frank Nielsen,et al.  On Clustering Histograms with k-Means by Using Mixed α-Divergences , 2014, Entropy.

[22]  Tao Wu,et al.  Automated Graph Regularized Projective Nonnegative Matrix Factorization for Document Clustering , 2014, IEEE Transactions on Cybernetics.

[23]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[24]  Swagatam Das,et al.  k-Means clustering with a new divergence-based distance metric: Convergence and performance analysis , 2017, Pattern Recognit. Lett..

[25]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[26]  James C. Bezdek,et al.  Sequential Competitive Learning and the Fuzzy c-Means Clustering Algorithms , 1996, Neural Networks.

[27]  Frank Klawonn,et al.  A contribution to convergence theory of fuzzy c-means and derivatives , 2003, IEEE Trans. Fuzzy Syst..

[28]  Frank Nielsen,et al.  On Conformal Divergences and Their Population Minimizers , 2013, IEEE Transactions on Information Theory.

[29]  Jerry M. Mendel,et al.  Optimality tests for the fuzzy c-means algorithm , 1994, Pattern Recognit..