Axiomatic generalization of the membership degree weighting function for fuzzy C means clustering: Theoretical development and convergence analysis

For decades practitioners have been using the center-based partitional clustering algorithms like Fuzzy C Means (FCM), which rely on minimizing an objective function, comprising of an appropriately weighted sum of distances of each data point from the cluster representatives. Numerous generalizations in terms of choice of the distance function have been introduced for FCM since its inception. In a stark contrast to this fact, to the best of our knowledge, there has not been any significant effort to address the issue of convergence of the algorithm under the structured generalization of the weighting function. Here, by generalization we mean replacing the conventional weighting function uijm (where uij indicates the membership of data xi to cluster Cj and m is the real-valued fuzzifier with 1 m < ) with a more general function g(uij) which can assume a wide variety of forms under some mild assumptions. In this article, for the first time, we present a novel axiomatic development of the general weighting function based on the membership degrees. We formulate the fuzzy clustering problem along with the intuitive justification of the technicalities behind the axiomatic approach. We develop an Alternative Optimization (AO) procedure to solve the main clustering problem by solving the partial optimization problems in an iterative way. The novelty of the article lies in the development of an axiomatic generalization of the weighting function of FCM, formulation of an AO algorithm for the fuzzy clustering problem with the extension to this general class of weighting functions, and a detailed convergence analysis of the proposed algorithm.

[1]  Frank Klawonn,et al.  A contribution to convergence theory of fuzzy c-means and derivatives , 2003, IEEE Trans. Fuzzy Syst..

[2]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[3]  James C. Bezdek,et al.  C-means Clustering with the Ll and L∞ Norms , 1991, IEEE Transactions on Systems, Man and Cybernetics.

[4]  W. T. Tucker,et al.  Convergence theory for fuzzy c-means: Counterexamples and repairs , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  Ka Yee Yeung,et al.  Details of the Adjusted Rand index and Clustering algorithms Supplement to the paper “ An empirical study on Principal Component Analysis for clustering gene expression data ” ( to appear in Bioinformatics ) , 2001 .

[6]  S. Miyamoto,et al.  Algorithms for L1 and Lp Fuzzy c-Means and Their Convergence , 1998 .

[7]  Frank Klawonn,et al.  What Is Fuzzy about Fuzzy Clustering? Understanding and Improving the Concept of the Fuzzifier , 2003, IDA.

[8]  Jian Xiao,et al.  Enhanced interval type-2 fuzzy c-means algorithm with improved initial center , 2014, Pattern Recognit. Lett..

[9]  Abdulkadir Sengür,et al.  NCM: Neutrosophic c-means clustering algorithm , 2015, Pattern Recognit..

[10]  Anil K. Jain Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[11]  James C. Bezdek,et al.  Generalized fuzzy c-means clustering strategies using Lp norm distances , 2000, IEEE Trans. Fuzzy Syst..

[12]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[13]  Jian Yu,et al.  Analysis of the weighting exponent in the FCM , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[14]  Athanasios A. Rontogiannis,et al.  A Novel Adaptive Possibilistic Clustering Algorithm , 2014, IEEE Transactions on Fuzzy Systems.

[15]  Jens Jäkel,et al.  A New Convergence Proof of Fuzzy c-Means , 2005, IEEE Trans. Fuzzy Syst..

[16]  James C. Bezdek,et al.  Convergence of Alternating Optimization , 2003, Neural Parallel Sci. Comput..

[17]  Ka Yee Yeung,et al.  Principal component analysis for clustering gene expression data , 2001, Bioinform..

[18]  Lawrence O. Hall,et al.  Convergence of the Single-Pass and Online Fuzzy C-Means Algorithms , 2011, IEEE Transactions on Fuzzy Systems.

[19]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[20]  Frank Nielsen,et al.  Sided and Symmetrized Bregman Centroids , 2009, IEEE Transactions on Information Theory.

[21]  Frank Klawonn,et al.  Fuzzy Clustering Based on Modified Distance Measures , 1999, IDA.

[22]  Qian Wang,et al.  The range of the value for the fuzzifier of the fuzzy c-means algorithm , 2012, Pattern Recognit. Lett..

[23]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[24]  Amit Konar,et al.  Clustering using vector membership: An extension of the Fuzzy C-Means algorithm , 2013, 2013 Fifth International Conference on Advanced Computing (ICoAC).

[25]  Cor J. Veenman,et al.  A Maximum Variance Cluster Algorithm , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Chih-Hung Wu,et al.  A New Fuzzy Clustering Validity Index With a Median Factor for Centroid-Based Clustering , 2015, IEEE Transactions on Fuzzy Systems.

[27]  Sanghamitra Bandyopadhyay,et al.  Classification and learning using genetic algorithms - applications in bioinformatics and web intelligence , 2007, Natural computing series.

[28]  Xin Guo,et al.  On the optimality of conditional expectation as a Bregman predictor , 2005, IEEE Trans. Inf. Theory.

[29]  Frank Klawonn,et al.  An alternative approach to the fuzzifier in fuzzy clustering to obtain better clustering , 2003, EUSFLAT Conf..

[30]  Athanasios A. Rontogiannis,et al.  Sparsity-Aware Possibilistic Clustering Algorithms , 2015, IEEE Transactions on Fuzzy Systems.

[31]  Frank Nielsen,et al.  On Conformal Divergences and Their Population Minimizers , 2013, IEEE Transactions on Information Theory.

[32]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[33]  I. Csiszár $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[34]  Frank Nielsen,et al.  The Burbea-Rao and Bhattacharyya Centroids , 2010, IEEE Transactions on Information Theory.

[35]  Marc Teboulle,et al.  A Unified Continuous Optimization Framework for Center-Based Clustering Methods , 2007, J. Mach. Learn. Res..

[36]  Paulo Fazendeiro,et al.  Observer-Biased Fuzzy Clustering , 2015, IEEE Transactions on Fuzzy Systems.

[37]  Ujjwal Maulik,et al.  Genetic clustering for automatic evolution of clusters and application to image classification , 2002, Pattern Recognit..

[38]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[39]  L. Bregman The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[40]  Swagatam Das,et al.  Geometric divergence based fuzzy clustering with strong resilience to noise features , 2016, Pattern Recognit. Lett..

[41]  Jianhua Lin,et al.  Divergence measures based on the Shannon entropy , 1991, IEEE Trans. Inf. Theory.

[42]  James C. Bezdek,et al.  Extending Information-Theoretic Validity Indices for Fuzzy Clustering , 2017, IEEE Transactions on Fuzzy Systems.

[43]  Hui Xiong,et al.  A Generalization of Distance Functions for Fuzzy $c$ -Means Clustering With Centroids of Arithmetic Means , 2012, IEEE Transactions on Fuzzy Systems.

[44]  Pasi Fränti,et al.  Iterative shrinking method for clustering problems , 2006, Pattern Recognit..

[45]  Jian Yu,et al.  Analysis of Parameter Selection for Gustafson–Kessel Fuzzy Clustering Using Jacobian Matrix , 2015, IEEE Transactions on Fuzzy Systems.

[46]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[47]  J. Bezdek,et al.  c-means clustering with the l/sub l/ and l/sub infinity / norms , 1991 .