Fuzzy J-Means: a new heuristic for fuzzy clustering

Abstract A fuzzy clustering problem consists of assigning a set of patterns to a given number of clusters with respect to some criteria such that each of them may belong to more than one cluster with different degrees of membership. In order to solve it, we first propose a new local search heuristic, called Fuzzy J-Means, where the neighbourhood is defined by all possible centroid-to-pattern relocations. The “integer” solution is then moved to a continuous one by an alternate step, i.e., by finding centroids and membership degrees for all patterns and clusters. To alleviate the difficulty of being stuck in local minima of poor value, this local search is then embedded into the Variable Neighbourhood Search metaheuristic. Results on five standard test problems from the literature are reported and compared with those obtained with the well-known Fuzzy C-Means heuristic. It appears that solutions of substantially better quality are obtained with the proposed methods than with this former one.

[1]  Khaled S. Al-Sultan,et al.  A tabu search-based algorithm for the fuzzy clustering problem , 1997, Pattern Recognit..

[2]  Pierre Hansen,et al.  An Interior Point Algorithm for Minimum Sum-of-Squares Clustering , 1997, SIAM J. Sci. Comput..

[3]  Ching-Tang Hsieh,et al.  Generalized Fuzzy Kohonen Clustering Networks , 1998 .

[4]  B. Jaumard,et al.  Cluster Analysis and Mathematical Programming , 2003 .

[5]  Witold Pedrycz,et al.  Conditional Fuzzy C-Means , 1996, Pattern Recognit. Lett..

[6]  Peter A. Burrough,et al.  High-resolution landform classification using fuzzy k-means , 2000, Fuzzy Sets Syst..

[7]  Polly Bart,et al.  Heuristic Methods for Estimating the Generalized Vertex Median of a Weighted Graph , 1968, Oper. Res..

[8]  Witold Pedrycz,et al.  Conditional fuzzy clustering in the design of radial basis function neural networks , 1998, IEEE Trans. Neural Networks.

[9]  Pierre Hansen,et al.  Improvement and Comparison of Heuristics for Solving the Uncapacitated Multisource Weber Problem , 2000, Oper. Res..

[10]  Alberto Cardoso,et al.  Supervision and c-Means clustering of PID controllers for a solar power plant , 1999, Int. J. Approx. Reason..

[11]  Klaus Danzer,et al.  Fuzzy cluster analysis by simulated annealing , 1996 .

[12]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[13]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[14]  R. Bellman,et al.  Abstraction and pattern classification , 1996 .

[15]  R. Horst,et al.  Global Optimization: Deterministic Approaches , 1992 .

[16]  L O Hall,et al.  Review of MR image segmentation techniques using pattern recognition. , 1993, Medical physics.

[17]  K. Chou,et al.  Predicting protein structural classes from amino acid composition: application of fuzzy clustering. , 1995, Protein engineering.

[18]  James C. Bezdek,et al.  Clustering with a genetically optimized approach , 1999, IEEE Trans. Evol. Comput..

[19]  Michio Sugeno,et al.  A fuzzy-logic-based approach to qualitative modeling , 1993, IEEE Trans. Fuzzy Syst..

[20]  Shokri Z. Selim,et al.  A global algorithm for the fuzzy clustering problem , 1993, Pattern Recognit..

[21]  Witold Pedrycz,et al.  Fuzzy set technology in knowledge discovery , 1998, Fuzzy Sets Syst..

[22]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[23]  Pierre Hansen,et al.  Variable Neighborhood Search , 2018, Handbook of Heuristics.

[24]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[25]  Silvano Martello,et al.  Meta-Heuristics: Advances and Trends in Local Search Paradigms for Optimization , 2012 .

[26]  James C. Bezdek,et al.  Fuzzy mathematics in pattern classification , 1973 .

[27]  G. Celeux,et al.  A Classification EM algorithm for clustering and two stochastic versions , 1992 .

[28]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[29]  Gerhard Reinelt,et al.  TSPLIB - A Traveling Salesman Problem Library , 1991, INFORMS J. Comput..

[30]  James C. Bezdek,et al.  Optimization of clustering criteria by reformulation , 1995, IEEE Trans. Fuzzy Syst..

[31]  Nenad Mladenović,et al.  An Introduction to Variable Neighborhood Search , 1997 .

[32]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[33]  Robert F. Ling,et al.  Cluster analysis algorithms for data reduction and classification of objects , 1981 .

[34]  James C. Bezdek,et al.  Optimization of fuzzy clustering criteria using genetic algorithms , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[35]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[36]  James C. Bezdek,et al.  Feature selection for binary data: medical diagnosis with fuzzy sets , 1976, AFIPS '76.

[37]  Pierre Hansen,et al.  J-MEANS: a new local search heuristic for minimum sum of squares clustering , 1999, Pattern Recognit..

[38]  Hans-Jürgen Zimmermann,et al.  Fuzzy rule based classification with FeatureSelector and modified threshold accepting , 2000, Eur. J. Oper. Res..

[39]  李幼升,et al.  Ph , 1989 .

[40]  James C. Bezdek,et al.  Sequential Competitive Learning and the Fuzzy c-Means Clustering Algorithms , 1996, Neural Networks.

[41]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[42]  James C. Bezdek,et al.  A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Pierre Hansen,et al.  Variable Neighborhood Decomposition Search , 1998, J. Heuristics.

[44]  J. Richardt,et al.  Connections between fuzzy theory, simulated annealing, and convex duality , 1998, Fuzzy Sets Syst..

[45]  Pierre Hansen,et al.  Variable neighborhood search: Principles and applications , 1998, Eur. J. Oper. Res..

[46]  Fu Guoyao,et al.  Optimization methods for fuzzy clustering , 1998 .

[47]  Michael Spann,et al.  A new approach to clustering , 1990, Pattern Recognit..

[48]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.