AFS Fuzzy Clustering Analysis

In this chapter, we apply the AFS theory to propose an elementary algorithm of fuzzy clustering. In the proposed approach, each cluster is interpreted by taking advantage of the semantics captured by the AFS logic. Within the framework of AFS theory, we develop new techniques of feature selection, concept categorization and characteristic description (i.e.,the characteristic description of an object or a group of objects using the fuzzy concepts) which are often encountered in tasks of machine learning and pattern recognition. The elementary fuzzy clustering algorithm is evolved to three more elaborate fuzzy clustering techniques by incorporating new techniques of feature selection, concept categorization and characteristic description. We show that they are simpler and produce more interpretable results when contrasted with some existing techniques. Several benchmark data and the evaluation data of 30 companies are considered to evaluate the effectiveness of the proposed AFS fuzzy clustering algorithms. We provide a detailed comparative analysis in which we compare the obtained results with those produced by some “conventional” methods such as FCM, k-means, and some newer algorithms including a two-level SOM-based clustering algorithm. The proposed algorithms can be applied to the data sets with mixed features such as sub-preference relations and even those including descriptions of human intuitive judgment. We show that the flexibility of the approach comes from the fact that the distance function and the class number need not be given beforehand. These two facets offers a far more higher flexible and contribute to a powerful framework for representing human knowledge and studying intelligent systems encountered in real world applications.

[1]  Xiaodong Liu,et al.  Approaches to the representations and logic operations of fuzzy concepts in the framework of axiomatic fuzzy set theory I , 2007, Inf. Sci..

[2]  Melody Y. Kiang,et al.  Extending the Kohonen self-organizing map networks for clustering analysis , 2002 .

[3]  Yilei Wu,et al.  A robust deterministic annealing algorithm for data clustering , 2007 .

[4]  Witold Pedrycz,et al.  Axiomatics fuzzy sets logic , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[5]  Mohamed A. Ismail,et al.  Fuzzy clustering for symbolic data , 1998, IEEE Trans. Fuzzy Syst..

[6]  Xiaodong Liu,et al.  The fuzzy sets and systems based on AFS.structure, EI algebra and EII algebra , 1998, Fuzzy Sets Syst..

[7]  James C. Bezdek,et al.  Fuzzy mathematics in pattern classification , 1973 .

[8]  Robert Jenssen,et al.  Information cut for clustering using a gradient descent approach , 2007, Pattern Recognit..

[9]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[10]  James C. Bezdek,et al.  A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[12]  Michael Kirby,et al.  Geometric Data Analysis: An Empirical Approach to Dimensionality Reduction and the Study of Patterns , 2000 .

[13]  James C. Bezdek,et al.  Fuzzy c-means clustering of incomplete data , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[14]  Gin-Shuh Liang,et al.  Computing, Artificial Intelligence and Information Technology Cluster analysis based on fuzzy equivalence relation , 2005 .

[15]  Witold Pedrycz,et al.  The development of fuzzy decision trees in the framework of Axiomatic Fuzzy Set logic , 2007, Appl. Soft Comput..

[16]  Xiaodong Liu,et al.  Novel artificial intelligent techniques via AFS theory: Feature selection, concept categorization and characteristic description , 2010, Appl. Soft Comput..

[17]  Xiaodong Liu,et al.  The fuzzy clustering analysis based on AFS theory , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Wang Guo-jun,et al.  Theory of topological molecular lattices , 1992 .

[19]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[20]  Ethem Alpaydin,et al.  Constructive feedforward ART clustering networks. I , 2002, IEEE Trans. Neural Networks.

[21]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[22]  Yen-Liang Chen,et al.  An overlapping cluster algorithm to provide non-exhaustive clustering , 2006, Eur. J. Oper. Res..

[23]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[24]  Francesco Marcelloni,et al.  Feature selection based on a modified fuzzy C-means algorithm with supervision , 2003, Inf. Sci..

[25]  Francesco Camastra,et al.  A Novel Kernel Method for Clustering , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Witold Pedrycz,et al.  Statistically grounded logic operators in fuzzy sets , 2009, Eur. J. Oper. Res..

[27]  Hsiao-Fan Wang,et al.  Fuzzy relations in Taiwan , 1992 .

[28]  Shaocheng Tong,et al.  On AFS algebra--Part II , 2004, Inf. Sci..

[29]  Yan Ren,et al.  Fuzzy Clustering Approaches Based on AFS Fuzzy Logic II , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[30]  Rui J. P. de Figueiredo,et al.  A new neural network for cluster-detection-and-labeling , 1998, IEEE Trans. Neural Networks.

[31]  Anil K. Jain,et al.  Simultaneous feature selection and clustering using mixture models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  James C. Bezdek,et al.  Generalized clustering networks and Kohonen's self-organizing scheme , 1993, IEEE Trans. Neural Networks.

[33]  Anil K. Jain,et al.  Online handwritten script recognition , 2004 .

[34]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Xiaodong Liu,et al.  THE DEVELOPMENT OF AFS THEORY UNDER PROBABILITY THEORY , 2007 .

[36]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Andrzej Bargiela,et al.  General fuzzy min-max neural network for clustering and classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[38]  Liu Xiaodong,et al.  The Fuzzy Theory Based on AFS Algebras and AFS Structure , 1998 .

[39]  Miin-Shen Yang,et al.  Alternative c-means clustering algorithms , 2002, Pattern Recognit..

[40]  Edwin Diday,et al.  Symbolic clustering using a new dissimilarity measure , 1991, Pattern Recognit..

[41]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[42]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[43]  Jing Hua,et al.  Localized feature selection for clustering , 2008, Pattern Recognit. Lett..

[44]  Hong-Zhong Huang,et al.  The representations of fuzzy concepts based on the fuzzy matrix theory and the AFS theory , 2003, Proceedings of the 2003 IEEE International Symposium on Intelligent Control.

[45]  Matthew King,et al.  Density based fuzzy C , 2006, Eur. J. Oper. Res..

[46]  Witold Pedrycz,et al.  The Development of Fuzzy Rough Sets with the Use of Structures and Algebras of Axiomatic Fuzzy Sets , 2009, IEEE Transactions on Knowledge and Data Engineering.

[47]  Yew-Soon Ong,et al.  Advances in Natural Computation, First International Conference, ICNC 2005, Changsha, China, August 27-29, 2005, Proceedings, Part I , 2005, ICNC.

[48]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[49]  Xiaodong Liu,et al.  Concept Lattice and AFS Algebra , 2006, FSKD.

[50]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[51]  Daphne Koller,et al.  Toward Optimal Feature Selection , 1996, ICML.

[52]  Xiaodong Liu,et al.  Concept analysis via rough set and AFS algebra , 2008, Inf. Sci..

[53]  Christophe Ambroise,et al.  Feature selection in robust clustering based on Laplace mixture , 2006, Pattern Recognit. Lett..

[54]  Z. Zenn Bien,et al.  Iterative Fuzzy Clustering Algorithm With Supervision to Construct Probabilistic Fuzzy Rule Base From Numerical Data , 2008, IEEE Transactions on Fuzzy Systems.

[55]  Didier Dubois,et al.  The three semantics of fuzzy sets , 1997, Fuzzy Sets Syst..

[56]  Theo Tryfonas,et al.  Frontiers in Artificial Intelligence and Applications , 2009 .

[57]  Xiaodong Liu,et al.  A new fuzzy model of pattern recognition and hitch diagnoses of complex systems , 1999, Fuzzy Sets Syst..

[58]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[59]  Jeff Fortuna,et al.  Improved support vector classification using PCA and ICA feature space modification , 2004, Pattern Recognit..

[60]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[61]  Ki Hang Kim Boolean matrix theory and applications , 1982 .

[62]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[63]  Xiaodong Liu,et al.  The Fuzzy Clustering Algorithm Based on AFS Topology , 2006, FSKD.

[64]  Xiaodong Liu,et al.  Credit Rating Analysis with AFS Fuzzy Logic , 2005, ICNC.

[65]  T. Kohonen Self-organized formation of topographically correct feature maps , 1982 .

[66]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[67]  Liu Xiaodong,et al.  The Topology of AFS Structure and AFS Algebras , 1998 .

[68]  Stephen J. Roberts,et al.  Maximum certainty data partitioning , 2000, Pattern Recognit..

[69]  Tommy W. S. Chow,et al.  Clustering of the self-organizing map using a clustering validity index based on inter-cluster and intra-cluster density , 2004, Pattern Recognit..

[70]  Jack E. Graver,et al.  Combinatorics with emphasis on the theory of graphs , 1977 .

[71]  Etienne E. Kerre,et al.  An overview of fuzzy quantifiers. (I). Interpretations , 1998, Fuzzy Sets Syst..