Granular self-organizing map (grSOM) for structure identification

This work presents a useful extension of Kohonen's Self-Organizing Map (KSOM) for structure identification in linguistic (fuzzy) system modeling applications. More specifically the granular SOM neural model is presented for inducing a distribution of nonparametric fuzzy interval numbers (FINs) from the data. A FIN can represent a local probability distribution function and/or a conventional fuzzy set; moreover, a FIN is interpreted as an information granule. Learning is based on a novel metric distance d(K)(.,.) between FINs. The metric d(K)(.,.) can be tuned nonlinearly by a mass function m(x), the latter attaches a weight of significance to a real number 'x' in a data dimension. Rigorous analysis is based on mathematical lattice theory. A grSOM can cope with ambiguity by processing linguistic (fuzzy) input data and/or intervals. This work presents a simple grSOM variant, namely greedy grSOM, for classification. A genetic algorithm (GA) introduces tunable nonlinearities during training. Extensive comparisons are shown with related work from the literature. The practical effectiveness of the greedy grSOM is demonstrated comparatively in three benchmark classification problems. Statistical evidence strongly suggests that the proposed techniques improve classification performance. In addition, the greedy grSOM induces descriptive decision-making knowledge (fuzzy rules) from the training data.

[1]  Ioannis B. Theocharis,et al.  Microgenetic algorithms as generalized hill-climbing operators for GA optimization , 2001, IEEE Trans. Evol. Comput..

[2]  Klaus Obermayer,et al.  Self-organizing maps and clustering methods for matrix data , 2004, Neural Networks.

[3]  Vassilios Petridis,et al.  FINkNN: A Fuzzy Interval Number k-Nearest Neighbor Classifier for Prediction of Sugar Production from Populations of Samples , 2003, J. Mach. Learn. Res..

[4]  Andrzej Bargiela,et al.  Granular clustering: a granular signature of data , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[5]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[6]  Gail A. Carpenter,et al.  ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases , 1998, Neural Networks.

[7]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Ioannis B. Theocharis,et al.  A GA-based fuzzy modeling approach for generating TSK models , 2002, Fuzzy Sets Syst..

[9]  Sankar K. Pal,et al.  Self-organizing neural network as a fuzzy classifier , 1994, IEEE Trans. Syst. Man Cybern..

[10]  Ben J. A. Kröse,et al.  Self-organizing mixture models , 2005, Neurocomputing.

[11]  Jakob J. Verbeek,et al.  Mixture models for clustering and dimension reduction , 2004 .

[12]  L. Zadeh.  Fuzzy sets as a basis for a theory of possibility , 1999 .

[13]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[14]  M. Sugeno,et al.  Structure identification of fuzzy model , 1988 .

[15]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[16]  Ludmila I. Kuncheva,et al.  On the Equivalence between fuzzy and Statistical Classifiers , 1996, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[17]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[18]  Petri Vuorimaa,et al.  Fuzzy self-organizing map , 1994 .

[19]  A. Banerjee.  Convex Analysis and Optimization , 2006 .

[20]  Daniel Polani.  On the Optimization of Self-Organizing Maps by Genetic Algorithms , 1999 .

[21]  Sushmita Mitra,et al.  Neuro-fuzzy rule generation: survey in soft computing framework , 2000, IEEE Trans. Neural Networks Learn. Syst..

[22]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[23]  B. Z. Vulikh,et al.  Introduction to the Theory of Partially Ordered Spaces , 1967 .

[24]  Frank Hoffmann,et al.  Evolutionary algorithms for fuzzy control system design , 2001, Proc. IEEE.

[25]  John Yen,et al.  Application of statistical information criteria for optimal fuzzy model construction , 1998, IEEE Trans. Fuzzy Syst..

[26]  James C. Bezdek,et al.  An integrated approach to fuzzy learning vector quantization and fuzzy c-means clustering , 1997, IEEE Trans. Fuzzy Syst..

[27]  Marie Cottrell,et al.  SOM-based algorithms for qualitative variables , 2004, Neural Networks.

[28]  Mu-Chun Su,et al.  Application of neural networks incorporated with real-valued genetic algorithms in knowledge acquisition , 2000, Fuzzy Sets Syst..

[29]  Magne Setnes,et al.  Supervised fuzzy clustering for rule extraction , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[30]  Witold Pedrycz,et al.  Fuzzy descriptive models: an interactive framework of information granulation [ECG data] , 2002, IEEE Trans. Fuzzy Syst..

[31]  Chin-Teng Lin,et al.  An ART-based fuzzy adaptive learning control network , 1997, IEEE Trans. Fuzzy Syst..

[32]  Krassimir T. Atanassov,et al.  Intuitionistic Fuzzy Sets - Theory and Applications , 1999, Studies in Fuzziness and Soft Computing.

[33]  Panu Somervuo.  Online algorithm for the self-organizing map of symbol strings , 2004, Neural Networks.

[34]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[35]  D. Ralescu,et al.  Statistical Modeling, Analysis and Management of Fuzzy Data , 2001 .

[36]  Lotfi A. Zadeh,et al.  From Computing with Numbers to Computing with Words - from Manipulation of Measurements to Manipulation of Perceptions , 2005, Logic, Thought and Action.

[37]  James C. Bezdek,et al.  Generalized clustering networks and Kohonen's self-organizing scheme , 1993, IEEE Trans. Neural Networks.

[38]  J. Príncipe,et al.  Local dynamic modeling with self-organizing maps and applications to nonlinear system identification and control , 1998, Proc. IEEE.

[39]  V. Petridis,et al.  Learning and decision-making in the framework of fuzzy lattices , 2002 .

[40]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[41]  Spiridon D. Likothanassis,et al.  Kernel-based Self-organized Maps Trained with Supervised Bias for Gene Expression Data Analysis , 2004, J. Bioinform. Comput. Biol..

[42]  Vassilios Petridis,et al.  Fuzzy Lattice Neurocomputing (FLN) models , 2000, Neural Networks.

[43]  Vassilis G. Kaburlasos.  FINs: lattice theoretic tools for improving prediction of sugar production from populations of measurements , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[44]  L Xiao.  Structure Optimization of Fuzzy Neural Network Based on Tabu Search , 2006 .

[45]  Vassilios Petridis,et al.  Clustering and Classification in Structured Data Domains Using Fuzzy Lattice Neurocomputing (FLN) , 2001, IEEE Trans. Knowl. Data Eng..

[46]  D. Dubois,et al.  Fuzzy sets and statistical data , 1986 .

[47]  Chuen-Tsai Sun,et al.  Neuro-fuzzy modeling and control , 1995, Proc. IEEE.

[48]  H. Ishigami,et al.  Structure optimization of fuzzy neural network by genetic algorithm , 1995 .

[49]  T. Martin McGinnity,et al.  An approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network , 2005, Fuzzy Sets Syst..

[50]  Sankar K. Pal,et al.  Fuzzy self-organization, inferencing, and rule generation , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[51]  James C. Bezdek,et al.  Fuzzy Kohonen clustering networks , 1994, Pattern Recognit..

[52]  Elias N. Houstis,et al.  On neurobiological, neuro-fuzzy, machine learning, and statistical pattern recognition techniques , 1997, IEEE Trans. Neural Networks.

[53]  Vassilis G. Kaburlasos,et al.  grSOM: a granular extension of the self-organizing map for structure identification applications , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[54]  Hujun Yin,et al.  Image denoising using self-organizing map-based nonlinear independent component analysis , 2002, Neural Networks.

[55]  Abraham Kandel,et al.  Granular neural networks for numerical-linguistic data fusion and knowledge discovery , 2000, IEEE Trans. Neural Networks Learn. Syst..

[56]  David G. Stork,et al.  Pattern Classification , 1973 .

[57]  Samuel Kaski,et al.  Self organization of a massive document collection , 2000, IEEE Trans. Neural Networks Learn. Syst..

[58]  Panu Somervuo,et al.  Self-organizing maps of symbol strings , 1998, Neurocomputing.

[59]  H. Ritter.  Self-Organizing Maps on non-euclidean Spaces , 1999 .

[60]  Héctor Pomares,et al.  Structure identification in complete rule-based fuzzy systems , 2002, IEEE Trans. Fuzzy Syst..

[61]  Wolfgang Näther,et al.  On the variance of random fuzzy variables , 2002 .

[62]  Hung T. Nguyen,et al.  Fuzziness and randomness , 2002 .

[63]  Vojislav Kecman,et al.  Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models , 2001 .

[64]  Erkki Oja,et al.  Kohonen Maps , 1999, Encyclopedia of Machine Learning.

[65]  Peter N. Brett,et al.  Estimation of the stapes-bone thickness in the stapedotomy surgical procedure using a machine-learning technique , 1999, IEEE Transactions on Information Technology in Biomedicine.

[66]  Anastasios Bezerianos,et al.  Ischemia detection with a self-organizing map supplemented by supervised learning , 2001, IEEE Trans. Neural Networks.

[67]  F. Riesz.  Sur la décomposition des opérations fonctionnelles linéaires , 1929 .

[68]  J. Dieudonne,et al.  Encyclopedic Dictionary of Mathematics , 1979 .

[69]  Vassilios Petridis,et al.  Fuzzy lattice neural network (FLNN): a hybrid model for learning , 1998, IEEE Trans. Neural Networks.

[70]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[71]  Lakhmi C. Jain,et al.  New Learning Paradigms in Soft Computing , 2002 .

[72]  T. Fukuda,et al.  Self-tuning fuzzy modeling with adaptive membership function, rules, and hierarchical structure based on genetic algorithm , 1995 .