Granular fuzzy modeling with evolving hyperboxes in multi-dimensional space of numerical data

Clustering has been applied to numerous areas, including signal and image processing. Many approaches have been developed over the years to efficiently construct granular models on a basis of numerical experimental data. In this study, we propose a novel approach to construct a granular model that is fundamentally designed around information granules regarded as hyperboxes. Several studies have been focused on building a set of hyperboxes around data; one of them being a Min-Max Neural Network (NN) algorithm. Here we develop two different methods to construct these information granules, nevertheless some essential similarities to previous studies can be found. In particular, hyperboxes are constructed by using some reference data, and they are endowed with some parametric flexibility to facilitate controlling their size, whereas the construction of the hyperboxes involve elimination or reduction of possible overlaps between them. In the proposed approach, given a set of input and output data pairs, we construct interval-based information granules to partition the output space (viz. the space of the output variable). On a basis of these intervals, we carry out a so-called context-based Fuzzy C-Means algorithm to construct cluster centers (prototypes) in the multivariable input space. These prototypes serve as hyperbox cores. To construct the information granules, two methods are studied: one develops a family of hyperboxes by realizing some constrictions, while the other one engages Differential Evolution (DE) to realize further optimization. To reduce overlap, two methods are tested: one being previously proposed for the min-max NN and a new one, which engages DE to optimize the overlap reduction. Experimental studies involve synthetic data and publicly available real-world data. The results are compared with the outcomes produced by the algorithm proposed by Simpson. The performance of the method is quantified and it is demonstrated that the obtained results are substantially better when dealing with multi-dimensional data.

[1]  Jonathan Lawry,et al.  FUZZ-IEEE 2000 , 2000 .

[2]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[3]  Shigeo Abe,et al.  A fuzzy classifier based on partitioned hyperboxes , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[4]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[5]  Muharram Mansoorizadeh,et al.  Multi-Level Fuzzy Min-Max Neural Network Classifier , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks - Part 2: Clustering , 1993, IEEE Trans. Fuzzy Syst..

[7]  Witold Pedrycz,et al.  Conditional fuzzy clustering in the design of radial basis function neural networks , 1998, IEEE Trans. Neural Networks.

[8]  Xiaohua Hu,et al.  Exploiting the Social Tagging Network for Web Clustering , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  G. Martinelli,et al.  Clustering with unconstrained hyperboxes , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[10]  Antonello Rizzi,et al.  Generalized min-max classifier , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[11]  B. Chatterjee,et al.  Rough-granular approach for impulse fault classification of transformers using cross-wavelet transform , 2008, IEEE Transactions on Dielectrics and Electrical Insulation.

[12]  Hong He,et al.  A two-stage genetic algorithm for automatic clustering , 2012, Neurocomputing.

[13]  Nong Sang,et al.  Using clustering analysis to improve semi-supervised classification , 2013, Neurocomputing.

[14]  Witold Pedrycz,et al.  Conditional Fuzzy C-Means , 1996, Pattern Recognit. Lett..

[15]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[16]  Andrzej Bargiela,et al.  General fuzzy min-max neural network for clustering and classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[17]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[18]  Jinwen Ma,et al.  Feature extraction through contourlet subband clustering for texture classification , 2013, Neurocomputing.

[19]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[20]  Rama Chellappa,et al.  Entropy-Rate Clustering: Cluster Analysis via Maximizing a Submodular Function Subject to a Matroid Constraint , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Witold Pedrycz.  Genetic tolerance fuzzy neural networks: From data to fuzzy hyperboxes , 2007, Neurocomputing.

[22]  Liangpei Zhang,et al.  An Adaptive Memetic Fuzzy Clustering Algorithm With Spatial Information for Remote Sensing Imagery , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[23]  Yanqing Zhang,et al.  Recursive Fuzzy Granulation for Gene Subsets Extraction and Cancer Classification , 2008, IEEE Transactions on Information Technology in Biomedicine.

[24]  P. K. Simpson,et al.  Fuzzy min-max neural networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[25]  Prabir Kumar Biswas,et al.  A Granular Reflex Fuzzy Min–Max Neural Network for Classification , 2009, IEEE Transactions on Neural Networks.

[26]  Andrzej Bargiela,et al.  Adaptation of exclusion / inclusion hyperboxes for classification of complex data , 2004 .

[27]  Wenjun Zhou,et al.  Spectral clustering of high-dimensional data exploiting sparse representation vectors , 2014, Neurocomputing.

[28]  S. Abe,et al.  A classifier using fuzzy rules extracted directly from numerical data , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[29]  Andrzej Bargiela,et al.  Exclusion/Inclusion Fuzzy Classification Network , 2003, KES.

[30]  Daoqiang Zhang,et al.  A Multiobjective Simultaneous Learning Framework for Clustering and Classification , 2010, IEEE Transactions on Neural Networks.

[31]  Chee Peng Lim,et al.  An Enhanced Fuzzy Min–Max Neural Network for Pattern Classification , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Andrzej Bargiela,et al.  Granular clustering: a granular signature of data , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[33]  Jon Atli Benediktsson,et al.  Spectral–Spatial Classification of Hyperspectral Imagery Based on Partitional Clustering Techniques , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[34]  R. Pace,et al.  Sparse spatial autoregressions , 1997 .

[35]  Vincent S. Tseng,et al.  A Novel Similarity-Based Fuzzy Clustering Algorithm by Integrating PCM and Mountain Method , 2007, IEEE Transactions on Fuzzy Systems.

[36]  Oscar Castillo,et al.  A review on type-2 fuzzy logic applications in clustering, classification and pattern recognition , 2014, Appl. Soft Comput..

[37]  Juan Humberto Sossa Azuela,et al.  Efficient training for dendrite morphological neural networks , 2014, Neurocomputing.

[38]  Shigeo Abe,et al.  A function approximator using fuzzy rules extracted directly from numerical data , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[39]  Yangyang Li,et al.  A particle swarm optimization based simultaneous learning framework for clustering and classification , 2014, Pattern Recognit..

[40]  Chrisina Jayne,et al.  Evaluation of hyperbox neural network learning for classification , 2014, Neurocomputing.

[41]  E. Fridell,et al.  The mechanism for NOx storage , 2000 .

[42]  P. K. Simpson.  Fuzzy Min-Max Neural Networks-Part 1 : Classification , 1992 .

[43]  Themos Stafylakis,et al.  A Study of the Cosine Distance-Based Mean Shift for Telephone Speech Diarization , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[44]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[45]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[46]  Andrzej Bargiela,et al.  An inclusion/exclusion fuzzy hyperbox classifier , 2004, Int. J. Knowl. Based Intell. Eng. Syst..

[47]  Huiping Cao,et al.  Comprehensive Clustering of Disturbance Events Recorded by Phasor Measurement Units , 2014, IEEE Transactions on Power Delivery.

[48]  Ming-Syan Chen,et al.  Adaptive Clustering for Multiple Evolving Streams , 2006, IEEE Transactions on Knowledge and Data Engineering.

[49]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks. I. Classification , 1992, IEEE Trans. Neural Networks.

[50]  Witold Pedrycz,et al.  Knowledge transfer in system modeling and its realization through an optimal allocation of information granularity , 2012, Appl. Soft Comput..

[51]  Jemal H. Abawajy,et al.  Multistage approach for clustering and classification of ECG data , 2013, Comput. Methods Programs Biomed..

[52]  David B. Fogel,et al.  Evolving fuzzy clusters , 1993, IEEE International Conference on Neural Networks.

[53]  Andrzej Bargiela,et al.  Granular Clustering with partial supervision – the development of abstract models for simulation , 2001 .

[54]  Dazhong Ma,et al.  Data-Core-Based Fuzzy Min–Max Neural Network for Pattern Classification , 2011, IEEE Transactions on Neural Networks.

[55]  Kaoru Hirota,et al.  Hyperbox clustering with Ant Colony Optimization (HACO) method and its application to medical risk profile recognition , 2009, Appl. Soft Comput..