GT2FC: An Online Growing Interval Type-2 Self-Learning Fuzzy Classifier

In this paper, we propose a Growing Type-2 Fuzzy Classifier (GT2FC) for online rule learning from real-time data streams. While in batch rule learning, the training data are assumed to be drawn from a stationary distribution, in online rule learning, data can dynamically change over time becoming potentially nonstationary. To accommodate dynamic change, GT2FC relies on a new semi-supervised online learning algorithm called Growing Gaussian Mixture Model (2G2M). In particular, 2G2M is used to generate the type-2 fuzzy membership functions to build the type-2 fuzzy rules. GT2FC is designed to accommodate data online and to reconcile labeled and unlabeled data using self-learning. Moreover, GT2FC maintains low complexity of the rule base using online optimization and feature selection mechanisms. GT2FC is tested on data obtained from an ambient intelligence application, where the goal is to exploit sensed data to monitor the living space on behalf of the inhabitants. Because sensors are prone to faults and noise, type-2 fuzzy modeling is very suitable for dealing with such an application. Thus, GT2FC offers the advantage of dealing with uncertainty in addition to self-adaptation in an online manner. For illustration purposes, GT2FC is also validated on synthetic and classic UCI data-sets. The detailed empirical study shows that GT2FC performs very well under various experimental settings.

[1]  Alexey Tsymbal,et al.  The problem of concept drift: definitions and related work , 2004 .

[2]  Shin Ishii,et al.  On-line EM Algorithm for the Normalized Gaussian Network , 2000, Neural Computation.

[3]  Peter A. Hall,et al.  A method to add Gaussian mixture models , 2004 .

[4]  Roland Mittermeir,et al.  Towards Incremental Fuzzy Classifiers , 2006, Soft Comput..

[5]  Oscar,et al.  [Studies in Fuzziness and Soft Computing] Type-2 Fuzzy Logic: Theory and Applications Volume 223 || 3 Type-2 Fuzzy Logic , 2008 .

[6]  Giovanna Castellano,et al.  Mining Diagnostic Rules Using Fuzzy Clustering , 2007 .

[7]  Chia-Feng Juang,et al.  A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning , 2008, IEEE Transactions on Fuzzy Systems.

[8]  Jerry M. Mendel,et al.  Operations on type-2 fuzzy sets , 2001, Fuzzy Sets Syst..

[9]  S. Salzberg.  A nearest hyperrectangle learning method , 2004, Machine Learning.

[10]  A. Krone,et al.  Data-based extraction of unidimensional fuzzy sets for fuzzy rule generation , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[11]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[12]  Stephen J. Roberts,et al.  Minimum-Entropy Data Partitioning Using Reversible Jump Markov Chain Monte Carlo , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[14]  Mark Zwolinski,et al.  Mutual Information Theory for Adaptive Mixture Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Plamen P. Angelov,et al.  A fuzzy controller with evolving structure , 2004, Inf. Sci..

[16]  Dar-Shyang Lee,et al.  Effective Gaussian mixture learning for video background subtraction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Hani Hagras,et al.  A type-2 fuzzy embedded agent to realise ambient intelligence in ubiquitous computing environments , 2005, Inf. Sci..

[18]  Arthur L. Dexter,et al.  On-line identification of computationally undemanding evolving fuzzy models , 2007, Fuzzy Sets Syst..

[19]  Witold Pedrycz,et al.  Type-2 Fuzzy Logic: Theory and Applications , 2007, 2007 IEEE International Conference on Granular Computing (GRC 2007).

[20]  Plamen P. Angelov,et al.  Evolving Fuzzy-Rule-Based Classifiers From Data Streams , 2008, IEEE Transactions on Fuzzy Systems.

[21]  Gerhard Widmer,et al.  Islands of Gaussians: The Self Organizing Map and Gaussian Music Similarity Features , 2010, ISMIR.

[22]  Refractor.  Uncertainty , 2001, The Lancet.

[23]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[24]  A. Klose,et al.  Semi-supervised induction of fuzzy rules applied to image segmentation , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[25]  Arianna Mencattini,et al.  Estimation of Uncertainty in Measurement by means of Type-2 Fuzzy Variables , 2007, 2007 IEEE International Fuzzy Systems Conference.

[26]  Plamen P. Angelov,et al.  An approach for fuzzy rule-base adaptation using on-line clustering , 2004, Int. J. Approx. Reason..

[27]  Hani Hagras,et al.  Dynamic Profile-Selection for zSlices based type-2 fuzzy agents controlling multi-user Ambient Intelligent Environments , 2012, 2012 IEEE International Conference on Fuzzy Systems.

[28]  Hani Hagras,et al.  Toward General Type-2 Fuzzy Logic Systems Based on zSlices , 2010, IEEE Transactions on Fuzzy Systems.

[29]  Cornelius T. Leondes,et al.  Fuzzy Theory Systems: Techniques and Applications , 1999 .

[30]  Hani Hagras,et al.  An Incremental Adaptive Life Long Learning Approach for Type-2 Fuzzy Embedded Agents in Ambient Intelligent Environments , 2007, IEEE Transactions on Fuzzy Systems.

[31]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Paul Rogers,et al.  Adaptive Mechanisms for Classification Problems with Drifting Data , 2007, KES.

[33]  Jerry M. Mendel,et al.  Computing the centroid of a general type-2 fuzzy set by means of the centroid-flow algorithm , 2011, IEEE Transactions on Fuzzy Systems.

[34]  Abdelhamid Bouchachia,et al.  An evolving classification cascade with self-learning , 2010, Evol. Syst..

[35]  U. Mahmood,et al.  Type-2 Fuzzy Classification of Blood Pressure Parameters , 2007, 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information.

[36]  Jerry M. Mendel,et al.  On KM Algorithms for Solving Type-2 Fuzzy Set Problems , 2013, IEEE Transactions on Fuzzy Systems.

[37]  Hongbin Wang,et al.  Highly efficient incremental estimation of Gaussian mixture models for online data stream clustering , 2005, SPIE Defense + Commercial Sensing.

[38]  Hani Hagras,et al.  An approach for the generation and adaptation of zSlices based general type-2 fuzzy sets from interval type-2 fuzzy sets to model agreement with application to Intelligent Environments , 2010, International Conference on Fuzzy Systems.

[39]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[40]  Edwin Lughofer,et al.  Evolving Fuzzy Systems - Methodologies, Advanced Concepts and Applications , 2011, Studies in Fuzziness and Soft Computing.

[41]  Witold Pedrycz,et al.  Advances in Fuzzy Clustering and its Applications , 2007 .

[42]  Mario A. Góngora,et al.  A generalised type-2 fuzzy logic system embedded board and integrated development environment , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[43]  Plamen Angelov,et al.  Evolving Rule-Based Models: A Tool For Design Of Flexible Adaptive Systems , 2002 .

[44]  Abdelhamid Bouchachia,et al.  Fuzzy classification in dynamic environments , 2011, Soft Comput..

[45]  Sankar K. Pal,et al.  Feature analysis: Neural network and fuzzy set theoretic approaches , 1997, Pattern Recognit..

[46]  Alexander H. Waibel,et al.  Growing Gaussian mixture models for pose invariant face recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[47]  Justus H. Piater,et al.  Online Learning of Gaussian Mixture Models - a Two-Level Approach , 2008, VISAPP.

[48]  Francisco Herrera,et al.  Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures , 2011, Inf. Sci..

[49]  Jerry M. Mendel,et al.  Type-2 fuzzy sets and systems: an overview , 2007, IEEE Computational Intelligence Magazine.

[50]  Abdelhamid Bouchachia,et al.  Incremental learning with multi-level adaptation , 2011, Neurocomputing.

[51]  Bogdan Gabrys,et al.  Learning hybrid neuro-fuzzy classifier models from data: to combine or not to combine? , 2004, Fuzzy Sets Syst..

[52]  Jerry M. Mendel,et al.  Uncertainty, fuzzy logic, and signal processing , 2000, Signal Process..

[53]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[54]  L. Williams,et al.  Contents , 2020, Ophthalmology (Rochester, Minn.).

[55]  Roberto Cipolla,et al.  Incremental Learning of Temporally-Coherent Gaussian Mixture Models , 2005, BMVC.

[56]  Gin-Der Wu,et al.  A Vectorization-Optimization-Method-Based Type-2 Fuzzy Neural Network for Noisy Data Classification , 2013, IEEE Transactions on Fuzzy Systems.

[57]  Giovanna Castellano,et al.  Distinguishability quantification of fuzzy sets , 2007, Inf. Sci..