pClass+: A Novel Evolving Semi-Supervised Classifier

A novel evolving semi-supervised classifier, namely Parsimonious Classifier+ (pClass+), is proposed in this paper. pClass+ enhances a recently developed classifier, namely pClass, for a semi-supervised learning scenario. As with its predecessor, pClass+ is capable of initiating its learning process from scratch with an empty rule base and adopts an open network structure, where fuzzy rules are evolved, pruned, and recalled automatically on demands. The novelty of pClass+ lies in an online active learning technique, which decreases operator’s annotation efforts and expedites its training process. pClass+ is also equipped with a new parameter identification strategy to cope with the class overlapping situation. The efficacy of pClass+ has been experimentally validated with numerous synthetic and real-world study cases, confirmed by thorough statistical tests and comparisons against state-of-the art classifiers, where pClass+ outperforms its counterparts in achieving the best trade-off between accuracy and complexity.

[1]  Narasimhan Sundararajan,et al.  A Metacognitive Neuro-Fuzzy Inference System (McFIS) for Sequential Classification Problems , 2013, IEEE Transactions on Fuzzy Systems.

[2]  Plamen P. Angelov,et al.  Handling drifts and shifts in on-line data streams with evolving fuzzy systems , 2011, Appl. Soft Comput..

[3]  Mahardhika Pratama,et al.  pClass: An Effective Classifier for Streaming Examples , 2015, IEEE Transactions on Fuzzy Systems.

[4]  Sundaram Suresh,et al.  A Meta-cognitive Interval Type-2 fuzzy inference system classifier and its projection based learning algorithm , 2013, 2013 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS).

[5]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[6]  Mahardhika Pratama,et al.  Recurrent Classifier Based on an Incremental Metacognitive-Based Scaffolding Algorithm , 2015, IEEE Transactions on Fuzzy Systems.

[7]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[8]  Edwin Lughofer,et al.  Reliable All-Pairs Evolving Fuzzy Classifiers , 2013, IEEE Transactions on Fuzzy Systems.

[9]  Edwin Lughofer On-line active learning based on enhanced reliability concepts , 2012, 2012 IEEE Conference on Evolving and Adaptive Intelligent Systems.

[10]  J. Nazuno Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[11]  Mahardhika Pratama,et al.  Evolving fuzzy rule-based classifier based on GENEFIS , 2013, 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[12]  Gregory Ditzler,et al.  Incremental Learning of Concept Drift from Streaming Imbalanced Data , 2013, IEEE Transactions on Knowledge and Data Engineering.

[13]  E. Lughofer,et al.  Evolving fuzzy classifiers using different model architectures , 2008, Fuzzy Sets Syst..

[14]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[15]  Kwok-Wo Wong,et al.  Generalized RLS approach to the training of neural networks , 2006, IEEE Trans. Neural Networks.

[16]  Rafael Santos,et al.  Creating fuzzy rules for image classification using biased data clustering , 1999, Electronic Imaging.

[17]  Edwin Lughofer,et al.  On-line evolving image classifiers and their application to surface inspection , 2010, Image Vis. Comput..

[18]  M. Omair Ahmad,et al.  Optimizing the kernel in the empirical feature space , 2005, IEEE Transactions on Neural Networks.

[19]  F. Bartlett,et al.  Remembering: A Study in Experimental and Social Psychology , 1932 .

[20]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[22]  Hazem Tawfik,et al.  Handoff algorithms based on fuzzy classifiers , 2000, IEEE Trans. Veh. Technol..

[23]  Abdelhamid Bouchachia,et al.  GT2FC: An Online Growing Interval Type-2 Self-Learning Fuzzy Classifier , 2014, IEEE Transactions on Fuzzy Systems.

[24]  Sundaram Suresh,et al.  Sequential Projection-Based Metacognitive Learning in a Radial Basis Function Network for Classification Problems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Boaz Lerner,et al.  The Bayesian ARTMAP , 2007, IEEE Transactions on Neural Networks.

[26]  Edwin Lughofer,et al.  On-line incremental feature weighting in evolving fuzzy classifiers , 2011, Fuzzy Sets Syst..

[27]  Plamen P. Angelov,et al.  PANFIS: A Novel Incremental Learning Machine , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[28]  N. Sundararajan,et al.  Extended sequential adaptive fuzzy inference system for classification problems , 2011, Evol. Syst..

[29]  Chuen-Chien Lee FUZZY LOGIC CONTROL SYSTEMS: FUZZY LOGIC CONTROLLER - PART I , 1990 .

[30]  Meng Joo Er,et al.  Data driven modeling based on dynamic parsimonious fuzzy neural network , 2013, Neurocomputing.

[31]  Xin Yao,et al.  The Impact of Diversity on Online Ensemble Learning in the Presence of Concept Drift , 2010, IEEE Transactions on Knowledge and Data Engineering.

[32]  Lei Wang,et al.  Fuzzy Passive-Aggressive classification: A robust and efficient algorithm for online classification problems , 2013, Inf. Sci..

[33]  Walmir M. Caminhas,et al.  Multivariable Gaussian Evolving Fuzzy Modeling System , 2011, IEEE Transactions on Fuzzy Systems.

[34]  Edwin Lughofer,et al.  Evolving Fuzzy Systems - Methodologies, Advanced Concepts and Applications , 2011, Studies in Fuzziness and Soft Computing.

[35]  William Nick Street,et al.  A streaming ensemble algorithm (SEA) for large-scale classification , 2001, KDD '01.

[36]  Mahardhika Pratama,et al.  An Incremental Classifier from Data Streams , 2014, SETN.

[37]  Chee Peng Lim,et al.  Improved GART Neural Network Model for Pattern Classification and Rule Extraction With Application to Power Systems , 2011, IEEE Transactions on Neural Networks.

[38]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[39]  Sundaram Suresh,et al.  A meta-cognitive interval type-2 fuzzy inference system and its projection based learning algorithm , 2014, Evol. Syst..

[40]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[41]  Edwin Lughofer,et al.  Single-pass active learning with conflict and ignorance , 2012, Evolving Systems.

[42]  Xin Yao,et al.  DDD: A New Ensemble Approach for Dealing with Concept Drift , 2012, IEEE Transactions on Knowledge and Data Engineering.

[43]  Mahardhika Pratama,et al.  GENEFIS: Toward an Effective Localist Network , 2014, IEEE Transactions on Fuzzy Systems.

[44]  Geoff Holmes,et al.  Active Learning With Drifting Streaming Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Abdelhamid Bouchachia,et al.  An evolving classification cascade with self-learning , 2010, Evol. Syst..