An incremental meta-cognitive-based scaffolding fuzzy neural network

The idea of meta-cognitive learning has enriched the landscape of evolving systems, because it emulates three fundamental aspects of human learning: what-to-learn; how-to-learn; and when-to-learn. However, existing meta-cognitive algorithms still exclude Scaffolding theory, which can realize a plug-and-play classifier. Consequently, these algorithms require laborious pre- and/or post-training processes to be carried out in addition to the main training process. This paper introduces a novel meta-cognitive algorithm termed GENERIC-Classifier (gClass), where the how-to-learn part constitutes a synergy of Scaffolding Theory - a tutoring theory that fosters the ability to sort out complex learning tasks, and Schema Theory - a learning theory of knowledge acquisition by humans. The what-to-learn aspect adopts an online active learning concept by virtue of an extended conflict and ignorance method, making gClass an incremental semi-supervised classifier, whereas the when-to-learn component makes use of the standard sample reserved strategy. A generalized version of the Takagi-Sugeno Kang (TSK) fuzzy system is devised to serve as the cognitive constituent. That is, the rule premise is underpinned by multivariate Gaussian functions, while the rule consequent employs a subset of the non-linear Chebyshev polynomial. Thorough empirical studies, confirmed by their corresponding statistical tests, have numerically validated the efficacy of gClass, which delivers better classification rates than state-of-the-art classifiers while having less complexity.

[1]  E. Lughofer,et al.  Evolving fuzzy classifiers using different model architectures , 2008, Fuzzy Sets Syst..

[2]  William Nick Street,et al.  A streaming ensemble algorithm (SEA) for large-scale classification , 2001, KDD '01.

[3]  I. Škrjanc,et al.  Self-tuning of 2 DOF control based on evolving fuzzy model , 2014, Appl. Soft Comput..

[4]  Mahardhika Pratama,et al.  pClass: An Effective Classifier for Streaming Examples , 2015, IEEE Transactions on Fuzzy Systems.

[5]  Geoff Holmes,et al.  MOA: Massive Online Analysis , 2010, J. Mach. Learn. Res..

[6]  Boaz Lerner,et al.  The Bayesian ARTMAP , 2007, IEEE Transactions on Neural Networks.

[7]  M. Omair Ahmad,et al.  Optimizing the kernel in the empirical feature space , 2005, IEEE Transactions on Neural Networks.

[8]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[10]  Plamen P. Angelov,et al.  Simpl_eClass: Simplified potential-free evolving fuzzy rule-based classifiers , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[11]  Brian J. Reiser,et al.  Scaffolding Complex Learning: The Mechanisms of Structuring and Problematizing Student Work , 2004, The Journal of the Learning Sciences.

[12]  Edwin Lughofer,et al.  Learning in Non-Stationary Environments: Methods and Applications , 2012 .

[13]  Mahardhika Pratama,et al.  Evolving fuzzy rule-based classifier based on GENEFIS , 2013, 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[14]  Mineichi Kudo,et al.  Data compression by volume prototypes for streaming data , 2010, Pattern Recognit..

[15]  Chuang Liu,et al.  Endpoint prediction model for basic oxygen furnace steel-making based on membrane algorithm evolving extreme learning machine , 2014, Appl. Soft Comput..

[16]  L. Vygotsky Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[17]  Edwin Lughofer,et al.  Reliable All-Pairs Evolving Fuzzy Classifiers , 2013, IEEE Transactions on Fuzzy Systems.

[18]  Edwin Lughofer,et al.  Autonomous data stream clustering implementing split-and-merge concepts - Towards a plug-and-play approach , 2015, Inf. Sci..

[19]  Narasimhan Sundararajan,et al.  A Metacognitive Neuro-Fuzzy Inference System (McFIS) for Sequential Classification Problems , 2013, IEEE Transactions on Fuzzy Systems.

[20]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[21]  Xiaodong Lin,et al.  Active Learning From Stream Data Using Optimal Weight Classifier Ensemble , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  O. J. Dunn Multiple Comparisons among Means , 1961 .

[23]  Edwin Lughofer,et al.  Single-pass active learning with conflict and ignorance , 2012, Evolving Systems.

[24]  Gin-Der Wu,et al.  A TS-Type Maximizing-Discriminability-Based Recurrent Fuzzy Network for Classification Problems , 2011, IEEE Transactions on Fuzzy Systems.

[25]  Edwin Lughofer,et al.  Hybrid active learning for reducing the annotation effort of operators in classification systems , 2012, Pattern Recognit..

[26]  Sundaram Suresh,et al.  A Meta-cognitive Interval Type-2 fuzzy inference system classifier and its projection based learning algorithm , 2013, 2013 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS).

[27]  Edwin Lughofer,et al.  Flexible Evolving Fuzzy Inference Systems from Data Streams (FLEXFIS , 2012 .

[28]  Sundaram Suresh,et al.  A meta-cognitive interval type-2 fuzzy inference system and its projection based learning algorithm , 2014, Evol. Syst..

[29]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[30]  Jesús S. Aguilar-Ruiz,et al.  Knowledge discovery from data streams , 2009, Intell. Data Anal..

[31]  See-Kiong Ng,et al.  ARPOP: An Appetitive Reward-Based Pseudo-Outer-Product Neural Fuzzy Inference System Inspired From the Operant Conditioning of Feeding Behavior in Aplysia , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Walmir M. Caminhas,et al.  Adaptive fault detection and diagnosis using an evolving fuzzy classifier , 2013, Inf. Sci..

[33]  Kwok-Wo Wong,et al.  Generalized RLS approach to the training of neural networks , 2006, IEEE Trans. Neural Networks.

[34]  Meng Joo Er,et al.  Data driven modeling based on dynamic parsimonious fuzzy neural network , 2013, Neurocomputing.

[35]  Sundaram Suresh,et al.  A Metacognitive Complex-Valued Interval Type-2 Fuzzy Inference System , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Xin Yao,et al.  The Impact of Diversity on Online Ensemble Learning in the Presence of Concept Drift , 2010, IEEE Transactions on Knowledge and Data Engineering.

[37]  Lei Wang,et al.  Fuzzy Passive-Aggressive classification: A robust and efficient algorithm for online classification problems , 2013, Inf. Sci..

[38]  Alex ChiChung Kot,et al.  Nonlinear dynamic system identification using Chebyshev functional link artificial neural networks , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[39]  R. Iman,et al.  Approximations of the critical region of the fbietkan statistic , 1980 .

[40]  Lihong Li,et al.  Unbiased online active learning in data streams , 2011, KDD.

[41]  Sundaram Suresh,et al.  Sequential Projection-Based Metacognitive Learning in a Radial Basis Function Network for Classification Problems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[42]  N. Sundararajan,et al.  Extended sequential adaptive fuzzy inference system for classification problems , 2011, Evol. Syst..

[43]  Plamen P. Angelov,et al.  Handling drifts and shifts in on-line data streams with evolving fuzzy systems , 2011, Appl. Soft Comput..

[44]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[45]  Edwin Lughofer,et al.  Self-adaptive and local strategies for a smooth treatment of drifts in data streams , 2014, Evol. Syst..

[46]  Éric Anquetil,et al.  ILClass: Error-driven antecedent learning for evolving Takagi-Sugeno classification systems , 2014, Appl. Soft Comput..

[47]  Mahardhika Pratama,et al.  A novel meta-cognitive-based scaffolding classifier to sequential non-stationary classification problems , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[48]  Plamen Angelov,et al.  Evolving Takagi-Sugeno fuzzy systems from data streams (eTS+). , 2010 .

[49]  Sundaram Suresh,et al.  A meta-cognitive sequential learning algorithm for neuro-fuzzy inference system , 2012, Appl. Soft Comput..

[50]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.

[51]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[52]  F. Bartlett,et al.  Remembering: A Study in Experimental and Social Psychology , 1932 .

[53]  H. Liu,et al.  Distributive Equations of Fuzzy Implications Based on Continuous Triangular Conorms Given as Ordinal Sums , 2013, IEEE Transactions on Fuzzy Systems.

[54]  Meng Joo Er,et al.  Generalized Single-Hidden Layer Feedforward Networks for Regression Problems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[55]  Chin-Teng Lin,et al.  Identification and Prediction of Dynamic Systems Using an Interactively Recurrent Self-Evolving Fuzzy Neural Network , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[56]  Meng Joo Er,et al.  Parsimonious Extreme Learning Machine Using Recursive Orthogonal Least Squares , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[57]  Sundaram Suresh,et al.  A Meta-Cognitive Learning Algorithm for an Extreme Learning Machine Classifier , 2013, Cognitive Computation.

[58]  Plamen P. Angelov,et al.  Evolving Fuzzy-Rule-Based Classifiers From Data Streams , 2008, IEEE Transactions on Fuzzy Systems.

[59]  J. Flavell Piaget's Legacy , 1996 .

[60]  Ganapati Panda,et al.  Identification of nonlinear dynamic systems using functional link artificial neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[61]  T. O. Nelson Metamemory: A Theoretical Framework and New Findings , 1990 .

[62]  Xin Yao,et al.  DDD: A New Ensemble Approach for Dealing with Concept Drift , 2012, IEEE Transactions on Knowledge and Data Engineering.

[63]  Mahardhika Pratama,et al.  GENEFIS: Toward an Effective Localist Network , 2014, IEEE Transactions on Fuzzy Systems.

[64]  Edwin Lughofer,et al.  On-line incremental feature weighting in evolving fuzzy classifiers , 2011, Fuzzy Sets Syst..

[65]  Vipin Kumar,et al.  Chapman & Hall/CRC Data Mining and Knowledge Discovery Series , 2008 .

[66]  Claudia-Adina Dragos,et al.  Online identification of evolving Takagi-Sugeno-Kang fuzzy models for crane systems , 2014, Appl. Soft Comput..

[67]  Edwin Lughofer,et al.  Evolving Fuzzy Systems - Methodologies, Advanced Concepts and Applications , 2011, Studies in Fuzziness and Soft Computing.

[68]  Gaël Richard,et al.  Multiclass Feature Selection With Kernel Gram-Matrix-Based Criteria , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[69]  Meng Joo Er,et al.  A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks , 2009, Neurocomputing.

[70]  Ning Wang,et al.  A Generalized Ellipsoidal Basis Function Based Online Self-constructing Fuzzy Neural Network , 2011, Neural Processing Letters.

[71]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[72]  Plamen P. Angelov,et al.  PANFIS: A Novel Incremental Learning Machine , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[73]  Gin-Der Wu,et al.  A Maximizing-Discriminability-Based Self-Organizing Fuzzy Network for Classification Problems , 2010, IEEE Transactions on Fuzzy Systems.

[74]  Sundaram Suresh,et al.  Meta-cognitive RBF Network and its Projection Based Learning algorithm for classification problems , 2013, Appl. Soft Comput..

[75]  Edwin Lughofer,et al.  On-line assurance of interpretability criteria in evolving fuzzy systems - Achievements, new concepts and open issues , 2013, Inf. Sci..