A Meta-Cognitive Learning Algorithm for an Extreme Learning Machine Classifier

This paper presents an efficient fast learning classifier based on the Nelson and Narens model of human meta-cognition, namely ‘Meta-cognitive Extreme Learning Machine (McELM).’ McELM has two components: a cognitive component and a meta-cognitive component. The cognitive component of McELM is a three-layered extreme learning machine (ELM) classifier. The neurons in the hidden layer of the cognitive component employ the q-Gaussian activation function, while the neurons in the input and output layers are linear. The meta-cognitive component of McELM has a self-regulatory learning mechanism that decides what-to-learn, when-to-learn, and how-to-learn in a meta-cognitive framework. As the samples in the training set are presented one-by-one, the meta-cognitive component receives the monitory signals from the cognitive component and chooses suitable learning strategies for the sample. Thus, it either deletes the sample, uses the sample to add a new neuron, or updates the output weights based on the sample, or reserves the sample for future use. Therefore, unlike the conventional ELM, the architecture of McELM is not fixed a priori, instead, the network is built during the training process. While adding a neuron, McELM chooses the centers based on the sample, and the width of the Gaussian function is chosen randomly. The output weights are estimated using the least square estimate based on the hinge-loss error function. The hinge-loss error function facilitates prediction of posterior probabilities better than the mean-square error and is hence preferred to develop the McELM classifier. While updating the network parameters, the output weights are updated using a recursive least square estimate. The performance of McELM is evaluated on a set of benchmark classification problems from the UCI machine learning repository. Performance study results highlight that meta-cognition in ELM framework enhances the decision-making ability of ELM significantly.

[1]  Paramasivan Saratchandran,et al.  Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm , 1998, IEEE Trans. Neural Networks.

[2]  Firdaus E. Udwadia,et al.  Generalized LM-inverse of a matrix augmented by a column vector , 2007, Appl. Math. Comput..

[3]  Sundaram Suresh,et al.  A meta-cognitive sequential learning algorithm for neuro-fuzzy inference system , 2012, Appl. Soft Comput..

[4]  Dong Sun Park,et al.  Online sequential extreme learning machine with forgetting mechanism , 2012, Neurocomputing.

[5]  Pedro Antonio Gutiérrez,et al.  Parameter estimation of q-Gaussian Radial Basis Functions Neural Networks with a Hybrid Algorithm for binary classification , 2012, Neurocomputing.

[6]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[7]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Chi Cheng,et al.  Extreme learning machines for intrusion detection , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[10]  Narasimhan Sundararajan,et al.  A Sequential Learning Algorithm for Complex-Valued Self-Regulating Resource Allocation Network-CSRAN , 2011, IEEE Transactions on Neural Networks.

[11]  Ye Yuan,et al.  An OS-ELM based distributed ensemble classification framework in P2P networks , 2011, Neurocomputing.

[12]  R. Venkatesh Babu,et al.  No-reference image quality assessment using modified extreme learning machine classifier , 2009, Appl. Soft Comput..

[13]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[14]  Erik Cambria,et al.  Sentic Activation: A Two-Level Affective Common Sense Reasoning Framework , 2012, AAAI.

[15]  Narasimhan Sundararajan,et al.  A fast learning Fully Complex-valued Relaxation Network (FCRN) , 2011, The 2011 International Joint Conference on Neural Networks.

[16]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[17]  Sundaram Suresh,et al.  A meta-cognitive learning algorithm for a Fully Complex-valued Relaxation Network , 2012, Neural Networks.

[18]  Sundaram Suresh,et al.  Meta-cognitive RBF Network and its Projection Based Learning algorithm for classification problems , 2013, Appl. Soft Comput..

[19]  Sundaram Suresh,et al.  A Fully Complex-Valued Radial Basis Function Network and its Learning Algorithm , 2009, Int. J. Neural Syst..

[20]  Randy M. Isaacson,et al.  Metacognitive Knowledge Monitoring and Self-Regulated Learning: Academic Success and Reflections on Learning. , 2006 .

[21]  Anamitra Makur,et al.  Online adaptive radial basis function networks for robust object tracking , 2010, Comput. Vis. Image Underst..

[22]  Sundaram Suresh,et al.  Sequential Projection-Based Metacognitive Learning in a Radial Basis Function Network for Classification Problems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Buyurman Baykal,et al.  Complexity reduction in radial basis function (RBF) networks by using radial B-spline functions , 1998, Neurocomputing.

[24]  Sundaram Suresh,et al.  Metacognitive Learning in a Fully Complex-Valued Radial Basis Function Neural Network , 2012, Neural Computation.

[25]  T. O. Nelson.  Metamemory: A Theoretical Framework and New Findings , 1990 .

[26]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[27]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Sundaram Suresh,et al.  Performance enhancement of extreme learning machine for multi-category sparse data classification problems , 2010, Eng. Appl. Artif. Intell..

[29]  Michael T. Cox.  Metacognition in computation: A selected research review , 2005, Artif. Intell..

[30]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[31]  H. J. Kim,et al.  A sequential learning algorithm for self-adaptive resource allocation network classifier , 2010, Neurocomputing.

[32]  Narasimhan Sundararajan,et al.  ICGA-PSO-ELM Approach for Accurate Multiclass Cancer Classification Resulting in Reduced Gene Sets in Which Genes Encoding Secreted Proteins Are Highly Represented , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[33]  Narasimhan Sundararajan,et al.  Risk-sensitive loss functions for sparse multi-category classification problems , 2008, Inf. Sci..

[34]  C. Siew,et al.  Extreme Learning Machine with Randomly Assigned RBF Kernels , 2005 .

[35]  Sundaram Suresh,et al.  A sequential learning algorithm for meta-cognitive neuro-fuzzy inference system for classification problems , 2011, The 2011 International Joint Conference on Neural Networks.

[36]  Haiyang Li,et al.  A Reconstruction Approach to CT with Cauchy RBFs Network , 2004, ISNN.

[37]  Yi Zhao,et al.  A protein secondary structure prediction framework based on the Extreme Learning Machine , 2008, Neurocomputing.

[38]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  K. Subramanian,et al.  Human Action Recognition using MetaCognitive Neuro-Fuzzy Inference System , 2012 .

[40]  Danilo P. Mandic,et al.  An Augmented Extended Kalman Filter Algorithm for Complex-Valued Recurrent Neural Networks , 2006, Neural Computation.

[41]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[42]  Judith Redi,et al.  Circular-ELM for the reduced-reference assessment of perceived image quality , 2013, Neurocomputing.

[43]  Sundaram Suresh,et al.  Meta-cognitive Neural Network for classification problems in a sequential learning framework , 2012, Neurocomputing.

[44]  Narasimhan Sundararajan,et al.  A sequential multi-category classifier using radial basis function networks , 2008, Neurocomputing.

[45]  Narasimhan Sundararajan,et al.  Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[46]  Xiaojun Qi,et al.  A Radial Basis Function and Semantic Learning Space Based Composite Learning Approach to Image Retrieval , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[47]  Sundaram Suresh,et al.  Human action recognition using Meta-Cognitive Neuro-Fuzzy Inference System , 2012, IJCNN.