Binary Output Layer of Extreme Learning Machine for Solving Multi-class Classification Problems

Considered in this paper is the design of output layer nodes of extreme learning machine (ELM) for solving multi-class classification problems with r ( $$r\ge 3$$ r ≥ 3 ) classes of samples. The common and conventional setting of output layer, called “ one - to - one approach ” in this paper, is as follows: The output layer contains r output nodes corresponding to the r classes. And for an input sample of the i th class ( $$1\le i\le r$$ 1 ≤ i ≤ r ), the ideal output is 1 for the i th output node, and 0 for all the other output nodes. We propose in this paper a new “ binary approach ”: Suppose $$2^{q-1}< r\le 2^q$$ 2 q - 1 < r ≤ 2 q with $$q\ge 2$$ q ≥ 2 , then we let the output layer contain q output nodes, and let the ideal outputs for the r classes be designed in a binary manner. Numerical experiments carried out in this paper show that our binary approach does equally good job as, but uses less output nodes and hidden-output weights than, the traditional one-to-one approach.

[1]  Jacek M. Zurada,et al.  Batch gradient method with smoothing L1/2 regularization for training of feedforward neural networks , 2014, Neural Networks.

[2]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[3]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[4]  Fei Pei,et al.  Prediction Research of Transformer Fault Based on Regular Extreme Learning Machine , 2014 .

[5]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[6]  R. Law.  Back-propagation learning in improving the accuracy of neural network-based tourism demand forecasting , 2000 .

[7]  Yi Lu Murphey,et al.  Multi-class pattern classification using neural networks , 2007, Pattern Recognit..

[8]  Shumin Fei,et al.  Neural network for multi-class classification by boosting composite stumps , 2015, Neurocomputing.

[9]  Chee Peng Lim,et al.  A neural network-based multi-agent classifier system , 2009, Neurocomputing.

[10]  Chi-Man Vong,et al.  Sparse Bayesian Extreme Learning Machine for Multi-classification , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[12]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[13]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[14]  Rajen B. Bhatt,et al.  User Localization in an Indoor Environment Using Fuzzy Hybrid of Particle Swarm Optimization & Gravitational Search Algorithm with Neural Networks , 2016, SocProS.

[15]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[16]  Jacek M. Zurada,et al.  Convergence of online gradient method for feedforward neural networks with smoothing L1/2 regularization penalty , 2014, Neurocomputing.

[17]  Larry Bull,et al.  Building anticipations in an accuracy-based learning classifier system by use of an artificial neural network , 2005, 2005 IEEE Congress on Evolutionary Computation.

[18]  D. F. Specht,et al.  Generalization accuracy of probabilistic neural networks compared with backpropagation networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[19]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[20]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[21]  Chao Zhang,et al.  Binary Output Layer of Feedforward Neural Networks for Solving Multi-Class Classification Problems , 2019, IEEE Access.

[22]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[23]  D. Serre.  Matrices: Theory and Applications , 2002 .

[24]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[25]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[26]  Sundaram Suresh,et al.  Meta-cognitive Neural Network for classification problems in a sequential learning framework , 2012, Neurocomputing.

[27]  Wei Wu,et al.  The Binary Output Units of Neural Network , 2013, ISNN.

[28]  Alan J. Mayne,et al.  Generalized Inverse of Matrices and its Applications , 1972 .