Extreme learning machine: algorithm, theory and applications

Extreme learning machine (ELM) is a new learning algorithm for the single hidden layer feedforward neural networks. Compared with the conventional neural network learning algorithm it overcomes the slow training speed and over-fitting problems. ELM is based on empirical risk minimization theory and its learning process needs only a single iteration. The algorithm avoids multiple iterations and local minimization. It has been used in various fields and applications because of better generalization ability, robustness, and controllability and fast learning rate. In this paper, we make a review of ELM latest research progress about the algorithms, theory and applications. It first analyzes the theory and the algorithm ideas of ELM, then tracking describes the latest progress of ELM in recent years, including the model and specific applications of ELM, finally points out the research and development prospects of ELM in the future.

[1]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[2]  Pedro Antonio Gutiérrez,et al.  MELM-GRBF: A modified version of the extreme learning machine for generalized radial basis function neural networks , 2011, Neurocomputing.

[3]  Zexuan Zhu,et al.  A fast pruned-extreme learning machine for classification problem , 2008, Neurocomputing.

[4]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[5]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[6]  Shuai Li,et al.  Selective Positive–Negative Feedback Produces the Winner-Take-All Competition in Recurrent Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Guang-Bin Huang,et al.  Learning capability and storage capacity of two-hidden-layer feedforward networks , 2003, IEEE Trans. Neural Networks.

[8]  Q. M. Jonathan Wu,et al.  Human face recognition based on multidimensional PCA and extreme learning machine , 2011, Pattern Recognit..

[9]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[10]  Shuai Li,et al.  Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks , 2012, Neurocomputing.

[11]  Stephen Grossberg,et al.  Adaptive Resonance Theory , 2010, Encyclopedia of Machine Learning.

[12]  Enrique Romero,et al.  Comparing error minimized extreme learning machines and support vector sequential feed-forward neural networks , 2012, Neural Networks.

[13]  Hong Zhu,et al.  Optimizing radial basis function neural network based on rough sets and affinity propagation clustering algorithm , 2012, Journal of Zhejiang University SCIENCE C.

[14]  Lili Liu,et al.  Research of neural network algorithm based on factor analysis and cluster analysis , 2011, Neural Computing and Applications.

[15]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[16]  Guang-Bin Huang,et al.  Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions , 1998, IEEE Trans. Neural Networks.

[17]  Cai Lei.  Comparison of the Extreme Learning Machine with the Support Vector Machine for Reservoir Permeability Prediction , 2010 .

[18]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[19]  Guang-Bin Huang,et al.  Neuron selection for RBF neural network classifier based on data structure preserving criterion , 2005, IEEE Transactions on Neural Networks.

[20]  Shuai Li,et al.  Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function , 2012, Neural Processing Letters.

[21]  Jing Zhong,et al.  A Classification Approach Based on Evolutionary Neural Networks , 2006 .

[22]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[23]  Han Min.  Fusion of Thermal Infrared and Multispectral Remote Sensing Images via Neural Network Regression , 2010 .

[24]  José David Martín-Guerrero,et al.  Regularized extreme learning machine for regression problems , 2011, Neurocomputing.

[25]  Urszula Markowska-Kaczmar,et al.  Fuzzy logic and evolutionary algorithm - two techniques in rule extraction from neural networks , 2005, Neurocomputing.

[26]  KahramanliHumar,et al.  Rule extraction from trained adaptive neural networks using artificial immune systems , 2009 .

[27]  Lv Zhe.  Soft Sensing Modeling Based on Extreme Learning Machine for Biochemical Processes , 2007 .

[28]  Teresa Bernarda Ludermir,et al.  An evolutionary extreme learning machine based on group search optimization , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[29]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[30]  Pan Hua-xian.  Lithologic identification based on ELM , 2010 .

[31]  Zhang Xian,et al.  Incremental regularized extreme learning machine based on Cholesky factorization and its application to time series prediction , 2011 .

[32]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[33]  Narasimhan Sundararajan,et al.  Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[34]  Chee Peng Lim,et al.  A modified fuzzy min-max neural network with rule extraction and its application to fault detection and classification , 2008, Appl. Soft Comput..

[35]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[36]  Zhang Xian,et al.  Selective forgetting extreme learning machine and its application to time series prediction , 2011 .

[37]  Li Xu,et al.  An optimizing method of RBF neural network based on genetic algorithm , 2011, Neural Computing and Applications.

[38]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[39]  Lin Chen,et al.  Research on Extreme Learning of Neural Networks: Research on Extreme Learning of Neural Networks , 2010 .

[40]  S. Grossberg.  Adaptive Resonance Theory , 2006 .

[41]  Chen Shi-fu,et al.  A Classification Approach Based on Evolutionary Neural Networks , 2005 .

[42]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[43]  Kezhi Mao,et al.  RBF neural network center selection based on Fisher ratio class separability measure , 2002, IEEE Trans. Neural Networks.

[44]  De-Shuang Huang,et al.  Improved extreme learning machine for function approximation by encoding a priori information , 2006, Neurocomputing.

[45]  Marghny H. Mohamed,et al.  Rules extraction from constructively trained neural networks based on genetic algorithms , 2011, Neurocomputing.

[46]  Shifei Ding,et al.  An optimizing BP neural network algorithm based on genetic algorithm , 2011, Artificial Intelligence Review.

[47]  Qinghua Zheng,et al.  Ordinal extreme learning machine , 2010, Neurocomputing.

[48]  FengGuorui,et al.  Error minimized extreme learning machine with growth of hidden nodes and incremental learning , 2009 .

[49]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[50]  Feilong Cao,et al.  A study on effectiveness of extreme learning machine , 2011, Neurocomputing.

[51]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[52]  Yuan Lan,et al.  Two-stage extreme learning machine for regression , 2010, Neurocomputing.

[53]  Shuai Li,et al.  A nonlinear model to generate the winner-take-all competition , 2013, Commun. Nonlinear Sci. Numer. Simul..

[54]  Narasimhan Sundararajan,et al.  On-Line Sequential Extreme Learning Machine , 2005, Computational Intelligence.

[55]  Novruz Allahverdi,et al.  Rule extraction from trained adaptive neural networks using artificial immune systems , 2009, Expert Syst. Appl..

[56]  Yaonan Wang,et al.  Rough Neural Network Based on Bottom-Up Fuzzy Rough Data Analysis , 2009, Neural Processing Letters.

[57]  Deng Wan.  Research on Extreme Learning of Neural Networks , 2010 .