Improved variable-Length Particle Swarm Optimization for Structure-Adjustable Extreme Learning Machine

Extreme learning machine (ELM) is one of the single hidden layer feed-forward neural networks (SLFNs). It has been widely used for multiclass classification because of the preferable generalization performance and its faster learning speed. The parameters (including the input weights, hidden biases and the number of hidden neurons) have great impact on the generalization performance of ELM classifier. An improved variable-length particle swarm optimization (IVPSO) algorithm is proposed in this paper to automatically select the optimal structure of ELM classifier (the number of hidden neurons with the corresponding input weights and hidden biases) for maximizing the accuracy of validation data and minimizing the norm of output weights. It has been verified in the experimental results that the new algorithm IVPSO-ELM significantly increases the testing accuracy of many classification problems that we choose in UCI machine learning repository.

[1]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[2]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[3]  Xizhao Wang,et al.  Dynamic ensemble extreme learning machine based on sample entropy , 2012, Soft Comput..

[4]  Guang-Bin Huang,et al.  Classification ability of single hidden layer feedforward neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[5]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[6]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[7]  김용수,et al.  Extreme Learning Machine 기반 퍼지 패턴 분류기 설계 , 2015 .

[8]  Fei Han,et al.  An improved evolutionary extreme learning machine based on particle swarm optimization , 2013, Neurocomputing.

[9]  Zakariya M. Al-Hamouz,et al.  Application of Particle Swarm Optimization Algorithm for Optimal Reactive Power Planning , 2007, Control. Intell. Syst..

[10]  Rini Akmeliawati,et al.  Robust State Feedback Control Design via PSO-Based Constrained Optimization , 2011, Control. Intell. Syst..

[11]  Feilong Cao,et al.  A study on effectiveness of extreme learning machine , 2011, Neurocomputing.

[12]  Visakan Kadirkamanathan,et al.  A Function Estimation Approach to Sequential Learning with Neural Networks , 1993, Neural Computation.

[13]  Yuan Lan,et al.  Constructive hidden nodes selection of extreme learning machine for regression , 2010, Neurocomputing.

[14]  Han Wang,et al.  Ensemble Based Extreme Learning Machine , 2010, IEEE Signal Processing Letters.

[15]  Alexandros Iosifidis,et al.  Minimum Class Variance Extreme Learning Machine for Human Action Recognition , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[17]  Vasant Honavar,et al.  Learn++: an incremental learning algorithm for supervised neural networks , 2001, IEEE Trans. Syst. Man Cybern. Part C.

[18]  Jason Jianjun Gu,et al.  An improved extreme learning machine based on Variable-length Particle Swarm Optimization , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[19]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[20]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[21]  Nan Liu,et al.  Voting based extreme learning machine , 2012, Inf. Sci..

[22]  R. Venkatesh Babu,et al.  No-reference image quality assessment using modified extreme learning machine classifier , 2009, Appl. Soft Comput..

[23]  Min Han,et al.  A modified fast recursive hidden nodes selection algorithm for ELM , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[24]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[26]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[27]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[28]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[29]  Min Liu,et al.  A new online learning algorithm for structure-adjustable extreme learning machine , 2010, Comput. Math. Appl..

[30]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[31]  Yang Shu,et al.  Evolutionary Extreme Learning Machine : Based on Particle Swarm Optimization , 2006 .