Particle Swarm Optimization Based Selective Ensemble of Online Sequential Extreme Learning Machine

A novel particle swarm optimization based selective ensemble (PSOSEN) of online sequential extreme learning machine (OS-ELM) is proposed. It is based on the original OS-ELM with an adaptive selective ensemble framework. Two novel insights are proposed in this paper. First, a novel selective ensemble algorithm referred to as particle swarm optimization selective ensemble is proposed, noting that PSOSEN is a general selective ensemble method which is applicable to any learning algorithms, including batch learning and online learning. Second, an adaptive selective ensemble framework for online learning is designed to balance the accuracy and speed of the algorithm. Experiments for both regression and classification problems with UCI data sets are carried out. Comparisons between OS-ELM, simple ensemble OS-ELM (EOS-ELM), genetic algorithm based selective ensemble (GASEN) of OS-ELM, and the proposed particle swarm optimization based selective ensemble of OS-ELM empirically show that the proposed algorithm achieves good generalization performance and fast learning speed.

[1]  P. Saratchandran,et al.  Multicategory Classification Using An Extreme Learning Machine for Microarray Gene Expression Cancer Diagnosis , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[2]  Han Wang,et al.  Ensemble Based Extreme Learning Machine , 2010, IEEE Signal Processing Letters.

[3]  Zexuan Zhu,et al.  A fast pruned-extreme learning machine for classification problem , 2008, Neurocomputing.

[4]  Kit Po Wong,et al.  A Reliable Intelligent System for Real-Time Dynamic Security Assessment of Power Systems , 2012, IEEE Transactions on Power Systems.

[5]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Tao Chen,et al.  Fast online learning algorithm for landmark recognition based on BoW framework , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[7]  Jiuwen Cao,et al.  Protein Sequence Classification with Improved Extreme Learning Machine Algorithms , 2014, BioMed research international.

[8]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[9]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[10]  Kwontaeg Choi,et al.  Incremental face recognition for large-scale social network services , 2012, Pattern Recognit..

[11]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[12]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[13]  Feng Xu,et al.  Online ship roll motion prediction based on grey sequential extreme learning machine , 2014, Neurocomputing.

[14]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[17]  Berkman Sahiner,et al.  Dual system approach to computer-aided detection of breast masses on mammograms. , 2006, Medical physics.

[18]  Cüneyt Güzelis,et al.  A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering , 2014, Neural Computing and Applications.

[19]  Anders Krogh,et al.  Learning with ensembles: How overfitting can be useful , 1995, NIPS.

[20]  Jianhua Yang,et al.  Genetic ensemble of extreme learning machine , 2014, Neurocomputing.

[21]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[22]  Min Han,et al.  Online sequential extreme learning machine with kernels for nonstationary time series prediction , 2014, Neurocomputing.

[23]  J. Kennedy,et al.  Matching algorithms to problems: an experimental test of the particle swarm and some genetic algorithms on the multimodal problem generator , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).