Hierarchical extreme learning machine for feedforward neural network

An approach, named extended extreme learning machine (ELM), is proposed for training the weights of a class of hierarchical feedforward neural network (HFNN). Unlike conventional single-hidden-layer feedforward networks (SLFNs), this hierarchical ELM (HELM) is based on the hierarchical structure which is capable of hierarchical learning of sequential information online, and one may simply choose hidden layers and then only need to adjust the output weights linking the hidden layer and the output layer. In such HELM implementations, the extended ELM provides better generalization performance during the learning process. Moreover, the proposed extended ELM method is efficient not only for HFNNs with sigmoid hidden nodes but also for HFNNs with radial basis function (RBF) hidden nodes. Finally, the HELM is applied to the activated sludge wastewater treatment processes (WWTPs) for predicting the water qualities. Experimental results and the performance comparison demonstrate the effectiveness of the proposed HELM.

[1]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[2]  J. Qiao,et al.  Prediction of activated sludge bulking based on a self-organizing RBF neural network☆ , 2012 .

[3]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[4]  Xizhao Wang,et al.  Dynamic ensemble extreme learning machine based on sample entropy , 2012, Soft Comput..

[5]  Yong Yu,et al.  Sales forecasting using extreme learning machine with applications in fashion retailing , 2008, Decis. Support Syst..

[6]  Zongben Xu,et al.  Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Dianhui Wang,et al.  Global Convergence of Online BP Training With Dynamic Learning Rate , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Amaury Lendasse,et al.  TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization , 2011, Neurocomputing.

[9]  Håkan Hjalmarsson,et al.  How to Make Bias and Variance Errors Insensitive to System and Model Complexity in Identification , 2011, IEEE Transactions on Automatic Control.

[10]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[11]  Huaicheng Guo,et al.  Water quality modeling for load reduction under uncertainty: a Bayesian approach. , 2008, Water research.

[12]  Junfei Qiao,et al.  Model predictive control of dissolved oxygen concentration based on a self-organizing RBF neural network , 2012 .

[13]  Zhiming Li,et al.  Modeling and Monitoring of Dynamic Processes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[15]  Junfei Qiao,et al.  An efficient self-organizing RBF neural network for water quality prediction , 2011, Neural Networks.

[16]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[17]  Ivica Kostanic,et al.  Principles of Neurocomputing for Science and Engineering , 2000 .

[18]  Yonggwan Won,et al.  An Improvement of Extreme Learning Machine for Compact Single-Hidden-Layer Feedforward Neural Networks , 2008, Int. J. Neural Syst..

[19]  Leszek Rutkowski,et al.  A fast training algorithm for neural networks , 1998 .

[20]  Nor Ashidi Mat Isa,et al.  Modified Recursive Least Squares algorithm to train the Hybrid Multilayered Perceptron (HMLP) network , 2010, Appl. Soft Comput..

[21]  Antonio J. Serrano,et al.  BELM: Bayesian Extreme Learning Machine , 2011, IEEE Transactions on Neural Networks.

[22]  Tommy W. S. Chow,et al.  Neural Networks and Computing - Learning Algorithms and Applications , 2007, Series in Electrical and Computer Engineering.

[23]  Shuzhi Sam Ge,et al.  Adaptive output feedback NN control of a class of discrete-time MIMO nonlinear systems with unknown control directions , 2009, 2009 7th Asian Control Conference.

[24]  Junfei Qiao,et al.  A Self-Organizing Fuzzy Neural Network Based on a Growing-and-Pruning Algorithm , 2010, IEEE Transactions on Fuzzy Systems.

[25]  M. Perrier,et al.  Prediction of activated sludge filamentous bulking using ATP data and neural networks , 2011 .

[26]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[27]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Yaonan Wang,et al.  Bidirectional Extreme Learning Machine for Regression Problem and Its Learning Effectiveness , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[29]  George A. Rovithakis,et al.  Adaptive Dynamic Output Feedback Neural Network Control of Uncertain MIMO Nonlinear Systems With Prescribed Performance , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Feilong Cao,et al.  A study on effectiveness of extreme learning machine , 2011, Neurocomputing.

[31]  Long Cheng,et al.  Adaptive neural network tracking control for manipulators with uncertain kinematics, dynamics and actuator model , 2009, Autom..

[32]  Erkki Oja,et al.  GPU-accelerated and parallelized ELM ensembles for large-scale regression , 2011, Neurocomputing.

[33]  Andrew Chi-Sing Leung,et al.  Convergence Analyses on On-Line Weight Noise Injection-Based Training Algorithms for MLPs , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Srinivasa Lingireddy,et al.  Backfilling missing microbial concentrations in a riverine database using artificial neural networks. , 2007, Water research.

[35]  E. N. Banadda,et al.  Dynamic modeling of filamentous bulking in lab-scale activated sludge processes , 2004 .

[36]  Zexuan Zhu,et al.  A fast pruned-extreme learning machine for classification problem , 2008, Neurocomputing.

[37]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[38]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[39]  R. Goel,et al.  Biocontrol of biomass bulking caused by Haliscomenobacter hydrossis using a newly isolated lytic bacteriophage. , 2011, Water research.

[40]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[41]  Junfei Qiao,et al.  Adaptive dissolved oxygen control based on dynamic structure neural network , 2011, Appl. Soft Comput..

[42]  Zhihong Man,et al.  Robust Single-Hidden Layer Feedforward Network-Based Pattern Classifier , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Christos S. Akratos,et al.  An artificial neural network model and design equations for BOD and COD removal prediction in horizontal subsurface flow constructed wetlands , 2008 .

[44]  Junfei Qiao,et al.  A Repair Algorithm for Radial Basis Function Neural Network and its Application to Chemical oxygen Demand Modeling , 2010, Int. J. Neural Syst..