Ensemble delta test-extreme learning machine (DT-ELM) for regression

Extreme learning machine (ELM) has shown its good performance in regression applications with a very fast speed. But there is still a difficulty to compromise between better generalization performance and smaller complexity of the ELM (a number of hidden nodes). This paper proposes a method called Delta Test-ELM (DT-ELM), which operates in an incremental way to create less complex ELM structures and determines the number of hidden nodes automatically. It uses Bayesian Information Criterion (BIC) as well as Delta Test (DT) to restrict the search as well as to consider the size of the network and prevent overfitting. Moreover, ensemble modeling is used on different DT-ELM models and it shows good test results in Experiments section.

[1]  Amaury Lendasse,et al.  Ensemble KNNs for Bankruptcy Prediction , 2009 .

[2]  Amaury Lendasse,et al.  TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization , 2011, Neurocomputing.

[3]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[4]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  Chee Kheong Siew,et al.  Extreme learning machine: RBF network case , 2004, ICARCV 2004 8th Control, Automation, Robotics and Vision Conference, 2004..

[7]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[8]  Nenad Koncar,et al.  A note on the Gamma test , 1997, Neural Computing & Applications.

[9]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[10]  Antonia J. Jones,et al.  New tools in non-linear modelling and prediction , 2004, Comput. Manag. Sci..

[11]  김용수,et al.  Extreme Learning Machine 기반 퍼지 패턴 분류기 설계 , 2015 .

[12]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[13]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[15]  Benoît Frénay,et al.  Ensembles of Local Linear Models for Bankruptcy Analysis and Prediction , 2011 .

[16]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[17]  Zexuan Zhu,et al.  A fast pruned-extreme learning machine for classification problem , 2008, Neurocomputing.

[18]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[19]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[20]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .

[21]  C. Siew,et al.  Extreme Learning Machine with Randomly Assigned RBF Kernels , 2005 .

[22]  Ping Zhang.  On the convergence rate of model selection criteria , 1993 .

[23]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[24]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[25]  D. B. Preston.  Spectral Analysis and Time Series , 1983 .

[26]  C. R. Rao,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[27]  Erkki Oja,et al.  GPU-accelerated and parallelized ELM ensembles for large-scale regression , 2011, Neurocomputing.

[28]  Yuan Lan,et al.  Random search enhancement of error minimized extreme learning machine , 2010, ESANN.

[29]  Yuan Lan,et al.  Constructive hidden nodes selection of extreme learning machine for regression , 2010, Neurocomputing.

[30]  Nan Liu,et al.  Patient Outcome Prediction with Heart Rate Variability and Vital Signs , 2011, J. Signal Process. Syst..

[31]  Yuan Lan,et al.  Extreme Learning Machine based bacterial protein subcellular localization prediction , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[32]  Chee Kheong Siew,et al.  Incremental extreme learning machine with fully complex hidden nodes , 2008, Neurocomputing.

[33]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[34]  Nan Liu,et al.  Voting based extreme learning machine , 2012, Inf. Sci..

[35]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[36]  Guang-Bin Huang,et al.  Face recognition based on extreme learning machine , 2011, Neurocomputing.

[37]  Yoram Reich,et al.  Ensemble modelling or selecting the best model: Many could be better than one , 1999, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[38]  Michel Verleysen,et al.  Ensemble Modeling with a Constrained Linear System of Leave-One-Out Outputs , 2010, ESANN.