Evolutionary Ordinal Extreme Learning Machine

Recently the ordinal extreme learning machine (ELMOR) algorithm has been proposed to adapt the extreme learning machine (ELM) algorithm to ordinal regression problems (problems where there is an order arrangement between categories). In addition, the ELM standard model has the drawback of needing many hidden layer nodes in order to achieve suitable performance. For this reason, several alternatives have been proposed, such as the evolutionary extreme learning machine (EELM). In this article we present an evolutionary ELMOR that improves the performance of ELMOR and EELM for ordinal regression. The model is integrated in the differential evolution algorithm of EELM, and it is extended to allow the use of a continuous weighted RMSE fitness function which is proposed to guide the optimization process. This favors classifiers which predict labels as close as possible (in the ordinal scale) to the real one. The experiments include eight datasets, five methods and three specific performance metrics. The results show the performance improvement of this type of neural networks for specific metrics which consider both the magnitude of errors and class imbalance.

[1]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[2]  Gianluca Pollastri,et al.  A neural network approach to ordinal regression , 2007, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[3]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[4]  Wei Chu,et al.  Gaussian Processes for Ordinal Regression , 2005, J. Mach. Learn. Res..

[5]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[6]  María Pérez-Ortiz,et al.  An Experimental Study of Different Ordinal Regression Methods and Measures , 2012, HAIS.

[7]  Jaime S. Cardoso,et al.  The unimodal model for the classification of ordinal data , 2008, Neural Networks.

[8]  Luc De Raedt,et al.  Machine Learning: ECML 2001 , 2001, Lecture Notes in Computer Science.

[9]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[10]  Andrea Esuli,et al.  Evaluation Measures for Ordinal Regression , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[11]  Eyke Hüllermeier,et al.  Is an ordinal class structure useful in classifier learning? , 2008, Int. J. Data Min. Model. Manag..

[12]  Qinghua Zheng,et al.  Ordinal extreme learning machine , 2010, Neurocomputing.

[13]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[14]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[15]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[16]  Sung-Bae Cho,et al.  Hybrid Artificial Intelligent Systems , 2015, Lecture Notes in Computer Science.

[17]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Hung Keng Pung,et al.  Universal Approximation and QoS Violation Application of Extreme Learning Machine , 2008, Neural Processing Letters.

[19]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[20]  Pedro Antonio Gutiérrez,et al.  Exploitation of Pairwise Class Distances for Ordinal Classification , 2013, Neural Computation.

[21]  Pedro Antonio Gutiérrez,et al.  Weighting Efficient Accuracy and Minimum Sensitivity for Evolving Multi-Class Classifiers , 2011, Neural Processing Letters.

[22]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..