An Analytical Study on Reasoning of Extreme Learning Machine for Classification from Its Inductive Bias

Since extreme learning machine (ELM) was proposed, hundreds of studies have been conducted on this subject in various areas, from theoretical researches to practical applications. However, there are very few papers in the literature to reveal the reasons why in ELM classification the class with the highest output value is being chosen as the predicted class for a given input. In order to give a clear insight into this question, this paper analyzes the rationality of ELM reasoning from the perspective of its inductive bias. The analysis results show that the choice of highest output in ELM is reasonable for both binary and multiclass classification problems. In addition, to deal with multiclass problems ELM uses the well-known one-against-all strategy, in which unclassifiable regions may exist. This paper also gives a clear explanation on how ELM resolves the unclassifiable regions, through both analysis and experiments.

[1]  Shigeo Abe.  Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[2]  Hui Liu,et al.  Four wind speed multi-step forecasting models using extreme learning machines and signal decomposing algorithms , 2015 .

[3]  Qian Du,et al.  Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Han Wang,et al.  Ensemble Based Extreme Learning Machine , 2010, IEEE Signal Processing Letters.

[5]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[6]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[7]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[8]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[10]  Chi-Man Vong,et al.  Fast and accurate face detection by sparse Bayesian extreme learning machine , 2014, Neural Computing and Applications.

[11]  Lin Qi,et al.  Extended Extreme Learning Machine for Biometric Signal Classification , 2015 .

[12]  Tong Zhang,et al.  Statistical Analysis of Some Multi-Category Large Margin Classification Methods , 2004, J. Mach. Learn. Res..

[13]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[14]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[15]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.