A novel extreme learning machine using privileged information

Extreme learning machine (ELM) is a competitive machine learning technique, which is much more efficient and usually lead to better generalization performance compared to the traditional classifiers. In order to further improve its performance, we proposed a novel ELM called ELM+ which introduces the privileged information to the traditional ELM method. This privileged information, which is ignored by the classical ELM but often exists in human teaching and learning, will optimize the training stage by constructing a set of correcting functions. We demonstrate the performance of ELM+ on datasets from UCI machine learning repository, Mackey-Glass time series and radar emitter recognition and also present the comparison with SVM, ELM and SVM+. The experimental results indicate the validity and advantage of our method.

[1]  Klaus Neumann,et al.  Optimizing extreme learning machines via ridge regression and batch intrinsic plasticity , 2013, Neurocomputing.

[2]  Uwe Aickelin,et al.  Privileged information for data clustering , 2012, Inf. Sci..

[3]  Vladimir Cherkassky,et al.  Learning Using Structured Data: Application to fMRI Data Analysis , 2007, 2007 International Joint Conference on Neural Networks.

[4]  Ping Zhong,et al.  A new one-class SVM based on hidden information , 2014, Knowl. Based Syst..

[5]  Amaury Lendasse,et al.  Regularized extreme learning machine for regression with missing data , 2013, Neurocomputing.

[6]  Lin Li,et al.  Radar emitter recognition based on cyclostationary signatures and sequential iterative least-square estimation , 2011, Expert Syst. Appl..

[7]  Chin-Teng Lin,et al.  A vector neural network for emitter identification , 2002 .

[8]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[9]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Narasimhan Sundararajan,et al.  Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Feilong Cao,et al.  A study on effectiveness of extreme learning machine , 2011, Neurocomputing.

[12]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[13]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[14]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[15]  Christoph H. Lampert,et al.  Learning to Rank Using Privileged Information , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Vladimir Cherkassky,et al.  Connection between SVM+ and multi-task learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[17]  Punyaphol Horata,et al.  Robust extreme learning machine , 2013, Neurocomputing.

[18]  Dong Sun Park,et al.  Online sequential extreme learning machine with forgetting mechanism , 2012, Neurocomputing.

[19]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[20]  Zhihong Man,et al.  Robust Single-Hidden Layer Feedforward Network-Based Pattern Classifier , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Qinghua Zheng,et al.  Regularized Extreme Learning Machine , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[22]  Saeed Zolfaghari,et al.  Chaotic time series prediction with residual analysis method using hybrid Elman-NARX neural networks , 2010, Neurocomputing.

[23]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .

[24]  Jiwen Dong,et al.  Time-series prediction using a local linear wavelet neural network , 2006, Neurocomputing.

[25]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[26]  Vladimir Vapnik,et al.  A new learning paradigm: Learning using privileged information , 2009, Neural Networks.

[27]  Min Han,et al.  Partial Lanczos extreme learning machine for single-output regression problems , 2009, Neurocomputing.

[28]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).