Knowledge-based extreme learning machines

Abstract By incorporating prior knowledge in the form of implications into extreme learning machine (ELM), a novel knowledge-based extreme learning machine (KBELM) formulation is proposed in this work. In this approach, the nonlinear prior knowledge implications are converted into linear inequalities and are then included as linear equality constraints in the ELM formulation. The proposed KBELM formulation has the advantage that it leads to solving a system of linear equations. Effectiveness of the proposed approach is demonstrated on three synthetic and the publicly available Wisconsin Prognostic Breast Cancer datasets by comparing their results with ELM and optimally pruned ELM using additive and radial basis function hidden nodes.

[1]  Jude W. Shavlik,et al.  Knowledge-Based Kernel Approximation , 2004, J. Mach. Learn. Res..

[2]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[3]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[4]  Olvi L. Mangasarian,et al.  Nonlinear Knowledge in Kernel Approximation , 2007, IEEE Transactions on Neural Networks.

[5]  Amaury Lendasse,et al.  TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization , 2011, Neurocomputing.

[6]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[7]  Haigang Zhang,et al.  An Improved ELM Algorithm Based on EM-ELM and Ridge Regression , 2013, IScIDE.

[8]  Haigang Zhang,et al.  An Improved ELM Algorithm Based on PCA Technique , 2015 .

[9]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .

[10]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[11]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Glenn Fung,et al.  Knowledge-Based Nonlinear Kernel Classifiers , 2003, COLT.

[13]  C. R. Rao,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[14]  Reshma Khemchandani,et al.  Knowledge based proximal support vector machines , 2009, Eur. J. Oper. Res..

[15]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[16]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[17]  Feilong Cao,et al.  Optimization approximation solution for regression problem based on extreme learning machine , 2011, Neurocomputing.

[18]  Yuan Lan,et al.  Two-stage extreme learning machine for regression , 2010, Neurocomputing.

[19]  Glenn Fung,et al.  Proximal Knowledge‐based Classification , 2009, Stat. Anal. Data Min..

[20]  S. Balasundaram,et al.  1-Norm extreme learning machine for regression and multiclass classification using Newton method , 2014, Neurocomputing.

[21]  Glenn Fung,et al.  Knowledge-Based Support Vector Machine Classifiers , 2002, NIPS.

[22]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[23]  Yong Shi,et al.  A group of knowledge-incorporated multiple criteria linear programming classifiers , 2011, J. Comput. Appl. Math..

[24]  Zhixiang Chen,et al.  A modified extreme learning machine with sigmoidal activation functions , 2012, Neural Computing and Applications.

[25]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[26]  Olvi L. Mangasarian,et al.  Nonlinear knowledge in kernel machines , 2008 .

[27]  S. Balasundaram,et al.  Application of error minimized extreme learning machine for simultaneous learning of a function and its derivatives , 2011, Neurocomputing.

[28]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[29]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[30]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.