Tensor extreme learning design via generalized Moore–Penrose inverse and triangular type-2 fuzzy sets

A tensor-based extreme learning machine is proposed, which is referred to as tensor-based type-2 extreme learning machine (TT2-ELM). In contrast to the work on ELM, regularized ELM (RELM), weighted regularized ELM (WRELM) and least squares support vector machine (LS-SVM), which are the most often used learning algorithm in regression problems, TT2-ELM adopts the tensor structure to construct the ELM for type-2 fuzzy sets, Moore–Penrose inverse of tensor is used to obtain the tensor regression result. No further type-reduction method is needed to obtain the coincide type-1 fuzzy sets, and type-2 fuzzy structure can be seamlessly incorporated into the ELM scheme. Experimental results are carried out on two Sinc functions, a nonlinear system identification problem and four real-world regression problems, results show that TT2-ELM performs at competitive level of generalized performance as the ELM, RELM, WRELM and LS-SVM on the small- and moderate-scale data sets.

[1]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[2]  Yimin Wei,et al.  Weighted Moore-Penrose inverses and fundamental theorem of even-order tensors with Einstein product , 2017 .

[3]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[4]  Amaury Lendasse,et al.  TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization , 2011, Neurocomputing.

[5]  Lexin Li,et al.  Regularized matrix regression , 2012, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[6]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[7]  J. Mendel.  Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions , 2001 .

[8]  Lili Guo,et al.  Extreme learning machine with kernel model based on deep learning , 2017, Neural Computing and Applications.

[9]  Nikos D. Sidiropoulos,et al.  ParCube: Sparse Parallelizable Tensor Decompositions , 2012, ECML/PKDD.

[10]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[11]  Yew-Soon Ong,et al.  A Fast Reduced Kernel Extreme Learning Machine , 2016, Neural Networks.

[12]  Guang-Bin Huang,et al.  A Fast SVD-Hidden-nodes based Extreme Learning Machine for Large-Scale Data Analytics , 2016, Neural Networks.

[13]  Tamara G. Kolda,et al.  Scalable Tensor Decompositions for Multi-aspect Data Mining , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[14]  Tingzhu Huang,et al.  A non-convex tensor rank approximation for tensor completion , 2017 .

[15]  Narasimhan Sundararajan,et al.  Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Noureddine Zerhouni,et al.  SW-ELM: A summation wavelet extreme learning machine algorithm with a priori parameter initialization , 2014, Neurocomputing.

[17]  Changjiang Bu,et al.  Moore–Penrose inverse of tensors via Einstein product , 2016 .

[18]  Qing-Song Xu,et al.  libPLS: An integrated library for partial least squares regression and linear discriminant analysis , 2018 .

[19]  Renato Pajarola,et al.  On best rank one approximation of tensors , 2013, Numer. Linear Algebra Appl..

[20]  Debasisha Mishra,et al.  Further results on generalized inverses of tensors via the Einstein product , 2016, 1604.02675.

[21]  Na Li,et al.  Solving Multilinear Systems via Tensor Inversion , 2013, SIAM J. Matrix Anal. Appl..

[22]  Hongming Zhou,et al.  Optimization method based extreme learning machine for classification , 2010, Neurocomputing.

[23]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[24]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[25]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[26]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Qinghua Zheng,et al.  Regularized Extreme Learning Machine , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[28]  Han Liu,et al.  Provable sparse tensor decomposition , 2015, 1502.01425.

[29]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .

[30]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[31]  Shmuel Friedland,et al.  A New Approach to Generalized Singular Value Decomposition , 2005, SIAM J. Matrix Anal. Appl..