Correntropy induced loss based sparse robust graph regularized extreme learning machine for cancer classification

Background As a machine learning method with high performance and excellent generalization ability, extreme learning machine (ELM) is gaining popularity in various studies. Various ELM-based methods for different fields have been proposed. However, the robustness to noise and outliers is always the main problem affecting the performance of ELM. Results In this paper, an integrated method named correntropy induced loss based sparse robust graph regularized extreme learning machine (CSRGELM) is proposed. The introduction of correntropy induced loss improves the robustness of ELM and weakens the negative effects of noise and outliers. By using the L 2,1 -norm to constrain the output weight matrix, we tend to obtain a sparse output weight matrix to construct a simpler single hidden layer feedforward neural network model. By introducing the graph regularization to preserve the local structural information of the data, the classification performance of the new method is further improved. Besides, we design an iterative optimization method based on the idea of half quadratic optimization to solve the non-convex problem of CSRGELM. Conclusions The classification results on the benchmark dataset show that CSRGELM can obtain better classification results compared with other methods. More importantly, we also apply the new method to the classification problems of cancer samples and get a good classification effect.

[1]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[2]  Jiguo Yu,et al.  Characteristic gene selection via L2,1-norm Sparse Principal Component Analysis , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[3]  Tieniu Tan,et al.  Half-Quadratic-Based Iterative Minimization for Robust Sparse Representation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[5]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[6]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[7]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[8]  Mikhail Belkin,et al.  Beyond the point cloud: from transductive to semi-supervised learning , 2005, ICML.

[9]  Na Li,et al.  Multi-label Text Categorization Using $$L_{21}$$-norm Minimization Extreme Learning Machine , 2017 .

[10]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Mikhail Belkin,et al.  Laplacian Support Vector Machines Trained in the Primal , 2009, J. Mach. Learn. Res..

[12]  Yves Goussard,et al.  On global and local convergence of half-quadratic algorithms , 2006, IEEE Transactions on Image Processing.

[13]  Rong Zhu,et al.  Co-differential Gene Selection and Clustering Based on Graph Regularized Multi-View NMF in Cancer Genomic Data , 2018, Genes.

[14]  Chun-Hou Zheng,et al.  Dual-network sparse graph regularized matrix factorization for predicting miRNA-disease associations. , 2019, Molecular omics.

[15]  Bao-Gang Hu,et al.  Robust C-Loss Kernel Classifiers , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Mirza Cilimkovic.  Neural Networks And Back Propagation Algorithm , 2010 .

[17]  Punyaphol Horata,et al.  Robust extreme learning machine , 2013, Neurocomputing.

[18]  Guang-Bin Huang,et al.  What are Extreme Learning Machines? Filling the Gap Between Frank Rosenblatt’s Dream and John von Neumann’s Puzzle , 2015, Cognitive Computation.

[19]  Qiang Liu,et al.  Random Fourier extreme learning machine with ℓ2, 1-norm regularization , 2016, Neurocomputing.

[20]  Fuzhen Zhuang,et al.  Clustering in extreme learning machine feature space , 2014, Neurocomputing.

[21]  Yi Lu,et al.  ELM-based gene expression classification with misclassification cost , 2013, Neural Computing and Applications.

[22]  Weifeng Liu,et al.  Correntropy: A Localized Similarity Measure , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[23]  Bo Liu,et al.  Image classification based on effective extreme learning machine , 2013, Neurocomputing.

[24]  Zhiping Lin,et al.  Extreme Learning Machine for Joint Embedding and Clustering , 2018, Neurocomputing.

[25]  Zhihong Man,et al.  A New Adaptive Backpropagation Algorithm Based on Lyapunov Stability Theory for Neural Networks , 2006, IEEE Transactions on Neural Networks.

[26]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[27]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[28]  Na Li,et al.  Multi-label text categorization using L21-norm minimization extreme learning machine , 2017, Neurocomputing.

[29]  Xianzhong Long,et al.  Discriminative graph regularized extreme learning machine and its application to face recognition , 2015, Neurocomputing.

[30]  Paul Honeine,et al.  Correntropy-based robust multilayer extreme learning machines , 2018, Pattern Recognition.

[31]  Vikas Sindhwani,et al.  An RKHS for multi-view learning and manifold co-regularization , 2008, ICML '08.

[32]  Gokhan Bilgin,et al.  MCK-ELM: multiple composite kernel extreme learning machine for hyperspectral images , 2019, Neural Computing and Applications.

[33]  Zhiping Lin,et al.  Extreme Learning Machine for Clustering , 2015 .

[34]  Jason Jianjun Gu,et al.  An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine , 2017, IEEE Transactions on Cybernetics.

[35]  Cheng Wu,et al.  Discriminative clustering via extreme learning machine , 2015, Neural Networks.

[36]  Jianzhong Wang,et al.  Adaptive multiple graph regularized semi-supervised extreme learning machine , 2018, Soft Comput..

[37]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[38]  Baojun Zhao,et al.  Blind Noisy Image Quality Assessment Using Sub-Band Kurtosis , 2020, IEEE Transactions on Cybernetics.

[39]  Zhe Yang,et al.  C-loss based extreme learning machine for estimating power of small-scale turbojet engine , 2019, Aerospace Science and Technology.

[40]  Juan Wang,et al.  Graph regularized robust non-negative matrix factorization for clustering and selecting differentially expressed genes , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[41]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[42]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[43]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[44]  Jin-Xing Liu,et al.  Hypergraph Regularized Discriminative Nonnegative Matrix Factorization on Sample Classification and Co-Differentially Expressed Gene Selection , 2019, Complex..

[45]  Zhuo Ren,et al.  Correntropy-based robust extreme learning machine for classification , 2018, Neurocomputing.

[46]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[47]  Yafei Song,et al.  $L_{21}$ -Norm Based Loss Function and Regularization Extreme Learning Machine , 2019, IEEE Access.

[48]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[49]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[50]  José Carlos Príncipe,et al.  The C-loss function for pattern classification , 2014, Pattern Recognit..