Robust Broad Learning System for Uncertain Data Modeling

Broad Learning System (BLS) has achieved good performance in classification and regression problems, and the computational efficiency is especially outstanding. However, there exists various outliers or noise in the sampling data, which puts a robust requirement on the algorithms. Standard BLS is sensitive to the contaminated data because of its composition structure. In this paper, we propose a robust version of BLS called RBLS to improve its generalization on contaminated data modeling. In RBLS, the ℓ2-norm based cost function will be replaced by ℓ1-norm style cost function. The Augmented Lagrange Multiplier (ALM) method is applied to optimize the new model iteratively. The experiments on function approximation and real-world regression demonstrated that the RBLS method has better modeling performance for sampling data with outliers or noise.

[1]  C. L. Philip Chen,et al.  A Fuzzy Restricted Boltzmann Machine: Novel Learning Algorithms Based on the Crisp Possibilistic Mean Value of Fuzzy Numbers , 2018, IEEE Transactions on Fuzzy Systems.

[2]  C. L. Philip Chen,et al.  A rapid supervised learning neural network for function interpolation and approximation , 1996, IEEE Trans. Neural Networks.

[3]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[4]  Ivan Tyukin,et al.  Approximation with random bases: Pro et Contra , 2015, Inf. Sci..

[5]  Jiye Liang,et al.  A simple and effective outlier detection algorithm for categorical data , 2014, Int. J. Mach. Learn. Cybern..

[6]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[7]  Ling Shao,et al.  Discriminative Elastic-Net Regularized Linear Regression , 2017, IEEE Transactions on Image Processing.

[8]  H. Jin Kim,et al.  Target Localization Using Ensemble Support Vector Regression in Wireless Sensor Networks , 2010, 2010 IEEE Wireless Communication and Networking Conference.

[9]  Yong Dou,et al.  Robust regularized extreme learning machine for regression using iteratively reweighted least squares , 2017, Neurocomputing.

[10]  Minxia Luo,et al.  Outlier-robust extreme learning machine for regression problems , 2015, Neurocomputing.

[11]  C. L. Philip Chen,et al.  An integration of neural network and rule-based systems for design and planning of mechanical assemblies , 1993, IEEE Trans. Syst. Man Cybern..

[12]  Xuelong Li,et al.  Regularized Label Relaxation Linear Regression , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Minghui Huang,et al.  Robust Least-Squares Support Vector Machine With Minimization of Mean and Variance of Modeling Error , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Arash Miranian,et al.  Developing a Local Least-Squares Support Vector Machines-Based Neuro-Fuzzy Model for Nonlinear and Chaotic Time Series Prediction , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[15]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[16]  Anders P. Eriksson,et al.  Efficient computation of robust low-rank matrix approximations in the presence of missing data using the L1 norm , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Fei Cheng,et al.  A Bayesian approach to support vector machines for the binary classification , 2008, Neurocomputing.

[18]  C. L. Philip Chen,et al.  Robust Broad Learning System for Uncertain Data Modeling , 2018, SMC.

[19]  Yuan Yan Tang,et al.  Robust Face Recognition via Minimum Error Entropy-Based Atomic Representation , 2015, IEEE Transactions on Image Processing.

[20]  Aleksandr Ometov,et al.  Visualizing Big Data with augmented and virtual reality: challenges and research agenda , 2015, Journal of Big Data.

[21]  C. L. Philip Chen,et al.  Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  J. M. Aldaz,et al.  Advances in operator cauchy-schwarz inequalities and their reverses , 2015 .

[23]  S. Van Huffel,et al.  Partial singular value decomposition algorithm ( Algorithm 32 ) , 1990 .

[24]  C. L. Philip Chen,et al.  Discriminative graph regularized broad learning system for image recognition , 2018, Science China Information Sciences.

[25]  Shuang Feng,et al.  Fuzzy Broad Learning System: A Novel Neuro-Fuzzy Model for Regression and Classification , 2020, IEEE Transactions on Cybernetics.

[26]  Shuo Yang,et al.  An improved Id3 algorithm for medical data classification , 2017, Comput. Electr. Eng..

[27]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[28]  Jian Yang,et al.  Regularized Robust Coding for Face Recognition , 2012, IEEE Transactions on Image Processing.

[29]  Fabio Roli,et al.  Multiple classifier systems for robust classifier design in adversarial environments , 2010, Int. J. Mach. Learn. Cybern..

[30]  Yun Zhang,et al.  Saturated Nussbaum Function Based Approach for Robotic Systems With Unknown Actuator Dynamics , 2016, IEEE Transactions on Cybernetics.

[31]  Ming Li,et al.  Robust stochastic configuration networks with kernel density estimation for uncertain data regression , 2017, Inf. Sci..

[32]  Xiong Luo,et al.  Regression and classification using extreme learning machine based on L1-norm and L2-norm , 2016, Neurocomputing.

[33]  C. L. Philip Chen,et al.  Metaface block sparse Bayesian learning for face recognition , 2017, 2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC).

[34]  Dianhui Wang,et al.  A probabilistic learning algorithm for robust modeling using neural networks with random weights , 2015, Inf. Sci..

[35]  Shun-Feng Su,et al.  The annealing robust backpropagation (ARBP) learning algorithm , 2000, IEEE Trans. Neural Networks Learn. Syst..

[36]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[37]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[38]  C. L. Philip Chen,et al.  Regularization parameter estimation for feedforward neural networks , 2003 .

[39]  Lei Zhang,et al.  A Probabilistic Collaborative Representation Based Approach for Pattern Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Haixian Wang,et al.  Block principal component analysis with L1-norm for image analysis , 2012, Pattern Recognit. Lett..

[41]  Yuan Yan Tang,et al.  Weighted Joint Sparse Representation for Removing Mixed Noise in Image , 2017, IEEE Transactions on Cybernetics.

[42]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[43]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[44]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[45]  C. L. Philip Chen,et al.  Universal Approximation Capability of Broad Learning System and Its Structural Variations , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Qinghua Zheng,et al.  Regularized Extreme Learning Machine , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[47]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[48]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[49]  C. L. Philip Chen,et al.  Data-intensive applications, challenges, techniques and technologies: A survey on Big Data , 2014, Inf. Sci..