Robust Clustered Support Vector Machine With Applications to Modeling of Practical Processes

Real datasets are often distributed nonlinearly. Although many least squares support vector machine (LS-SVM) methods have successfully modeled this kind of data using a divide-and-conquer strategy, they are often ineffective when nonlinear data are subject to noise due to a lack of robustness within each sub-model. In this paper, a robust clustered LS-SVM is proposed to model this type of data. First, the clustering method is used to divide the sample data into several sub-datasets. A local robust LS-SVM model is then developed to capture the local dynamics of the corresponding sub-dataset and to be robust to noise. Subsequently, a global regularization is constructed to intelligently coordinate all local models. These new features ensure that the global model is smooth and continuous and has a good generalization and robustness. Through the use of both artificial and real cases, the effectiveness of the proposed robust clustered LS-SVM is demonstrated.

[1]  Lu You,et al.  A New Robust Least Squares Support Vector Machine for Regression with Outliers , 2011 .

[2]  Bin Fan,et al.  A novel LS-SVM control for unknown nonlinear systems with application to complex forging process , 2017 .

[3]  Bernhard Sick,et al.  The responsibility weighted Mahalanobis kernel for semi-supervised training of support vector machines for classification , 2015, Inf. Sci..

[4]  Johan A. K. Suykens,et al.  Efficient multiple scale kernel classifiers , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[5]  Pasi Fränti,et al.  A fast minimum spanning tree algorithm based on K-means , 2015, Inf. Sci..

[6]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[7]  Reshma Khemchandani,et al.  TWSVR: Regression via Twin Support Vector Machine , 2016, Neural Networks.

[8]  B. Muthén,et al.  Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study , 2007 .

[9]  Thanh-Nghi Do,et al.  Non-linear Classification of Massive Datasets with a Parallel Algorithm of Local Support Vector Machines , 2015, ICCSAMA.

[10]  Chuanfa Chen,et al.  A robust weighted least squares support vector regression based on least trimmed squares , 2015, Neurocomputing.

[11]  C. L. Philip Chen,et al.  Fuzzy density weight-based support vector regression for image denoising , 2016, Inf. Sci..

[12]  Minghui Huang,et al.  A Novel LS-SVM Modeling Method for a Hydraulic Press Forging Process With Multiple Localized Solutions , 2015, IEEE Transactions on Industrial Informatics.

[13]  Yong Shi,et al.  Twin support vector machine with Universum data , 2012, Neural Networks.

[14]  Chien-Liang Liu,et al.  An HMM-Based Algorithm for Content Ranking and Coherence-Feature Extraction , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Korris Fu-Lai Chung,et al.  Multi-view L2-SVM and its multi-view core vector machine , 2016, Neural Networks.

[16]  Chuanhou Gao,et al.  Design of a multiple kernel learning algorithm for LS-SVM by convex programming , 2011, Neural Networks.

[17]  Tian Sheng-feng.  A Simplification Algorithm to Support Vector Machines for Regression , 2002 .

[18]  Maurice K. Wong,et al.  Algorithm AS136: A k-means clustering algorithm. , 1979 .

[19]  Li Zhang,et al.  Fisher-regularized support vector machine , 2016, Inf. Sci..

[20]  Ramiro Saltos Atiencia,et al.  A Rough-Fuzzy approach for Support Vector Clustering , 2016, Inf. Sci..

[21]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[22]  Johan A. K. Suykens,et al.  Approximate Confidence and Prediction Intervals for Least Squares Support Vector Regression , 2011, IEEE Transactions on Neural Networks.

[23]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[24]  Sheng-De Wang,et al.  Training algorithms for fuzzy support vector machines with noisy data , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[25]  Han-Xiong Li,et al.  Probabilistic support vector machines for classification of noise affected data , 2013, Inf. Sci..

[26]  Shie-Jue Lee,et al.  A weighted LS-SVM based learning system for time series forecasting , 2015, Inf. Sci..

[27]  Minghui Huang,et al.  Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system , 2016, Neurocomputing.

[28]  Xinjiang Lu,et al.  Integration of weighted LS-SVM and manifold learning for fuzzy modeling , 2017, Neurocomputing.

[29]  R. Farebrother.  Linear least squares computations , 1988 .

[30]  Zne-Jung Lee,et al.  Hybrid robust support vector machines for regression with outliers , 2011, Appl. Soft Comput..

[31]  Xiaodong Liu,et al.  A parsimony fuzzy rule-based classifier using axiomatic fuzzy set theory and support vector machines , 2011, Inf. Sci..

[32]  Yaonan Wang,et al.  A Three-Domain Fuzzy Support Vector Regression for Image Denoising and Experimental Studies , 2014, IEEE Transactions on Cybernetics.

[33]  Senjian An,et al.  Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression , 2007, Pattern Recognit..

[34]  J. Suykens.  Nonlinear modelling and support vector machines , 2001, IMTC 2001. Proceedings of the 18th IEEE Instrumentation and Measurement Technology Conference. Rediscovering Measurement in the Age of Informatics (Cat. No.01CH 37188).

[35]  Albert Y. Zomaya,et al.  Tensor-Based Big Data Management Scheme for Dimensionality Reduction Problem in Smart Grid Systems: SDN Perspective , 2018, IEEE Transactions on Knowledge and Data Engineering.

[36]  Xiaowei Yang,et al.  A robust least squares support vector machine for regression and classification with noise , 2014, Neurocomputing.

[37]  Li Xie,et al.  Electricity price forecasting by clustering-LSSVM , 2007, 2007 International Power Engineering Conference (IPEC 2007).

[38]  Han-Xiong Li,et al.  Probabilistic Inference-Based Least Squares Support Vector Machine for Modeling Under Noisy Environment , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[39]  Han-Xiong Li,et al.  Spatio-Temporal Modeling of Nonlinear Distributed Parameter Systems: A Time/Space Separation Based Approach , 2011 .

[40]  Javier Montero,et al.  A Divide-and-Link algorithm for hierarchical clustering in networks , 2015, Inf. Sci..

[41]  Avory Bryant,et al.  RNN-DBSCAN: A Density-Based Clustering Algorithm Using Reverse Nearest Neighbor Density Estimates , 2018, IEEE Transactions on Knowledge and Data Engineering.

[42]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[43]  Xiaowei Yang,et al.  Robust least squares support vector machine based on recursive outlier elimination , 2010, Soft Comput..

[44]  Han-Xiong Li,et al.  Data-Driven Robust Design for a Curing Oven , 2014, IEEE Transactions on Components, Packaging and Manufacturing Technology.

[45]  Minghui Huang,et al.  Dempster-Shafer theory-based robust least squares support vector machine for stochastic modelling , 2016, Neurocomputing.

[46]  Lev V. Utkin,et al.  Binary classification SVM-based algorithms with interval-valued training data using triangular and Epanechnikov kernels , 2016, Neural Networks.

[47]  Yingjie Tian,et al.  A divide-and-combine method for large scale nonparallel support vector machines , 2016, Neural Networks.

[48]  George C. Runger,et al.  CRAFTER: A Tree-Ensemble Clustering Algorithm for Static Datasets with Mixed Attributes and High Dimensionality , 2018, IEEE Transactions on Knowledge and Data Engineering.

[49]  T. Lane,et al.  A Framework for Multiple Kernel Support Vector Regression and Its Applications to siRNA Efficacy Prediction , 2009, TCBB.

[50]  Jiawei Han,et al.  Clustered Support Vector Machines , 2013, AISTATS.

[51]  Thomas E Mallouk,et al.  Water electrolysis: Divide and conquer. , 2013, Nature chemistry.

[52]  Xiaowei Yang,et al.  A Kernel Fuzzy c-Means Clustering-Based Fuzzy Support Vector Machine Algorithm for Classification Problems With Outliers or Noises , 2011, IEEE Transactions on Fuzzy Systems.

[53]  Reshma Khemchandani,et al.  Robust least squares twin support vector machine for human activity recognition , 2016, Appl. Soft Comput..

[54]  Fu Li,et al.  A pruning method of refining recursive reduced least squares support vector regression , 2015, Inf. Sci..