Integration of weighted LS-SVM and manifold learning for fuzzy modeling

Abstract In this paper, a robust fuzzy modeling method is proposed for strongly nonlinear systems in the presence of noise and/or outliers. The proposed method integrates the advantages of the fuzzy structure, the manifold learning, and the weighted least squares support vector machine (LS-SVM). First, the Gustafson–Kessel clustering algorithm (GKCA) is applied to split the training data set into several subsets to determine the fuzzy rules and premise parameters. Then, a new objective function is constructed based on the fuzzy structure, the weighted LS-SVM, and the manifold regularization, which takes into account robustness and the intrinsic geometry of the data. A solving method is further developed, from which the fuzzy model is achieved and can effectively approximate a nonlinear system with various types of random noise. The proposed method is applied to an artificial case as well as a practical hydraulic actuator, demonstrating its effectiveness in modeling of a nonlinear system even under noise.

[1]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[2]  Bing Liu,et al.  Manifold regularized extreme learning machine , 2015, Neural Computing and Applications.

[3]  Lianfa Bai,et al.  Semi-supervised classification via discriminative sparse manifold regularization , 2016, Signal Process. Image Commun..

[4]  Deli Zhao,et al.  Linear local tangent space alignment and application to face recognition , 2007, Neurocomputing.

[5]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  Xinjiang Lu,et al.  Operation-Region-Decomposition-Based Singular Value Decomposition/Neural Network Modeling Method for Complex Hydraulic Press Machines , 2013 .

[7]  H. A. David.  Early sample measures of variability , 1998 .

[8]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[9]  Kazuo Tanaka,et al.  Fuzzy Control Systems Design and Analysis: A Linear Matrix Inequality Approach , 2008 .

[10]  Emil Levi,et al.  Identification of complex systems based on neural and Takagi-Sugeno fuzzy model , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Yao Sun,et al.  Towards designing risk-based safe Laplacian Regularized Least Squares , 2016, Expert Syst. Appl..

[12]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[13]  Chin-Wang Tao,et al.  Hybrid robust approach for TSK fuzzy modeling with outliers , 2009, Expert Syst. Appl..

[14]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[15]  Qiang Yang,et al.  Discriminatively regularized least-squares classification , 2009, Pattern Recognit..

[16]  Igor Skrjanc,et al.  Recursive fuzzy c-means clustering for recursive fuzzy identification of time-varying processes. , 2011, ISA transactions.

[17]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[18]  Kheireddine Chafaa,et al.  Fuzzy modelling using Kalman filter , 2007 .

[19]  Euntai Kim,et al.  A new approach to fuzzy modeling , 1997, IEEE Trans. Fuzzy Syst..

[20]  R. Kruse,et al.  An extension to possibilistic fuzzy cluster analysis , 2004, Fuzzy Sets Syst..

[21]  Zhihui Lai,et al.  Manifold discriminant regression learning for image classification , 2015, Neurocomputing.

[22]  Minghui Huang,et al.  A KL/Subspace Modeling Method for Integrated Design and Control in Curing Processes , 2014 .

[23]  John Yen,et al.  Improving the interpretability of TSK fuzzy models by combining global learning and local learning , 1998, IEEE Trans. Fuzzy Syst..

[24]  Chia-Feng Juang,et al.  A Fuzzy System Constructed by Rule Generation and Iterative Linear SVR for Antecedent and Consequent Parameter Optimization , 2012, IEEE Transactions on Fuzzy Systems.

[25]  Shifei Ding,et al.  An overview on twin support vector machines , 2012, Artificial Intelligence Review.

[26]  Nikhil R. Pal,et al.  Simultaneous Structure Identification and Fuzzy Rule Generation for Takagi–Sugeno Models , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Huajing Fang,et al.  A novel cost function based on decomposing least-square support vector machine for Takagi-Sugeno fuzzy system identification , 2014 .

[28]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yifei Wang,et al.  A normal least squares support vector machine (NLS-SVM) and its learning algorithm , 2009, Neurocomputing.

[30]  Hujun Bao,et al.  Laplacian Regularized Gaussian Mixture Model for Data Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[31]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[32]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[33]  Paulo J. G. Lisboa,et al.  Making nonlinear manifold learning models interpretable: The manifold grand tour , 2015, Expert Syst. Appl..

[34]  Chih-Wei Lee,et al.  Kernel local Fisher discriminant analysis based manifold-regularized SVM model for financial distress predictions , 2012, Expert Syst. Appl..

[35]  Magne Setnes,et al.  Rule-based modeling: precision and transparency , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[36]  Hui Xue,et al.  Semi-supervised classification learning by discrimination-aware manifold regularization , 2015, Neurocomputing.

[37]  Korris Fu-Lai Chung,et al.  Support vector machine with manifold regularization and partially labeling privacy protection , 2015, Inf. Sci..

[38]  XinJiang Lu,et al.  Two-Level Modeling Based Intelligent Integration Control for Time-Varying Forging Processes , 2015 .

[39]  Yong Peng,et al.  Discriminative manifold extreme learning machine and applications to image and EEG signal classification , 2016, Neurocomputing.

[40]  XinJiang Lu,et al.  Integrated Design and Control under Uncertainty: A Fuzzy Modeling Approach , 2010 .

[41]  Yuan-Hai Shao,et al.  Laplacian smooth twin support vector machine for semi-supervised classification , 2013, International Journal of Machine Learning and Cybernetics.