Emphatic Constraints Support Vector Machine

In this paper, a new support vector machine, ESVM, with more emphasis on constraints is presented. The constraints are fuzzy inequalities. With this scheme, two problems are solved: training samples with some degree of uncertainty, and samples with tolerance. Also, the fuzzy SVM (FSVM) model is modified with emphasis constraints. The new model is called fuzzy ESVM (FESVM), in this paper. With this scheme we will able to consider importance degree for samples both in the cost function and constraints, simultaneously. Necessary experiments are performed and the results show the superiority of the proposed methods. ESVM and fuzzy ESVM are strongly recommended to the researchers who work on data sets with noisy or low degree of certainty samples. Support vector machines (SVMs) are one of the most powerful methods that deliver state-of-the-art performance in real world pattern recognition and data mining applications. In the SVM solution, a pattern recognition problem is converted to a constraint quadratic programming. Support vector machine is originally introduced by Vapnik (1) within the area of statistical learning theory. It is bases on structural risk minimization (SRM) principle and finds a classifier with minimized Vapnik-Chervonenkis (VC) dimension. In pattern recognition, SVMs have been developed for data classification, feature reduction, and function estimation (2,3). The SVM classifier is very sensitive to outliers and noisy samples since the penalty term of SVM treats every data point equally in the training process. Increasing the values of slack variables, help in reducing the effect of noisy support vectors. Without introducing these slack variables, in the presence of noisy data, SVM may not be able to determine a hyperplane between two classes. The generalized formulation of the SVM model which can tolerate noisy data close to separating hyperplane is as follows:

[1]  Sheng-De Wang,et al.  Training algorithms for fuzzy support vector machines with noisy data , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[2]  Kemal Polat,et al.  A new medical decision making system: Least square support vector machine (LSSVM) with Fuzzy Weighting Pre-processing , 2007, Expert Syst. Appl..

[3]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[4]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[5]  Xindong Wu,et al.  Mining With Noise Knowledge: Error-Aware Data Mining , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[6]  Jian-Ping Sun,et al.  Support Vector Machine for Classification Based on Fuzzy Training Data , 2006 .

[7]  Reshma Khemchandani,et al.  Fuzzy linear proximal support vector machines for multi-category data classification , 2005, Neurocomputing.

[8]  Maoliu Lin,et al.  Research and application of noise suppression based on support vector machine , 2005, IEEE International Symposium on Communications and Information Technology, 2005. ISCIT 2005..

[9]  Chengdong Wu,et al.  A fuzzy support vector machine based on geometric model , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[10]  Massimiliano Pontil,et al.  Support Vector Machines: Theory and Applications , 2001, Machine Learning and Its Applications.

[11]  Jian Zhang,et al.  Nonlinear Noise Filtering with Support Vector Regression , 2006, Sixth International Conference on Intelligent Systems Design and Applications.

[12]  Dipti Prasad Mukherjee,et al.  Advances in Pattern Recognition , 2005, Pattern Recognit. Lett..

[13]  Noise Reduction of Chaotic Systems Based on Least Squares Support Vector Machines , 2006, 2006 International Conference on Communications, Circuits and Systems.

[14]  Kin Keung Lai,et al.  A new fuzzy support vector machine to evaluate credit risk , 2005, IEEE Transactions on Fuzzy Systems.

[15]  Hadi Sadoghi Yazdi,et al.  The probabilistic constraints in the support vector machine , 2007, Appl. Math. Comput..

[16]  Tai-Yue Wang,et al.  Fuzzy support vector machine for multi-class text categorization , 2007, Inf. Process. Manag..

[17]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[18]  Xindong Wu,et al.  Mining with Noise Knowledge: Error Aware Data Mining , 2007, 2007 International Conference on Computational Intelligence and Security (CIS 2007).

[19]  R. Casey,et al.  Advances in Pattern Recognition , 1971 .

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  Eric R. Ziegel,et al.  Mastering Data Mining , 2001, Technometrics.