Training of support vector machine with the use of multivariate normalization

Graphical abstractDisplay Omitted HighlightsWe analyze SVM (support vector machines) techniques.We propose a multivariable normalization of the inputs both during the training and classification processes.The multivariable normalization applied to a real SVM is equivalent to the use of a SVM that uses the Mahalanobis distance measure.The study confirms the improvement achieved in the classification processes. SVM (support vector machines) techniques have recently arrived to complete the wide range of classification methods for complex systems. These classification systems offer similar performances to other classifiers (such as the neuronal networks or classic statistical classifiers) and they are becoming a valuable tool in industry for the resolution of real problems. One of the fundamental elements of this type of classifier is the metric used for determining the distance between samples of the population to be classified. Although the Euclidean distance measure is the most natural metric for solving problems, it presents certain disadvantages when trying to develop classification systems that can be adapted as the characteristics of the sample space change. Our study proposes a means of avoiding this problem using the multivariate normalization of the inputs (both during the training and classification processes). Using experimental results produced from a significant number of populations, the study confirms the improvement achieved in the classification processes. Lastly, the study demonstrates that the multivariate normalization applied to a real SVM is equivalent to the use of a SVM that uses the Mahalanobis distance measure, for non-normalized data.

[1]  Shigeo Abe,et al.  Support Vector Machines for Pattern Classification (Advances in Pattern Recognition) , 2005 .

[2]  Davide Anguita,et al.  Neural network learning for analog VLSI implementations of support vector machines: a survey , 2003, Neurocomputing.

[3]  Daehyon Kim,et al.  Prediction performance of support vector machines on input vector normalization methods , 2004, Int. J. Comput. Math..

[4]  José Antonio Torres Arriaza,et al.  Multilevel RBF to resolve classification problems with large training sets: new pseudo-exact procedure , 2014, Soft Comput..

[5]  Seung-Jae Lee,et al.  Input Vector Normalization Methods in Support Vector Machines for Automatic Incident Detection , 2007 .

[6]  Daniel Peña Sánchez de Rivera Análisis de datos multivariantes , 2002 .

[7]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[8]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[9]  Beiji Zou,et al.  Optimal Approach for Texture Analysis and Classification based on Wavelet Transform and Neural Network , 2011, J. Inf. Hiding Multim. Signal Process..

[10]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  Dong Xu,et al.  Twin Mahalanobis distance-based support vector machines for pattern recognition , 2012, Inf. Sci..

[13]  Zhenyu Liu,et al.  A method of SVM with Normalization in Intrusion Detection , 2011 .

[14]  Chuan-Yu Chang,et al.  Applications of Block Linear Discriminant Analysis for Face Recognition , 2011, J. Inf. Hiding Multim. Signal Process..

[15]  Shigeo Abe Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[16]  Francisco Arteaga,et al.  Prediccin de hipertensin arterial usando mquinas de vectores de soporte , 2006 .

[17]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[18]  Alexander J. Smola,et al.  Classification in a normalized feature space using support vector machines , 2003, IEEE Trans. Neural Networks.

[19]  Elzbieta Pekalska,et al.  Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Kate Smith-Miles,et al.  Improved Support Vector Machine Generalization Using Normalized Input Space , 2006, Australian Conference on Artificial Intelligence.

[21]  Hsiang-Cheh Huang,et al.  A refactoring method for cache-efficient swarm intelligence algorithms , 2012, Inf. Sci..

[22]  Silvio Borer,et al.  Normalization in Support Vector Machines , 2001, DAGM-Symposium.