A feature transformation method based on multi objective particle swarm optimization for reducing support vector machine error

Discriminative methods are used for increasing pattern recognition and classification accuracy. These methods can be used as discriminative transformations applied to features or they can be used as discriminative learning algorithms for the classifiers. Most of discriminative feature transformation measures don't consider the classification method errors and information. In this paper, we propose a feature transformation method for support vector machine to consider both features discrimination and classification error. To this end, we use Multi-Objective Particle Swarm Optimization (Multi-PSO), where we consider two mentioned criteria as objectives in Multi-PSO fitness function. Experimental results on UCI dataset show that the proposed Multi-PSO based feature transformation method outperform other conventional methods of feature transformation when it is used as a preprocessing step for SVM.

[1]  Ching-Shih Tsou,et al.  Using Crowding Distance to Improve Multi-Objective PSO with Local Search , 2007 .

[2]  Seiichi Nakagawa,et al.  Power linear discriminant analysis , 2007, 2007 9th International Symposium on Signal Processing and Its Applications.

[3]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[4]  George Saon,et al.  Maximum likelihood discriminant feature spaces , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[5]  Anil K. Jain,et al.  Incremental nonlinear dimensionality reduction by manifold learning , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Maja Pohar Perme,et al.  Comparison of logistic regression and linear discriminant analysis , 2004, Advances in Methodology and Statistics.

[7]  C. Singh,et al.  Reserve-Constrained Multiarea Environmental/Economic Dispatch Using Enhanced Particle Swarm Optimization , 2006, 2006 IEEE Systems and Information Engineering Design Symposium.

[8]  Jieping Ye,et al.  Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems , 2005, J. Mach. Learn. Res..

[9]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[10]  S. Khudanpur,et al.  On projections of Gaussian distributions using maximum likelihood criteria , 2009, 2009 Information Theory and Applications Workshop.

[11]  Andreas G. Andreou,et al.  Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition , 1998, Speech Commun..

[12]  Kun-Huang Chen,et al.  An improved particle swarm optimization for feature selection , 2011, Intell. Data Anal..

[13]  Andreas G. Andreou,et al.  A semi-supervised version of heteroscedastic linear discriminant analysis , 2009, INTERSPEECH.

[14]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[15]  Susana Cecilia Esquivel,et al.  Multi-objective optimization with a Gaussian PSO algorithm , 2008 .

[16]  Mengjie Zhang,et al.  Multi-objective particle swarm optimisation (PSO) for feature selection , 2012, GECCO '12.

[17]  Carlos A. Coello Coello,et al.  Improving PSO-Based Multi-objective Optimization Using Crowding, Mutation and epsilon-Dominance , 2005, EMO.

[18]  Ahmad Akbari,et al.  Improving linear discriminant analysis with artificial immune system-based evolutionary algorithms , 2012, Inf. Sci..