Feature selection using binary particle swarm optimization and support vector machines for medical diagnosis

Abstract In this article, we propose a feature selection strategy using a binary particle swarm optimization algorithm for the diagnosis of different medical diseases. The support vector machines were used for the fitness function of the binary particle swarm optimization. We evaluated our proposed method on four databases from the machine learning repository, including the single proton emission computed tomography heart database, the Wisconsin breast cancer data set, the Pima Indians diabetes database, and the Dermatology data set. The results indicate that, with selected less number of features, we obtained a higher accuracy in diagnosing heart, cancer, diabetes, and erythematosquamous diseases. The results were compared with the traditional feature selection methods, namely, the F-score and the information gain, and a superior accuracy was obtained with our method. Compared to the genetic algorithm for feature selection, the results of the proposed method show a higher accuracy in all of the data, except in one. In addition, in comparison with other methods that used the same data, our approach has a higher performance using less number of features.

[1]  Chang Jui-Fang A PERFORMANCE COMPARISON BETWEEN GENETIC ALGORITHMS AND PARTICLE SWARM OPTIMIZATION APPLIED IN CONSTRUCTING EQUITY PORTFOLIOS , 2009 .

[2]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[3]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[4]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[5]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[6]  George Forman,et al.  An Extensive Empirical Study of Feature Selection Metrics for Text Classification , 2003, J. Mach. Learn. Res..

[7]  Hussein A. Abbass,et al.  An evolutionary artificial neural networks approach for breast cancer diagnosis , 2002, Artif. Intell. Medicine.

[8]  T. Yıldırım,et al.  MEDICAL DIAGNOSIS ON PIMA INDIAN DIABETES USING GENERAL REGRESSION NEURAL NETWORKS , 2003 .

[9]  Hussein A. Abbass,et al.  C-Net: A Method for Generating Non-deterministic and Dynamic Multivariate Decision Trees , 2001, Knowledge and Information Systems.

[10]  D.H. Werner,et al.  Particle swarm optimization versus genetic algorithms for phased array synthesis , 2004, IEEE Transactions on Antennas and Propagation.

[11]  Lukasz Kurgan,et al.  Ensemble of classifiers to improve accuracy of the CLIP4 machine-learning algorithm , 2002, SPIE Defense + Commercial Sensing.

[12]  Li-Yeh Chuang,et al.  Feature Selection using PSO-SVM , 2007, IMECS.

[13]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Kenneth DeJong Evolutionary computation: a unified approach , 2007, GECCO.

[15]  Chu Kiong Loo,et al.  A Global k-means Approach for Autonomous Cluster Initialization of Probabilistic Neural Network , 2008, Informatica.

[16]  Smita Pathak Optimal Power Flow by Particle Swarm Optimization for Reactive Loss Minimization , 2012 .

[17]  Zexuan Zhu,et al.  Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[19]  Lukasz A. Kurgan,et al.  Knowledge discovery approach to automated cardiac SPECT diagnosis , 2001, Artif. Intell. Medicine.

[20]  Fernando De la Torre,et al.  Optimal feature selection for support vector machines , 2010, Pattern Recognit..

[21]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[22]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[23]  M.M.B.R. Vellasco,et al.  Inverted hierarchical neuro-fuzzy BSP system: a novel neuro-fuzzy model for pattern classification and rule extraction in databases , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[24]  D B Fogel,et al.  Evolving neural networks for detecting breast cancer. , 1995, Cancer letters.

[25]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  H.E. Osman,et al.  Online incremental random forests , 2007, 2007 International Conference on Machine Vision.

[27]  Krzysztof J. Cios,et al.  CLIP3: Cover learning using integer programming , 1997 .

[28]  Ludmil Mikhailov,et al.  Evolving fuzzy medical diagnosis of Pima Indians diabetes and of dermatological diseases , 2010, Artif. Intell. Medicine.

[29]  Enrique Alba,et al.  Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[30]  He Zhen-ya,et al.  A multiple objective optimization based echo state network tree and application to intrusion detection , 2005, Proceedings of 2005 IEEE International Workshop on VLSI Design and Video Technology, 2005..

[31]  Pasi Luukka,et al.  Similarity classifier with generalized mean applied to medical data , 2006, Comput. Biol. Medicine.

[32]  Kemal Polat,et al.  The Medical Applications of Attribute Weighted Artificial Immune System (AWAIS): Diagnosis of Heart and Diabetes Diseases , 2005, ICARIS.

[33]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[34]  Chih-Jen Lin,et al.  Combining SVMs with Various Feature Selection Strategies , 2006, Feature Extraction.

[35]  Chu Kiong Loo,et al.  Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP , 2005, IEEE Transactions on Knowledge and Data Engineering.

[36]  H. Altay Güvenir,et al.  Learning differential diagnosis of erythemato-squamous diseases using voting feature intervals , 1998, Artif. Intell. Medicine.

[37]  Dimitrios Gunopulos,et al.  Incremental support vector machine construction , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[38]  Qiangfu Zhao,et al.  Evolving NNTrees More Efficiently , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[39]  Jin Young Choi,et al.  Novel Incremental Principal Component Analysis with Improved Performance , 2008, SSPR/SPR.

[40]  John Platt,et al.  Large Margin DAG's for Multiclass Classification , 1999 .

[41]  O. Weck,et al.  A COMPARISON OF PARTICLE SWARM OPTIMIZATION AND THE GENETIC ALGORITHM , 2005 .

[42]  O. Mangasarian,et al.  Pattern Recognition Via Linear Programming: Theory and Application to Medical Diagnosis , 1989 .

[43]  Kemal Polat,et al.  A cascade learning system for classification of diabetes disease: Generalized Discriminant Analysis and Least Square Support Vector Machine , 2008, Expert Syst. Appl..

[44]  Chih-Jen Lin,et al.  A Simple Decomposition Method for Support Vector Machines , 2002, Machine Learning.

[45]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .