Simultaneous classification and feature selection via convex quadratic programming with application to HIV-associated neurocognitive disorder assessment

Support vector machines (SVMs), that utilize a mixture of the L1-norm and the L2-norm penalties, are capable of performing simultaneous classification and selection of highly correlated features. These SVMs, typically set up as convex programming problems, are re-formulated here as simple convex quadratic minimization problems over non-negativity constraints, giving rise to a new formulation - the pq-SVM method. Solutions to our re-formulation are obtained efficiently by an extremely simple algorithm. Computational results on a range of publicly available datasets indicate that these methods allow greater classification accuracy in addition to selecting groups of highly correlated features. These methods were also compared on a new dataset assessing HIV-associated neurocognitive disorder in a group of 97 HIV-infected individuals.

[1]  David R. Musicant,et al.  Lagrangian Support Vector Machines , 2001, J. Mach. Learn. Res..

[2]  Lucette A. Cysique,et al.  Neurodegeneration and Ageing in the HAART Era , 2009, Journal of Neuroimmune Pharmacology.

[3]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[4]  Glenn Fung,et al.  Data selection for support vector machine classifiers , 2000, KDD '00.

[5]  B. Navia,et al.  The AIDS dementia complex: I. Clinical features , 1986, Annals of neurology.

[6]  Suvrit Sra,et al.  Efficient Large Scale Linear Programming Support Vector Machines , 2006, ECML.

[7]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[8]  R. Price,et al.  The AIDS dementia complex. , 1988, The Journal of infectious diseases.

[9]  Xiaodong Lin,et al.  Gene expression Gene selection using support vector machines with non-convex penalty , 2005 .

[10]  H. Zou,et al.  The doubly regularized support vector machine , 2006 .

[11]  David R. Musicant,et al.  Active Support Vector Machine Classification , 2000, NIPS.

[12]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[13]  John T. Ormerod,et al.  Knowledge-based semidefinite linear programming classifiers , 2006, Optim. Methods Softw..

[14]  Gene H. Golub,et al.  Matrix computations , 1983 .

[15]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[16]  Li Wang,et al.  Hybrid huberized support vector machines for microarray classification , 2007, ICML '07.

[17]  Glenn Fung,et al.  A Feature Selection Newton Method for Support Vector Machine Classification , 2004, Comput. Optim. Appl..

[18]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[19]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[20]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[21]  David R. Musicant,et al.  Robust Linear and Support Vector Regression , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Glenn Fung,et al.  Finite Newton method for Lagrangian support vector machine classification , 2003, Neurocomputing.