Sparse support vector machines with L0 approximation for ultra-high dimensional omics data

Omics data usually have ultra-high dimension (p) and small sample size (n). Standard support vector machines (SVMs), which minimize the L2 norm for the primal variables, only lead to sparse solutions for the dual variables. L1 based SVMs, directly minimizing the L1 norm, have been used for feature selection with omics data. However, most current methods directly solve the primal formulations of the problem, which are not computationally scalable. The computational complexity increases with the number of features. In addition, L1 norm is known to be asymptotically biased and not consistent for feature selection. In this paper, we develop an efficient method for sparse support vector machines with L0 norm approximation. The proposed method approximates the L0 minimization through solving a series of L2 optimization problems, which can be formulated with dual variables. It finds the optimal solution for p primal variables through estimating n dual variables, which is more efficient as long as the sample size is small. L0 approximation leads to sparsity in both dual and primal variables, and can be used for both feature and sample selections. The proposed method identifies much less number of features and achieves similar performances in simulations. We apply the proposed method to feature selections with metagenomic sequencing and gene expression data. It can identify biologically important genes and taxa efficiently.

[1]  Martin Jaggi,et al.  An Equivalence between the Lasso and Support Vector Machines , 2013, ArXiv.

[2]  Amir Abbas Najafi,et al.  Support vector machine with Dirichlet feature mapping , 2018, Neural Networks.

[3]  Dechang Chen,et al.  Class Prediction and Feature Selection with Linear Optimization for Metagenomic Count Data , 2013, PloS one.

[4]  Bilwaj Gaonkar,et al.  Interpreting support vector machine models for multivariate group wise analysis in neuroimaging , 2015, Medical Image Anal..

[5]  Reshma Khemchandani,et al.  TWSVR: Regression via Twin Support Vector Machine , 2016, Neural Networks.

[6]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[7]  Farida Zehraoui,et al.  Towards a piRNA prediction using multiple kernel fusion and support vector machine , 2014, Bioinform..

[8]  Cheng Wang,et al.  A filter feature selection method based on the Maximal Information Coefficient and Gram-Schmidt Orthogonalization for biomedical data mining , 2017, Comput. Biol. Medicine.

[9]  Gang Li,et al.  Efficient Regularized Regression with L 0 Penalty for Variable Selection and Network Construction , 2016, Comput. Math. Methods Medicine.

[10]  Rob Knight,et al.  A Modular Organization of the Human Intestinal Mucosal Microbiota and Its Association with Inflammatory Bowel Disease , 2013, PloS one.

[11]  Makoto Fujii,et al.  Reduced Abundance of Butyrate-Producing Bacteria Species in the Fecal Microbial Community in Crohn's Disease , 2016, Digestion.

[12]  Zhao Wu,et al.  Feature selection method based on support vector machine and shape analysis for high-throughput medical data , 2017, Comput. Biol. Medicine.

[13]  Yangyang Xu,et al.  Proximal gradient method for huberized support vector machine , 2015, Pattern Analysis and Applications.

[14]  Frank R. Burden,et al.  Relevance Vector Machines: Sparse Classification Methods for QSAR , 2015, J. Chem. Inf. Model..

[15]  Xiao-Lei Xia,et al.  Training sparse least squares support vector machines by the QR decomposition , 2018, Neural Networks.

[16]  Aitor González,et al.  LedPred: an R/bioconductor package to predict regulatory sequences using support vector machines , 2016, Bioinform..

[17]  Kathrin Heikamp,et al.  Support vector machines for drug discovery , 2014, Expert opinion on drug discovery.

[18]  Bo Ding,et al.  MIEC-SVM: automated pipeline for protein peptide/ligand interaction prediction , 2016, Bioinform..

[19]  Huiqin Chen,et al.  Support Vector Machine Classification of Drunk Driving Behaviour , 2017, International journal of environmental research and public health.

[20]  Heikki Huttunen,et al.  Comparison of Feature Selection Techniques in Machine Learning for Anatomical Brain MRI in Dementia , 2016, Neuroinformatics.

[21]  Yixin Chen,et al.  A Reduction of the Elastic Net to Support Vector Machines with an Application to GPU Computing , 2014, AAAI.

[22]  Fariza Tahi,et al.  miRBoost: boosting support vector machines for microRNA precursor classification , 2015, RNA.

[23]  Homay Danaei Mehr,et al.  Diagnosis of Chronic Kidney Disease Based on Support Vector Machine by Feature Selection Methods , 2017, Journal of Medical Systems.

[24]  S. Jaffe.  Planning for US Precision Medicine Initiative underway , 2015, The Lancet.

[25]  R. Knight,et al.  Analysis of the Gut Microbiota in the Old Order Amish and Its Relation to the Metabolic Syndrome , 2012, PloS one.

[26]  Xiaohui Xie,et al.  Efficient variable selection in support vector machines via the alternating direction method of multipliers , 2011, AISTATS.

[27]  Ivor W. Tsang,et al.  Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets , 2010, ICML.

[28]  H. Sokol,et al.  Identification of an anti-inflammatory protein from Faecalibacterium prausnitzii, a commensal bacterium deficient in Crohn’s disease , 2015, Gut.

[29]  T. Vatanen,et al.  Dysbiosis, inflammation, and response to treatment: a longitudinal study of pediatric subjects with newly diagnosed inflammatory bowel disease , 2016, Genome Medicine.

[30]  Jiwen Lu,et al.  Simultaneous Local Binary Feature Learning and Encoding for Homogeneous and Heterogeneous Face Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Fernando De la Torre,et al.  Optimal feature selection for support vector machines , 2010, Pattern Recognit..

[32]  Euan A Ashley,et al.  The precision medicine initiative: a new national effort. , 2015, JAMA.

[33]  Syed Muhammad Anwar,et al.  Wrapper method for feature selection to classify cardiac arrhythmia , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[34]  Mohd Salman Leong,et al.  An improved wrapper-based feature selection method for machinery fault diagnosis , 2017, PloS one.

[35]  Bernhard Schölkopf,et al.  Use of the Zero-Norm with Linear Models and Kernel Methods , 2003, J. Mach. Learn. Res..

[36]  M. Peppelenbosch,et al.  Similar Depletion of Protective Faecalibacterium prausnitzii in Psoriasis and Inflammatory Bowel Disease, but not in Hidradenitis Suppurativa. , 2016, Journal of Crohn's & colitis.

[37]  L. Waldron,et al.  Creation of a Human Secretome: A Novel Composite Library of Human Secreted Proteins: Validation Using Ovarian Cancer Gene Expression Data and a Virtual Secretome Array , 2015, Clinical Cancer Research.

[38]  Yuanyuan Shen,et al.  Support Vector Machines with L1 penalty for detecting gene-gene interactions , 2012, Int. J. Data Min. Bioinform..