Towards Optimal Descriptor Subset Selection with Support Vector Machines in Classification and Regression

In this paper we present a novel method for selecting descriptor subsets by means of Support Vector Machines in classification and regression - the Incremental Regularized Risk Minimization (IRRM) algorithm. In contrast to many other wrapper methods it is fully deterministic and computationally efficient. We compare our method to existing algorithms and present results on a Human Intestinal Absorption (HIA) classification data set and the Huuskonen regression data set for aqueous solubility.

[1]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[2]  Stuart J. Russell,et al.  NP-Completeness of Searches for Smallest Possible Feature Sets , 1994 .

[3]  N. Nikolova,et al.  International Union of Pure and Applied Chemistry, LUMO energy ± The Lowest Unoccupied Molecular Orbital (LUMO) , 2022 .

[4]  B. Müller,et al.  Drug Liposome Partitioning as a Tool for the Prediction of Human Passive Intestinal Absorption , 1999, Pharmaceutical Research.

[5]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[6]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[7]  A. Zell,et al.  Feature subset selection for support vector machines by incremental regularized risk minimization , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[8]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[9]  Bernhard Schölkopf,et al.  Feature selection for support vector machines by means of genetic algorithm , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[10]  Jarmo Huuskonen,et al.  Estimation of Aqueous Solubility for a Diverse Set of Organic Compounds Based on Molecular Topology , 2000, J. Chem. Inf. Comput. Sci..

[11]  Haleh Vafaie,et al.  Evolutionary Feature Space Transformation , 1998 .

[12]  A. S. Weigend,et al.  Selecting Input Variables Using Mutual Information and Nonparemetric Density Estimation , 1994 .

[13]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[14]  Kristina Luthman,et al.  Polar Molecular Surface Properties Predict the Intestinal Absorption of Drugs in Humans , 1997, Pharmaceutical Research.

[15]  Shiyin Yee,et al.  In Vitro Permeability Across Caco-2 Cells (Colonic) Can Predict In Vivo (Small Intestinal) Absorption in Man—Fact or Myth , 1997, Pharmaceutical Research.

[16]  Jens Sadowski,et al.  Comparison of Support Vector Machine and Artificial Neural Network Systems for Drug/Nondrug Classification , 2003, J. Chem. Inf. Comput. Sci..

[17]  Andreas Zell,et al.  Feature Selection for Descriptor Based Classification Models. 1. Theory and GA-SEC Algorithm , 2004, J. Chem. Inf. Model..

[18]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[19]  M. Yazdanian,et al.  Correlating Partitioning and Caco-2 Cell Permeability of Structurally Diverse Small Molecular Weight Compounds , 1998, Pharmaceutical Research.

[20]  Bernhard Schölkopf,et al.  Use of the Zero-Norm with Linear Models and Kernel Methods , 2003, J. Mach. Learn. Res..

[21]  Ron Kohavi,et al.  Irrelevant Features and the Subset Selection Problem , 1994, ICML.

[22]  Bernhard Schölkopf,et al.  Extracting Support Data for a Given Task , 1995, KDD.

[23]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[24]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[25]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[26]  Alain Rakotomamonjy,et al.  Variable Selection Using SVM-based Criteria , 2003, J. Mach. Learn. Res..

[27]  Andreas Zell,et al.  Feature Selection for Descriptor Based Classification Models. 2. Human Intestinal Absorption (HIA) , 2004, J. Chem. Inf. Model..

[28]  Fernando Pérez-Cruz,et al.  Feature Selection via Genetic Optimization , 2002, ICANN.

[29]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[30]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[31]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[32]  Peter C. Jurs,et al.  Prediction of Human Intestinal Absorption of Drug Compounds from Molecular Structure , 1998, J. Chem. Inf. Comput. Sci..

[33]  K Gubernator,et al.  Physicochemical high throughput screening: parallel artificial membrane permeation assay in the description of passive absorption processes. , 1998, Journal of medicinal chemistry.

[34]  Andreas Zell,et al.  Prediction of Aqueous Solubility and Partition Coefficient Optimized by a Genetic Algorithm Based Descriptor Selection Method , 2003, J. Chem. Inf. Comput. Sci..