Evolutionary Learning for Soft Margin Problems: A Case Study on Practical Problems with Kernels

This paper addresses two practical problems: the classification and prediction of properties for polymer and glass materials, as a case study of evolutionary learning for tackling soft margin problems. The presented classifier is modelled by support vectors as well as various kernel functions, with its hard restrictions relaxed by slack variables to be soft restrictions in order to achieve higher performance. We have compared evolutionary learning with traditional gradient methods on standard, dual and soft margin support vector machines, built by polynomial, Gaussian, and ANOVA kernels. Experimental results for data on 434 polymers and 1,441 glasses show that both gradient and evolutionary learning approaches have their advantages. We show that within this domain the chosen gradient methodology is beneficial for standard linear classification problems, whilst the evolutionary methodology is more effective in addressing highly non-linear and complex problems, such as the soft margin problem.

[1]  Vojislav Kecman,et al.  Kernel Based Algorithms for Mining Huge Data Sets: Supervised, Semi-supervised, and Unsupervised Learning , 2006, Studies in Computational Intelligence.

[2]  J. van Leeuwen,et al.  Evolutionary Multi-Criterion Optimization , 2003, Lecture Notes in Computer Science.

[3]  Wenqing Zhang,et al.  Data mining-aided materials discovery and optimization , 2017 .

[4]  Hui-Ling Huang,et al.  ESVM: Evolutionary support vector machine for automatic feature selection and classification of microarray data , 2007, Biosyst..

[5]  DebK.,et al.  A fast and elitist multiobjective genetic algorithm , 2002 .

[6]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[7]  Ingo Mierswa,et al.  Evolutionary learning with kernels: a generic solution for large margin problems , 2006, GECCO '06.

[8]  Nancy Arana-Daniel,et al.  Support Vector Machines Trained with Evolutionary Algorithms Employing Kernel Adatron for Large Scale Classification of Protein Structures , 2016, Evolutionary bioinformatics online.

[9]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[10]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[11]  Kalyanmoy Deb,et al.  A Niched-Penalty Approach for Constraint Handling in Genetic Algorithms , 1999, ICANNGA.

[12]  Chiho Kim,et al.  A polymer dataset for accelerated property prediction and design , 2016, Scientific Data.

[13]  Gary B. Lamont,et al.  Evolutionary algorithms for solving multi-objective problems, Second Edition , 2007, Genetic and evolutionary computation series.

[14]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[15]  Dedy Dwi Prastyo,et al.  Support Vector Machines with Evolutionary Feature Selection for Default Prediction , 2012 .

[16]  KalyanmoyDebandSamirAgrawal KanpurGeneticAlgorithmsLaboratory,et al.  A Niched-Penalty Approach for Constraint Handling in Genetic Algorithms , 2002 .

[17]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[18]  Dumitru Dumitrescu,et al.  Evolutionary Support Vector Regression Machines , 2006, 2006 Eighth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[19]  Qingfu Zhang,et al.  An Effective Ensemble Framework for Multiobjective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[20]  Yang Yu,et al.  Evolutionary Learning: Advances in Theories and Algorithms , 2019 .

[21]  Xin Yao,et al.  Evolving Neural Network Ensembles by Minimization of Mutual Information , 2004, Int. J. Hybrid Intell. Syst..

[22]  G. Wahba,et al.  Smoothing spline ANOVA for exponential families, with application to the Wisconsin Epidemiological Study of Diabetic Retinopathy : the 1994 Neyman Memorial Lecture , 1995 .

[23]  Wenjun Wang,et al.  Towards machine learning approaches for predicting the self-healing efficiency of materials , 2019, Computational Materials Science.

[24]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[25]  Madson Luiz Dantas Dias,et al.  Evolutionary support vector machines: A dual approach , 2016, CEC.

[26]  P. Massart,et al.  Statistical performance of support vector machines , 2008, 0804.0551.

[27]  Michael Rabadi,et al.  Kernel Methods for Machine Learning , 2015 .