On the optimization of the support vector machine regression hyperparameters setting for gas sensors array applications

Abstract Support Vector Machine Regression (SVR) has been shown to be more accurate compared to other machine learning techniques that are commonly used for chemical sensors arrays applications. However, the performance of SVR depends strongly on the selection of its hyperparameters. Most of time, researchers in this field rely on trivial grid search methods to find suitable values of SVR hyperparameters by minimizing the cross-validation prediction error. This method is not a practical solution because of the large domain of possible parameter values, which is further exacerbated by the lack of prior knowledge on the data. In this article, we investigate the optimization of SVR hyperparameters by combining the SVR algorithm with a simple algorithm for SVR parameters selection. We begin by studying the influence of each hyperparameter on SVR performance. We then propose the Generalized Pattern Search algorithm (GPS) as a faster alternative to determine these hyperparameters. Finally, we demonstrate that the proposed GPS algorithm, with its simplicity and robustness, gives similar results compared to more complicated alternatives, such as Genetic Algorithms, Simulating Annealing, Bayesian Optimization or Particle Swarm Optimization.

[1]  Radu Ionescu,et al.  Exhaled breath analysis using electronic nose and gas chromatography–mass spectrometry for non-invasive diagnosis of chronic kidney disease, diabetes mellitus and healthy subjects , 2018 .

[2]  Yunqian Ma,et al.  Practical selection of SVM parameters and noise estimation for SVM regression , 2004, Neural Networks.

[3]  Martine Lumbreras,et al.  Optimization of an electronic nose for rapid quantitative recognition , 2014, 2014 International Conference on Control, Decision and Information Technologies (CoDIT).

[4]  Julian W. Gardner,et al.  Electronic noses: a review of signal processing techniques , 1999 .

[5]  C Cevoli,et al.  Classification of Pecorino cheeses using electronic nose combined with artificial neural network and comparison with GC-MS analysis of volatile compounds. , 2011, Food chemistry.

[6]  Kourosh Mozafari,et al.  Hepatitis disease diagnosis using a novel hybrid method based on support vector machine and simulated annealing (SVM-SA) , 2012, Comput. Methods Programs Biomed..

[7]  Ramón Huerta,et al.  Chemical Discrimination in Turbulent Gas Mixtures with MOX Sensors Validated by Gas Chromatography-Mass Spectrometry , 2014, Sensors.

[8]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[9]  Jun Wang,et al.  Comparison of ELM, RF, and SVM on E-nose and E-tongue to trace the quality status of mandarin (Citrus unshiu Marc.) , 2015 .

[10]  Taavi Saviauk,et al.  Electronic Nose in the Detection of Wound Infection Bacteria from Bacterial Cultures: A Proof-of-Principle Study , 2018, European Surgical Research.

[11]  Qi Ye,et al.  Classification of multiple indoor air contaminants by an electronic nose and a hybrid support vector machine , 2012 .

[12]  Min Zhang,et al.  Application of electronic tongue for fresh foods quality evaluation: A review , 2018 .

[13]  Jun Wang,et al.  The prediction of food additives in the fruit juice based on electronic nose with chemometrics. , 2017, Food chemistry.

[14]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[15]  Eric Michielssen,et al.  Genetic algorithm optimization applied to electromagnetics: a review , 1997 .

[16]  A. D. Wilson,et al.  Review of electronic-nose technologies and algorithms to detect hazardous chemicals in the environment , 2012 .

[17]  Cosimo Distante,et al.  Drift counteraction with multiple self-organising maps for an electronic nose , 2004 .

[18]  Łukasz Lentka,et al.  DETERMINATION OF GAS MIXTURE COMPONENTS USING FLUCTUATION ENHANCED SENSING AND THE LS-SVM REGRESSION ALGORITHM , 2015 .

[19]  Qinghai Bai,et al.  Analysis of Particle Swarm Optimization Algorithm , 2010, Comput. Inf. Sci..

[20]  A. Durán,et al.  Development of an electronic nose to characterize odours emitted from different stages in a wastewater treatment plant. , 2018, Water research.

[21]  Changying Li,et al.  Neural network and Bayesian network fusion models to fuse electronic nose and surface acoustic wave sensor data for apple defect detection , 2007 .

[22]  M. Siadat,et al.  A comparison between SVM and PLS for E-nose based gas concentration monitoring , 2018, 2018 IEEE International Conference on Industrial Technology (ICIT).

[23]  Yan Shi,et al.  Comparison of SVM, RF and ELM on an Electronic Nose for the Intelligent Evaluation of Paraffin Samples , 2018, Sensors.

[24]  M. Siadat,et al.  Reduction of drift impact in gas sensor response to improve quantitative odor analysis , 2017, 2017 IEEE International Conference on Industrial Technology (ICIT).

[25]  Silvia Coradeschi,et al.  Direct Identification of Bacteria in Blood Culture Samples Using an Electronic Nose , 2010, IEEE Transactions on Biomedical Engineering.

[26]  A. Gutierrez-Galvez,et al.  Signal and Data Processing for Machine Olfaction and Chemical Sensing: A Review , 2012, IEEE Sensors Journal.

[27]  Mahdi Ghasemi-Varnamkhasti,et al.  Performance Comparison of Fuzzy ARTMAP and LDA in Qualitative Classification of Iranian Rosa damascena Essential Oils by an Electronic Nose , 2016, Sensors.

[28]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[29]  Alphus D. Wilson,et al.  Applications and Advances in Electronic-Nose Technologies , 2009, Sensors.

[30]  Sheng-wei Fei,et al.  Fault diagnosis of power transformer based on support vector machine with genetic algorithm , 2009, Expert Syst. Appl..

[31]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[32]  Qiang Cai,et al.  Rapid Classification of Hairtail Fish and Pork Freshness Using an Electronic Nose Based on the PCA Method , 2011, Sensors.