Support vector regression in sum space for multivariate calibration

Abstract In this paper, a support vector regression algorithm in the sum of reproducing kernel Hilbert spaces (SVRSS) is proposed for multivariate calibration. In SVRSS, the target regression function is represented as the sum of several single kernel decision functions, where each single kernel function with specific scale can approximate certain component of the target function. For sum spaces with two Gaussian kernels, the proposed method is compared, in terms of RMSEP, to traditional chemometric PLS calibration methods and recent promising SVR, GPR and ELM methods on a simulated data set and four real spectroscopic data sets. Experimental results demonstrate that SVR methods outperform PLS methods for spectroscopy regression problems. Moreover, SVRSS method with multi-scale kernels improves the single kernel SVR method and shows superiority over GPR and ELM methods.

[1]  L. Buydens,et al.  Comparing support vector machines to PLS for spectral regression applications , 2004 .

[2]  F Despagne,et al.  Neural networks in multivariate calibration. , 1998, The Analyst.

[3]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Søren Balling Engelsen,et al.  Rapid Spectroscopic Analysis of Marzipan—Comparative Instrumentation , 2004 .

[5]  D. Massart,et al.  Elimination of uninformative variables for multivariate calibration. , 1996, Analytical chemistry.

[6]  Ulf G. Indahl,et al.  A twist to partial least squares regression , 2005 .

[7]  Roberto Kawakami Harrop Galvão,et al.  A method for calibration and validation subset partitioning. , 2005, Talanta.

[8]  M. Dyrby,et al.  Chemometric Quantitation of the Active Substance (Containing C≡N) in a Pharmaceutical Tablet Using Near-Infrared (NIR) Transmittance and NIR FT-Raman Spectra , 2002 .

[9]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[10]  Jiangtao Peng,et al.  Multivariate Calibration with Support Vector Regression Based on Random Projection , 2012, Int. J. Wavelets Multiresolution Inf. Process..

[11]  E. Martin,et al.  Gaussian process regression for multivariate spectroscopic calibration , 2007 .

[12]  Jiaxin Wang,et al.  Non-flat function estimation with a multi-scale support vector regression , 2006, Neurocomputing.

[13]  R. Leardi,et al.  Genetic algorithms applied to feature selection in PLS regression: how and when to use them , 1998 .

[14]  I. Talavera,et al.  Support vector regression for functional data in multivariate calibration problems. , 2009, Analytica chimica acta.

[15]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[16]  S. Wold,et al.  The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses , 1984 .

[17]  Lu Liu,et al.  Least Square Regularized Regression in Sum Space , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[19]  Jean-Pierre Gauchi,et al.  Selecting both latent and explanatory variables in the PLS1 regression model , 2003 .

[20]  L. Brás,et al.  A bootstrap‐based strategy for spectral interval selection in PLS regression , 2008 .

[21]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[22]  Yuanyan Tang,et al.  Combination of activation functions in extreme learning machines for multivariate calibration , 2013 .

[23]  H. Wold.  Soft Modelling by Latent Variables: The Non-Linear Iterative Partial Least Squares (NIPALS) Approach , 1975, Journal of Applied Probability.

[24]  E. V. Thomas,et al.  Partial least-squares methods for spectral analyses. 1. Relation to other quantitative calibration methods and the extraction of qualitative information , 1988 .