LP-SVR Model Selection Using an Inexact Globalized Quasi-Newton Strategy

In this paper we study the problem of model selection for a linear programming-based support vector machine for regression. We propose generalized method that is based on a quasi-Newton method that uses a globalization strategy and an inexact computation of first order information. We explore the case of two-class, multi-class, and regression problems. Simulation results among standard datasets suggest that the algorithm achieves insignificant variability when measuring residual statistical properties.

[1]  S. Sathiya Keerthi,et al.  Evaluation of simple performance measures for tuning SVM hyperparameters , 2003, Neurocomputing.

[2]  HighWire Press.  Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[3]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[4]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[5]  Zheng Pi-e.  Method for Selecting Parameters of Least Squares Support Vector Machines Based on GA and Bootstrap , 2008 .

[6]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[7]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[8]  A. J. Barret,et al.  Methods of Mathematical Physics, Volume I . R. Courant and D. Hilbert. Interscience Publishers Inc., New York. 550 pp. Index. 75s. net. , 1954, The Journal of the Royal Aeronautical Society.

[9]  Zhao Lu,et al.  Linear programming support vector regression with wavelet kernel: A new approach to nonlinear dynamical systems identification , 2009, Math. Comput. Simul..

[10]  Jose Gerardo Rosiles,et al.  Algorithms for training large-scale linear programming support vector regression and classification , 2011 .

[11]  Massimiliano Pontil,et al.  Support Vector Machines: Theory and Applications , 2001, Machine Learning and Its Applications.

[12]  Shigeo Abe,et al.  Decomposition techniques for training linear programming support vector machines , 2009, Neurocomputing.

[13]  Davide Anguita,et al.  Theoretical and Practical Model Selection Methods for Support Vector Classifiers , 2004 .

[14]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[15]  Magnus R. Hestenes,et al.  Pseudoinversus and conjugate gradients , 1975, CACM.

[16]  Gavin C. Cawley,et al.  Leave-One-Out Cross-Validation Based Model Selection Criteria for Weighted LS-SVMs , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[17]  Miguel Argáez,et al.  A new infeasible interior-point algorithm for linear programming , 2003, TAPIA '03.

[18]  Li Zhang,et al.  On the sparseness of 1-norm support vector machines , 2010, Neural Networks.

[19]  Davide Anguita,et al.  Hyperparameter design criteria for support vector classifiers , 2003, Neurocomputing.

[20]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .