Support Vector Optimization through Hybrids: Heuristics and Math Approach

This paper presents a strategy to optimize the learning phase of the Support Vector Machines algorithm (SVM). The SVM algorithm is widely used in solving different tasks like classification, regression, density estimation and clustering problems. However, the algorithm presents important disadvantages when learning large scale problems. Training a SVM involves finding the solution of a quadratic optimization problem (QP), which is very resource consuming. What is more, during the learning step, the best working set must be selected, which is a hard to perform task. In this work, we combine a heuristic approach, which selects the best working set data, with a projected conjugate gradient method, which is a fast and easy to implement algorithm that solves the quadratic programming problem involved in the SVM algorithm. We compare the performances of the optimization strategies using some well-known benchmark databases.