Novel Frank-Wolfe Methods for SVM Learning

SVM classifiers are very effective mining tools, but scalability for massive datasets is still an open issue. The Frank-Wolfe (FW) method deals with large-scale instances of the general problem of maximizing a concave function on the unit simplex, and can be specialized to SVM training to obtain algorithms with remarkable theoretical properties and competitive performance in practice. We present and analyze a variant of the FW method designed to obtain improved performance on large-scale SVM problems. The algorithm is based on a new way to perform away steps, a well-known strategy employed to accelerate the convergence of the basic FW method. We demonstrate that the method matches the guarantees in terms of convergence rate and number of iterations obtained by using classic away steps. In particular, the method enjoys a linear rate of convergence. On the practical side, we provide experiments on several classification datasets, and evaluate the results using statistical tests. Experiments show that our method is faster than the FW method and works well even in the cases in which classic away steps slow down the algorithm. Furthermore, these improvements are obtained without sacrificing the predictive accuracy of the obtained SVM model.