Surrogate-Based Support Vector Machine Method

Surrogate-based method (SBM) is used to train a support vector machine (SVM) for discriminating between the elements of two classes of input points. The key idea to develop the algorithm is to replace the minimization of the cost function at each iteration by the minimization of a surrogate function, leading to a guaranteed decrease in the cost function. SBM simultaneously update all of points, which is very different from Platt’s sequential minimal optimization (SMO) and Joachims’ SVM light. The former handles one point at a time and the latter handles a small number of points at a time. In contrast to the sequential methods, SBM is easy to parallelize. The proposed algorithm has some favorable properties, including the monotonic decrease of the cost function, the self-constraining in the feasible region, and the absence of a predetermined step size and any additional parameter. This paper theoretically proves that the iteration sequence will converge to a sole global solution. Encouraging numerical results are presented on data sets, and SBM provides a performance comparable with that of other commonly used methods as concerns convergence speed and computational cost. Keywords-surrogate; Kuhn-Tucker (KT); conditions; regularization technique; sparsity/smoothness penalty