Optimal Locality Regularized Least Squares Support Vector Machine via Alternating Optimization

The least squares support vector machine (LSSVM), like standard support vector machine (SVM) which is based on structural risk minimization, can be obtained by solving a simpler optimization problem than that in SVM. However, local structure information of data samples, especially intrinsic manifold structure, is not taken full consideration in LSSVM. To address this problem and inspired by manifold learning technique, we propose a novel iterative least squares classifier, coined optimal locality preserving least squares support vector machine (OLP-LSSVM). The idea is to combine structural risk minimization and locality preserving criterion in a unified framework to take advantage of the manifold structure of data samples to enhance LSSVM. Furthermore, inspired by the recent development of simultaneous optimization technique, adjacent graph of locality preserving criterion is optimized simultaneously to give rise to improved discriminative performance. The resulting model can be solved by alternating optimization method. The experimental results on several publicly available benchmark data sets show the feasibility and effectiveness of the proposed method.