Constraint Projections for Discriminative Support Vector Machines

Abstract— The objection of the standard support vector machine (SVM) is to achieve the maximum margin and control the generalization capability of SVM classifier. It only considers the punishment constraints, but does not introduce the within-class and between-class constraint projections. A novel robust discriminative support vector machines algorithm with pairwise constraint projections called CPSVM is discussed for linearly separable and linearly non-separable datasets. CPSVM can preserve the structure of the original samples data as well as the pairwise constraints in the projective feature spaces. The central idea is to find a projective vector such that can assure the maximum margin of SVM hyperplane, and simultaneously considers improving the tightness among the distances between the similar patterns under the must-link constraints, while expanding the distances between those dissimilar pairs under the cannot-link constraints after the restricted data are projected onto the projection vector in the high-dimensional kernel spaces. The projection process can make the nonlinear data separable in the feature spaces. We demonstrate the practical usefulness and good performance of CPSVM algorithm in data visualization and classification tasks through extensive simulations on eleven UCI datasets. Experimental results show that CPSVM algorithm can almost always achieve the highest accuracies and has better robustness to the distant data noise.