A new combined KSVM and KFD model for classification and recognition

The Kernel Support Vector Machine (KSVM) is a powerful nonlinear classification methodology where, the Support Vectors (SVs) fully describe the decision surface by incorporating local information in the Kernel space. On the other hand, the Kernel Fisher Discriminant(KFD) is a non-linear classifier which has proven to be powerful and competitive to several state-of-the-art classifiers. This paper proposes a novel KSVM + KFD model which combines these two methods. This model can be viewed as an extension to the KSVM by incorporating 'global' characteristics of the data to estimate the decision boundary in the Kernel space. On the other hand, this new model could also be considered as an improvement to the KFD by incorporating the Support Vectors (local margin concept) into the KFD formulation. The KSVM + KFD model can be reduced to the classical KSVM model so that existing KSVM software can be used for easy implementation. An extensive comparison of the KSVM + KFD to the KFD, KSVM, Linear Discriminant Analysis (LDA), Linear Support Vector Machine (LSVM) and the combined LSVM and LDA, performed on real data sets, has shown the advantages of our proposed model. In particular, the experiments on face recognition have clearly shown the superiority of the KSVM + KFD over other methods.