pRBF Kernels: A Framework for the Incorporation of Task-Specific Properties into Support Vector Methods

The incorporation of prior-knowledge into support vector machines (SVM) in order to compensate for inadequate training data has been the focus of previous research works and many found a kernel-based approach to be the most appropriate. However, they are more adapted to deal with broad domain knowledge (e.g. ``sets are invariant to permutations of the elements'') rather than task-specific properties (e.g. ``the weight of a person is cubically related to her height''). In this paper, we present the partially RBF (pRBF) kernels, our original framework for the incorporation of prior-knowledge about correlation patterns between specific features and the output label. pRBF kernels are based upon the tensor-product combination of the standard radial basis function (RBF) kernel with more specialized kernels and provide a natural way for the incorporation of a commonly available type of prior-knowledge. In addition to a theoretical validation of our framework, we propose an detailed empirical evaluation on real-life biological data which illustrates its ease-of-use and effectiveness. Not only pRBF kernels were able to improve the learning results in general but they also proved to perform particularly well when the training data set was very small or strongly biased, significantly broadening the field of application of SVMs.