An Ensemble of Kernel Ridge Regression for Multi-class Classification

Abstract We propose an ensemble of kernel ridge regression based classifiers in this paper. Kernel ridge regression admits a closed form solution making it faster to compute and also making it suitable to use for ensemble methods for small and medium sized data sets. Our method uses random vector functional link network to generate training samples for kernel ridge regression classifiers. Several kernel ridge regression classifiers are constructed from different training subsets in each base classifier. The partitioning of the training samples into different subsets leads to a reduction in computational complexity when calculating matrix inverse compared with the standard approach of using all N samples for kernel matrix inversion. The proposed method is evaluated using well known multi-class UCI data sets. Experimental results show the proposed ensemble method outperforms the single kernel ridge regression classifier and its bagging version.