EXACT AND EFFICIENT LEAVE-PAIR-OUT CROSS-VALIDATION FOR RANKING RLS

In this paper, we introduce an efficient cross-validation algorithm for RankRLS, a kernel-based ranking algorithm. Cross-validation (CV) is one of the most useful methods for model selection and performance assessment of machine learning algorithms, especially when the number of labeled data is small. A natural way to measure the performance of ranking algorithms by CV is to hold each data point pair out from the training set at a time and measure the performance with the held out pair. This approach is known as leave-pair-out cross-validation (LPOCV). We present a computationally efficient algorithm for performing LPOCV for RankRLS. If RankRLS is already trained with the whole training set, the computational complexity of the algorithm isO(m). Further, if there are d outputs to be learned simultaneously, the computational complexity of performing LPOCV isO(md). An approximativeO(m) time LPOCV algorithm for RankRLS has been previously proposed, but our method is the first exact solution to this problem. We introduce a general framework for developing and analysing hold-out and cross-validation techniques for quadratically regularized kernel-based learning algorithms. The framework is constructed using a value regularization based variant of the representer theorem. We provide a simple proof for this variant using matrix calculus. Our cross-validation algorithm can be seen as an instance of this framework.