Graph-Based Pairwise Learning to Rank for Video Search

Learning-based ranking is a promising approach to a variety of search tasks, which is aimed at automatically creating the ranking model based on training samples and machine learning techniques. However, the problem of lacking training samples labeled with relevancy degree or ranking orders is frequently encountered. To address this problem, we propose a novel graph-based learning to rank (GLRank) for video search by leveraging the vast amount of unlabeled samples. A relation graph is constructed by using sample (i.e., video shot) pairs rather than individual samples as vertices. Each vertex in this graph represents the "relevancy relation" between two samples in a pair (i.e., which sample is more relevant to the given query). Such relevancy relation is discovered through a set of pre-trained concept detectors and then propagated among the pairs. When all the pairs, constructed with the samples to be searched, receive the propagated relevancy relation, a round robin criterion is proposed to obtain the final ranking list. We have conducted comprehensive experiments on automatic video search task over TRECVID 2005-2007 benchmarks and shown significant and consistent improvements over the other state-of-the-art ranking approaches.