Optimizing video search reranking via minimum incremental information loss

This paper is concerned with video search reranking - the task of reordering the initial ranked documents (video shots) to improve the search performance - in an optimization framework. Conventional supervised reranking approaches empirically convert the reranking as a classification problem in which each document is determined relevant or not, followed by reordering the documents according to the confidence scores of classification. We argue that reranking is essentially an optimization problem in which the ranked list is globally optimal if any two arbitrary documents from the list are correctly ranked in terms of relevance, rather than simply classifying a document into relevant or not. Therefore, we propose in this paper to directly optimize video search reranking from a novel viewpoint of information theory, that is, to identify an optimal set of correctly-ranked document pairs which maximally preserves the relevant information and simultaneously carries the irrelevant information as little as possible. The final reranked list is then directly recovered from this optimal set of pairs. Under the framework, we further propose an effective algorithm, called minimum incremental information loss (MIIL) reranking, to solve the optimization problem more practically. We conducted comprehensive experiments on automatic video search task over TRECVID 2005-2007 benchmarks, and showed significant and consistent improvements over the text search baseline and other reranking approaches.