Click-boosting multi-modality graph-based reranking for image search

Image reranking is an effective way for improving the retrieval performance of keyword-based image search engines. A fundamental issue underlying the success of existing image reranking approaches is the ability in identifying potentially useful recurrent patterns from the initial search results. Ideally, these patterns can be leveraged to upgrade the ranks of visually similar images, which are also likely to be relevant. The challenge, nevertheless, originates from the fact that keyword-based queries are used to be ambiguous, resulting in difficulty in predicting the search intention. Mining useful patterns without understanding query is risky, and may lead to incorrect judgment in reranking. This paper explores the use of click-through data, which can be viewed as the footprints of user searching behavior, as an effective means of understanding query, for providing the basis on identifying the recurrent patterns that are potentially helpful for reranking. A new reranking algorithm, named click-boosting multi-modality graph-based reranking, is proposed. The algorithm leverages clicked images to locate similar images that are not clicked, and reranks them in a multi-modality graph-based learning scheme. Encouraging results are reported for image reranking on a real-world image dataset collected from a commercial search engine with click-through data.