Retrieving Social Images using Relevance Filtering and Diverse Selection

Retrieving relevant and diverse images from a large set of images is problem of interest in social media. Given a set of images pertaining to a location or a concept, a subset of diverse image can summarize the attributes of the corresponding location/concept. In this work, we present a two step image retrieval model involving relevance filtering followed by diverse selection. Based on the visual features, textual descriptions and Flickr rank, relevance filtering initially determines a subset of images which have correspondence to a topic of interest. Subsequently, diverse selection determines a smaller subset of images to provide a diverse perspective of the concept. We obtain an F1 score of .509 on a test set containing 139 concepts, when computed over the top 20 images output by our system. We analyze the outcomes of our system and investigate the utility of image metadata (reviews, Flickr content) when combined with visual descriptors.