User specific context construction for personalized multimedia retrieval

The onset of Web 2.0 has given the freedom of tagging to the users. The popularization of social networking and the expansion of the smartphone market in the past decade has led to an increase of data being accumulated on the social media platforms, particularly images and videos. The exponential and ever increasing data have made information retrieval cumbersome, especially for the social network users, and this has turned out to be a huge challenge in the evolution of algorithms and technologies. In this paper, we present a novel framework and techniques for retrieving user’s multimedia content like images from the user’s profile using the context of the image/media file. We apply the Logical Itemset mining on the image Metadata consisting of the textual data (Hashtags, Caption, Date and Time) associated with the images. Through this work, we intend to bridge the semantic gap between the images and the data representation that the user associated with them. Our framework also addresses the paraphrase problem of variation in words (synonyms) used to describe a context of a media file. To evaluate the applicability of our framework, we performed tests on large Instagram image dataset extracted from various user profiles containing monolingual metadata, which show promising results for real-time applications. Furthermore, we compare and evaluate our framework with another context-based image retrieval framework, Krumbs.