Personalized multimedia content retrieval through relevance feedback techniques for enhanced user experience

Emerging multimedia interactive services inherently call for user-centered design approaches, where the involved high degree of interactivity requires the implementation of efficient and effective information retrieval approaches. In this paper, a multimodal content retrieval framework is introduced that employs personalization along with relevance feedback techniques in order to enhance provided QoE, by retrieving and offering multimedia content tailored to individual users' characteristics and/or preferences. The developed Relevance Feedback mechanism engages the user into assessing the relevance of the initially retrieved results list of the original query, and through one or more iterations to present him with the most relevant result list based on his feedback. Our proposed framework implements a similarity learning scheme to improve multimedia content retrieval, towards increasing user experience. A model for implicit relevance feedback is formulated and a confidence level parameter is introduced to classify the results, based on the Jaccard similarities of the results that did not receive explicit feedback with those that did. This relevance feedback mechanism acts complementary to the personalized search by reranking the initial retrieved set in iterative rounds. The performance and effectiveness of the proposed framework was evaluated and demonstrated through an extensive experimental study, utilizing an interactive multimodal multimedia web-based system, with media files consisting of a set of 3D movies containing audio-visual content with high and low level semantic annotations.

[1]  Rong Yan,et al.  Video Retrieval Based on Semantic Concepts , 2008, Proceedings of the IEEE.

[2]  Mounia Lalmas,et al.  A survey on the use of relevance feedback for information access systems , 2003, The Knowledge Engineering Review.

[3]  Lucjan Janowski,et al.  Modeling subjective tests of quality of experience with a Generalized Linear Model , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[4]  Simon Dooms,et al.  Dynamic generation of personalized hybrid recommender systems , 2013, RecSys.

[5]  Piotr Romaniak.  Towards Realization of a Framework for Integrated Video Quality of Experience Assessment , 2009, IEEE INFOCOM Workshops 2009.

[6]  ChengXiang Zhai,et al.  Implicit user modeling for personalized search , 2005, CIKM '05.

[7]  Susan Gauch,et al.  Personalizing Search Based on User Search Histories , 2004 .

[8]  Stephen E. Robertson,et al.  Relevance weighting of search terms , 1976, J. Am. Soc. Inf. Sci..

[9]  Jianmin Zhao,et al.  A personalized multimedia retrieval frame based on user interest , 2008, 2008 First IEEE International Conference on Ubi-Media Computing.

[10]  Susan T. Dumais,et al.  Personalizing search via automated analysis of interests and activities , 2005, SIGIR '05.

[11]  Zheng Wei,et al.  A New Approach Measuring Users' QoE in the IPTV , 2009, 2009 Pacific-Asia Conference on Circuits, Communications and Systems.

[12]  William Haseman,et al.  An Empirical Investigation of the Influences of the Degree of Interactivity on User-Outcomes in a Multimedia Environment , 2002, Inf. Resour. Manag. J..

[13]  Yong Yu,et al.  Learning the Gain Values and Discount Factors of DCG , 2012, ArXiv.

[14]  Jean-Marie Bonnin,et al.  Quality of Experience Measurements for Video Streaming over Wireless Networks , 2009, 2009 Sixth International Conference on Information Technology: New Generations.

[15]  Simone Santini,et al.  Similarity Measures , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Anastasios Tefas,et al.  Person Identity Label Propagation in Stereo Videos , 2014, IEEE Transactions on Multimedia.

[17]  Gabriella Pasi.  Implicit Feedback through User-system Interactions for Defining User Models in Personalized Search , 2014, IHCI.