Multimedia Information Retrieval Based on Late Semantic Fusion Approaches: Experiments on a Wikipedia Image Collection

Main goal of this work is to show the improvement of using a textual pre-filtering combined with an image re-ranking in a Multimedia Information Retrieval task. The defined three step-based retrieval processes and a well-selected combination of visual and textual techniques help the developed Multimedia Information Retrieval System to overcome the semantic gap in a given query. In the paper, five different late semantic fusion approaches are discussed and experimented in a realistic scenario for multimedia retrieval like the one provided by the publicly available ImageCLEF Wikipedia Collection.

[1]  Ana M. García-Serrano,et al.  Some Results Using Different Approaches to Merge Visual and Text-Based Features in CLEF'08 Photo Collection , 2008, CLEF.

[2]  Ana M. García-Serrano,et al.  Multimodal Information Approaches for the Wikipedia Collection at ImageCLEF 2011 , 2011, CLEF.

[3]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[4]  Javed A. Aslam,et al.  Condorcet fusion for improved retrieval , 2002, CIKM '02.

[5]  Ronald R. Yager,et al.  On ordered weighted averaging aggregation operators in multicriteria decision-making , 1988 .

[6]  Gabriela Csurka,et al.  XRCE's Participation at Wikipedia Retrieval of ImageCLEF 2011 , 2011, CLEF.

[7]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[8]  Esther de Ves,et al.  Applying logistic regression to relevance feedback in image retrieval systems , 2007, Pattern Recognit..

[9]  Shengli Wu,et al.  Evaluating Score Normalization Methods in Data Fusion , 2006, AIRS.

[10]  Ana M. García-Serrano,et al.  Multimedia Retrieval by Means of Merge of Results from Textual and Content Based Retrieval Subsystems , 2009, CLEF.

[11]  Gabriela Csurka,et al.  Semantic combination of textual and visual information in multimedia retrieval , 2011, ICMR.

[12]  Adrian Popescu,et al.  Overview of the Wikipedia Retrieval Task at ImageCLEF 2010 , 2010, CLEF.

[13]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[14]  Henning Müller,et al.  Information Fusion for Combining Visual and Textual Image Retrieval , 2010, 2010 20th International Conference on Pattern Recognition.

[15]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[16]  Henning Müller,et al.  Fusion Techniques for Combining Textual and Visual Information Retrieval , 2010, ImageCLEF.

[17]  Adrian Popescu,et al.  Overview of the Wikipedia Image Retrieval Task at ImageCLEF 2011 , 2011, CLEF.

[18]  Stéphane Marchand-Maillet,et al.  Information Fusion in Multimedia Information Retrieval , 2007, Adaptive Multimedia Retrieval.

[19]  Javed A. Aslam,et al.  Models for metasearch , 2001, SIGIR '01.

[20]  Yiannis S. Boutalis,et al.  Accurate Image Retrieval Based on Compact Composite Descriptors and Relevance Feedback Information , 2010, Int. J. Pattern Recognit. Artif. Intell..

[21]  Michael Grubinger,et al.  Analysis and evaluation of visual information systems performance , 2007 .

[22]  Hermann Ney,et al.  Features for image retrieval: an experimental comparison , 2008, Information Retrieval.

[23]  Paul Clough,et al.  ImageCLEF: Experimental Evaluation in Visual Information Retrieval , 2010 .

[24]  Edward A. Fox,et al.  Combination of Multiple Searches , 1993, TREC.

[25]  Ana M. García-Serrano,et al.  Experiences at ImageCLEF 2010 using CBIR and TBIR Mixing Information Approaches , 2010, CLEF.

[26]  Hugo Jair Escalante,et al.  Late fusion of heterogeneous methods for multimedia image retrieval , 2008, MIR '08.