Enhancing query interpretation by combining textual and visual analyses

Query analysis is an important phase in image retrieval process especially for ambiguous queries. This paper describes a query analysis process that manages textual and visual queries. The main idea is to select the most appropriate concepts. For the textual part, we extract keywords. Then, we deduce the most relevant concepts related to such keyword by performing a semantic similarity computing based on ontology structure. A similar process is carried out on the visual part based on the associated annotation. The concept set deduced from each part are then merged. Finally, based on a semantic inter-concept graph, we attempt to refine the query by expanding or reweighting the concepts list. Our approach is evaluated in ImagCLEF2012 benchmark. The experiments show encouraging results.

[1]  Chokri Ben Amar,et al.  Effective concept detection using Second order Co-occurence Flickr context similarity measure SOCFCS , 2012, 2012 10th International Workshop on Content-Based Multimedia Indexing (CBMI).

[2]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[3]  Adel M. Alimi,et al.  Semi-automatic soft collaborative annotation for semantic video indexing , 2011, 2011 IEEE EUROCON - International Conference on Computer as a Tool.

[4]  Rong Yan,et al.  Semantic concept-based query expansion and re-ranking for multimedia retrieval , 2007, ACM Multimedia.

[5]  Wei-Ying Ma,et al.  Probabilistic query expansion using query logs , 2002, WWW '02.

[6]  Chong-Wah Ngo,et al.  Concept-Driven Multi-Modality Fusion for Video Search , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  W. Bruce Croft,et al.  Learning concept importance using a weighted dependence model , 2010, WSDM '10.

[8]  ChengXiang Zhai,et al.  Mining term association patterns from search logs for effective query reformulation , 2008, CIKM '08.

[9]  Emmanuel Dellandréa,et al.  LIRIS-Imagine at ImageCLEF 2011 Photo Annotation Task , 2011, CLEF.

[10]  Benjamin Rey,et al.  Generating query substitutions , 2006, WWW '06.

[11]  Adel M. Alimi,et al.  A fuzzy ontology: based framework for reasoning in visual video content analysis and indexing , 2011, MDMKDD '11.

[12]  W. Bruce Croft,et al.  Improving verbose queries using subset distribution , 2010, CIKM.

[13]  Massih-Reza Amini,et al.  Exploiting Visual Concepts to Improve Text-Based Image Retrieval , 2009, ECIR.

[14]  W. Bruce Croft,et al.  Parameterized concept weighting in verbose queries , 2011, SIGIR.

[15]  James Allan,et al.  Regression Rank: Learning to Meet the Opportunity of Descriptive Queries , 2009, ECIR.

[16]  Chokri Ben Amar,et al.  Flickr-based semantic context to refine automatic photo annotation , 2012, 2012 3rd International Conference on Image Processing Theory, Tools and Applications (IPTA).

[17]  W. Bruce Croft,et al.  Discovering key concepts in verbose queries , 2008, SIGIR '08.

[18]  Iadh Ounis,et al.  Combining fields for query expansion and adaptive query expansion , 2007, Inf. Process. Manag..

[19]  Chokri Ben Amar,et al.  REGIMvid at ImageCLEF2012: Concept-based Query Refinement and Relevance-based Ranking Enhancement for Image Retrieval , 2012, CLEF.

[20]  James Allan,et al.  Effective and efficient user interaction for long queries , 2008, SIGIR '08.

[21]  Yongdong Zhang,et al.  Graph-based multi-space semantic correlation propagation for video retrieval , 2010, The Visual Computer.