Analysis of Keywords used in Image Understanding Tasks 15 : 10 – 15 : 35 Katerina Pastra Image-Language Association : are we looking at the right features ? 15 : 35 – 16 : 00

In this paper, several strategies for cross-language image indexing and terminological glossary compilation are presented. The process starts form a source language indexed image. CBIR is proposed as a means to find similar images in target language documents in the web. The text surrounding the target matched image is chunked and the chunks are classified into concrete and abstract nouns by means of a discriminant analysis. The number of images retrieved by each chunk and the edit distance between each chunk and each image file name are taken as differentiating variables; a 74.4% rate of correctly classified labeled examples shows the adequacy of these variables. Nouns classified as concrete are used to retrieve images from the web and each retrieved image is compared with the image in the target document. When a positive matching occurs, the chunk used to retrieve the matched image is assigned as the index for the image in the target document and as the target language equivalent for the source image index. As the experiments are carried out in specialized domains, a systematic and recursive use of the approach is used to build terminological glossaries by storing images with their respective cross-language indices.

[1]  P. Kay,et al.  Basic Color Terms: Their Universality and Evolution , 1973 .

[2]  D Breznik,et al.  Methodology of the study , 1976 .

[3]  M. Minsky.  The Society of Mind , 1986 .

[4]  R. Jackendoff.  On beyond Zebra: The relation of linguistic and visual information , 1987, Cognition.

[5]  George Lakoff,et al.  Women, Fire, and Dangerous Things , 1987 .

[6]  David R. Karger,et al.  Scatter/Gather: a cluster-based approach to browsing large document collections , 1992, SIGIR '92.

[7]  B. Landau,et al.  “What” and “where” in spatial language and spatial cognition , 1993 .

[8]  Stephen E. Robertson,et al.  Okapi at TREC-4 , 1995, TREC.

[9]  Donna K. Harman,et al.  Overview of the Fourth Text REtrieval Conference (TREC-4) , 1995, TREC.

[10]  Kevin Humphreys,et al.  XI: A Simple Prolog-based Language for Cross-Classification and Inheritance , 1996 .

[11]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[12]  Kathleen R. McKeown,et al.  Linear segmentation and segment relevence , 1998 .

[13]  W. Bruce Croft,et al.  Deriving concept hierarchies from text , 1999, SIGIR '99.

[14]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Clement H. C. Leung,et al.  Benchmarking for Content-Based Visual Information Search , 2000, VISUAL.

[16]  Anil K. Jain,et al.  Image classification for content-based indexing , 2001, IEEE Trans. Image Process..

[17]  Clement H. C. Leung,et al.  Structured natural-language descriptions for semantic content retrieval of visual materials , 2001, J. Assoc. Inf. Sci. Technol..

[18]  Thierry Pun,et al.  Performance evaluation in content-based image retrieval: overview and proposals , 2001, Pattern Recognit. Lett..

[19]  Jane Hunter,et al.  Adding Multimedia to the Semantic Web: Building an MPEG-7 ontology , 2001, SWWS.

[20]  Laura A. Carlson,et al.  Grounding spatial language in perception: an empirical and computational investigation. , 2001, Journal of experimental psychology. General.

[21]  Bob J. Wielinga,et al.  Ontology-Based Photo Annotation , 2001, IEEE Intell. Syst..

[22]  Corinne Jörgensen.  TOWARDS AN IMAGE TESTBED FOR BENCHMARKING IMAGE INDEXING AND RETRIEVAL SYSTEMS , 2001 .

[23]  Mark Sanderson,et al.  Building, Testing, and Applying Concept Hierarchies , 2002 .

[24]  Thierry Pun,et al.  The Truth about Corel - Evaluation in Image Retrieval , 2002, CIVR.

[25]  Sven Wachsmuth,et al.  Towards a Framework for Learning Structured Shape Models from Text-Annotated Images , 2003, HLT-NAACL 2003.

[26]  Yorick Wilks,et al.  Intelligent Indexing of Crime Scene Photographs , 2003, IEEE Intell. Syst..

[27]  Judith L. Klavans,et al.  Methods for precise named entity matching in digital collections , 2003, 2003 Joint Conference on Digital Libraries, 2003. Proceedings..

[28]  Vipin Kumar,et al.  Finding Topics in Collections of Documents: A Shared Nearest Neighbor Approach , 2003, Clustering and Information Retrieval.

[29]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[30]  Shu-Yuan Chen,et al.  Image classification using color, texture and regions , 2003, Image Vis. Comput..

[31]  Ching-Yung Lin,et al.  Video Collaborative Annotation Forum: Establishing Ground-Truth Labels on Large Multimedia Datasets , 2003, TRECVID.

[32]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Kevin Li,et al.  Faceted metadata for image search and browsing , 2003, CHI '03.

[34]  Yorick Wilks,et al.  Vision-Language Integration in AI: A Reality Check , 2004, ECAI.

[35]  Mark Sanderson,et al.  A Study of User Interaction with a Concept-Based Interactive Query Expansion Support Tool , 2004, ECIR.

[36]  Michael G. Strintzis,et al.  An Ontology Framework For Knowledge-Assisted Semantic Video Analysis and Annotation , 2004, SemAnnot@ISWC.

[37]  Mark Sanderson,et al.  The CLEF 2004 Cross-Language Image Retrieval Track , 2004, CLEF.

[38]  Wei-Ying Ma,et al.  Data-driven approach for bridging the cognitive gap in image retrieval , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[39]  Karen Spärck Jones.  A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[40]  Mohan S. Kankanhalli,et al.  Benchmarking Multimedia Databases , 1997, Multimedia Tools and Applications.

[41]  Wei-Ying Ma,et al.  Hierarchical clustering of WWW image search results using visual, textual and link information , 2004, MULTIMEDIA '04.

[42]  Ekaterini Pastra.  Vision - language integration : a double-grounding case , 2004 .

[43]  Nando de Freitas,et al.  A Statistical Model for General Contextual Object Recognition , 2004, ECCV.

[44]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..

[45]  Paul Over,et al.  The TREC VIdeo Retrieval Evaluation (TRECVID): A Case Study and Status Report , 2004, RIAO.

[46]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[47]  Carol Peters,et al.  Cross-Language Evaluation Forum: Objectives, Results, Achievements , 2004, Information Retrieval.

[48]  David Sinclair,et al.  Language-based querying of image collections on the basis of an extensible ontology , 2004, Image Vis. Comput..

[49]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[50]  Paul Over,et al.  Multimedia retrieval benchmarks , 2004, IEEE MultiMedia.

[51]  Eero Sormunen,et al.  End-User Searching Challenges Indexing Practices in the Digital Newspaper Photo Archive , 2004, Information Retrieval.

[52]  Marius Tico,et al.  A Test Collection for the Evaluation of Content-Based Image Retrieval Algorithms—A User and Task-Based Approach , 2001, Information Retrieval.

[53]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[54]  Alexander G. Hauptmann,et al.  Towards a Large Scale Concept Ontology for Broadcast Video , 2004, CIVR.

[55]  Antonio Criminisi,et al.  Object categorization by learned universal visual dictionary , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[56]  Beatriz Marcotegui,et al.  Fast Implementation of Waterfall Based on Graphs , 2005, ISMM.

[57]  Svitlana Zinger,et al.  Extracting an ontology of portrayable objects from Wordnet , 2005 .

[58]  S. Kollias,et al.  ONTOLOGY FOR MULTIMEDIA REASONING , 2005 .

[59]  Thomas Martin Deserno,et al.  The CLEF 2005 Cross-Language Image Retrieval Track , 2005, CLEF.

[60]  Heung-Kyu Lee,et al.  Re-ranking algorithm using post-retrieval clustering for content-based image retrieval , 2005, Inf. Process. Manag..

[61]  Yiannis Kompatsiaris,et al.  Using a Multimedia Ontology Infrastructure for Semantic Annotation of Multimedia Content , 2005, SemAnnot@ISWC.

[62]  Christopher K. I. Williams,et al.  Pascal Visual Object Classes Challenge Results , 2005 .

[63]  Dan I. Moldovan,et al.  Exploiting ontologies for automatic image annotation , 2005, SIGIR '05.

[64]  Gregory Grefenstette,et al.  Evaluating Content Based Image Retrieval Techniques with the One Million Images CLIC TestBed , 2005, WEC.

[65]  Paul D. Clough,et al.  The IAPR Benchmark for Assessing Image Retrieval Performance in Cross Language Evaluation Tasks , 2005 .

[66]  Paul Buitelaar,et al.  Feature Representation for Cross-Lingual, Cross-Media Semantic Web Application , 2005, SemAnnot@ISWC.

[67]  A. Hanbury.  Review of Image Annotation for the Evaluation of Computer Vision Algorithms , 2006 .

[68]  Isabelle Bloch,et al.  Workshop on Language Resources for Content-based Image Retrieval during LREC 2006 , 2006 .

[69]  Katerina Pastra.  An Alternative Suggestion for Vision-Language Integration in Intelligent Agents , 2006, SETN.

[70]  G. Grefenstette,et al.  Clustering and semantically filtering web images to create a large-scale image ontology , 2006, Electronic Imaging.