Going Beyond the Surrounding Text to Semantically Annotate and Search Digital Images

Digital objects such as images and videos are fundamental resources in digital library. To assist in retrieving such objects usually they are being tagged by some keywords or sentences. The popular approach to tag digital objects is based on associated text. However, relying on associated text alone such as the surrounding text unable to semantically describe such objects. This paper discusses the use of WordNet and ConceptNet to tag digital images beyond terms available in the surrounding text. WordNet is used to disambiguate concepts or terms from the associated text and ConceptNet is meant to infer topics or common-sense knowledge from summarizing the text that describe the images. However, relying on WordNet alone is not sufficed particularly when it comes to disambiguate specific or domain dependent concepts. As such the Name Entity Recognition (NER) technique is required to annotate important entities such as name of a person, location and organization. Our work focused on on-lines news images that are richly described with textual description.

[1]  Push Singh,et al.  Common Sense Conversations: Understanding Casual Conversation using a Common Sense Database , 2003 .

[2]  Hanqing Lu,et al.  Semantic knowledge extraction and annotation for web images , 2005, MULTIMEDIA '05.

[3]  Charles E. Kahn,et al.  Automated semantic indexing of figure captions to improve radiology image retrieval. , 2009, Journal of the American Medical Informatics Association : JAMIA.

[4]  Hiroyuki Sakai,et al.  QUARK: A Question and Answering System using Newspaper Corpus as a Knowledge Source , 2002, NTCIR.

[5]  Tzu-Chuan Chou,et al.  CanFind-a semantic image indexing and retrieval system , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[6]  Euripides G. M. Petrakis,et al.  Semantic similarity methods in wordNet and their application to information retrieval on the web , 2005, WIDM '05.

[7]  Marie-Francine Moens,et al.  Text Analysis for Automatic Image Annotation , 2007, ACL.

[8]  Wenfei Fan,et al.  Keys with Upward Wildcards for XML , 2001, DEXA.

[9]  Ross Purves,et al.  Eliciting concepts of place for text-based image retrieval , 2007, GIR '07.

[10]  Hugo Liu,et al.  ConceptNet — A Practical Commonsense Reasoning Tool-Kit , 2004 .

[11]  Henry Lieberman,et al.  Robust Photo Retrieval Using World Semantics , 2002 .

[12]  Hsin-Hsi Chen,et al.  Information retrieval with commonsense knowledge , 2006, SIGIR.

[13]  Zhiguo Gong,et al.  Web Query Expansion by WordNet , 2005, DEXA.

[14]  Zhiguo Gong,et al.  Web image indexing by using associated texts , 2005, Knowledge and Information Systems.

[15]  Shih-Fu Chang,et al.  Semantic knowledge construction from annotated image collections , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[16]  John Tait,et al.  Word sense disambiguation in information retrieval revisited , 2003, SIGIR.

[17]  Paul Buitelaar,et al.  A Cross Language Document Retrieval System Based on Semantic Annotation , 2003, EACL.

[18]  Changhu Wang,et al.  Scalable search-based image annotation , 2008, Multimedia Systems.

[19]  Chin-Hui Lee,et al.  Automatic Image Annotation through Multi-Topic Text Categorization , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[20]  Hsin-Hsi Chen,et al.  Query Expansion with ConceptNet and WordNet: An Intrinsic Comparison , 2006, AIRS.

[21]  Yansong Feng,et al.  Automatic Image Annotation Using Auxiliary Text Information , 2008, ACL.