Generic Multimedia Database Architecture Based upon Semantic Libraries

Semantic-based storage and retrieval of multimedia data requires accurate annotation of the data. Annotation can be done either manually or automatically. The retrieval performance of the manual annotation based approaches is quite good, as compared to approaches based on automatic annotation. However, manual annotation is time consuming and labor extensive. Therefore, it is quite difficult to apply this technique on huge volume of multimedia data. On the other hand, automatic annotation is commonly used to annotate the multimedia data based on low level features, which obviously lacks the semantic nature of the multimedia data. Yet, we have not come across with any such system which automatically annotate the multimedia data based on the extracted semantics accurately. In this paper, we have performed automatic annotation of the images by extracting their semantics (high level features) with the help of semantic libraries. Semantic libraries use semantic graphs. Each graph consists of related concepts along with their relationships. We have also demonstrated with the help of a case study that our proposed approach ensures an improvement in the semantic based retrieval of multimedia data.

[1]  Thomas Peltier,et al.  NIST Special Publications , 2003 .

[2]  László Böszörményi,et al.  The Life Cycle of Multimedia Metadata , 2005, IEEE Multim..

[3]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[4]  Xiaomeng Su,et al.  A Comparative Study of Ontology Languages and Tools , 2002, CAiSE.

[5]  Lynne Dunckley,et al.  Multimedia Databases: An Object Relational Approach , 2003 .

[6]  David Schach,et al.  XML Query Language (XQL) , 1998, QL.

[7]  Ilaria Bartolini,et al.  Windsurf: region-based image retrieval using wavelets , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[8]  Bernd Heisele,et al.  Component based recognition of objects in an office environment , 2003 .

[9]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Michael G. Strintzis,et al.  An Ontology Framework For Knowledge-Assisted Semantic Video Analysis and Annotation , 2004, SemAnnot@ISWC.

[11]  XML parsing: a threat to database performance , 2003, CIKM '03.

[12]  Shih-Fu Chang,et al.  Visually Searching the Web for Content , 1997, IEEE Multim..

[13]  Yehoshua Sagiv,et al.  XSEarch: A Semantic Search Engine for XML , 2003, VLDB.

[14]  Sven Loncaric,et al.  A survey of shape analysis techniques , 1998, Pattern Recognit..

[15]  Alin Deutsch,et al.  XML-QL: A Query Language for XML , 1998 .

[16]  Muhammad Abdul Qadir,et al.  Generic Multimedia Database Architecture , 2005, WEC.

[17]  Mathias Lux,et al.  XML and MPEG-7 for Interactive Annotation and Retrieval using Semantic Meta-data , 2002, J. Univers. Comput. Sci..

[18]  Rong Jin,et al.  Regularizing translation models for better automatic image annotation , 2004, CIKM '04.

[19]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[20]  Laura M. Haas,et al.  The Garlic project , 1996, SIGMOD '96.

[21]  Aidong Zhang,et al.  SemQuery: Semantic Clustering and Querying on Heterogeneous Features for Visual Data , 2002, IEEE Trans. Knowl. Data Eng..

[22]  Yixin Chen,et al.  A Region-Based Fuzzy Feature Matching Approach to Content-Based Image Retrieval , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[24]  Steven J. DeRose,et al.  XML Path Language (XPath) , 1999 .

[25]  László Böszörményi,et al.  The SMOOTH video DB - demonstration of an integrated generic indexing approach , 2000, MM 2000.

[26]  Zhi-Hua Zhou,et al.  SOM Based Image Segmentation , 2003, RSFDGrC.

[27]  Forouzan Golshani,et al.  Rx for semantic video database retrieval , 1994, MULTIMEDIA '94.

[28]  Henryk Palus Clustering techniques in colour image segmentation , 2003 .

[29]  David J. DeWitt,et al.  On supporting containment queries in relational database management systems , 2001, SIGMOD '01.

[30]  K. Wakimoto,et al.  Efficient and Effective Querying by Image Content , 1994 .

[31]  Karen M. Drabenstott,et al.  Browse and Search Patterns in a Digital Image Database , 2004, Information Retrieval.

[32]  Rong Yan,et al.  IBM multimedia analysis and retrieval system , 2008, CIVR '08.

[33]  de Arjen Vries,et al.  Mirror: Multimedia query processing in extensible databases , 1998 .

[34]  Sven Groppe,et al.  XPath query transformation based on XSLT stylesheets , 2003, WIDM '03.

[35]  Djoerd Hiemstra,et al.  Conceptual Language Models for Context-Aware Text Retrieval , 2004, TREC.

[36]  Georg Gottlob,et al.  The complexity of XPath query evaluation , 2003, PODS.

[37]  Akifumi Makinouchi,et al.  Semantic Approach to Image Database Classification and Retrieval (「夏のデータベースワークショップ(DBWS2003)」一般) , 2003 .

[38]  Ilaria Bartolini,et al.  FeedbackBypass: A New Approach to Interactive Similarity Query Processing , 2001, VLDB.

[39]  Thomas S. Huang,et al.  Supporting content-based queries over images in MARS , 1997, Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[40]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Salima Benbernou,et al.  Semantic retrieval of multimedia data , 2004, MMDB '04.

[42]  Werner Kuhn,et al.  Semantic reference systems , 2003, Int. J. Geogr. Inf. Sci..

[43]  Alin Deutsch,et al.  A Query Language for XML , 1999, Comput. Networks.

[44]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[45]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[46]  Lloyd Rutledge,et al.  Media semantics: who needs it and why? , 2002, MULTIMEDIA '02.

[47]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[48]  Ying Liu,et al.  Region-Based Image Retrieval with High-Level Semantic Color Names , 2005, 11th International Multimedia Modelling Conference.

[49]  Zhi-Hua Zhou,et al.  SOM Ensemble-Based Image Segmentation , 2004, Neural Processing Letters.

[50]  Stefan M. Rüger,et al.  Performance Comparison of Different Similarity Models for CBIR with Relevance Feedback , 2003, CIVR.

[51]  Frank van Harmelen,et al.  Ontology languages for the Semantic Web , 2005 .

[52]  Anil K. Jain,et al.  Image classification for content-based indexing , 2001, IEEE Trans. Image Process..

[53]  Jintao Li,et al.  A Generic Framework for Semantic Sports Video Analysis Using Dynamic Bayesian Networks , 2005, 11th International Multimedia Modelling Conference.

[54]  Asunción Gómez-Pérez,et al.  Evaluation of ontologies , 2001, International Journal of Intelligent Systems.

[55]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Nico Roos,et al.  Intelligent Information Retrieval and Presentation with Multimedia Databases , 2003 .

[57]  Thomas R. Gruber,et al.  Toward principles for the design of ontologies used for knowledge sharing? , 1995, Int. J. Hum. Comput. Stud..

[58]  Asunción Gómez-Pérez,et al.  A Roadmap to Ontology Specification Languages , 2000, EKAW.

[59]  Steven J. DeRose,et al.  XML Path Language (XPath) Version 1.0 , 1999 .

[60]  Tomaso A. Poggio,et al.  Example-Based Object Detection in Images by Components , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Yong Rui,et al.  Multimedia Analysis and Retrieval System , 1997 .

[62]  James Ze Wang,et al.  Studying digital imagery of ancient paintings by mixtures of stochastic models , 2004, IEEE Transactions on Image Processing.

[63]  Shaukat Ali,et al.  Context-Aware Querying in Multimedia Databases - A Futuristic Approach , 2005, WEC.

[64]  Yixin Chen,et al.  Content-based image retrieval by clustering , 2003, MIR '03.

[65]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[66]  Wei-Ying Ma,et al.  Learning similarity measure for natural image retrieval with relevance feedback , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[67]  Milan Sonka,et al.  Image processing analysis and machine vision [2nd ed.] , 1999 .

[68]  Raimondo Schettini,et al.  Image annotation using SVM , 2003, IS&T/SPIE Electronic Imaging.

[69]  Antonio Torralba,et al.  Contextual Priming for Object Detection , 2003, International Journal of Computer Vision.

[70]  Donald D. Chamberlin,et al.  XQuery: a query language for XML , 2003, SIGMOD '03.

[71]  Christian Becker,et al.  Quality of Service in Distributed Object Systems and Distributed Multimedia Object/Component Systems , 2001, ECOOP Workshops.

[72]  Fausto Giunchiglia,et al.  Semantic Matching: Algorithms and Implementation , 2007, J. Data Semant..

[73]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Jake K. Aggarwal,et al.  Feature Integration, Multi-image Queries and Relevance Feedback in Image Retrieval , 2003 .

[75]  Steffen Staab,et al.  Ontology Learning for the Semantic Web , 2002, IEEE Intell. Syst..

[76]  Antonio Torralba,et al.  Statistical Context Priming for Object Detection , 2001, ICCV.