A relational vector space model using an advanced weighting scheme for image retrieval

In this paper, we lay out a relational approach for indexing and retrieving photographs from a collection. The increase of digital image acquisition devices, combined with the growth of the World Wide Web, requires the development of information retrieval (IR) models and systems that provide fast access to images searched by users in databases. The aim of our work is to develop an IR model suited to images, integrating rich semantics for representing this visual data and user queries, which can also be applied to large corpora. Our proposal merges the vector space model of IR - widely tested in textual IR - with the conceptual graph (CG) formalism, based on the use of star graphs (i.e. elementary CGs made up of a single relation connected to some concepts representing image objects). A novel weighting scheme for star graphs, based on image objects size, position, and image heterogeneity is outlined. We show that integrating relations into the vector space model through star graphs increases the system's precision, and that the results are comparable to those from graph projection systems, and also that they shorten processing time for user queries.

[1]  Joo-Hwee Lim.  Photograph retrieval and classification by visual keywords and thesaurus , 2009, New Generation Computing.

[2]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[3]  Jong-Hak Lee,et al.  Analyses of multiple evidence combination , 1997, SIGIR '97.

[4]  Jean-Marc Odobez,et al.  Natural Scene Image Modeling Using Color and Texture Visterms , 2006, CIVR.

[5]  Iadh Ounis,et al.  RELIEF: combining expressiveness and rapidity into a single system , 1998, SIGIR '98.

[6]  Elisa Bertino,et al.  A constraint-based approach to shape management in multimedia databases , 1998, Multimedia Systems.

[7]  Marcel Worring,et al.  Classification of user image descriptions , 2004, Int. J. Hum. Comput. Stud..

[8]  Antonio Adán,et al.  3D scene analysis from a single range image through occlusion graphs , 2008, Pattern Recognit. Lett..

[9]  Rainer Lienhart,et al.  Image retrieval on large-scale image databases , 2007, CIVR '07.

[10]  M. Chein,et al.  Conceptual graphs: fundamental notions , 1992 .

[11]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[12]  Wen Gao,et al.  Effective and efficient object-based image retrieval using visual phrases , 2006, MM '06.

[13]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[14]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[15]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[17]  Vilas Wuwongse,et al.  Fuzzy Conceptual Graphs , 1993, ICCS.

[18]  James Ze Wang,et al.  RF/sup */IPF: a weighting scheme for multimedia information retrieval , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[19]  S. K. Morton.  Conceptual graphs and fuzziness in artificial intelligence , 1987 .

[20]  Stéphane Ayache,et al.  Classifier Fusion for SVM-Based Multimedia Semantic Indexing , 2007, ECIR.

[21]  Mohan S. Kankanhalli,et al.  Advances in Digital Home Photo Albums , 2004 .

[22]  László Czúni,et al.  Depth-Based Indexing and Retrieval of Photographic Images , 2003, VLBV.

[23]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[24]  Jean-Marc Odobez,et al.  A Thousand Words in a Scene , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[26]  Shin'ichi Satoh,et al.  Media objects for user-centered similarity matching , 2008, Multimedia Tools and Applications.

[27]  Iadh Ounis,et al.  Photograph indexing and retrieval using star-graphs , 2003 .

[28]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[30]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[32]  M. Mugnier,et al.  Représenter des connaissances et raisonner avec des graphes , 1996 .

[33]  Shin'ichi Satoh,et al.  Using Visual-Textual Mutual Information and Entropy for Inter-modal Document Indexing , 2007, ECIR.

[34]  Pinar Duygulu Sahin,et al.  Recognizing Objects and Scenes in News Videos , 2006, CIVR.

[35]  Tae-Sun Choi,et al.  A novel algorithm for estimation of depth map using image focus for 3D shape recovery in the presence of noise , 2008, Pattern Recognit..

[36]  Kerry Rodden,et al.  How do people manage their digital photographs? , 2003, CHI '03.

[37]  Bart M. ter Haar Romeny,et al.  Front-End Vision and Multi-Scale Image Analysis , 2003, Computational Imaging and Vision.

[38]  Francesco M. Donini,et al.  Structured Knowledge Representation for Image Retrieval , 2011, J. Artif. Intell. Res..

[39]  Sagarmay Deb.  Multimedia Systems and Content-Based Image Retrieval , 2003 .

[40]  Iadh Ounis,et al.  A Weighting Scheme for Star-Graphs , 2003, ECIR.

[41]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[42]  Vijay V. Raghavan,et al.  A critical analysis of vector space model for information retrieval , 1986, J. Am. Soc. Inf. Sci..

[43]  David A. Hull.  Improving text retrieval for the routing problem using latent semantic indexing , 1994, SIGIR '94.

[44]  Kerry Rodden.  How Do People Organise Their Photographs? , 1999, BCS-IRSG Annual Colloquium on IR Research.

[45]  Eric L. Schwartz,et al.  Design considerations for a space-variant visual sensor with complex-logarithmic geometry , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[46]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[47]  Susan T. Dumais,et al.  Richard Harshman Indexing by Latent Semantic Analysis , 1990 .

[48]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[49]  Joo-Hwee Lim.  Building Visual Vocabulary for Image Indexation and Query Formulation , 2001, Pattern Analysis & Applications.

[50]  Bernt Schiele,et al.  International Journal of Computer Vision manuscript No. (will be inserted by the editor) Semantic Modeling of Natural Scenes for Content-Based Image Retrieval , 2022 .

[51]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[52]  Joo-Hwee Lim,et al.  Symbolic photograph content-based retrieval , 2002, CIKM '02.

[53]  Bertrand Chupeau,et al.  Depth-based segmentation , 1997, IEEE Trans. Circuits Syst. Video Technol..

[54]  Gerard Salton,et al.  The SMART Retrieval System , 1971 .

[55]  Philippe Mulhem,et al.  A model for weighting image objects in home photographs , 2005, CIKM '05.

[56]  Mourad Mechkour,et al.  EMIR2: An Extended Model for Image Representation and Retrieval , 1995, DEXA.

[57]  Allan Kuchinsky,et al.  Quality is in the eye of the beholder: meeting users' requirements for Internet quality of service , 2000, CHI.

[58]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[59]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .