Media objects for user-centered similarity matching

The increase of digital image and video acquisition devices, combined with the growth of the World Wide Web, requires the definition of user-relevant similarity matching methods providing meaningful access to documents searched by users among large amounts of data. The aim of our work is to define media objects for document description suited to images and videos, integrating a user-centered definition of importance for similarity matching. The importance is defined according to criteria and hypotheses, which have been experimentally validated. This leads to a definition of a weighting scheme for media objects (based on objects size, position, and scene homogeneity), which has also been validated with users in a second experiment. This model allows for meaningful similarity matching between document pairs and between users’ queries and documents.

[1]  Paul S. Martin,et al.  Measuring Behaviour: An Introductory Guide , 1986 .

[2]  Joo-Hwee Lim.  Building Visual Vocabulary for Image Indexation and Query Formulation , 2001, Pattern Analysis & Applications.

[3]  Chris Buckley,et al.  Improving automatic query expansion , 1998, SIGIR '98.

[4]  Bernt Schiele,et al.  International Journal of Computer Vision manuscript No. (will be inserted by the editor) Semantic Modeling of Natural Scenes for Content-Based Image Retrieval , 2022 .

[5]  Marcel Worring,et al.  Classification of user image descriptions , 2004, Int. J. Hum. Comput. Stud..

[6]  Rainer Lienhart,et al.  Image retrieval on large-scale image databases , 2007, CIVR '07.

[7]  Hong Guo,et al.  Background removal in image indexing and retrieval , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[8]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[9]  Hans-Peter Frei,et al.  Concept based query expansion , 1993, SIGIR.

[10]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Wen Gao,et al.  Effective and efficient object-based image retrieval using visual phrases , 2006, MM '06.

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Joo-Hwee Lim,et al.  Home Photo Content Modeling for Personalized Event-Based Retrieval , 2003, IEEE Multim..

[15]  Mohan S. Kankanhalli,et al.  Advances in Digital Home Photo Albums , 2004 .

[16]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[17]  Bernard Mérialdo,et al.  Comparison of Multiepisode Video Summarization Algorithms , 2003, EURASIP J. Adv. Signal Process..

[18]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[19]  Jean-Marc Odobez,et al.  Natural Scene Image Modeling Using Color and Texture Visterms , 2006, CIVR.

[20]  Gerard Salton,et al.  The SMART Retrieval System , 1971 .

[21]  Kerry Rodden,et al.  How do people manage their digital photographs? , 2003, CHI '03.

[22]  Philippe Mulhem,et al.  A model for weighting image objects in home photographs , 2005, CIKM '05.

[23]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Eric L. Schwartz,et al.  Design considerations for a space-variant visual sensor with complex-logarithmic geometry , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[25]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[26]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[27]  Kerry Rodden.  How Do People Organise Their Photographs? , 1999, BCS-IRSG Annual Colloquium on IR Research.

[28]  W. Buxton.  Human-Computer Interaction , 1988, Springer Berlin Heidelberg.

[29]  Anthony J. Maeder,et al.  Automatic identification of perceptually important regions in an image , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[30]  Fred Stentiford.  Attention-based image similarity measure with application to content-based information retrieval , 2003, IS&T/SPIE Electronic Imaging.

[31]  Iadh Ounis,et al.  Finding the Best Parameters for Image Ranking: a User-Oriented Approach , 1998 .

[32]  Jean-Marc Odobez,et al.  A Thousand Words in a Scene , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Pinar Duygulu Sahin,et al.  Recognizing Objects and Scenes in News Videos , 2006, CIVR.

[34]  Bart M. ter Haar Romeny,et al.  Front-End Vision and Multi-Scale Image Analysis , 2003, Computational Imaging and Vision.

[35]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[37]  F. Stentiford.  An attention based similarity measure with application to content-based information retrieval , 2002 .

[38]  Joo-Hwee Lim.  Photograph retrieval and classification by visual keywords and thesaurus , 2009, New Generation Computing.

[39]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[40]  Shin'ichi Satoh,et al.  Using Visual-Textual Mutual Information and Entropy for Inter-modal Document Indexing , 2007, ECIR.

[41]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[42]  James Ze Wang,et al.  RF/sup */IPF: a weighting scheme for multimedia information retrieval , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[43]  Jong-Hak Lee,et al.  Analyses of multiple evidence combination , 1997, SIGIR '97.

[44]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Stéphane Ayache,et al.  Classifier Fusion for SVM-Based Multimedia Semantic Indexing , 2007, ECIR.

[46]  Iadh Ounis,et al.  Photograph indexing and retrieval using star-graphs , 2003 .

[47]  Bernard Mérialdo,et al.  Comparison of multi-episode video summarisation algorithms , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).