Classification of user image descriptions

In order to resolve the mismatch between user needs and current image retrieval techniques, we conducted a study to get more information about what users look for in images. First, we developed a framework for the classification of image descriptions by users, based on various classification methods from the literature. The classification framework distinguishes three related viewpoints on images, namely nonvisual metadata, perceptual descriptions and conceptual descriptions. For every viewpoint a set of descriptive classes and relations is specified. We used the framework in an empirical study, in which image descriptions were formulated by 30 participants. The resulting descriptions were split into fragments and categorized in the framework. The results suggest that users prefer general descriptions as opposed to specific or abstract descriptions. Frequently used categories were objects, events and relations between objects in the image.

[1]  P. Bryan Heidorn,et al.  The identification of index terms in natural language object descriptions , 1999 .

[2]  T. Wickens.  Multiway Contingency Tables Analysis for the Social Sciences , 1989 .

[3]  John P. Eakins,et al.  Towards intelligent image retrieval , 2002, Pattern Recognit..

[4]  Erwin Panofsky,et al.  Studies in Iconology , 1962 .

[5]  Shih-Fu Chang,et al.  Conceptual framework for indexing visual information at multiple levels , 1999, Electronic Imaging.

[6]  Konrad Saur,et al.  IFLA Study Group on the Functional Requirements for Bibliographic Records , 1998 .

[7]  Amanda Spink,et al.  Image searching on the Excite Web search engine , 2001, Inf. Process. Manag..

[8]  C. Olivia Frost,et al.  Browsing images using broad classification categories , 1998 .

[9]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Lyn Condron,et al.  Functional Requirements for Bibliographic Records , 2004 .

[11]  Patrick Le Boeuf,et al.  Functional Requirements for Bibliographic Records , 2005 .

[12]  Peter G. B. Enser,et al.  Analysis of user need in image archives , 1997, J. Inf. Sci..

[13]  Pertti Vakkari,et al.  Cognition and changes of search terms and tactics during task performance: A longitudinal case study , 2000, RIAO.

[14]  Hsin-Liang Chen,et al.  An analysis of image queries in the field of art history , 2001, J. Assoc. Inf. Sci. Technol..

[15]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[16]  Bob J. Wielinga,et al.  Ontology-Based Photo Annotation , 2001, IEEE Intell. Syst..

[17]  Ivar Jacobson,et al.  The Unified Modeling Language User Guide , 1998, J. Database Manag..

[18]  Peter Stanchev,et al.  Content-Based Image Retrieval Systems , 2001 .

[19]  Amanda Spink,et al.  Searching for multimedia: analysis of audio, video and image Web queries , 2000, World Wide Web.

[20]  Sara Shatford,et al.  Analyzing the Subject of a Picture: A Theoretical Approach , 1986 .

[21]  Raya Fidel,et al.  The image retrieval task: implications for the design and evaluation of image databases , 1997, New Rev. Hypermedia Multim..

[22]  Susanne Ornager.  The newspaper image database: empirical supported analysis of users' typology and word association clusters , 1995, SIGIR '95.

[23]  Corinne Jörgensen,et al.  Attributes of Images in Describing Tasks , 1998, Inf. Process. Manag..

[24]  J. Eakins.  Techniques for image retrieval , 1998 .

[25]  Edie M. Rasmussen,et al.  Users' relevance criteria in image retrieval in American history , 2002, Inf. Process. Manag..

[26]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .