User-generated descriptions of individual images versus labels of groups of images: A comparison using basic level theory

Although images are visual information sources with little or no text associated with them, users still tend to use text to describe images and formulate queries. This is because digital libraries and search engines provide mostly text query options and rely on text annotations for representation and retrieval of the semantic content of images. While the main focus of image research is on indexing and retrieval of individual images, the general topic of image browsing and indexing, and retrieval of groups of images has not been adequately investigated. Comparisons of descriptions of individual images as well as labels of groups of images supplied by users using cognitive models are scarce. This work fills this gap. Using the basic level theory as a framework, a comparison of the descriptions of individual images and labels assigned to groups of images by 180 participants in three studies found a marked difference in their level of abstraction. Results confirm assertions by previous researchers in LIS and other fields that groups of images are labeled using more superordinate level terms while individual image descriptions are mainly at the basic level. Implications for design of image browsing interfaces, taxonomies, thesauri, and similar tools are discussed.

[1]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[2]  Claire Kramsch,et al.  Language, Thought, and Culture , 2008 .

[3]  G. Lakoff,et al.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[4]  Corinne Jörgensen Image attributes: an investigation , 1995 .

[5]  James M. Turner Comparing User-Assigned Terms with Indexer-Assigned Terms for Storage and Retrieval of Moving Images: Research Results. , 1995 .

[6]  Eleanor H. Finch 65th Annual Meeting of the Society , 1971, American Journal of International Law.

[7]  Marti A. Hearst,et al.  Searching and browsing text collections with large category hierarchies , 1997, CHI Extended Abstracts.

[8]  Susan T. Dumais,et al.  Exploring personal information , 2006, Commun. ACM.

[9]  A. Tversky Features of Similarity , 1977 .

[10]  Hemalata Iyer,et al.  Classificatory structures: Concepts, relations and representation , 1995 .

[11]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[12]  E. Markman,et al.  Developmental differences in the acquisition of basic and superordinate categories. , 1980 .

[13]  G. Murphy,et al.  Categorizing objects in isolation and in scenes: what a superordinate is good for. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[14]  Sara Shatford Layne,et al.  Some Issues in the Indexing of Images , 1994, J. Am. Soc. Inf. Sci..

[15]  Anthony Peter Macmillan Coxon,et al.  Sorting Data: Collection and Analysis , 1999 .

[16]  Charles H. Davis American Society for Information Science , 1984 .

[17]  Eero Sormunen,et al.  End-User Searching Challenges Indexing Practices in the Digital Newspaper Photo Archive , 2004, Information Retrieval.

[18]  Mari Laine-Hernandez,et al.  Image semantics in the description and categorization of journalistic photographs , 2007, ASIST.

[19]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[20]  Erwin Panofsky,et al.  Meaning in the Visual Arts: Papers in and on Art History , 1955 .

[21]  Brian C. O'Connor,et al.  Modelling what users see when they look at images: a cognitive viewpoint , 2002, J. Documentation.

[22]  James Ze Wang,et al.  Content-based image retrieval: approaches and trends of the new age , 2005, MIR '05.

[23]  Kevin Li,et al.  Faceted metadata for image search and browsing , 2003, CHI '03.

[24]  Dedre Gentner,et al.  Why Nouns Are Learned before Verbs: Linguistic Relativity Versus Natural Partitioning. Technical Report No. 257. , 1982 .

[25]  Hsin-Liang Chen,et al.  An analysis of image retrieval tasks in the field of art history , 2001, Inf. Process. Manag..

[26]  Sara Shatford,et al.  Analyzing the Subject of a Picture: A Theoretical Approach , 1986 .

[27]  Edie M. Rasmussen,et al.  Searching for images: The analysis of users' queries for image retrieval in American history , 2003, J. Assoc. Inf. Sci. Technol..

[28]  Brian C. O'Connor,et al.  What do users see? Exploring the cognitive nature of functional image retrieval , 2005, ASIST.

[29]  Hemalata Iyer,et al.  Theories of cognition and image categorization: What category labels reveal about basic level theory , 2008, J. Assoc. Inf. Sci. Technol..

[30]  Samantha Kelly Hastings,et al.  Query Categories in a Study of Intellectual Access to Digitized Art Images. , 1995 .

[31]  Marius Tico,et al.  A Test Collection for the Evaluation of Content-Based Image Retrieval Algorithms—A User and Task-Based Approach , 2001, Information Retrieval.

[32]  Ali Shiri,et al.  HILT: A Pilot Terminology Mapping Service with a DDC Spine , 2006 .

[33]  R. Weber Basic content analysis, 2nd ed. , 1990 .

[34]  Roger B. Wyatt,et al.  Photo Provocations: Thinking In, With, and About Photographs , 2004 .

[35]  James M. Turner,et al.  Determining the subject content of still and moving image documents for storage and retrieval : an experimental investigation , 1994 .

[36]  June Abbas,et al.  User Reactions as Access Mechanism: An Exploration Based on Captions for Images , 1999, J. Am. Soc. Inf. Sci..

[37]  J. Tanaka,et al.  Object categories and expertise: Is the basic level in the eye of the beholder? , 1991, Cognitive Psychology.

[38]  Besiki Stvilia,et al.  End-user collection building behavior in Flickr , 2008, ASIST.

[39]  Dirk Neumann,et al.  Image retrieval and perceptual similarity , 2006, TAP.

[40]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[41]  Krystyna K. Matusiak Towards user-centered indexing in digital image collections , 2006, OCLC Syst. Serv..

[42]  Abdus Sattar Chaudhry,et al.  Enhancing access to digital information resources on heritage: A case of development of a taxonomy at the Integrated Museum and Archives System in Singapore , 2005, J. Documentation.

[43]  Corinne Jörgensen Image Retrieval: Theory and Research , 2003 .

[44]  Edward J. Wisniewski,et al.  Superordinate and basic category names in discourse: A textual analysis , 1989 .

[45]  E. Rosch Cognitive Representations of Semantic Categories. , 1975 .

[46]  Amanda Spink,et al.  Web search engine multimedia functionality , 2008, Inf. Process. Manag..

[47]  Arthur B. Markman,et al.  Similar and Different: The Differentiation of Basic-Level Categories , 1997 .

[48]  Corinne Jörgensen,et al.  Attributes of Images in Describing Tasks , 1998, Inf. Process. Manag..

[49]  Raya Fidel User-centered indexing , 1994 .

[50]  Edie M. Rasmussen,et al.  Users' relevance criteria in image retrieval in American history , 2002, Inf. Process. Manag..

[51]  Marti A. Hearst Clustering versus faceted categories for information exploration , 2006, Commun. ACM.

[52]  Marcel Worring,et al.  Classification of user image descriptions , 2004, Int. J. Hum. Comput. Stud..

[53]  Peter G. B. Enser,et al.  Analysis of user need in image archives , 1997, J. Inf. Sci..

[54]  D. C. Blair,et al.  Language and Representation in Information Retrieval , 1990 .

[55]  Hsin-Liang Chen,et al.  An analysis of image queries in the field of art history , 2001, J. Assoc. Inf. Sci. Technol..

[56]  Corinne Jörgensen,et al.  Indexing Images: Testing an Image Description Template. , 1996 .

[57]  Peter G. B. Enser,et al.  Visual image retrieval: seeking the alliance of concept-based and content-based paradigms , 2000, J. Inf. Sci..

[58]  Ben Shneiderman,et al.  Find that photo! , 2006, Commun. ACM.

[59]  H. Brownell,et al.  Category differentiation in object recognition: typicality constraints on the basic category advantage. , 1985, Journal of experimental psychology. Learning, memory, and cognition.

[60]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[61]  C. Mervis,et al.  Order of acquisition of subordinate-, basic-, and superordinate-level categories. , 1982 .

[62]  Rebecca Green Vocabulary Alignment via Basic Level Concepts , 2006 .

[63]  Raya Fidel,et al.  The image retrieval task: implications for the design and evaluation of image databases , 1997, New Rev. Hypermedia Multim..

[64]  Shih-Fu Chang,et al.  A conceptual framework and empirical research for classifying visual descriptors , 2001, J. Assoc. Inf. Sci. Technol..

[65]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[67]  Dagobert Soergel,et al.  Organizing information - principles of data base and retrieval systems , 1985 .

[68]  Amanda Spink,et al.  Image searching on the Excite Web search engine , 2001, Inf. Process. Manag..

[69]  R. Weber Basic Content Analysis , 1986 .