Image semantics in the description and categorization of journalistic photographs

This paper reports a study on the description and categorization of images. The aim of the study was to evaluate existing indexing frameworks in the context of reportage photographs and to find out how the use of this particular image genre influences the results. The effect of different tasks on image description and categorization was also studied. Subjects performed keywording and free description tasks and the elicited terms were classified using the most extensive one of the reviewed frameworks. Differences were found in the terms used in constrained and unconstrained descriptions. Summarizing terms such as abstract concepts, themes, settings and emotions were used more frequently in keywording than in free description. Free descriptions included more terms referring to locations within the images, people and descriptive terms due to the narrative form the subjects used without prompting. The evaluated framework was found to lack some syntactic and semantic classes present in the data and modifications were suggested. According to the results of this study image categorization is based on high-level interpretive concepts, including affective and abstract themes. The results indicate that image genre influences categorization and keywording modifies and truncates natural image description.

[1]  Raya Fidel,et al.  The image retrieval task: implications for the design and evaluation of image databases , 1997, New Rev. Hypermedia Multim..

[2]  Kenneth Kobre,et al.  Photojournalism: The Professionals' Approach , 1980 .

[3]  Shih-Fu Chang,et al.  Conceptual framework for indexing visual information at multiple levels , 1999, Electronic Imaging.

[4]  Kalervo Järvelin,et al.  The Perceived Similarity of Photos - A Test-Collection Based Evaluation Framework for the Content-Based Image Retrieval Algorithms , 1999, MIRA.

[5]  Neff Walker,et al.  Classifying visual knowledge representations: a foundation for visualization research , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[6]  Aleksandra Mojsilovic,et al.  Psychophysical approach to modeling image semantics , 2001, IS&T/SPIE Electronic Imaging.

[7]  Neff Walker,et al.  A classification of visual representations , 1994, CACM.

[8]  Aleksandra Mojsilovic,et al.  Capturing image semantics with low-level descriptors , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[9]  Corinne Jörgensen,et al.  Attributes of Images in Describing Tasks , 1998, Inf. Process. Manag..

[10]  Corinne Joergensen.  Retrieving the unretrievable in electronic imaging systems: emotions, themes, and stories , 1999, Electronic Imaging.

[11]  Brian C. O'Connor,et al.  Modelling what users see when they look at images: a cognitive viewpoint , 2002, J. Documentation.

[12]  Susanne Ornager.  Image Retrieval: Theoretical Analysis and Empirical User Studies on Accessing Information in Images. , 1997 .

[13]  M. Markkula,et al.  Searching for Photos - Journalists' Practices in Pictorial IR , 1998 .

[14]  Pamela Briggs,et al.  Image Retrieval Interfaces: A User Perspective , 2004, CIVR.

[15]  Samantha K. Hastings,et al.  Free sorting of images: Attributes used for categorization , 2004, ASIST.

[16]  Marcel Worring,et al.  Classification of user image descriptions , 2004, Int. J. Hum. Comput. Stud..

[17]  S. Süsstrunk,et al.  Measuring colourfulness in natural images , 2003 .

[18]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..

[19]  Eero Sormunen,et al.  End-User Searching Challenges Indexing Practices in the Digital Newspaper Photo Archive , 2004, Information Retrieval.

[20]  Sabine Süsstrunk,et al.  Measuring colorfulness in natural images , 2003, IS&T/SPIE Electronic Imaging.

[21]  Charles A. Bouman,et al.  Perceptual image similarity experiments , 1998, Electronic Imaging.