Towards a Comprehensive Survey of the Semantic Gap in Visual Image Retrieval

This paper adopts the premise that the 'semantic gap' is an incompletely surveyed feature in the landscape of visual image retrieval, and proposes a framework within which this deficiency might be made good. Simple classifications of types of image and of types of user are proposed. Consideration is then given in outline to how semantic content is realised by each class of user within each class of image. The argument is advanced that this realisation finds expression in perceptual, generic interpretive and specific interpretive content. This analytic framework provides the basis for the specification of a broadly encompassing evaluation study, which will employ the image/user type classification and the expert domain knowledge of selected user groups in the construction of segmented test collections of real queries, images and relevance judgements. From this study should come a better-informed view on the nature of semantic information need, and on the representation and recovery of semantic content across a broad spectrum of image retrieval activity.

[1]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[2]  M. Krause.  Intellectual problems of indexing picture collections , 1988 .

[3]  Maurice B. Line,et al.  PROGRESS IN DOCUMENTATION: ‘obsolescence’ and changes in the use of literature with time , 1974 .

[4]  Samantha Kelly Hastings,et al.  Evaluation of Image Retrieval Systems: Role of User Feedback , 1999, Libr. Trends.

[5]  Raya Fidel,et al.  The image retrieval task: implications for the design and evaluation of image databases , 1997, New Rev. Hypermedia Multim..

[6]  Susanne Ornager.  The newspaper image database: empirical supported analysis of users' typology and word association clusters , 1995, SIGIR '95.

[7]  Samantha Kelly Hastings,et al.  Query Categories in a Study of Intellectual Access to Digitized Art Images. , 1995 .

[8]  Peter G. B. Enser,et al.  VIRAMI - Visual Information Retrieval for Archival Moving Imagery , 2001, ICHIM.

[9]  Eero Sormunen,et al.  End-User Searching Challenges Indexing Practices in the Digital Newspaper Photo Archive , 2004, Information Retrieval.

[10]  Peter G. B. Enser,et al.  Analysis of user need in image archives , 1997, J. Inf. Sci..

[11]  J. P. Eakins.  Design criteria for a shape retrieval system , 1993 .

[12]  Babu M. Mehtre,et al.  Content-based retrieval for trademark registration , 1996, Multimedia Tools and Applications.

[13]  Wei-Ying Ma,et al.  Image and Video Retrieval , 2003, Lecture Notes in Computer Science.

[14]  R. Barthes.  Elements of Semiology , 1967 .

[15]  M. Markkula,et al.  Searching for Photos - Journalists' Practices in Pictorial IR , 1998 .

[16]  Herbert Coblans,et al.  Progress in Documentation. , 1972 .

[17]  Brian C. O'Connor,et al.  Modelling what users see when they look at images: a cognitive viewpoint , 2002, J. Documentation.

[18]  Peter G. B. Enser,et al.  Progress in Documentation Pictorial Information Retrieval , 1995, J. Documentation.

[19]  Corinne Jörgensen,et al.  Indexing Images: Testing an Image Description Template. , 1996 .

[20]  Peter G. B. Enser.  Pictorial information retrieval , 1995 .

[21]  Peter G. B. Enser.  Pictorial Information Retrieval (Progress in Documentation) , 1995 .

[22]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[23]  Peter G. B. Enser,et al.  Retrieval of Archival Moving Imagery - CBIR Outside the Frame? , 2002, CIVR.

[24]  Progress in Visual Information Access and Retrieval , 1999 .

[25]  John P. Eakins,et al.  Similarity Retrieval of Trademark Images , 1998, IEEE Multim..

[26]  Jack Kranz Mls.  Enhanced Access to Pamphlets , 1985 .

[27]  Sara Shatford,et al.  Analyzing the Subject of a Picture: A Theoretical Approach , 1986 .