Semi-Automatic Image Annotation

A novel approach to semi-automatically and progressively annotating images with keywords is presented. The progressive annotation process is embedded in the course of integrated keyword-based and content-based image retrieval and user feedback. When the user submits a keyword query and then provides relevance feedback, the search keywords are automatically added to the images that receive positive feedback and can then facilitate keyword-based image retrieval in the future. The coverage and quality of image annotation in such a database system is improved progressively as the cycle of search and feedback increases. The strategy of semi-automatic image annotation is better than manual annotation in terms of efficiency and better than automatic annotation in terms of accuracy. A performance study is presented which shows that high annotation coverage can be achieved with this approach, and a preliminary user study is described showing that users view annotations as important and will likely use them in image retrieval. The user study also suggested user interface enhancements needed to support relevance feedback. We believe that similar approaches could also be applied to annotating and managing other forms of multimedia objects.

[1]  Ramesh C. Jain,et al.  Similarity measures for image databases , 1995, Electronic Imaging.

[2]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[3]  Thomas S. Huang,et al.  A novel relevance feedback technique in image retrieval , 1999, MULTIMEDIA '99.

[4]  Yihong Gong,et al.  An image database system with content capturing and fast image indexing abilities , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[5]  Benjamin B. Bederson,et al.  Does zooming improve image browsing? , 1999, DL '99.

[6]  Donna K. Harman,et al.  Relevance Feedback and Other Query Modification Techniques , 1992, Information retrieval (Boston).

[7]  Ben Shneiderman,et al.  Direct annotation: a drag-and-drop strategy for labeling photos , 2000, 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics.

[8]  Nicholas J. Belkin,et al.  A case for interaction: a study of interactive information retrieval behavior and effectiveness , 1996, CHI.

[9]  Kerry Rodden.  How Do People Organise Their Photographs? , 1999, BCS-IRSG Annual Colloquium on IR Research.

[10]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[11]  Beng Chin Ooi,et al.  Giving meanings to WWW images , 2000, ACM Multimedia.

[12]  L. R. Rasmussen,et al.  In information retrieval: data structures and algorithms , 1992 .

[13]  Ingemar J. Cox,et al.  PicHunter: Bayesian relevance feedback for image retrieval , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[14]  Nuno Vasconcelos,et al.  Learning from User Feedback in Image Retrieval Systems , 1999, NIPS.

[15]  Masao Sakauchi,et al.  A flexible content-based image retrieval system with combined scene description keyword , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[16]  Beng Chin Ooi,et al.  Giving meanings to WWW images , 2000, MM 2000.

[17]  Myron Flickner,et al.  Query by Image and Video Content , 1995 .

[18]  Thomas S. Huang,et al.  Optimizing learning in image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).