SmartAlbum: a multi-modal photo annotation system

This demonstration presents a novel application (called SmartAlbum) for photo indexing and retrieval that unifies two different image indexing approaches. The system uses two modalities to extract information about a digital photograph; i.e. content-based and speech annotation for image description. The result is a powerful image retrieval tool that has capabilities beyond what current single-mode retrieval systems can offer. We show on a corpus of 1200 images the interest of our approach.

[1]  Philippe Mulhem,et al.  A Method for Photograph Indexing Using Speech Annotation , 2001, IEEE Pacific Rim Conference on Multimedia.

[2]  Philippe Mulhem,et al.  Image query system using object probes , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[3]  Carla E. Brodley,et al.  Testing for human perceptual categories in a physician-in-the-loop CBIR system for medical imagery , 1999, Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL'99).

[4]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[5]  Michael L. Creech,et al.  FotoFile: a consumer multimedia organization and retrieval system , 1999, CHI '99.

[6]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[7]  Alexander C. Loui,et al.  Using event segmentation to improve indexing of consumer photographs , 2001, SIGIR '01.

[8]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Electronic Imaging.