Personal photo album summarization for global and local photo annotation

Although content-based media retrieval tools are continuously improving, personalized image annotation is still one of the most reliable ways to index large image archives. Unfortunately, it is also a time consuming and repetitive operation. Using content to facilitate the user in media annotation may lead to reduced effort and more accurate results. In this paper we propose a content-based interactive tool that supports a user in annotating his personal photo albums. The system provides two main functionalities: to summarize a photo collection in salient moments, and to annotate pictures in a semi-supervised way based on their global and local content. The summarization is based on a bottom-up unsupervised hierarchical clustering that exploits two different matrices of visual distances, while the local tagging uses an object retrieval method based on local image features. Experiments on personal photo collections show that the proposed technique produces good results in terms of organization and access to data.