EasyAlbum: an interactive photo annotation system based on face clustering and re-ranking

Digital photo management is becoming indispensable for the explosively growing family photo albums due to the rapid popularization of digital cameras and mobile phone cameras. In an effective photo management system photo annotation is the most challenging task. In this paper, we develop several innovative interaction techniques for semi-automatic photo annotation. Compared with traditional annotation systems, our approach provides the following new features: "cluster annotation" puts similar faces or photos with similar scene together, and enables user label them in one operation; "contextual re-ranking" boosts the labeling productivity by guessing the user intention; "ad hoc annotation" allows user label photos while they are browsing or searching, and improves system performance progressively through learning propagation. Our results show that these technologies provide a more user friendly interface for the annotation of person name, location, and event, and thus substantially improve the annotation performance especially for a large photo album.

[1]  J. C. Platt.  AutoAlbum: clustering digital photographs using probabilistic model merging , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[2]  LONGBIN CHEN,et al.  Face Annotation for Family Photo Album Management , 2003, Int. J. Image Graph..

[3]  Mingjing Li,et al.  Automated annotation of human faces in family albums , 2003, MULTIMEDIA '03.

[4]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[5]  Michael I. Jordan,et al.  Learning Spectral Clustering , 2003, NIPS.

[6]  Xiaogang Wang,et al.  Random Sampling for Subspace Face Recognition , 2006, International Journal of Computer Vision.

[7]  Ben Shneiderman,et al.  Direct annotation: a drag-and-drop strategy for labeling photos , 2000, 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics.

[8]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[9]  Harry Shum,et al.  A Bayesian mixture model for multi-view face alignment , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Michael L. Creech,et al.  FotoFile: a consumer multimedia organization and retrieval system , 1999, CHI '99.

[11]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[13]  Benjamin B. Bederson,et al.  Semi-Automatic Image Annotation Using Event and Torso Identification , 2004 .

[14]  Xiaoou Tang,et al.  Imlooking: image-based face retrieval in online dating profile search , 2006, CHI EA '06.

[15]  Xiaogang Wang,et al.  A unified framework for subspace face recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  John Adcock,et al.  Leveraging face recognition technology to find and organize photos , 2004, MIR '04.

[17]  Yi Zhou,et al.  Bayesian tangent shape model: estimating shape and pose parameters via Bayesian inference , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  Andreas Paepcke,et al.  Time as essence for photo browsing through personal digital libraries , 2002, JCDL '02.

[19]  Kerry Rodden,et al.  How do people manage their digital photographs? , 2003, CHI '03.

[20]  B. K. Julsing,et al.  Face Recognition with Local Binary Patterns , 2012 .