Interactive tool for image annotation using a semi-supervised and hierarchical approach

This paper presents a semi-automatic tool, called IGAnn (Interactive image ANNotation), that assists users in annotating textual labels with images. IGAnn performs an interactive retrieval-like procedure: the system presents the user with images that have higher confidences, and then the user determines which images are actually relevant or irrelevant for a specified label. By collecting relevant and irrelevant images of iterations, a hierarchical classifier associated with the specified label is built using our proposed semi-supervised approach to compute confidence values of unlabeled images. This paper describes the system interface of IGAnn and also demonstrates quantitative experiments of our proposed approach.

[1]  R. Manmatha,et al.  Multiple Bernoulli relevance models for image and video annotation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[2]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[3]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[4]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[5]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[6]  Wei-Ying Ma,et al.  A probabilistic semantic model for image annotation and multi-modal image retrieval , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  James Ze Wang,et al.  Real-Time Computerized Annotation of Pictures , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[9]  Wei-Ying Ma,et al.  A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieva , 2005, ICCV.

[10]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[11]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  David G. Stork,et al.  Pattern Classification , 1973 .

[13]  Y. Mori,et al.  Image-to-word transformation based on dividing and vector quantizing images with words , 1999 .

[14]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[15]  Latifur Khan,et al.  Standards for image annotation using Semantic Web , 2007, Comput. Stand. Interfaces.

[16]  W. Bruce Croft,et al.  Relevance-Based Language Models , 2001, SIGIR '01.

[17]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[18]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[19]  Claudio Gutierrez,et al.  Survey of graph database models , 2008, CSUR.

[20]  Dan I. Moldovan,et al.  Exploiting ontologies for automatic image annotation , 2005, SIGIR '05.

[21]  Demin Wang,et al.  A multiscale gradient algorithm for image segmentation using watershelds , 1997, Pattern Recognit..

[22]  Ronald Rosenfeld,et al.  Semi-supervised learning with graphs , 2005 .

[23]  Bo Zhang,et al.  An efficient and effective region-based image retrieval framework , 2004, IEEE Transactions on Image Processing.

[24]  Christos Diou,et al.  Image annotation using clickthrough data , 2009, CIVR '09.

[25]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[26]  James Ze Wang,et al.  Content-based image retrieval: approaches and trends of the new age , 2005, MIR '05.

[27]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[28]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[29]  Xuming He,et al.  Learning structured prediction models for image labeling , 2008 .

[30]  Raymond J. Mooney,et al.  Integrating constraints and metric learning in semi-supervised clustering , 2004, ICML.