Automatic image semantic interpretation using social action and tagging data

The plethora of social actions and annotations (tags, comments, ratings) from online media sharing Websites and collaborative games have induced a paradigm shift in the research on image semantic interpretation. Social inputs with their added context represent a strong substitute for expert annotations. Novel algorithms have been designed to fuse visual features with noisy social labels and behavioral signals. In this survey, we review nearly 200 representative papers to identify the current trends, challenges as well as opportunities presented by social inputs for research on image semantics. Our study builds on an interdisciplinary confluence of insights from image processing, data mining, human computer interaction, and sociology to describe the folksonomic features of users, annotations and images. Applications are categorized into four types: concept semantics, person identification, location semantics and event semantics. The survey concludes with a summary of principle research directions for the present and the future.

[1]  John Riedl,et al.  The quest for quality tags , 2007, GROUP.

[2]  Simon King,et al.  Towards context-aware face recognition , 2005, MULTIMEDIA '05.

[3]  Stephen E. Robertson,et al.  Rethinking the ESP game , 2009, CHI Extended Abstracts.

[4]  Sourav S. Bhowmick,et al.  Image tag clarity: in search of visual-representative tags for social images , 2009, WSM@MM.

[5]  Mor Naaman,et al.  ZoneTag's Collaborative Tag Suggestions: What is This Person Doing in My Phone? , 2008, IEEE MultiMedia.

[6]  James Surowiecki.  The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations Doubleday Books. , 2004 .

[7]  Xin Chen,et al.  Tag dictionary and its applications , 2010, MIR '10.

[8]  Shonali Krishnaswamy,et al.  Mining data streams: a review , 2005, SGMD.

[9]  Eugene Agichtein,et al.  A few bad votes too many?: towards robust ranking in social media , 2008, AIRWeb '08.

[10]  James Ze Wang,et al.  Real-Time Computerized Annotation of Pictures , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Mor Naaman,et al.  How flickr helps us make sense of the world: context and content in community-contributed media collections , 2007, ACM Multimedia.

[12]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[13]  Mor Naaman,et al.  From Where to What: Metadata Sharing for Digital Photographs with Geographic Coordinates , 2003, OTM.

[14]  James Ze Wang,et al.  Tagging over time: real-world image annotation by lightweight meta-learning , 2007, ACM Multimedia.

[15]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[16]  James D. Hamilton.  Time Series Analysis , 1994 .

[17]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[18]  Natasha Gelfand,et al.  Visual summaries of popular landmarks from community photo collections , 2009 .

[19]  R. Nisbett,et al.  Culture and systems of thought: holistic versus analytic cognition. , 2001, Psychological review.

[20]  Peter Pirolli,et al.  An elementary social information foraging model , 2009, CHI.

[21]  Stefanie N. Lindstaedt,et al.  Recommending Tags for Pictures Based on Text, Visual Content and User Context , 2008, 2008 Third International Conference on Internet and Web Applications and Services.

[22]  Rong Yan,et al.  Hybrid Tagging and Browsing Approaches for Efficient Manual Image Annotation , 2009, IEEE MultiMedia.

[23]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[24]  Ling Chen,et al.  Using Co-occurence of Tags and Resources to Identify Spammers , 2008 .

[25]  Kilian Q. Weinberger,et al.  Reliable tags using image similarity: mining specificity and expertise from large-scale multimedia databases , 2009, WSMC '09.

[26]  Dong Liu,et al.  Smart batch tagging of photo albums , 2009, MM '09.

[27]  VARUN CHANDOLA,et al.  Outlier Detection : A Survey , 2007 .

[28]  Pinar Duygulu Sahin,et al.  Tag Suggestr: Automatic Photo Tag Expansion Using Visual Information for Photo Sharing Websites , 2008, SAMT.

[29]  Bernardo A. Huberman,et al.  Usage patterns of collaborative tagging systems , 2006, J. Inf. Sci..

[30]  Hinrich Schütze,et al.  Introduction to Information Retrieval: Index compression , 2008 .

[31]  Oded Nov,et al.  Why do people tag? , 2010, Commun. ACM.

[32]  Ja-Ling Wu,et al.  SheepDog: group and tag recommendation for flickr photos by automatic search-based learning , 2008, ACM Multimedia.

[33]  Wei Dong,et al.  Cultural difference in image tagging , 2010, CHI.

[34]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[35]  I. Kompatsiaris,et al.  Leveraging social media for training object detectors , 2009, 2009 16th International Conference on Digital Signal Processing.

[36]  Jure Leskovec,et al.  Statistical properties of community structure in large social and information networks , 2008, WWW.

[37]  Patrick F. Reidy.  An Introduction to Latent Semantic Analysis , 2009 .

[38]  Qi Tian,et al.  What are the high-level concepts with small semantic gaps? , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Scott A. Golder.  Measuring social networks with digital photograph collections , 2008, Hypertext.

[40]  Claudio Gutierrez,et al.  Survey of graph database models , 2008, CSUR.

[41]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[42]  James Allan,et al.  Topic detection and tracking: event-based information organization , 2002 .

[43]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[44]  Jiebo Luo,et al.  Mining GPS traces and visual words for event classification , 2008, MIR '08.

[45]  Wai-Tat Fu.  The microstructures of social tagging: a rational model , 2008, CSCW.

[46]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[47]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[48]  Anthony Hoogs,et al.  Evaluation of Localized Semantics: Data, Methodology, and Experiments , 2008, International Journal of Computer Vision.

[49]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[50]  Roberto Navigli,et al.  Word sense disambiguation: A survey , 2009, CSUR.

[51]  Ingmar Weber,et al.  Personalized, interactive tag recommendation for flickr , 2008, RecSys '08.

[52]  Susan T. Dumais,et al.  Improving Web Search Ranking by Incorporating User Behavior Information , 2019, SIGIR Forum.

[53]  Alexander Tuzhilin,et al.  Towards the Next Generation of Recommender Systems , 2010, ICE-B 2010.

[54]  Lei Zhang,et al.  IGroup: presenting web image search results in semantic clusters , 2007, CHI.

[55]  David R. Karger,et al.  Scatter/Gather: a cluster-based approach to browsing large document collections , 1992, SIGIR '92.

[56]  Hans-Peter Kriegel,et al.  Knowledge Discovery in Spatial Databases , 1999, DAGM-Symposium.

[57]  Latifur Khan,et al.  Image annotations by combining multiple evidence & wordNet , 2005, ACM Multimedia.

[58]  Susan T. Dumais,et al.  The vocabulary problem in human-system communication , 1987, CACM.

[59]  Graeme Hirst,et al.  Evaluating WordNet-based Measures of Lexical Semantic Relatedness , 2006, CL.

[60]  Jiebo Luo,et al.  Annotating collections of photos using hierarchical event and scene models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Reiner Lenz,et al.  Emotion related structures in large image databases , 2010, CIVR '10.

[62]  Shuicheng Yan,et al.  Inferring semantic concepts from community-contributed images and noisy tags , 2009, ACM Multimedia.

[63]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[64]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[65]  Eugene Agichtein,et al.  Learning to recognize reliable users and content in social media with coupled mutual reinforcement , 2009, WWW '09.

[66]  Shih-Fu Chang,et al.  To search or to label?: predicting the performance of search-based automatic image classifiers , 2006, MIR '06.

[67]  Marcel Ausloos,et al.  Contextualising tags in collaborative tagging systems , 2009, HT '09.

[68]  Mor Naaman,et al.  World explorer: visualizing aggregate data from unstructured text in geo-referenced collections , 2007, JCDL '07.

[69]  Ben Shneiderman,et al.  Motivating Annotation for Digital Photographs: Lowering Barriers While Raising Incentives (2004) , 2005 .

[70]  Yong Man Ro,et al.  Face annotation for personal photos using context-assisted face recognition , 2008, MIR '08.

[71]  Jiawei Han,et al.  Efficient and Effective Clustering Methods for Spatial Data Mining , 1994, VLDB.

[72]  Yuxiao Hu,et al.  Efficient propagation for face annotation in family albums , 2004, MULTIMEDIA '04.

[73]  Jiebo Luo,et al.  Annotating photo collections by label propagation according to multiple similarity cues , 2008, ACM Multimedia.

[74]  Yiannis Kompatsiaris,et al.  SEMSOC: SEMantic, SOcial and Content-Based Clustering in Multimedia Collaborative Tagging Systems , 2008, 2008 IEEE International Conference on Semantic Computing.

[75]  Benjamin Z. Yao,et al.  Introduction to a Large-Scale General Purpose Ground Truth Database: Methodology, Annotation Tool and Benchmarks , 2007, EMMCVPR.

[76]  Fred Stentiford,et al.  Using context and similarity for face and location identification , 2006, Electronic Imaging.

[77]  John M. Carroll,et al.  Enhancing information scent: identifying and recommending quality tags , 2009, GROUP.

[78]  Andreas Hotho,et al.  Information Retrieval in Folksonomies: Search and Ranking , 2006, ESWC.

[79]  Christoph Meinel,et al.  Telling experts from spammers: expertise ranking in folksonomies , 2009, SIGIR.

[80]  Larry S. Davis,et al.  Detection and analysis of hair , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  John Adcock,et al.  Leveraging face recognition technology to find and organize photos , 2004, MIR '04.

[82]  Andreas Hotho,et al.  Tag Recommendations in Folksonomies , 2007, LWA.

[83]  P. Pirolli.  Information Foraging Theory: Adaptive Interaction with Information , 2007 .

[84]  Wei-Ying Ma,et al.  Optimizing web search using web click-through data , 2004, CIKM '04.

[85]  Ramesh C. Jain,et al.  Automatic Person Annotation of Family Photo Album , 2006, CIVR.

[86]  Elisabeth Sylvan.  Predicting influence in an online community of creators , 2010, CHI.

[87]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[88]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[89]  Nuno Correia,et al.  A gesture based game for image tagging , 2008, CHI Extended Abstracts.

[90]  Daniel Gatica-Perez,et al.  Analyzing Flickr groups , 2008, CIVR '08.

[91]  Ebroul Izquierdo,et al.  Semantic structuring and retrieval of event chapters in social photo collections , 2010, MIR '10.

[92]  Wei-Ying Ma,et al.  AnnoSearch: Image Auto-Annotation by Search , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[93]  Myra Spiliopoulou,et al.  On exploiting the power of time in data mining , 2008, SKDD.

[94]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[95]  Mor Naaman,et al.  Towards automatic extraction of event and place semantics from flickr tags , 2007, SIGIR.

[96]  James Ze Wang,et al.  Quest for relevant tags using local interaction networks and visual content , 2010, MIR '10.

[97]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[98]  Jiebo Luo,et al.  Inferring generic activities and events from image content and bags of geo-tags , 2008, CIVR '08.

[99]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[100]  Valentin Robu,et al.  The complex dynamics of collaborative tagging , 2007, WWW '07.

[101]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[102]  Jiawei Han,et al.  Spatial Data Mining: Progress and Challenges , 1996, Workshop on Research Issues on Data Mining and Knowledge Discovery.

[103]  Wei-Ying Ma,et al.  Hierarchical clustering of WWW image search results using visual, textual and link information , 2004, MULTIMEDIA '04.

[104]  Stephen C. Y. Lu,et al.  Collaborative Tagging Applications and Approaches , 2008, IEEE MultiMedia.

[105]  Eyke Hüllermeier,et al.  Case-Based Multilabel Ranking , 2007, IJCAI.

[106]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.

[107]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[108]  Stefanie N. Lindstaedt,et al.  Automatic image annotation using visual content and folksonomies , 2009, Multimedia Tools and Applications.

[109]  Yinghui Xu,et al.  Automatic image tagging as a random walk with priors on the canonical correlation subspace , 2008, MIR '08.

[110]  Peter Pirolli,et al.  Remembrance of things tagged: how tagging effort affects tag production and human memory , 2009, CHI.

[111]  Kilian Q. Weinberger,et al.  Resolving tag ambiguity , 2008, ACM Multimedia.

[112]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[113]  John Riedl,et al.  Learning to recognize valuable tags , 2009, IUI.

[114]  Richard Szeliski,et al.  Finding People in Repeated Shots of the Same Scene , 2006, BMVC.

[115]  Ramesh C. Jain,et al.  Classification and annotation of digital photos using optical context data , 2008, CIVR '08.

[116]  Mathias Lux,et al.  An exploratory study on joint analysis of visual classification in narrow domains and the discriminative power of tags , 2008, MS '08.

[117]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[118]  Lillian Lee,et al.  Measures of Distributional Similarity , 1999, ACL.

[119]  Dimitrios Gunopulos,et al.  Identifying similarities, periodicities and bursts for online search queries , 2004, SIGMOD '04.

[120]  John F. Roddick,et al.  Higher order mining , 2008, SKDD.

[121]  Yang Song,et al.  Context-Aided Human Recognition - Clustering , 2006, ECCV.

[122]  Nuria Oliver,et al.  The role of tags and image aesthetics in social image search , 2009, WSM '09.

[123]  Mor Naaman,et al.  HT06, tagging paper, taxonomy, Flickr, academic article, to read , 2006, HYPERTEXT '06.

[124]  B. Shneiderman,et al.  Motivating Annotation for Personal Digital Photo Libraries : Lowering Barriers While Raising Incentives , 2022 .

[125]  P. Kollock,et al.  Communities in Cyberspace , 2002 .

[126]  Gang Wang,et al.  Web 2.0 dictionary , 2008, CIVR '08.

[127]  Susan T. Dumais,et al.  Statistical semantics: analysis of the potential performance of keyword information systems , 1984 .

[128]  Ted Pedersen,et al.  WordNet::Similarity - Measuring the Relatedness of Concepts , 2004, NAACL.

[129]  Pavel Serdyukov,et al.  Placing flickr photos on a map , 2009, SIGIR.

[130]  R. Manmatha,et al.  Automatic Image Annotation and Retrieval using CrossMedia Relevance Models , 2003 .

[131]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[132]  Mor Naaman,et al.  Generating summaries and visualization for large collections of geo-referenced photographs , 2006, MIR '06.

[133]  Simone Santini,et al.  Emergent Semantics through Interaction in Image Databases , 2001, IEEE Trans. Knowl. Data Eng..

[134]  Yiming Yang,et al.  A study of retrospective and on-line event detection , 1998, SIGIR '98.

[135]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[136]  Gennady L. Andrienko,et al.  Exploratory spatio-temporal visualization: an analytical review , 2003, J. Vis. Lang. Comput..

[137]  Ed H. Chi,et al.  Understanding the efficiency of social tagging systems using information theory , 2008, ICWSM.

[138]  Paul M. B. Vitányi,et al.  The Google Similarity Distance , 2004, IEEE Transactions on Knowledge and Data Engineering.

[139]  James Ze Wang,et al.  ACQUINE: aesthetic quality inference engine - real-time automatic rating of photo aesthetics , 2010, MIR '10.

[140]  Ciro Cattuto,et al.  Semantic Analysis of Tag Similarity Measures in Collaborative Tagging Systems , 2008, LWA.

[141]  Benjamin B. Bederson,et al.  Semi-automatic photo annotation strategies using event based clustering and clothing based person recognition , 2007, Interact. Comput..

[142]  Yair Weiss,et al.  Learning object detection from a small number of examples: the importance of good features , 2004, CVPR 2004.

[143]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[144]  Carla E. Brodley,et al.  Class Noise Mitigation Through Instance Weighting , 2007, ECML.

[145]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[146]  Helen Ashman,et al.  Co-active intelligence for image retrieval , 2005, MULTIMEDIA '05.

[147]  Marcel Worring,et al.  Learning tag relevance by neighbor voting for social image retrieval , 2008, MIR '08.

[148]  Alessandro Perina,et al.  Geo-located image analysis using latent representations , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[149]  Dragomir Anguelov,et al.  Contextual Identity Recognition in Personal Photo Albums , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[150]  Vittorio Loreto,et al.  Network properties of folksonomies , 2007, AI Commun..

[151]  John Riedl,et al.  tagging, communities, vocabulary, evolution , 2006, CSCW '06.

[152]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[153]  Dinan Gunawardena,et al.  Social tags: meaning and suggestions , 2008, CIKM '08.

[154]  Oded Nov,et al.  What drives content tagging: the case of photos on Flickr , 2008, CHI.

[155]  Meng Wang,et al.  Visual tag dictionary: interpreting tags with visual words , 2009, WSMC '09.

[156]  Stefanie Nowak,et al.  How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation , 2010, MIR '10.

[157]  Jiebo Luo,et al.  Photo classification by integrating image content and camera metadata , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[158]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[159]  Dong Liu,et al.  Tag quality improvement for social images , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[160]  Wei-Ying Ma,et al.  Image annotation by large-scale content-based image retrieval , 2006, MM '06.

[161]  Ebroul Izquierdo,et al.  An interactive framework for image annotation through gaming , 2010, MIR '10.

[162]  David G. Stork,et al.  Pattern Classification , 1973 .

[163]  B. S. Manjunath,et al.  Spirittagger: a geo-aware tag suggestion tool mined from flickr , 2008, MIR '08.

[164]  Mor Naaman,et al.  Leveraging context to resolve identity in photo albums , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[165]  Changhu Wang,et al.  Content-Based Image Annotation Refinement , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[166]  Edmund A. Mennis.  The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations , 2006 .

[167]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[168]  Dong Liu,et al.  Tag ranking , 2009, WWW '09.

[169]  Bamshad Mobasher,et al.  Personalized recommendation in social tagging systems using hierarchical clustering , 2008, RecSys '08.

[170]  Laura A. Dabbish,et al.  Designing games with a purpose , 2008, CACM.

[171]  Georgia Koutrika,et al.  Combating spam in tagging systems , 2007, AIRWeb '07.

[172]  Nenghai Yu,et al.  Learning to tag , 2009, WWW '09.

[173]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[174]  W. P. Jones.  The memory extender personal filing system , 1986, CHI '86.

[175]  Nenghai Yu,et al.  Flickr distance , 2008, ACM Multimedia.

[176]  Gang Wang,et al.  iRIN: image retrieval in image-rich information networks , 2010, WWW '10.

[177]  Luis von Ahn.  Games with a Purpose , 2006, Computer.

[178]  Thomas Hofmann,et al.  Unsupervised Learning by Probabilistic Latent Semantic Analysis , 2004, Machine Learning.

[179]  Tamara L. Berg,et al.  Automatic Ranking of Iconic Images , 2007 .

[180]  Ravi Kumar,et al.  Visualizing tags over time , 2006, WWW '06.

[181]  Padhraic Smyth,et al.  Learning to detect events with Markov-modulated poisson processes , 2007, TKDD.

[182]  Lexing Xie,et al.  Contextual wisdom: social relations and correlations for multimedia event annotation , 2007, ACM Multimedia.

[183]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[184]  Jaime G. Carbonell,et al.  Efficiently learning the accuracy of labeling sources for selective sampling , 2009, KDD.

[185]  Touradj Ebrahimi,et al.  Object-based tag propagation for semi-automatic annotation of images , 2010, MIR '10.

[186]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[187]  John F. Roddick,et al.  A Survey of Temporal Knowledge Discovery Paradigms and Methods , 2002, IEEE Trans. Knowl. Data Eng..

[188]  Ling Chen,et al.  Event detection from flickr data through wavelet-based spatial analysis , 2009, CIKM.

[189]  Cordelia Schmid,et al.  Image annotation with tagprop on the MIRFLICKR set , 2010, MIR '10.

[190]  Changhu Wang,et al.  Image annotation refinement using random walk with restarts , 2006, MM '06.

[191]  Wei-Ying Ma,et al.  Learning to cluster web search results , 2004, SIGIR '04.

[192]  David A. Forsyth,et al.  Utility data annotation with Amazon Mechanical Turk , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[193]  John F. Roddick,et al.  Geographic Data Mining and Knowledge Discovery , 2001 .

[194]  Manuel Blum,et al.  Peekaboom: a game for locating objects in images , 2006, CHI.

[195]  Pushpak Bhattacharyya,et al.  A model for handling approximate, noisy or incomplete labeling in text classification , 2005, ICML.

[196]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[197]  Alessandro Perina,et al.  Content visualization and management of geo-located image databases , 2008, CHI Extended Abstracts.

[198]  Jakob J. Verbeek,et al.  Ranking User-annotated Images for Multiple Query Terms , 2009, BMVC.

[199]  Andreas Hotho,et al.  FolkRank : A Ranking Algorithm for Folksonomies , 2006, LWA.