Intelligent tagging systems: machine learning for novel interaction

The Web puts a vast repository of information at users' fingertips, but the size and complexity of this information space can easily overwhelm users. Recommender systems and tagging systems represent two very different approaches to addressing this information overload. Recommender systems use machine learning and statistical models to automatically retrieve the items of most interest to a particular user. Tagging systems leverage the community's collective knowledge to help users explore the information space themselves. While both approaches can be very effective, they each have limitations. Recommender systems require little effort from users, but they leave users with little control over the recommendation process. Tagging systems put control in the hands of the user, but—because tags are applied by humans—tagging systems often suffer from issues of tag sparsity. This thesis explores intelligent tagging systems that combine the machine intelligence of recommender systems with the user control and comprehensibility of tagging systems. We first present Tagsplanations, tag-based explanations that help users understand why an item was recommended to them. We then introduce the Tag Genome, a novel data structure that uses machine learning to augment human judgments of the relationships between tags and items. Next we discuss Movie Tuner, a conversational recommender system based on the Tag Genome that enables users to provide multifaceted feedback using tags. For each system, we outline the design space of the problem and discuss our design decisions. We evaluate each system using both offline analyses as well as field studies involving thousands of users from MovieLens, a movie recommender system that also supports tagging of movies. Finally, we draw conclusions for the broader space of related applications.

[1]  John Riedl,et al.  Tagsplanations: explaining recommendations using tags , 2009, IUI.

[2]  Kristian J. Hammond,et al.  Knowledge-Based Navigation of Complex Information Spaces , 1996, AAAI/IAAI, Vol. 1.

[3]  Pearl Pu,et al.  A Comparative Study of Compound Critique Generation in Conversational Recommender Systems , 2006, AH.

[4]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[5]  Paul Lamere,et al.  Generating transparent, steerable recommendations from textual descriptions of items , 2009, RecSys '09.

[6]  Shilad Sen,et al.  Rating: how difficult is it? , 2011, RecSys '11.

[7]  John Riedl,et al.  Tag expression: tagging with feeling , 2010, UIST.

[8]  Raymond J. Mooney,et al.  Explaining Recommendations: Satisfaction vs. Promotion , 2005 .

[9]  Li Chen,et al.  Evaluating Critiquing-based Recommender Agents , 2006, AAAI.

[10]  Martin Halvey,et al.  An assessment of tag presentation techniques , 2007, WWW '07.

[11]  Koji Yatani,et al.  Review spotlight: a user interface for summarizing user-generated reviews using adjective-noun word pairs , 2011, CHI.

[12]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[13]  Carl Gutwin,et al.  Seeing things in the clouds: the effect of visual features on tag cloud selections , 2008, Hypertext.

[14]  Andrew McCallum,et al.  A comparison of event models for naive bayes text classification , 1998, AAAI 1998.

[15]  Judith Masthoff,et al.  Effective explanations of recommendations: user-centered design , 2007, RecSys '07.

[16]  Andreas Hotho,et al.  Tag Recommendations in Folksonomies , 2007, LWA.

[17]  Barry Smyth,et al.  Experiments in dynamic critiquing , 2005, IUI.

[18]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[19]  Manfred Tscheligi,et al.  Semantically structured tag clouds: an empirical evaluation of clustered presentation approaches , 2009, CHI.

[20]  Thierry Bertin-Mahieux,et al.  Automatic Generation of Social Tags for Music Recommendation , 2007, NIPS.

[21]  Weng-Keen Wong,et al.  Fixing the program my computer learned: barriers for end users, challenges for the machine , 2009, IUI.

[22]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[23]  Kevin Li,et al.  Faceted metadata for image search and browsing , 2003, CHI '03.

[24]  Vittorio Loreto,et al.  Semiotic dynamics and collaborative tagging , 2006, Proceedings of the National Academy of Sciences.

[25]  Susan T. Dumais,et al.  Challenges for Supporting Faceted Search in Large, Heterogeneous Corpora like the Web , 2008 .

[26]  Greg Linden,et al.  Interactive Assessment of User Preference Models: The Automated Travel Assistant , 1997 .

[27]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[28]  Duane Szafron,et al.  Visual Explanation of Evidence with Additive Classifiers , 2006, AAAI.

[29]  Mor Naaman,et al.  HT06, tagging paper, taxonomy, Flickr, academic article, to read , 2006, HYPERTEXT '06.

[30]  Bernardo A. Huberman,et al.  Usage patterns of collaborative tagging systems , 2006, J. Inf. Sci..

[31]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[32]  Yvonne Kammerer,et al.  Signpost from the masses: learning effects in an exploratory social tag search browser , 2009, CHI.

[33]  Boi Faltings,et al.  Designing example-critiquing interaction , 2004, IUI '04.

[34]  Barry Smyth,et al.  Compound Critiques for Conversational Recommender Systems , 2004, IEEE/WIC/ACM International Conference on Web Intelligence (WI'04).

[35]  John Riedl,et al.  tagging, communities, vocabulary, evolution , 2006, CSCW '06.

[36]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[37]  John D. Lafferty,et al.  A study of smoothing methods for language models applied to Ad Hoc information retrieval , 2001, SIGIR '01.

[38]  Ben Shneiderman,et al.  Dynamic queries for visual information seeking , 1994, IEEE Software.

[39]  John Riedl,et al.  Is seeing believing?: how recommender system interfaces affect users' opinions , 2003, CHI '03.

[40]  Eric Harslem,et al.  Designing the STAR User Interface , 1987, ECICS.

[41]  C. Borror Generalized Linear Models and Extensions, Second Edition , 2008 .

[42]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.

[43]  Mark Hingston,et al.  User Friendly Recommender Systems , 2006 .

[44]  Martin Mozina,et al.  Nomograms for Visualization of Naive Bayesian Classifier , 2004, PKDD.

[45]  Andrzej T. Galecki,et al.  Linear mixed-effects models using R , 2013 .

[46]  Michael J. Pazzani,et al.  A personal news agent that talks, learns and explains , 1999, AGENTS '99.

[47]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[48]  Judith Masthoff,et al.  A Survey of Explanations in Recommender Systems , 2007, 2007 IEEE 23rd International Conference on Data Engineering Workshop.

[49]  Nenghai Yu,et al.  Learning to tag , 2009, WWW '09.

[50]  John Riedl,et al.  Tagommenders: connecting users to items through tags , 2009, WWW '09.

[51]  John Riedl,et al.  The quest for quality tags , 2007, GROUP.

[52]  R. Fletcher Practical Methods of Optimization , 1988 .

[53]  Li Chen,et al.  Integrating tradeoff support in product search tools for e-commerce sites , 2005, EC '05.

[54]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[55]  Kristian J. Hammond,et al.  The FindMe Approach to Assisted Browsing , 1997, IEEE Expert.

[56]  Michael S. Bernstein,et al.  Collabio: a game for annotating people within social networks , 2009, UIST '09.

[57]  Nava Tintarev,et al.  Explanations of recommendations , 2007, RecSys '07.

[58]  Ewan Klein,et al.  Natural Language Processing with Python , 2009 .

[59]  Adrian Ulges,et al.  A System That Learns to Tag Videos by Watching Youtube , 2008, ICVS.

[60]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[61]  Eric J. Johnson,et al.  The adaptive decision maker , 1993 .

[62]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[63]  Michael J. Muller,et al.  Getting our head in the clouds: toward evaluation studies of tagclouds , 2007, CHI.

[64]  Leonard Adelman,et al.  How Web Site Decision Technology Affects Consumers , 2002, IEEE Internet Comput..

[65]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[66]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[67]  Hector Garcia-Molina,et al.  Social tag prediction , 2008, SIGIR '08.

[68]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[69]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[70]  Rashmi R. Sinha,et al.  The role of transparency in recommender systems , 2002, CHI Extended Abstracts.

[71]  Loriene Roy,et al.  Content-based book recommending using learning for text categorization , 1999, DL '00.

[72]  Shinichi Honiden,et al.  Web Page Recommender System based on Folksonomy Mining for ITNG ’06 Submissions , 2006, Third International Conference on Information Technology: New Generations (ITNG'06).

[73]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .