Human-Centered and Interactive: Expanding the Impact of Topic Models

Statistical topic modeling is a common tool for summarizing the themes in a document corpus. Due to the complexity of topic modeling algorithms, however, their results are not accessible to non-expert users. Recent work in interactive topic modeling looks to incorporate the user into the inference loop, for example, by allowing them to view a model then update it by specifying important words and words that should be ignored. However, the majority of interactive topic modeling work has been performed without fully understanding the needs of the end user and does not adequately consider challenges that arise in interactive machine learning. In this paper, we outline a subset of interactive machine learning design challenges with specific considerations for interactive topic modeling. For each challenge, we propose solutions based on prior work and our own preliminary findings and identify open questions to guide future work.

[1]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[2]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[3]  Rashmi R. Sinha,et al.  The role of transparency in recommender systems , 2002, CHI Extended Abstracts.

[4]  Jr. Robert Hoekman.  Designing the Obvious: A Common Sense Approach to Web Application Design , 2002 .

[5]  Jerry Alan Fails,et al.  Interactive machine learning , 2003, IUI '03.

[6]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[7]  Robert E. Kraut,et al.  Motivating participation by displaying the value of contribution , 2006, CHI.

[8]  Xiaojin Zhu,et al.  Incorporating domain knowledge into topic modeling via Dirichlet Forest priors , 2009, ICML '09.

[9]  Andrew McCallum,et al.  Rethinking LDA: Why Priors Matter , 2009, NIPS.

[10]  Matt Gardner.  The Topic Browser An Interactive Tool for Browsing Topic Models , 2010 .

[11]  David M. Blei,et al.  Probabilistic topic models , 2012, Commun. ACM.

[12]  Timothy Baldwin,et al.  Automatic Evaluation of Topic Coherence , 2010, NAACL.

[13]  Stephanie Rosenthal,et al.  Towards maximizing the accuracy of human-labeled sensor data , 2010, IUI '10.

[14]  Desney S. Tan,et al.  Effective End-User Interaction with Machine Learning , 2011, AAAI.

[15]  Andrew McCallum,et al.  Optimizing Semantic Coherence in Topic Models , 2011, EMNLP.

[16]  Aniket Kittur,et al.  TopicViz: interactive topic exploration in document collections , 2012, CHI Extended Abstracts.

[17]  David M. Blei,et al.  Visualizing Topic Models , 2012, ICWSM.

[18]  Jeffrey Heer,et al.  Termite: visualization techniques for assessing textual topic models , 2012, AVI.

[19]  John T. Stasko,et al.  iVisClustering: An Interactive Visual Document Clustering via Topic Modeling , 2012, Comput. Graph. Forum.

[20]  Jaegul Choo,et al.  UTOPIAN: User-Driven Topic Modeling Based on Interactive Nonnegative Matrix Factorization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[21]  Quentin Pleple,et al.  Interactive Topic Modeling , 2013 .

[22]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[23]  Jordan L. Boyd-Graber,et al.  12 Care and Feeding of Topic Models : Problems , Diagnostics , and Improvementes , 2014 .

[24]  Jeffrey Heer,et al.  TopicCheck: Interactive Alignment for Assessing Topic Model Stability , 2015, NAACL.

[25]  Ben Shneiderman,et al.  Visual Analysis of Topical Evolution in Unstructured Text: Design and Evaluation of TopicFlow , 2015, Applications of Social Media and Social Network Analysis.