Understanding How People Use Natural Language to Ask for Recommendations

The technical barriers for conversing with recommender systems using natural language are vanishing. Already, there are commercial systems that facilitate interactions with an AI agent. For instance, it is possible to say "what should I watch" to an Apple TV remote to get recommendations. In this research, we investigate how users initially interact with a new natural language recommender to deepen our understanding of the range of inputs that these technologies can expect. We deploy a natural language interface to a recommender system, we observe users' first interactions and follow-up queries, and we measure the differences between speaking- and typing-based interfaces. We employ qualitative methods to derive a categorization of users' first queries (objective, subjective, and navigation) and follow-up queries (refine, reformulate, start over). We employ quantitative methods to determine the differences between speech and text, finding that speech inputs are typically longer and more conversational.

[1]  Daqing He,et al.  How do users respond to voice input errors?: lexical and phonetic query reformulation in voice search , 2013, SIGIR.

[2]  H. Bernard,et al.  Techniques to Identify Themes , 2003 .

[3]  Peter Pirolli,et al.  Information Foraging , 2009, Encyclopedia of Database Systems.

[4]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[5]  Sean M. McNee,et al.  Making recommendations better: an analytic model for human-recommender interaction , 2006, CHI Extended Abstracts.

[6]  Robert E. Kraut,et al.  Expressive richness: a comparison of speech and text as media for revision , 1991, CHI.

[7]  Lorraine McGinty,et al.  On the Evolution of Critiquing Recommenders , 2011, Recommender Systems Handbook.

[8]  Li Chen,et al.  A user-centric evaluation framework for recommender systems , 2011, RecSys '11.

[9]  Barry Smyth,et al.  Compound Critiques for Conversational Recommender Systems , 2004, IEEE/WIC/ACM International Conference on Web Intelligence (WI'04).

[10]  Ido Guy,et al.  Searching by Talking: Analysis of Voice Queries on Mobile Web Search , 2016, SIGIR.

[11]  Mehrbakhsh Nilashi,et al.  Collaborative filtering recommender systems , 2013 .

[12]  John Riedl,et al.  The Tag Genome: Encoding Community Knowledge to Support Novel Interaction , 2012, TIIS.

[13]  F. Maxwell Harper,et al.  Understanding How People Use Natural Language to Ask for Recommendations: Query Dataset , 2017 .

[14]  Aaron Sedley,et al.  Survey Research in HCI , 2014, Ways of Knowing in HCI.

[15]  Shumeet Baluja,et al.  A large scale study of wireless search behavior: Google mobile search , 2006, CHI.

[16]  Mark S. Ackerman,et al.  The perfect search engine is not enough: a study of orienteering behavior in directed search , 2004, CHI.

[17]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[18]  Jia Zhang,et al.  Evaluating Item-Item Similarity Algorithms for Movies , 2016, CHI Extended Abstracts.

[19]  Pat Langley,et al.  A Personalized System for Conversational Recommendations , 2011, J. Artif. Intell. Res..

[20]  Amanda Spink,et al.  Determining the informational, navigational, and transactional intent of Web queries , 2008, Inf. Process. Manag..

[21]  Aaron E. Rosenberg,et al.  SCANMail: a voicemail interface that makes speech browsable, readable and searchable , 2002, CHI.

[22]  Daniel E. Rose,et al.  Understanding user goals in web search , 2004, WWW '04.

[23]  Ed H. Chi,et al.  Using information scent to model user information needs and actions and the Web , 2001, CHI.

[24]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[25]  Fabio Crestani,et al.  Written versus spoken queries: A qualitative and quantitative comparative analysis , 2006, J. Assoc. Inf. Sci. Technol..

[26]  Jane F. Dye,et al.  Constant Comparison Method: A Kaleidoscope of Data , 2000 .

[27]  Cynthia A. Thompson,et al.  Personalized Conversational Case-Based Recommendation , 2000, EWCBR.

[28]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.