All Work and No Play?

Many conversational agents (CAs) are developed to answer users' questions in a specialized domain. In everyday use of CAs, user experience may extend beyond satisfying information needs to the enjoyment of conversations with CAs, some of which represent playful interactions. By studying a field deployment of a Human Resource chatbot, we report on users' interest areas in conversational interactions to inform the development of CAs. Through the lens of statistical modeling, we also highlight rich signals in conversational interactions for inferring user satisfaction with the instrumental usage and playful interactions with the agent. These signals can be utilized to develop agents that adapt functionality and interaction styles. By contrasting these signals, we shed light on the varying functions of conversational interactions. We discuss design implications for CAs, and directions for developing adaptive agents based on users' conversational behaviors.

[1]  Mark Coeckelbergh,et al.  You, robot: on the linguistic construction of artificial others , 2011, AI & SOCIETY.

[2]  Sarah Sharples,et al.  "Do Animals Have Accents?": Talking with Agents in Multi-Party Conversation , 2017, CSCW.

[3]  Jodi Forlizzi,et al.  Receptionist or information kiosk: how do people talk with a robot? , 2010, CSCW '10.

[4]  Alexander I. Rudnicky,et al.  Ravenclaw: dialog management using hierarchical task decomposition and an expectation agenda , 2003, INTERSPEECH.

[5]  Susan T. Dumais,et al.  Personalizing Search via Automated Analysis of Interests and Activities , 2005, SIGIR.

[6]  J. M. Atkinson.  Transcript Notation - Structures of Social Action: Studies in Conversation Analysis , 1999 .

[7]  Csr Young,et al.  How to Do Things With Words , 2009 .

[8]  Nicole Shechtman,et al.  Media inequality in conversation: how people behave differently when interacting with computers and people , 2003, CHI '03.

[9]  Abigail Sellen,et al.  "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents , 2016, CHI.

[10]  Martin Porcheron.  Conversational agent use in a café , 2017 .

[11]  John T. Stasko,et al.  The role of choice and customization on users' interaction with embodied conversational agents: effects on perception and performance , 2007, CHI.

[12]  Mark G. Core,et al.  Coding Dialogs with the DAMSL Annotation Scheme , 1997 .

[13]  Robert Trappl,et al.  Your Virtual Butler: the making-of , 2013 .

[14]  J. Searle.  A classification of illocutionary acts , 1976, Language in Society.

[15]  Alfred Kobsa,et al.  User Models in Dialog Systems , 1989, Symbolic Computation.

[16]  John T. Stasko,et al.  An empirical study of the effect of agent competence on user performance and perception , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[17]  Robert Trappl,et al.  Your Virtual Butler , 2013, Lecture Notes in Computer Science.

[18]  Eric Gilbert,et al.  Phrases that signal workplace hierarchy , 2012, CSCW.

[19]  S. Shyam Sundar,et al.  Utilitarian vs. hedonic robots: Role of parasocial tendency and anthropomorphism in shaping user attitudes , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[20]  Bilge Mutlu,et al.  Pay attention!: designing adaptive agents that monitor and improve user engagement , 2012, CHI.

[21]  Joseph J. Martocchio,et al.  Microcomputer playfulness: development of a measure with workplace implications , 1992 .

[22]  Chenglei Yang,et al.  Effects of Pedagogical Agent's Personality and Emotional Feedback Strategy on Chinese Students' Learning Experiences and Performance: A Study Based on Virtual Tai Chi Training Studio , 2016, CHI.

[23]  Stefan Kopp,et al.  A Conversational Agent as Museum Guide - Design and Evaluation of a Real-World Application , 2005, IVA.

[24]  Yasaman Khazaeni,et al.  Leveraging Conversational Systems to Assists New Hires During Onboarding , 2017, INTERACT.

[25]  J. M. Atkinson.  Structures of Social Action: Contents , 1985 .

[26]  Jaime Teevan,et al.  Implicit feedback for inferring user preference: a bibliography , 2003, SIGF.

[27]  J. Cacioppo,et al.  On seeing human: a three-factor theory of anthropomorphism. , 2007, Psychological review.

[28]  Justine Cassell,et al.  Embodied Conversational Agents: Representation and Intelligence in User Interfaces , 2001, AI Mag..

[29]  Jane Webster,et al.  Making computer tasks at work more playful: Implications for systems analysts and designers , 1988, SIGCPR '88.

[30]  Joakim Gustafson,et al.  Speech technology on trial: Experiences from the August system , 2000, Natural Language Engineering.

[31]  A. White.  How to do things with words , 1963 .

[32]  Timothy W. Bickmore,et al.  Relational Agents Improve Engagement and Learning in Science Museum Visitors , 2011, IVA.

[33]  Ben Shneiderman,et al.  Direct manipulation vs. interface agents , 1997, INTR.

[34]  Amy Ogan,et al.  Rudeness and Rapport: Insults and Learning Gains in Peer Tutoring , 2012, ITS.

[35]  David R. Traum,et al.  Dialogues in Context: An Objective User-Oriented Evaluation Approach for Virtual Human Dialogue , 2010, LREC.

[36]  Joseph Weizenbaum,et al.  and Machine , 1977 .

[37]  Ran Zhao,et al.  Automatic Recognition of Conversational Strategies in the Service of a Socially-Aware Dialog System , 2016, SIGDIAL Conference.

[38]  Kostas Karpouzis,et al.  Investigating shared attention with a virtual agent using a gaze-based interface , 2010, Journal on Multimodal User Interfaces.

[39]  Justine Cassell,et al.  Negotiated Collusion: Modeling Social Language and its Relationship Effects in Intelligent Agents , 2003, User Modeling and User-Adapted Interaction.

[40]  J. O. Urmson,et al.  How to Do Things with Words@@@The William James Lectures , 1963 .

[41]  Timothy Bickmore,et al.  Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents , 2009, CHI.

[42]  David R. Traum,et al.  What would you Ask a conversational Agent? Observations of Human-Agent Dialogues in a Museum Setting , 2008, LREC.

[43]  J. Searle,et al.  Speech act theory and pragmatics , 1980 .

[44]  Marilyn A. Walker,et al.  PARADISE: A Framework for Evaluating Spoken Dialogue Agents , 1997, ACL.

[45]  Alexander Serenko,et al.  A model of user adoption of interface agents for email notification , 2008, Interact. Comput..

[46]  Filip Radlinski,et al.  A Theoretical Framework for Conversational Search , 2017, CHIIR.

[47]  John D. Lafferty,et al.  A risk minimization framework for information retrieval , 2006, Inf. Process. Manag..

[48]  Eric Gilbert,et al.  The language that gets people to give: phrases that predict success on kickstarter , 2014, CSCW.

[49]  Imed Zitouni,et al.  Automatic Online Evaluation of Intelligent Assistants , 2015, WWW.

[50]  William Yang Wang,et al.  “Love ya, jerkface”: Using Sparse Log-Linear Models to Build Positive and Impolite Relationships with Teens , 2012, SIGDIAL Conference.

[51]  Yang Song,et al.  Optimal rare query suggestion with implicit user feedback , 2010, WWW '10.

[52]  Amy Ogan,et al.  "Oh dear stacy!": social interaction, elaboration, and learning with teachable agents , 2012, CHI.

[53]  Stefan Kopp,et al.  To Err is Human(-like): Effects of Robot Gesture on Perceived Anthropomorphism and Likability , 2013, International Journal of Social Robotics.

[54]  Susanne van Mulken,et al.  The impact of animated interface agents: a review of empirical research , 2000, Int. J. Hum. Comput. Stud..

[55]  N. Sadat Shami,et al.  What Can You Do?: Studying Social-Agent Orientation and Agent Proactive Interactions with an Agent for Employees , 2016, Conference on Designing Interactive Systems.

[56]  Ana Paiva,et al.  Automatic analysis of affective postures and body motion to detect engagement with a game companion , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[57]  Niels Ole Bernsen,et al.  Domain-Oriented Conversation with H.C. Andersen , 2004, ADS.

[58]  Stefan Kopp,et al.  Towards Conversational Agents That Attend to and Adapt to Communicative User Feedback , 2011, IVA.