Understanding Affective Experiences with Conversational Agents

While previous studies of Conversational Agents (e.g. Siri, Google Assistant, Alexa and Cortana) have focused on evaluating usability and exploring capabilities of these systems, little work has examined users' affective experiences. In this paper we present a survey study with 171 participants to examine CA users' affective experiences. Specifically, we present four major usage scenarios, users' affective responses in these scenarios, and the factors which influenced the affective responses. We found that users' overall experience was positive with interest being the most salient positive emotion. Affective responses differed depending on the scenarios. Both pragmatic and hedonic qualities influenced affect. The factors underlying pragmatic quality are: helpfulness, proactivity, fluidity, seamlessness and responsiveness. The factors underlying hedonic quality are: comfort in human-machine conversation, pride of using cutting-edge technology, fun during use, perception of having a human-like assistant, concern about privacy and fear of causing distraction.

[1]  Florian Alt,et al.  Understanding Shoulder Surfing in the Wild: Stories from Users and Observers , 2017, CHI.

[2]  Marc Hassenzahl,et al.  The Inference of Perceived Usability From Beauty , 2010, Hum. Comput. Interact..

[3]  Sabine Payr,et al.  Virtual Butlers and Real People: Styles and Practices in Long-Term Use of a Companion , 2013, Your Virtual Butler.

[4]  Deborah Richards,et al.  Changing stigmatizing attitudes to mental health via education and contact with embodied conversational agents , 2017, Comput. Hum. Behav..

[5]  J. C. Flanagan.  Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[6]  JungKyoon Yoon,et al.  Design for Interest: Exploratory Study on a Distinct Positive Emotion in Human-Product Interaction , 2012 .

[7]  Sarah Diefenbach,et al.  Experience-Oriented and Product-Oriented Evaluation: Psychological Need Fulfillment, Positive Affect, and Product Perception , 2015, Int. J. Hum. Comput. Interact..

[8]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[9]  Jodi Forlizzi,et al.  Understanding experience in interactive systems , 2004, DIS '04.

[10]  Benjamin R. Cowan,et al.  "What can i help you with?": infrequent users' experiences of intelligent personal assistants , 2017, MobileHCI.

[11]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[12]  Kasper Hornbæk,et al.  Momentary Pleasure or Lasting Meaning?: Distinguishing Eudaimonic and Hedonic User Experiences , 2016, CHI.

[13]  Guang-Jie Ren,et al.  Conversational UX Design , 2017, CHI Extended Abstracts.

[14]  Shruti Sannon,et al.  "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo , 2017, CHI Extended Abstracts.

[15]  Jennifer Zamora,et al.  Rise of the Chatbots: Finding A Place for Artificial Intelligence in India and US , 2017, IUI Companion.

[16]  Andreas Beu,et al.  Engineering Joy , 2001, IEEE Softw..

[17]  Charles L. A. Clarke,et al.  Exploring Conversational Search With Humans, Assistants, and Wizards , 2017, CHI Extended Abstracts.

[18]  Marc Hassenzahl,et al.  MARC HASSENZAHL CHAPTER 3 The Thing and I: Understanding the Relationship Between User and Product , 2003 .

[19]  Ho Ming Lau,et al.  Embodied Conversational Agents in Clinical Psychology: A Scoping Review , 2017, Journal of medical Internet research.

[20]  Bettina Graf,et al.  Nombot: simplify food tracking , 2015, MUM.

[21]  A. Graesser,et al.  Assessing Collaborative Problem Solving Through Conversational Agents , 2017 .

[22]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[23]  Sarah Diefenbach,et al.  Needs, affect, and interactive products - Facets of user experience , 2010, Interact. Comput..

[24]  Kaisa Väänänen,et al.  Analysing user experience of personal mobile products through contextual factors , 2010, MUM.

[25]  Lawrence R. Burns,et al.  Facets of dynamic positive affect: differentiating joy, interest, and activation in the positive and negative affect schedule (PANAS). , 2003, Journal of personality and social psychology.

[26]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[27]  Ves Dimov,et al.  Smartphone Conversational Agents (Apple Siri, Google, Windows Cortana) and Questions about Allergy and Asthma Emergencies , 2017 .

[28]  R. J. Hunt,et al.  Percent Agreement, Pearson's Correlation, and Kappa as Measures of Inter-examiner Reliability , 1986, Journal of dental research.

[29]  Bert Vandenberghe.  Bot Personas as Off-The-Shelf Users , 2017, CHI Extended Abstracts.

[30]  Malte F. Jung.  Affective Grounding in Human-Robot Interaction , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[31]  Kennon M. Sheldon,et al.  What is satisfying about satisfying events? Testing 10 candidate psychological needs. , 2001, Journal of personality and social psychology.

[32]  Quynh N. Nguyen,et al.  AI capabilities and user experiences: a comparative study of user reviews for assistant and non-assistant mobile apps , 2017, AMCIS.

[33]  P. Desmet.  Faces of Product Pleasure: 25 Positive Emotions in Human-Product Interactions , 2012 .

[34]  Alexandre N. Tuch,et al.  Analyzing users' narratives to understand experience with interactive products , 2013, CHI.

[35]  Rana El Kaliouby,et al.  On the Future of Personal Assistants , 2016, CHI Extended Abstracts.

[36]  Klaus Opwis,et al.  Facets In HCI: Towards Understanding Eudaimonic UX -- Preliminary Findings , 2015, CHI Extended Abstracts.

[37]  Kristinn R. Thórisson,et al.  The Power of a Nod and a Glance: Envelope Vs. Emotional Feedback in Animated Conversational Agents , 1999, Appl. Artif. Intell..

[38]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[39]  Anton Nijholt,et al.  Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes , 2001, CHI.

[40]  Sandeep Avula,et al.  Searchbots: Using Chatbots in Collaborative Information-seeking Tasks , 2017, SIGIR.

[41]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[42]  Klaus Krippendorff,et al.  Computing Krippendorff's Alpha-Reliability , 2011 .

[43]  B. Fredrickson,et al.  Open hearts build lives: positive emotions, induced through loving-kindness meditation, build consequential personal resources. , 2008, Journal of personality and social psychology.

[44]  Imed Zitouni,et al.  Understanding User Satisfaction with Intelligent Assistants , 2016, CHIIR.

[45]  Abigail Sellen,et al.  "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents , 2016, CHI.

[46]  P. Jordan.  Designing Pleasurable Products: An Introduction to the New Human Factors , 2000 .

[47]  W. E. Scott.  Activation theory and task design , 1966 .