Faster is Not Always Better: Understanding the Effect of Dynamic Response Delays in Human-Chatbot Interaction

A key challenge in designing conversational user interfaces is to make the conversation between the user and the system feel natural and human-like. In order to increase perceived humanness, many systems with conversational user interfaces (e.g., chatbots) use response delays to simu-late the time it would take humans to respond to a message. However, delayed responses may also negatively impact user satisfaction, particularly in situations where fast response times are expected, such as in customer service. This paper reports the findings of an online experiment in a customer service context that investigates how user perceptions differ when interacting with a chatbot that sends dynamically delayed responses compared to a chatbot that sends near-instant responses. The dynamic delay length was calculated based on the complexity of the re-sponse and complexity of the previous message. Our results indicate that dynamic response de-lays not only increase users’ perception of humanness and social presence, but also lead to greater satisfaction with the overall chatbot interaction. Building on social response theory, we provide evidence that a chatbot’s response time represents a social cue that triggers social re-sponses shaped by social expectations. Our findings support researchers and practitioners in understanding and designing more natural human-chatbot interactions.

[1]  J. Dietz Satisfaction: A Behavioral Perspective on the Consumer , 1997 .

[2]  L. R. Goldberg THE DEVELOPMENT OF MARKERS FOR THE BIG-FIVE FACTOR STRUCTURE , 1992 .

[3]  Izak Benbasat,et al.  Effects of rational and social appeals of online recommendation agents on cognition- and affect-based trust , 2016, Decis. Support Syst..

[4]  T. M. Holtgraves,et al.  Perceiving artificial social agents , 2007, Comput. Hum. Behav..

[5]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[6]  Nicole C. Krämer,et al.  "It doesn't matter what you are!" Explaining social effects of agents and avatars , 2010, Comput. Hum. Behav..

[7]  John A. Hoxmeier,et al.  System Response Time and User Satisfaction: An Experimental Study of Browser-based Applications , 2000 .

[8]  Detmar W. Straub,et al.  A Practical Guide To Factorial Validity Using PLS-Graph: Tutorial And Annotated Example , 2005, Commun. Assoc. Inf. Syst..

[9]  Daniel P. Siewiorek,et al.  Using Crowd Sourcing to Measure the Effects of System Response Delays on User Engagement , 2016, CHI.

[10]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[11]  H. Ishiguro,et al.  The uncanny advantage of using androids in cognitive and social science research , 2006 .

[12]  Alessandro Bogliolo,et al.  The Rise of Bots: A Survey of Conversational Interfaces, Patterns, and Paradigms , 2017, Conference on Designing Interactive Systems.

[13]  Tibert Verhagen,et al.  Virtual Customer Service Agents: Using Social Presence and Personalization to Shape Online Service Encounters , 2014, J. Comput. Mediat. Commun..

[14]  Youngme Moon Intimate Exchanges: Using Computers to Elicit Self-Disclosure from Consumers , 2000 .

[15]  Jonathan Gratch,et al.  Exploring users' social responses to computer counseling interviewers' behavior , 2014, Comput. Hum. Behav..

[16]  Izak Benbasat,et al.  Evaluating Anthropomorphic Product Recommendation Agents: A Social Relationship Perspective to Designing Information Systems , 2009, J. Manag. Inf. Syst..

[17]  Milena M. Head,et al.  Manipulating perceived social presence through the web interface and its impact on attitude towards online shopping , 2007, Int. J. Hum. Comput. Stud..

[18]  G. Zinkhan,et al.  Determinants of Perceived Web Site Interactivity , 2008 .

[19]  Avi Rushinek,et al.  What makes users happy? , 1986, CACM.

[20]  Robert Dale,et al.  The return of the chatbots , 2016, Natural Language Engineering.

[21]  Guang-Jie Ren,et al.  Conversational UX Design , 2017, CHI Extended Abstracts.

[22]  Steve Muylle,et al.  The conceptualization and empirical validation of web site user satisfaction , 2004, Inf. Manag..

[23]  Claudio S. Pinhanez,et al.  Typefaces and the Perception of Humanness in Natural Language Chatbots , 2017, CHI.

[24]  David Gefen,et al.  Managing User Trust in B2C e-Services , 2003 .

[25]  Chuan-Hoo Tan,et al.  The Impact of Intra-Transaction Communication on Customer Purchase Behaviour in E-Commerce Context , 2013, ACIS.

[26]  Alicia A. Grandey,et al.  Service with a smile and encounter satisfaction: emotional contagion and appraisal mechanisms , 2006 .

[27]  Detmar W. Straub,et al.  Gender Differences in the Perception and Use of E-Mail: An Extension to the Technology Acceptance Model , 1997, MIS Q..

[28]  Michael Marien,et al.  Book Review: The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies , 2014 .

[29]  Milena M. Head,et al.  Exploring human images in website design: a multi-method approach , 2009 .

[30]  Berhane Teclehaimanot,et al.  Two Peas in a Pod? A Comparison of Face-to-Face and Web Based Classrooms , 2004 .

[31]  Youngme Moon,et al.  The effects of physical distance and response latency on persuasion in computer-mediated communication and human–computer communication. , 1999 .

[32]  Nicole C. Krämer,et al.  Does Humanity Matter? Analyzing the Importance of Social Cues and Perceived Agency of a Computer System for the Emergence of Social Reactions during Human-Computer Interaction , 2012, Adv. Hum. Comput. Interact..

[33]  Justin Scott Giboney,et al.  Facilitating Natural Conversational Agent Interactions: Lessons from a Deception Experiment , 2014, ICIS.

[34]  Thomas Holtgraves,et al.  A procedure for studying online conversational processing using a chat bot , 2007, Behavior research methods.

[35]  Bart P. Knijnenburg,et al.  Inferring Capabilities of Intelligent Agents , 2014 .

[36]  Graeme McLean,et al.  Evolving the online customer experience ... is there a role for online customer support? , 2015, Comput. Hum. Behav..

[37]  Fang Chen,et al.  Using language complexity to measure cognitive load for adaptive interaction design , 2010, IUI '10.

[38]  Mary Jo Bitner,et al.  Technology infusion in service encounters , 2000 .

[39]  Mark A. Fuller,et al.  Designing Interfaces with Social Presence: Using Vividness and Extraversion to Create Social Recommendation Agents , 2009, J. Assoc. Inf. Syst..

[40]  David Griol,et al.  The Conversational Interface: Talking to Smart Devices , 2016 .

[41]  Barbara H Wixom,et al.  A Theoretical Integration of User Satisfaction and Technology Acceptance , 2005, Inf. Syst. Res..

[42]  Asbjørn Følstad,et al.  Chatbots and the new world of HCI , 2017, Interactions.

[43]  Izak Benbasat,et al.  E-Commerce Product Recommendation Agents: Use, Characteristics, and Impact , 2007, MIS Q..

[44]  Xiuwen Liu,et al.  Computer-Mediated Deception: Strategies Revealed by Language-Action Cues in Spontaneous Communication , 2016, J. Manag. Inf. Syst..

[45]  Georgios Paltoglou,et al.  No Peanuts! Affective Cues for the Virtual Bartender , 2011, FLAIRS.

[46]  Joseph B. Walther,et al.  Selective self-presentation in computer-mediated communication: Hyperpersonal dimensions of technology, language, and cognition , 2007, Comput. Hum. Behav..

[47]  Alexander Maedche,et al.  Towards Designing Cooperative and Social Conversational Agents for Customer Service , 2017, ICIS.

[48]  Joseph Weizenbaum,et al.  and Machine , 1977 .

[49]  Dafydd Gibbon,et al.  Assessment of interactive systems. , 1998 .

[50]  Nicole Shechtman,et al.  Media inequality in conversation: how people behave differently when interacting with computers and people , 2003, CHI '03.

[51]  Alexander Maedche,et al.  Designing Conversational Agents for Energy Feedback , 2018, DESRIST.

[52]  Arne De Keyser,et al.  “Service Encounter 2.0”: an investigation into the roles of technology, employees and customers , 2017 .

[53]  Chayan Chakrabarti,et al.  Artificial conversations for customer service chatter bots: Architecture, algorithms, and evaluation metrics , 2015, Expert Syst. Appl..

[54]  Amy L. Ostrom,et al.  Domo Arigato Mr. Roboto , 2017 .

[55]  Jay F. Nunamaker,et al.  Detecting Deceptive Chat-Based Communication Using Typing Behavior and Message Cues , 2013, TMIS.

[56]  Tetsuo Ono,et al.  A humanoid robot that pretends to listen to route guidance from a human , 2007, Auton. Robots.

[57]  B. J. Fogg,et al.  Persuasive technology: using computers to change what we think and do , 2002, UBIQ.

[58]  Alexander Maedche,et al.  Advanced User Assistance Systems , 2016, Bus. Inf. Syst. Eng..

[59]  Bart P. Knijnenburg,et al.  Inferring Capabilities of Intelligent Agents from Their External Traits , 2016, ACM Trans. Interact. Intell. Syst..

[60]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[61]  Clifford Nass,et al.  Computers are social actors , 1994, CHI '94.

[62]  Steven M. Drucker,et al.  Alternative interfaces for chat , 1999, UIST '99.

[63]  Hal Burdick,et al.  THE LEXILE FRAMEWORK AS AN APPROACH FOR READING MEASUREMENT AND SUCCESS , 2004 .

[64]  Stephen J. Cox,et al.  Analysis of User Interaction with Service Oriented Chatbot Systems , 2007, HCI.

[65]  Mohammed Slim Ben Mimoun,et al.  Case study—Embodied virtual agents: An analysis on reasons for failure , 2012 .

[66]  M. Mori THE UNCANNY VALLEY , 2020, The Monster Theory Reader.

[67]  Poornima Madhavan,et al.  A note of caution regarding anthropomorphism in HCI agents , 2013, Comput. Hum. Behav..

[68]  Ruhi Sarikaya An overview of the system architecture and key components The Technology Behind Personal Digital Assistants , 2022 .

[69]  C. Nass,et al.  Machines and Mindlessness , 2000 .

[70]  Nils Urbach,et al.  Structural Equation Modeling in Information Systems Research Using Partial Least Squares , 2010 .

[71]  Youngme Moon,et al.  Don’t Blame the Computer: When Self-Disclosure Moderates the Self-Serving Bias , 2003 .

[72]  S. N. Alexander,et al.  Communication between Man and Machine , 1962, Proceedings of the IRE.