In the shades of the uncanny valley: An experimental study of human-chatbot interaction

Abstract This project has been carried out in the context of recent major developments in botics and more widespread usage of virtual agents in personal and professional sphere. The general purpose of the experiment was to thoroughly examine the character of the human–non-human interaction process. Thus, in the paper, we present a study of human–chatbot interaction, focusing on the affective responses of users to different types of interfaces with which they interact. The experiment consisted of two parts: measurement of psychophysiological reactions of chatbot users and a detailed questionnaire that focused on assessing interactions and willingness to collaborate with a bot. In the first quantitative stage, participants interacted with a chatbot, either with a simple text chatbot (control group) or an avatar reading its responses in addition to only presenting them on the screen (experimental group. We gathered the following psychophysiological data from participants: electromyography (EMG), respirometer (RSP), electrocardiography (ECG), and electrodermal activity (EDA). In the last, declarative stage, participants filled out a series of questionnaires related to the experience of interacting with (chat)bots and to the overall human–(chat)bot collaboration assessment. The theory of planned behaviour survey investigated attitude towards cooperation with chatbots in the future. The social presence survey checked how much the chatbot was considered to be a “real” person. The anthropomorphism scale measured the extent to which the chatbot seems humanlike. Our particular focus was on the so-called uncanny valley effect, consisting of the feeling of eeriness and discomfort towards a given medium or technology that frequently appears in various kinds of human–machine interactions. Our results show that participants were experiencing lesser uncanny effects and less negative affect in cooperation with a simpler text chatbot than with the more complex, animated avatar chatbot. The simple chatbot have also induced less intense psychophysiological reactions. Despite major developments in botics, the user’s affective responses towards bots have frequently been neglected. In our view, understanding the user’s side may be crucial for designing better chatbots in the future and, thus, can contribute to advancing the field of human–computer interaction.

[1]  John Woods,et al.  Survey on Chatbot Design Techniques in Speech Conversation Systems , 2015 .

[2]  Sven-Thomas Graupner,et al.  Virtual friend or threat? The effects of facial expression and gaze interaction on psychophysiological responses and emotional experience. , 2009, Psychophysiology.

[3]  Oliver C. Schultheiss,et al.  Implicit need for affiliation is associated with increased corrugator activity in a non-positive, but not in a positive social interaction , 2012 .

[4]  Slawomir Zadrozny,et al.  Computing With Words Is an Implementable Paradigm: Fuzzy Queries, Linguistic Data Summaries, and Natural-Language Generation , 2010, IEEE Transactions on Fuzzy Systems.

[5]  A. J. Fridlund,et al.  Guidelines for human electromyographic research. , 1986, Psychophysiology.

[6]  Victor R. Basili,et al.  Experimentation in software engineering , 1986, IEEE Transactions on Software Engineering.

[7]  M. Bradley,et al.  Measuring emotion: Behavior, feeling, and physiology , 2000 .

[8]  Edward Moemeka,et al.  Leveraging Cortana and Speech , 2015 .

[9]  H. Ishiguro,et al.  The uncanny advantage of using androids in cognitive and social science research , 2006 .

[10]  Lakshmi Kurup,et al.  Study of a Home Robot: JIBO , 2014 .

[11]  B. Appelhans,et al.  Heart Rate Variability as an Index of Regulated Emotional Responding , 2006 .

[12]  W. R. Ford,et al.  Real conversations with artificial intelligence: A comparison between human-human online conversations and human-chatbot conversations , 2015, Comput. Hum. Behav..

[13]  E. Hutchins.  Cognition in the wild , 1995 .

[14]  Amy J. C. Cuddy,et al.  A model of (often mixed) stereotype content: competence and warmth respectively follow from perceived status and competition. , 2002, Journal of personality and social psychology.

[15]  Sascha Topolinski,et al.  Corrugator activity confirms immediate negative affect in surprise , 2015, Front. Psychol..

[16]  Stephen J. Cox,et al.  Analysis of User Interaction with Service Oriented Chatbot Systems , 2007, HCI.

[17]  Emilia I. Barakova,et al.  Social Interaction in Robotic Agents Emulating the Mirror Neuron Function , 2007, IWINAC.

[18]  J. Cacioppo,et al.  Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. , 1986, Journal of personality and social psychology.

[19]  S. Shimojo,et al.  Interpersonal body and neural synchronization as a marker of implicit social interaction , 2012, Scientific Reports.

[20]  Peter Ford Dominey,et al.  I Reach Faster When I See You Look: Gaze Effects in Human–Human and Human–Robot Face-to-Face Cooperation , 2012, Front. Neurorobot..

[21]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[22]  Jun'ichiro Seyama,et al.  The Uncanny Valley: Effect of Realism on the Impression of Artificial Human Faces , 2007, PRESENCE: Teleoperators and Virtual Environments.

[23]  I. René J. A. te Boekhorst,et al.  Avoiding the uncanny valley: robot appearance, personality and consistency of behavior in an attention-seeking home scenario for a robot companion , 2008, Auton. Robots.

[24]  P. Pauli,et al.  Arousal, valence, and the uncanny valley: psychophysiological and self-report findings , 2015, Front. Psychol..

[25]  Y. Wilks,et al.  CONVERSE: a Conversational Companion , 1999 .

[26]  A. Gillespie,et al.  A truly human interface: interacting face-to-face with someone whose words are determined by a computer program , 2015, Front. Psychol..

[27]  Joseph Weizenbaum,et al.  and Machine , 1977 .

[28]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[29]  Chung How Kau,et al.  Genome-Wide Association Study Reveals Multiple Loci Influencing Normal Human Facial Morphology , 2016, PLoS genetics.

[30]  Alex Gillespie,et al.  The Body That Speaks: Recombining Bodies and Speech Sources in Unscripted Face-to-Face Communication , 2016, Front. Psychol..

[31]  Karl J. Friston,et al.  Modelling event-related skin conductance responses , 2010, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[32]  J. Decety,et al.  The Role of the Right Temporoparietal Junction in Social Interaction: How Low-Level Computational Processes Contribute to Meta-Cognition , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[33]  M. Bradley,et al.  Motivated attention: Affect, activation, and action. , 1997 .

[34]  James D. Hollan,et al.  Graphical multiscale Web histories: a study of padprints , 1998, HYPERTEXT '98.

[35]  Kellie Morrissey,et al.  'Realness' in Chatbots: Establishing Quantifiable Criteria , 2013, HCI.

[36]  Florin Dolcos,et al.  Brain imaging investigation of the neural correlates of observing virtual social interactions. , 2011, Journal of visualized experiments : JoVE.

[37]  Grzegorz Pochwatko,et al.  Polish Version of the Negative Attitude Toward Robots Scale (NARS-PL) , 2015, J. Autom. Mob. Robotics Intell. Syst..

[38]  D. Lakens,et al.  Why Psychologists Should by Default Use Welch's t-test Instead of Student's t-test with Unequal Group Sizes , 2017 .

[39]  M. Bartlett,et al.  Bridging the Mechanical and the Human Mind: Spontaneous Mimicry of a Physically Present Android , 2014, PloS one.